Decision Trees Cheat Sheet
A reference for decision trees covering scikit-learn classifiers and regressors, splitting criteria like Gini and entropy, pruning, and feature importance.
1 PageBeginnerMar 5, 2026
Classifier with scikit-learn
Fit and visualize a decision tree.
python
from sklearn.tree import DecisionTreeClassifier, plot_treeimport matplotlib.pyplot as pltclf = DecisionTreeClassifier( criterion='gini', max_depth=5, min_samples_leaf=10, random_state=42)clf.fit(X_train, y_train)plt.figure(figsize=(12, 8))plot_tree(clf, feature_names=feature_names, class_names=class_names, filled=True)plt.show()
Splitting Criteria
The math used to choose each split.
python
# Gini impurity: G = 1 - sum(p_i^2) over classes# Entropy: H = -sum(p_i * log2(p_i))# Information Gain = H(parent) - weighted_avg(H(children))from sklearn.tree import DecisionTreeRegressorreg = DecisionTreeRegressor(criterion='squared_error', max_depth=4)reg.fit(X_train, y_train) # regression trees split to minimize variance (MSE)
Feature Importance
Inspect which features drove the splits.
python
importances = clf.feature_importances_for name, imp in sorted(zip(feature_names, importances), key=lambda x: -x[1]): print(f'{name}: {imp:.3f}')
Key Concepts
Core theory behind decision trees.
- Gini impurity- Probability of misclassifying a randomly chosen sample; 0 means a perfectly pure node
- Entropy- Information-theoretic impurity measure; higher entropy means more disorder within a node
- Pruning- Reduces overfitting via max_depth limits (pre-pruning) or cost-complexity pruning with ccp_alpha (post-pruning)
- max_depth / min_samples_leaf- Core hyperparameters that trade off tree complexity against overfitting risk
- Overfitting- Unconstrained trees can memorize training data perfectly but generalize poorly
Pro Tip
A single unconstrained decision tree almost always overfits — tune max_depth, min_samples_leaf, or ccp_alpha cost-complexity pruning, or better yet use the tree only as a base learner inside a Random Forest or Gradient Boosting ensemble.
Was this cheat sheet helpful?
Explore Topics
#DecisionTrees#DecisionTreesCheatSheet#DataScience#Beginner#ClassifierWithScikitLearn#SplittingCriteria#FeatureImportance#KeyConcepts#DataStructures#MachineLearning#CheatSheet#SkillVeris
Advertisement
Sri Hayavadhana Info-Tech
Professional Web Designing Services
- Responsive Websites
- E-commerce Solutions
- SEO Friendly Design
- Fast & Secure
- Support & Maintenance
