LightGBM Cheat Sheet
LightGBM reference covering the scikit-learn and native Dataset APIs, leaf-wise tree growth, and the hyperparameters that control speed and overfitting.
2 PagesIntermediateApr 15, 2026
Scikit-learn API
Familiar fit/predict interface.
python
import lightgbm as lgbfrom lightgbm import LGBMClassifiermodel = LGBMClassifier( n_estimators=500, num_leaves=31, learning_rate=0.05, subsample=0.8, colsample_bytree=0.8, random_state=42,)model.fit( X_train, y_train, eval_set=[(X_val, y_val)], callbacks=[lgb.early_stopping(stopping_rounds=30)],)preds = model.predict(X_test)
Native Dataset API
Lower-level API with more control.
python
train_data = lgb.Dataset(X_train, label=y_train)val_data = lgb.Dataset(X_val, label=y_val, reference=train_data)params = { "objective": "binary", "metric": "binary_logloss", "num_leaves": 31, "learning_rate": 0.05,}booster = lgb.train( params, train_data, num_boost_round=500, valid_sets=[val_data], callbacks=[lgb.early_stopping(30), lgb.log_evaluation(50)],)
Key Hyperparameters
Parameters that matter most for tuning.
- num_leaves- main complexity control for leaf-wise tree growth
- max_depth- limits tree depth (default -1 = unlimited)
- learning_rate- shrinkage applied to each new tree
- feature_fraction- fraction of features sampled per iteration
- bagging_fraction- fraction of rows sampled per iteration
- min_data_in_leaf- minimum samples per leaf, prevents overfitting
- categorical_feature- column names/indices for native categorical handling
Cross-Validation
Built-in CV helper for the native API.
python
cv_results = lgb.cv( params, train_data, num_boost_round=500, nfold=5, stratified=True, callbacks=[lgb.early_stopping(30)],)print(min(cv_results["valid binary_logloss-mean"]))
Custom Objective & Feval
Plug in a custom loss and evaluation metric for the native API.
python
import numpy as npdef focal_loss_obj(preds, train_data): labels = train_data.get_label() p = 1.0 / (1.0 + np.exp(-preds)) gamma = 2.0 grad = p - labels # simplified focal-style gradient hess = p * (1.0 - p) * (1 + gamma * np.abs(labels - p)) return grad, hessdef feval_auc(preds, train_data): from sklearn.metrics import roc_auc_score labels = train_data.get_label() return "custom_auc", roc_auc_score(labels, preds), True # higher-is-betterbooster = lgb.train( {"num_leaves": 31}, train_data, num_boost_round=300, fobj=focal_loss_obj, feval=feval_auc, valid_sets=[val_data],)
GOSS & DART Boosting Types
Alternative boosting strategies for large data and overfitting control.
python
# GOSS: keeps large-gradient samples, samples the rest -- good for big datagoss_params = { "boosting_type": "goss", "top_rate": 0.2, "other_rate": 0.1, "num_leaves": 63,}# DART: drops trees during training to fight overfittingdart_params = { "boosting_type": "dart", "drop_rate": 0.1, "max_drop": 50, "skip_drop": 0.5,}booster = lgb.train(goss_params, train_data, num_boost_round=500, valid_sets=[val_data])
Native Categoricals & Monotone Constraints
Handle high-cardinality categoricals and enforce feature direction.
python
df["city"] = df["city"].astype("category") # pandas categorical dtypetrain_data = lgb.Dataset( df[features], label=df["target"], categorical_feature=["city", "device_type"], free_raw_data=False,)params = { "objective": "regression", "monotone_constraints": [1, -1, 0, 0, 0], # per-feature direction "monotone_constraints_method": "advanced", "cat_smooth": 10.0, # regularizes categorical splits on rare categories}booster = lgb.train(params, train_data, num_boost_round=400)
SHAP Contributions & Importance Types
Explain predictions and compare split-count vs gain-based importance.
python
# Native SHAP-style contributions, no extra dependencycontribs = booster.predict(X_test, pred_contrib=True)# last column is the expected value (bias term)# Two importance types report very different rankingssplit_imp = booster.feature_importance(importance_type="split") # times usedgain_imp = booster.feature_importance(importance_type="gain") # total gainimport pandas as pdpd.DataFrame({ "feature": booster.feature_name(), "split": split_imp, "gain": gain_imp,}).sort_values("gain", ascending=False)lgb.plot_tree(booster, tree_index=0, figsize=(20, 10))
Production & Imbalance Tuning Knobs
Options for deployment, class imbalance, and reproducibility.
- is_unbalance=True- auto-rebalances class weights for binary objectives (mutually exclusive with scale_pos_weight)
- scale_pos_weight- explicit positive-class weight ratio for imbalanced binary targets
- booster.save_model('model.txt')- text format for the native Booster; use booster_from_string to reload
- max_bin- number of histogram bins per feature; lower speeds training, may hurt accuracy
- linear_tree=True- fits linear models in the leaves instead of constants
- force_col_wise / force_row_wise- pins the parallelization strategy for reproducible speed on wide vs tall data
- deterministic=True- trades some speed for bit-exact reproducible results
Pro Tip
LightGBM grows trees leaf-wise (best-first) rather than level-wise, which trains faster and often reaches higher accuracy — but it's more prone to overfitting on small datasets, so tune num_leaves and min_data_in_leaf together.
Was this cheat sheet helpful?
Explore Topics
#LightGBM#LightGBMCheatSheet#DataScience#Intermediate#ScikitLearnAPI#NativeDatasetAPI#KeyHyperparameters#CrossValidation#DataStructures#MachineLearning#APIs#CheatSheet#SkillVeris
