CatBoost Cheat Sheet
CatBoost reference covering native categorical feature handling, the Pool data structure, ordered boosting, and built-in cross-validation.
2 PagesIntermediateApr 12, 2026
Basic Training
Train a classifier with native categorical support.
python
from catboost import CatBoostClassifiercat_features = ["city", "device_type"] # column names or indicesmodel = CatBoostClassifier( iterations=500, depth=6, learning_rate=0.05, loss_function="Logloss", eval_metric="AUC", cat_features=cat_features, verbose=50,)model.fit(X_train, y_train, eval_set=(X_val, y_val), early_stopping_rounds=30)preds = model.predict(X_test)proba = model.predict_proba(X_test)
Pool API
Efficient data container for training.
python
from catboost import Pooltrain_pool = Pool(X_train, label=y_train, cat_features=cat_features)val_pool = Pool(X_val, label=y_val, cat_features=cat_features)model = CatBoostClassifier(iterations=500, depth=6)model.fit(train_pool, eval_set=val_pool)model.save_model("model.cbm")
Cross-Validation
Built-in CV helper.
python
from catboost import cvparams = {"iterations": 500, "depth": 6, "loss_function": "Logloss"}cv_results = cv(train_pool, params, fold_count=5, early_stopping_rounds=30)print(cv_results["test-Logloss-mean"].min())
Key Features
What sets CatBoost apart.
- cat_features- pass raw categorical columns without manual one-hot encoding
- Pool- wraps features, labels, and categorical info for efficient training
- ordered boosting- reduces target leakage/overfitting versus classic gradient boosting
- SymmetricTree- default oblivious tree structure, fast inference
- get_feature_importance()- built-in feature importance including SHAP values
- grid_search() / randomized_search()- built-in hyperparameter tuning helpers
GPU Training & Task Type
Switch training to GPU and control device selection for large datasets.
python
model = CatBoostClassifier( iterations=2000, depth=8, learning_rate=0.03, task_type="GPU", devices="0:1", # use GPUs 0 and 1 gpu_ram_part=0.9, boosting_type="Plain", # Plain is faster on GPU than Ordered cat_features=cat_features,)model.fit(train_pool, eval_set=val_pool, use_best_model=True)print(model.get_best_iteration())
Text & Embedding Features
Let CatBoost tokenize raw text columns and mix in precomputed embeddings.
python
from catboost import Pooltrain_pool = Pool( X_train, label=y_train, cat_features=["city"], text_features=["review_text"], embedding_features=["user_embedding"],)model = CatBoostClassifier( iterations=800, tokenizers=[{"tokenizer_id": "Space", "lowercasing": "true"}], dictionaries=[{"dictionary_id": "BiGram", "gram_order": "2"}], feature_calcers=["BoW:top_tokens_count=1000"],)model.fit(train_pool)
SHAP Values & Feature Interactions
Explain individual predictions and quantify pairwise feature interaction strength.
python
shap_values = model.get_feature_importance( train_pool, type="ShapValues")# last column is the expected value (bias term)contribs, base_value = shap_values[:, :-1], shap_values[0, -1]interactions = model.get_feature_importance( train_pool, type="Interaction")top = sorted(interactions, key=lambda r: -r[2])[:5]for f1, f2, score in top: print(model.feature_names_[int(f1)], model.feature_names_[int(f2)], score)
Staged Predictions & Custom Eval Metrics
Inspect predictions at each boosting iteration and compute metrics after training.
python
for i, pred in enumerate(model.staged_predict_proba(val_pool, ntree_start=0, ntree_end=100, eval_period=10)): print(i * 10, pred[:3, 1])from catboost.utils import eval_metricauc = eval_metric(y_val.values, model.predict_proba(val_pool)[:, 1], "AUC")print("AUC:", auc[0])# compare two trained models on the same datafrom catboost import CatBoostCatBoost.compare(model, other_model, data=val_pool, metrics=["Logloss", "AUC"])
Advanced Hyperparameters
Knobs beyond depth/learning_rate that matter once you're tuning seriously.
- monotone_constraints- force predictions to be monotonic (+1/-1/0) in specific features
- grow_policy- SymmetricTree (default), Depthwise, or Lossguide for XGBoost-like leaf-wise growth
- bootstrap_type- Bayesian, Bernoulli, MVS, or Poisson (GPU) sampling strategy for each tree
- od_type / od_wait- overfitting detector: IncToDec or Iter, stops training after od_wait rounds without improvement
- l2_leaf_reg- L2 regularization on leaf values, higher values reduce overfitting
- border_count- number of splits considered per numeric feature (bins), trades accuracy for speed
- one_hot_max_size- categorical features with fewer unique values than this use one-hot instead of target stats
- best_model_min_trees- minimum number of trees kept even when use_best_model would truncate earlier
Pro Tip
Pass raw categorical columns directly via cat_features instead of one-hot or label encoding them yourself — CatBoost's ordered target statistics handle high-cardinality categoricals better and avoid leakage.
Was this cheat sheet helpful?
Explore Topics
#CatBoost#CatBoostCheatSheet#DataScience#Intermediate#BasicTraining#PoolAPI#CrossValidation#KeyFeatures#MachineLearning#CheatSheet#SkillVeris
