XGBoost
By the DMLC (Distributed Machine Learning Community)
XGBoost (Extreme Gradient Boosting) is an open-source, highly optimized implementation of the gradient boosting algorithm, widely used for structured/tabular data problems due to its speed, scalability, and predictive accuracy.
Definition
XGBoost (Extreme Gradient Boosting) is an open-source, highly optimized implementation of the gradient boosting algorithm, widely used for structured/tabular data problems due to its speed, scalability, and predictive accuracy.
Overview
XGBoost was created by Tianqi Chen and released in 2014, quickly becoming a dominant tool in applied machine learning after being used in a large share of winning solutions on Kaggle and other data science competitions. It implements gradient boosting with a number of engineering and algorithmic refinements that improve both training speed and generalization: a regularized learning objective that penalizes model complexity to reduce overfitting, a second-order (Newton) approximation of the loss function for more accurate tree splits, sparsity-aware split finding that natively handles missing values, and a histogram-based approximate split-finding algorithm that scales to very large datasets. XGBoost supports parallel and distributed training across CPU cores, clusters, and GPUs, along with out-of-core computation for datasets too large to fit in memory. It exposes bindings for Python, R, Java, Scala, and other languages, and integrates with standard ML tooling such as scikit-learn's API conventions, making it easy to drop into existing pipelines. Its objective function framework is highly flexible, supporting standard regression and classification losses as well as custom, user-defined objectives and evaluation metrics, and it natively supports ranking objectives for search and recommendation use cases. While newer competitors like LightGBM (from Microsoft) and CatBoost (from Yandex) offer faster training or better native handling of categorical features in some scenarios, XGBoost remains one of the most widely deployed machine learning libraries in production systems, particularly for tabular data where it frequently matches or exceeds the performance of deep neural networks while being far cheaper to train and easier to interpret using tools like SHAP values.
Key Features
- Regularized gradient boosting objective that penalizes model complexity
- Second-order gradient approximation for more accurate tree splits
- Sparsity-aware algorithm with native handling of missing values
- Histogram-based approximate split finding for large-scale training
- Parallel, distributed, and GPU-accelerated training support
- Out-of-core computation for datasets larger than available memory
- Bindings for Python, R, Java, Scala, and integration with scikit-learn
- Supports custom objective functions and native ranking objectives
Use Cases
Alternatives
History
XGBoost (eXtreme Gradient Boosting) is an open-source library that provides a fast, regularized gradient-boosting framework across languages including C++, Python, R, Java, and Scala. It was created by Tianqi Chen, then a PhD student at the University of Washington, and first released in 2014 under the Distributed (Deep) Machine Learning Community (DMLC). After a quiet start, its combination of accuracy, regularization, and scalability made it a dominant algorithm in Kaggle competitions and applied machine learning. Chen and Carlos Guestrin formalized the system in the influential 2016 paper "XGBoost: A Scalable Tree Boosting System" at the KDD conference. XGBoost runs on single machines and distributed frameworks such as Spark and Dask, and remains a workhorse for tabular-data modeling.
Sources
- XGBoost on GitHub — dmlc/xgboost · as of 2026-07-17
- DMLC (University of Washington) — "XGBoost: A Scalable Tree Boosting System" · as of 2026-07-17