> For the complete documentation index, see [llms.txt](https://machine-learning.paperspace.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://machine-learning.paperspace.com/wiki/gradient-boosting.md).

# Gradient Boosting

![Source: Hands-On Machine Learning with R](/files/-Lw70EB_T-Y3OCO-L_4o)

Gradient boosting (derived from the term *gradient boosting machines*) is a popular supervised machine learning technique for regression and classification problems that aggregates an ensemble of weak individual models to obtain a more accurate final model.  

Gradient boosting is a unique ensemble method since it involves identifying the shortcomings of weak models and incrementally or sequentially building a final ensemble model using a [loss function](/wiki/accuracy-and-loss.md#loss) that is optimized with [gradient descent](/wiki/gradient-descent.md).  Decision trees are typically the weak learners in gradient boosting and consequently, the technique is sometimes referred to as *gradient tree boosting*.  

![](/files/-Lw70ZzNl5AgerhBsp-v)

[XGBoost](/wiki/comparison-of-ai-frameworks.md#ml-frameworks) is a very popular gradient boosting framework that is fast, uses some clever tricks to obtain more accurate results, and is easy to parallelize.