# Gradient Boosting

![Source: Hands-On Machine Learning with R](https://2327526407-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LvBP1svpACTB1R1x_U4%2F-Lw6zezdliKWkGknCJ6R%2F-Lw70EB_T-Y3OCO-L_4o%2Fimage.png?alt=media\&token=a3edaf4a-d3d2-4c84-9a10-3d870c21d641)

Gradient boosting (derived from the term *gradient boosting machines*) is a popular supervised machine learning technique for regression and classification problems that aggregates an ensemble of weak individual models to obtain a more accurate final model.  

Gradient boosting is a unique ensemble method since it involves identifying the shortcomings of weak models and incrementally or sequentially building a final ensemble model using a [loss function](https://machine-learning.paperspace.com/accuracy-and-loss#loss) that is optimized with [gradient descent](https://machine-learning.paperspace.com/wiki/gradient-descent).  Decision trees are typically the weak learners in gradient boosting and consequently, the technique is sometimes referred to as *gradient tree boosting*.  

![](https://2327526407-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LvBP1svpACTB1R1x_U4%2F-Lw6zezdliKWkGknCJ6R%2F-Lw70ZzNl5AgerhBsp-v%2Fimage.png?alt=media\&token=5df440f0-a102-454e-9d43-e07549dd5334)

[XGBoost](https://machine-learning.paperspace.com/comparison-of-ai-frameworks#ml-frameworks) is a very popular gradient boosting framework that is fast, uses some clever tricks to obtain more accurate results, and is easy to parallelize.