Gradient boosting
- 2025-09-28
Gradient boosting is a powerful ensemble machine learning technique used for both regression and classification tasks. It builds a strong predictive model by combining a series of “weak” models, which are typically decision trees.
The key idea is that it builds the model in a sequential, stage-wise fashion. Each new weak model is trained to correct the errors - specifically, the residual errors - of the previous one. This process is called “boosting” because each step boosts the overall performance of the model.