Ensemble model

In ML, ensemble model is a model that combines multiple models to achieve better performance than any single model could.

Concepts

• Bagging creates diversity by generating random samples from the training observations and fitting the same model to each different sample - also known as homogeneous parallel ensembles.
• Boosting follows an iterative process by sequentially training each base model on the up-weighted errors of the previous base model, producing an additive model to reduce the final model errors.

Instances

• Random forest (an extension of bagging)
• Gradient boosting (boosting)