bootstrap aggregating

Bootstrap Aggregating (Bagging) is a meta-algorithm to improve classification and regression models in terms of stability and classification accuracy. Bagging also reduces variance and helps to avoid over-fitting. Although this method is usually applied to decision tree models, it is not limited to any model type. Bagging is a special case of model averaging approach. Given a standard training set D of size N, we generate L new training sets

D_i

also of size N by sampling examples uniformly from D, and with replacement. By sampling with replacement it is likely that some examples will be repeated in each

D_i

. This kind of sample is known as a bootstrap sample. The L models are fitted using the above L bootstrap samples and combined by averaging the output (in case of regression) or voting (in case of classification)

References

Leo Breiman. Bagging predictors. Machine Learning, 24(2):123140, 1996.

Bootstrap Aggregating

References

See also