Decision trees and their ensembles are widely used in machine learning, statistics and data analysis. Predictive models based on decision trees, show outstanding results in terms of quality and learning time. Especially on heterogeneous tabular data. Speed performance, simplicity and reliability make this family of predictive one of the most popular models in machine learning. important parameters when training ensembles of decision trees (random forest, gradient boosting, etc.) are: the number of trees and their maximum depth. These options are usually selected by a complete enumeration of all possible options on the training sample. The report will prove a theorem on expressive the possibilities of an ensemble of decision trees of limited depth. From of this result it follows that the depth of the decisive trees cannot be replaced by the size of the ensemble.

decision trees

Sokolov Andrey Pavlovich,  Lomonosov Moscow State University