![\"\"](\"https:\/\/dev.littlebigcode.fr\/wp-content\/uploads\/2022\/04\/Capture-de\u0301cran-2022-04-13-a\u0300-02.45.27-3.png\")
<\/p>\n
\n
While ensemble approaches give excellent results nowadays, Zhi-Hua Zhou<\/a> and Ji Feng<\/a> indicate in their work on a novel deep learning methodology that where a random forest produces decent results on your data, apply a deep forest and you will be pleasantly surprised. The proposal by Zhi-Hua Zhou and Ji Feng of a revolutionary deep learning framework dubbed Deep Forest (DF) can be considered as one of the most significant events in machine learning in 2017.<\/p>\n The DF employs a number of ensemble-based approaches, most notably Random Forests (RF) and Stacking, to generate a structure similar to that of a multi-layer neural network, except that each layer is made of RFs rather than neurons. DF is particularly favourable for training because it requires a small number of hyperparameters, does not require back-propagation, and outperforms some well-known techniques, including deep neural networks, when only small-scale training data is available` [1, 2].<\/p>\n Diversity enhancement is a technique that entails combining numerous “weak” or \u201cweak\u201d\/\u201dperformant\u201d learners into a single “strong” learner, with “weak” being relative. As a result, the stacking theory should minimise both bias and variation, and it is particularly effective at avoiding overfitting and variance. This is because of two reasons :<\/strong><\/p>\n Each ensemble learner will have a somewhat different manner of mapping features to outcomes, and the idea is that by combining them, a larger portion of the search area will be covered.<\/p>\n<\/li>\n Furthermore, taking the first and second models of an ensemble, both have a low bias but a high variance owing to overfitting, they have theoretically overfitted separate regions of the search space; after combining them, the overall variance would be reduced.<\/p>\n<\/li>\n<\/ul>\n As a consequence, ensemble learning with a focus on diversity (i.e., choose your learners carefully)<\/em> will result in a reduction in overall variance. Additionally, in practice, this is commonly added to ensemble by injecting randomness during training [11 – Section IV-B].<\/p>\n <\/p>\n Because I believe that the diversity of our models is the next breakthrough among others in machine learning, here is a more general description of how the DF includes diversity management and why it is critical in machine learning, or more precisely, ensemble approaches and stacking.<\/strong><\/p>\n \n The following article begins with an overview of how DF function, followed by a brief explanation of why diversity is important in DF, and then a quick introduction on how to enhance diversity with the DF framework (practically perspective).<\/p>\n \n \n <\/p>\n Because the original paper states it very effectively and succinctly, the following explanation will be broad in scope; consequently, for the mathematical component, refer to the original papers as well [1, 2]. Consequently, we will analyse the classification and prediction components of the Deep Forest architecture using the diagram below :<\/p>\n Nota bene : <\/strong>While the preceding processes are automated, the first step can be performed manually or automatically using the default ensemble combination specified by the framework’s designers. To do so manually, read the remainder of this article to learn how to do so.<\/p>\n \n \n <\/p>\n Diversity is a vast subject of study, and here we will discuss how an engineer should think while selecting his base learners (i.e., algorithms). Following which, we will conclude by showing how these three techniques apply “diversity” :<\/p>\n As a result, depending on the approach used to select learners for the ensemble, the diversity will be applied differently, depending on whether the view is sample-based, hyperparameter-based, or algorithm-based. Finally, the DF will be used in the final perspective when your ensemble learners are averaged to make one, at which point the diversity will be applied to the findings of the learners you previously selected using the brief techniques outlined above. To learn more, consult this scholarly article [11].<\/p>\n \n \n <\/p>\n Due to the DF theory’s primary focus on ensemble and stacking, diversity is another key component included in the package. However, why is it such a critical component of the framework?<\/p>\n From the gcForest to Deep Forest paper updated in 2019 [2]: `Ensemble learning<\/strong> [4] is a machine-learning paradigm<\/strong> in which a task is solved by training and combining several learners (e.g. classifiers). [\u2026]` Therefore, among the ensemble learning methods we have :<\/p>\n Due to the heterogeneous manner in which stacking occurs, diversity plays a critical part in allowing this. As briefly mentioned previously in the article, diversity enables the development of robust ensemble models; this is also stated in the DF original paper: `[\u2026] \u201cto build a strong ensemble, the individual learners must be precise and different\u201d.<\/p>\n Combining purely correct learners is frequently inferior than combining some accurate learners with some comparatively weaker learners, since complementarity takes precedence over pure accuracy.`[2]. Nonetheless, scientists continue to grapple with the same enigma: ‘What is diversity really?’ This question remains the field’s holy grail.<\/p>\n Finally, because diversity is a critical component of the DF framework, its lead engineer states in [5] that the framework contains diversity by default and has the capability to incorporate diversity into the learning process, allowing the user to completely customise it to their liking. As a result, the following section will discuss how to use the Deep Forest framework, formerly known as gcForest, to deal with diversity, which is sometimes referred to in the literature as ambiguity.<\/p>\n <\/p>\n <\/p>\n We presume below that you already have Python installed because deep forest is a Python framework; otherwise, please see the python \/ pip manual for information on how to install them. [6, 7] To install the Deep Forest 21<\/a> Framework, follow the steps below :<\/p>\n \n\n
How does the Deep Forest function in general ?<\/h1>\n
![\"\"](\"https:\/\/dev.littlebigcode.fr\/wp-content\/uploads\/2022\/04\/image-11.png\")
<\/p>\nLayer by layer, here’s how Deep Forest Architecture works :<\/span><\/h2>\n
\n
n<\/span><\/code>\u00a0estimators previously configured are executed<\/span>, each of which produces prediction vector (i.e., binary class classification: two outputs, multi-class classification:\u00a0n_class<\/code> outputs).<\/li>\nHow does diversity is applied in Deep Forest ?<\/h1>\n
\n
Why diversity in Deep Forest ?<\/h1>\n
What is an Ensemble learning and the Bagging\/Boosting and Stacking sub-paradigms ?<\/span><\/h2>\n
\n
Why does diversity has to deal with stacking ensemble ?<\/span><\/h2>\n
Deep Forest (DF) installation<\/h1>\n