21 Random Forests Interview Questions For ML Engineers

• Random Forests is a type of ensemble learning method for classification, regression, and other tasks.
• Random Forests works by constructing many decision trees at a training time. The way that this works is by averaging several decision trees at different parts of the same training set.

• Pruning is a data compression technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that are non-critical and redundant to classify instances.
• Random Forest usually does not require pruning because it will not over-fit like a single decision tree. This happens due to the fact that the trees are bootstrapped and that multiple random trees use random features so the individual trees are strong without being correlated with each other.

• Classification: The output of the Random Forest is the one selected by the most trees.
• Regression: The output of the Random Forest is the mean or average prediction of the individual trees.

• Random forest is an ensemble learning method that works by constructing a multitude of decision trees. A random forest can be constructed for both classification and regression tasks.
• Random forest outperforms decision trees, and it also does not have the habit of overfitting the data as decision trees do.
• A decision tree trained on a specific dataset will become very deep and cause overfitting. To create a random forest, decision trees can be trained on different subsets of the training dataset, and then the different decision trees can be averaged with the goal of decreasing the variance.

• Due to the observations being sampled with replacements, even if the size of the bootstrapped dataset is different, the datasets will be different.
• Due to this, the full size of the training data can be used.

Most of the time the best thing to do is not touch this hyperparameter.

• Ensemble methods is a machine learning technique that combines several base models in order to produce one optimal predictive model.
• Random Forest is a type of ensemble method.
• The number of component classifier in an ensemble has a great impact on the accuracy of the prediction, although there is a law of diminishing results in ensemble construction.

• Random Forest is very versatile and can be used in both regression and classification tasks. It can also handle all binary, categorical, and numerical features.
• The process is parallelizable where the process can be split to run in different machines.
• It performs better in high dimensionality since the work is on subsets of data.
• The training speed is faster than decision trees because they are working only on a subset of features. Even if there are hundreds of features the training speed will be significantly faster.
• The Random Forest is good at balancing errors for class population unbalanced data sets.
• It has low bias, but moderate variance because when all the trees are averaged in random forest, all the variances are also averaged so it has low bias but a moderate variance.

The Random Forest methods encourage two ways of handling missing values:

• Drop data points with missing values. This is not recommended due to the fact that all the available data points is not used.
• Fill in the missing values with the median (for numerical values) or mode (for categorical values). This method will brush too broad a stroke for datasets with many gaps and significant structure.

There are other methods of filling in missing values such as calculating the similarity between the missing features, and the missing values estimated by weighting.

• As part of their construction, random forest predictors naturally lead to a dissimilarity measure among the observations. One can also define a random forest dissimilarity measure between unlabeled data:

  • the idea is to construct a random forest predictor that distinguishes the observed data from suitably generated synthetic data.

• Many unsupervised learning methods require the inclusion of an input dissimilarity measure among the observations. Hence, if a dissimilarity matrix can be produced using Random Forest, unsupervised learning can be successfully implemented. The patterns found in the process will be used to make clusters.

Some things to try to improve the performance of Random Forest are:

• Using a higher quality dataset and feature engineering. Using too many features and data are not good for the model so sometimes it is important to perform some feature reduction too.
• Tuning the hyperparameters of the algorithm.
• Trying different algorithms.

• Proximity is the closeness or nearness between pairs of cases.
• Proximities are calculated for each pair of cases/observations/sample points. If two cases occupy the same terminal node through one tree, their proximity is increased by one. At the end of the run of all trees, the proximities are normalized by dividing by the number of trees.
• Proximities are used in replacing missing data, locating outliers, and producing illuminating low-dimensional views of the data.

Random forest is an extension of the bagging method as it utilizes both bagging and feature randomness to create an uncorrelated forest of decision trees. Hence, random forest is Random in the following ways:

• Each tree is trained on a random subset of features, which ensures low correlation among decision trees.
• Each tree in the forest is trained in 2/3-rd of the total training data and data points are drawn at random from the original dataset.

• The basic definition of entropy is a measure of disorder. The equation of entropy is as follows:

  $E(S) = \sum_{i=1}^c - p_i \log_2 p_i$

  where, $p_i$ is the probability of frequentist probability of the element i in the data.

• In machine learning models, the goal is to decrease uncertainties. Thus, the models should have less entropy.

• The reduction of entropy is defined as information gain. It is shown below:

• Information gain is just the subtraction of entropy of Y given X from the entropy of just Y. E(Y|X) corresponds to the information of Y that we already know. So, E(Y|X) is not new information for the model.

• Out-of-bag error (OOB) is a method of measuring the prediction of error of random forest.
• OOB error is calculated in random forest classifier trained using bootstrap aggregation, where each new tree is fit from a bootstrap sample of training observations.
• OOB error is the mean prediction error on each training sample x, using only trees that do not have x in their bootstrap sample.

• The OOB error for regression is given by:

  $OOB = \frac{1}{n}\sum_{i=1}^n{(Y_i - \hat{Y_i})^2}$

• The OOB error for classification is given by:

  $OOB = \frac{1}{n}\sum_{i=1}^n{1_{Y_i \neq \hat{Y_i}}}$

• Decision trees can be easily converted into rules which increase human interpretability of the results and explain why a decision was made.
• For Random Forest the general recommendation is to use as many trees as possible. In most cases, with hundreds of trees, you wouldn't be able to understand why did they collectively made the decision that they made.

• The difference between Random Forest and Bagging is the fact that for Random Forest only a subset of features out of all are selected in random and the best split feature from the subset is used to split each node in a tree.
• In bagging all the features are considered in splitting the node.

Due to the fact that bagging considers all the features, the training efficiency of random forest is better.