23 Classification Interview Questions (ANSWERED) For ML Engineers And Data Scientists

There is not a rule of thumb to choose a standard optimal k. This value depends and varies from dataset to dataset, but as a general rule, the main goal is to keep it:

• small enough to exclude the samples of the other classes but
• large enough to minimize any noise in the data.

A way to looking for this optimal parameter, commonly called the Elbow method, consist in creating a for loop that trains various KNN models with different k values, keeping track of the error for each of these models, and use the model with the k value that achieves the best accuracy.

In Logistic regression models, we are modeling the probability that an input (X) belongs to the default class (Y=1), that is to say:

where the P(X) values are given by the logistic function,

The β0 and β1 values are estimated during the training stage using maximum-likelihood estimation or gradient descent. Once we have it, we can make predictions by simply putting numbers into the logistic regression equation and calculating a result.

For example, let's consider that we have a model that can predict whether a person is male or female based on their height, such as if P(X) ≥ 0.5 the person is male, and if P(X) < 0.5 then is female.

During the training stage we obtain β0 = -100 and β1 = 0.6, and we want to evaluate what's the probability that a person with a height of 150cm is male, so with that intention we compute:

Given that logistic regression solves a classification task, we can use directly this value to predict that the person is a female.

A decision boundary is a line or a hyperplane that separates the classes. This is what we expect to obtain from logistic regression, as with any other classifier. With this, we can figure out some way to split the data to allow for an accurate prediction of a given observation’s class using the available information.

In the case of a generic two-dimensional example, the split might look something like this:

• K-nearest neighbors or KNN is a supervised classification algorithm. This means that we need labeled data to classify an unlabeled data point. It attempts to classify a data point based on its proximity to other K-data points in the feature space.

• K-means Clustering is an unsupervised classification algorithm. It requires only a set of unlabeled points and a threshold K, so it gathers and groups data into K number of clusters.

• Logistic regression: ideally used for classification of binary variables. Implements the sigmoid function to calculate the probability that a data point belongs to a certain class.

• K-Nearest Neighbours (kNN): calculate the distance of one data point from every other data point and then takes a majority vote from k-nearest neighbors of each data points to classify the output.

• Decision trees: use multiple if-else statements in the form of a tree structure that includes nodes and leaves. The nodes breaking down the one major structure into smaller structures and eventually providing the final outcome.

• Random Forest: uses multiple decision trees to predict the outcome of the target variable. Each decision tree provides its own outcome and then it takes the majority vote to classify the final outcome.

• Support Vector Machines: it creates an n-dimensional space for the n number of features in the dataset and then tries to create the hyperplanes such that it divides and classifies the data points with the maximum margin possible.

We call it naive because its assumptions (it assumes that all of the features in the dataset are equally important and independent) are really optimistic and rarely true in most real-world applications:

• we consider that these predictors are independent
• we consider that all the predictors have an equal effect on the outcome (like the day being windy does not have more importance in deciding to play golf or not)

• If the availability of data is a constraint, i.e. if the training data is smaller or if the dataset has a fewer number of observations and a higher number of features, we can choose algorithms with high bias/low variance like Naïve Bayes and Linear SVM.

• If the training data is sufficiently large and the number of observations is higher as compared to the number of features, one can go for low bias/high variance algorithms like K-Nearest Neighbors, Decision trees, Random forests, and kernel SVM.

Yes, depending on the problem and the context is possible to convert a regression to a classification problem and vice versa.

• To convert Regression into Classification we perform discretization: a process through which we can transform continuous variables into an ordered relationship (called ordinal). For example, amounts in a continuous range between $0 and $100 could be converted into 2 buckets:

  • Class 0: $0 to $49.
  • Class 1: $50 to $100.

• To convert Classification into Regression, a label can be converted into a continuous range, i.e. we perform the inverse operation described above:

  • $0 to $49 for Class 1.
  • $50 to $100 for Class 2.

  Here, we must be sure that the class labels in the classification problem do have a natural ordinal relationship. If not, the conversion from classification to regression may result in surprising or poor performance as the model may learn a false or non-existent mapping from inputs to the continuous output range.

• Logistic regression is a statistical model that utilizes logit function to model classification problems. It is a regression analysis to conduct when the dependent variable is binary. The logit function is shown below:

• Looking at the logit function, the next question that comes to mind is how to fit that graph/equation. The fitting of the logistic regression is done using the maximum likelihood function.
• In a supervised logistic regression, features are mapped onto the output. The output is usually a categorical value (which means that it is mapped with one-hot vectors or binary numbers).
• Since the logit function always outputs a value between 0 and 1, it gives the probability of the outcome.

• An ROC curve stands for Receiver Operating Characteristic curve.
• It shows the performance of a classification model at all classification thresholds. The curve plots the true positive rate and the false positive rate.
• The ROC curve plots TP (True Positives) Rate vs. FP (False Positives) Rate at different classification thresholds. Lowering the classification threshold classifies more items as positive, increasing both false positives and true positives.
• The graph below shows an ROC curve:

• AUC stands for the Area Under the ROC Curve.
• AUC calculates the entire two-dimensional area under the ROC curve.
• It provides an aggregate measure of performance across all possible classification thresholds. One way of interpreting AUC is as the probability that a model ranks a random positive example higher than a random negative example.
• An AUC is shown by the graph below:

• Accuracy: is the proportion of true positives among the total number of cases examined. Is suited for classification problems that are well balanced and not skewed.
• Precision: is the proportion of true positive values between all positive values. Is better to use when we want to be very sure of our prediction.
• Recall: measures the proportion of true positives correctly classified. This is a good metric to use when we want to capture as many positives as possible. For example: If we are building a system to predict if a person has cancer or not, we want to capture the disease even if we are not very sure.
• F1-score: is a number between 0 and 1 and is the harmonic mean of precision and recall. This metric maintains a balance between these two, so if the precision is low, the F1 is low, and if the recall is low again the F1 score is low. It is suited for imbalanced class distributions problems.

• Bagging:

  • Is a parallel ensemble method: the base learners are created independently for each other and are trained simultaneously.
  • Is based on the bootstrap aggregate: it aggregates several sampling subsets of the original dataset to train different learners chosen randomly with replacement.
  • Is focused on decreases variance, not bias, and solves over-fitting issues in a model.

• Boosting

  • Is a sequential ensemble method: the base learners depend on each other, so each learner influences the next one and general paternal behavior can be deduced.
  • Each base learner in the sequence is fitted giving more importance to observations in the dataset that were badly handled by the previous learners in the sequence. Intuitively, each new model focuses its efforts on the most difficult observations to fit up to now, so that we obtain, at the end of the process, a strong learner.
  • Is mainly focused on reducing bias, not variance, and can increase the overfitting in some cases.

• A Generative Model ‌explicitly models the actual distribution of each class. It ‌learns the joint probability distribution p(x,y) = P(y) * P(x|y), where both P(y) and P(x|y) can be estimated from the dataset by computing class frequencies. Some examples of generative models are:

  • Naïve Bayes
  • Bayesian networks
  • Markov random fields

• A Discriminative Model models the decision boundary between the classes by ‌learning the conditional probability distribution P(y|x) from the dataset. Some examples of such kinds of classifiers are:

  • ‌Logistic regression
  • Scalar Vector Machine
  • Traditional neural networks
  • KNN.

Both One-vs-Rest and One-vs-One are two techniques for splitting the multi-class dataset into multiple binary classification problems. It allows us to categorize the test data into multiple class labels present in trained data as a model prediction. One key difference in both techniques is the number of classifiers that are created in each one.

• In One-vs-Rest, for the N-class instances dataset, we have to generate the N binary classifier models. The predictions are then made using the model that is the most confident. For example, given a multi-class classification problem with examples for each class 'Red', 'Blue' and 'Green', the way to divide into three binary classifications could be as follows:

  • Classifier 1:- [Green] vs [Red, Blue].
  • Classifier 2:- [Blue] vs [Green, Red].
  • Classifier 3:- [Red] vs [Blue, Green].

• In One-vs-One, we split the primary dataset into one dataset for each class opposite to every other class, so for the N-class instances dataset, we have to generate N* (N-1)/2 binary classifier models. Taking the above example, we have a classification problem having three types: 'Green', 'Blue', and 'Red' (N=3). So we divide this problem into N* (N-1)/2 = 3 binary classifier problems:

  • Classifier 1: [Green] vs. [Blue].
  • Classifier 2: [Green] vs. [Red].
  • Classifier 3: [Blue] vs. [Red].

  Each binary classifier predicts one class label. When we input the test data to the classifier, then the model with the majority counts is concluded as a result.