23 Supervised Learning Interview Questions To Crush Data Science Interview

Supervised learning can be separated into two types of problems when data mining:

• Classification: It uses algorithms to assign the test data into specific categories. Common classification algorithms are linear classifiers, support vector machines (SVM), decision trees, k-nearest neighbor, and random forest.
• Regression: It is used to understand the relationship between dependent and independent variables. Linear regression, logistical regression, and polynomial regression are popular regression algorithms.

Support vector machines (SVMs) are a set of supervised learning methods used for classification, regression and outliers detection. The objective of the support vector machine algorithm is to find a hyperplane in an N-dimensional space(N — the number of features) that distinctly classifies the data points.

Support vector machines focus only on the points that are the most difficult to tell apart, whereas other classifiers pay attention to all of the points.

The intuition behind the support vector machine approach is that if a classifier is good at the most challenging comparisons (the points in B and A that are closest to each other), then the classifier will be even better at the easy comparisons (comparing points in B and A that are far away from each other).

• Supervised learning examples:

  • You get a bunch of photos with information about what is on them and then you train a model to recognize new photos.
  • You have a bunch of molecules and information about which are drugs and you train a model to answer whether a new molecule is also a drug.
  • Based on past information about spams, filtering out a new incoming email into Inbox (normal) or Junk folder (Spam)
  • Cortana or any speech automated system in your mobile phone trains your voice and then starts working based on this training.
  • Train your handwriting to OCR system and once trained, it will be able to convert your hand-writing images into text (till some accuracy obviously)

• Unsupervised learning examples:

  • You have a bunch of photos of 6 people but without information about who is on which one and you want to divide this dataset into 6 piles, each with the photos of one individual.
  • You have molecules, part of them are drugs and part are not but you do not know which are which and you want the algorithm to discover the drugs.
  • A friend invites you to his party where you meet totally strangers. Now you will classify them using unsupervised learning (no prior knowledge) and this classification can be on the basis of gender, age group, dressing, educational qualification or whatever way you would like. Why this learning is different from Supervised Learning? Since you didn't use any past/prior knowledge about people and classified them "on-the-go".
  • NASA discovers new heavenly bodies and finds them different from previously known astronomical objects - stars, planets, asteroids, blackholes etc. (i.e. it has no knowledge about these new bodies) and classifies them the way it would like to (distance from Milky way, intensity, gravitational force, red/blue shift or whatever)
  • Let's suppose you have never seen a Cricket match before and by chance watch a video on internet, now you can classify players on the basis of different criterion: Players wearing same sort of kits are in one class, Players of one style are in one class (batsmen, bowler, fielders), or on the basis of playing hand (RH vs LH) or whatever way you would observe and classify it.

• k-Nearest Neighbors is a supervised machine learning algorithm that can be used to solve both classification and regression problems.
• It assumes that similar things are closer to each other in certain feature spaces, in other words, similar things are in close proximity.

• The image above shows how similar points are closer to each other. KNN hinges on this assumption being true enough for the algorithm to be useful.
• There are many different ways of calculating the distance between the points, however, the straight line distance (Euclidean distance) is a popular and familiar choice.

In supervised machine learning an algorithm learns a model from training data.

The goal of any supervised machine learning algorithm is to best estimate the mapping function (f) for the output variable (Y) given the input data (X). The mapping function is often called the target function because it is the function that a given supervised machine learning algorithm aims to approximate.

Bias are the simplifying assumptions made by a model to make the target function easier to learn.

Generally, linear algorithms have a high bias making them fast to learn and easier to understand but generally less flexible.

• Examples of low-bias machine learning algorithms include: Decision Trees, k-Nearest Neighbors and Support Vector Machines.

• Examples of high-bias machine learning algorithms include: Linear Regression, Linear Discriminant Analysis and Logistic Regression.

• Overfitting refers to a model that models the training data too well.
• Overfitting happens when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data. This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. The problem is that these concepts do not apply to new data and negatively impact the model's ability to generalize.

• Underfitting refers to a model that can neither model the training data nor generalizes to new data.
• An underfit machine learning model is not a suitable model and will be obvious as it will have poor performance on the training data.
• Underfitting is often not discussed as it is easy to detect given a good performance metric. The remedy is to move on and try alternate machine learning algorithms.

• Supervised learning is when the data you feed your algorithm with is tagged or labelled, to help your logic make decisions.

  Example: a hypothetical non-machine learning algorithm for face detection in images would try to define what a face is (round skin-like-colored disk, with dark area where you expect the eyes etc). A machine learning algorithm would not have such coded definition, but would "learn-by-examples": you'll show several images of faces and not-faces and a good algorithm will eventually learn and be able to predict whether or not an unseen image is a face.

• Unsupervised learning are types of algorithms that try to find correlations without any external inputs other than the raw data (your examples are not labeled, i.e. you don't say anything). In such a case the algorithm itself cannot "invent" what a face is, but it can try to cluster the data into different groups, e.g. it can distinguish that faces are very different from landscapes, which are very different from horses.

• Classification is the problem of identifying which set of categories an observation belongs to.
• Regression is a set of statistical processes for estimating the relationships between a dependent variable and an independent variable.

• Classification is used to predict the values of a categorical variable, so the output is generally in the form of integers, or binary (0 or 1).
• Regression is used to predict a continuous variable, so the output is also a floating-point number (0.1, 0.74, 0.69, etc.).

• Covariance measures whether a variation in one variable results in a variation in another variable, and deals with the linear relationship of only 2 variables in the dataset. Its value can take range from -∞ to +∞. Simply speaking Covariance indicates the direction of the linear relationship between variables.

• Correlation measures how strongly two or more variables are related to each other. Its values are between -1 to 1. Correlation measures both the strength and direction of the linear relationship between two variables. Correlation is a function of the covariance.

• Logistic regression is a statistical model that utilizes logit function to model classification problems. It is a regression analysis to conduct when the dependent variable is binary. The logit function is shown below:

• Looking at the logit function, the next question that comes to mind is how to fit that graph/equation. The fitting of the logistic regression is done using the maximum likelihood function.
• In a supervised logistic regression, features are mapped onto the output. The output is usually a categorical value (which means that it is mapped with one-hot vectors or binary numbers).
• Since the logit function always outputs a value between 0 and 1, it gives the probability of the outcome.

Imagine you have 1500 labeled data points, and you want to estimate how well some classifier will work on new data.

What you do with with NO cross-validation:

Lets say you decided to train the model with 1000 of your examples, and evaluate with 500. Note that we only have predictions for the 500 test samples so our model performance estimate is overly optimistic. Why? Because accuracy obtained for one test set can be very different to the accuracy obtained for a different test set.

What you do with cross-validation:

Lets set 𝑘=3, so the data is split into three sets of 500 points (A, B and C). Use A & B to train a model, and get predictions for C with this model. Use B & C to train a model, to get predictions for A. Finally, use A & C to train a model, and get predictions for B. Now we have a prediction for every point in our labeled data that came from a model trained on different data.

• Gradient boosting is a machine learning technique for regression and classification. Gradient boosting is an ensemble of weak prediction models which create an accurate and good performing ensemble of models.
• From the term gradient boosting, boosting refers to the practice of using an ensemble of weak models to create strong models. Gradient boosting re-defines boosting as a numerical optimization problem where the objective is to minimize the loss function of the model by adding weak learners using gradient descent.
• Gradient boosting is a stage-wise additive model that generates learners during the learning process where new models are added one at a time without changing the existing models (the learners are usually trees).

Some challenges faced when using a supervised regression model are:

• Nonlinearities: Real-world data points are more complex and do not follow a linear relationship. Sometimes a non-linear model is better at fitting the dataset. So, it is a challenge to find the perfect equation for the dataset.
• Multicollinearity: Multicollinearity is a phenomenon where one predictor variable in a multiple regression model can be linearly predicted from the others with a substantial degree of accuracy. If there is a problem of multicollinearity then even the slightest change in the independent variable causes the output to change erratically. Thus, the accuracy of the model is affected and it undermines the quality of the whole model.
• Outliers: Outliers can change and make huge impact on the machine learning model. This happens because the regression model tries to fit the outliers into the model as well. The problem of outliers is shown in the figure below:

• Type I error is classified as a false positive conclusion. Here, a sample was incorrectly labeled as positive but it is actually negative. A type 1 error is also known as a false positive and occurs when a researcher incorrectly rejects a true null hypothesis. This means that your report that your findings are significant when in fact they have occurred by chance.
• Type II error is classified as a false negative. Here, positive tuples were incorrectly labeled as negative but actually, it is positive. A type II error is also known as a false negative and occurs when a researcher fails to reject a null hypothesis which is really false. Here a researcher concludes there is not a significant effect, when actually there really is.

• The consequences of making a Type I error mean that changes or interventions are made which are unnecessary, and thus waste time, resources, etc.
• Type II errors typically lead to the preservation of the status quo (i.e. interventions remain the same) when change is needed.

The main difference between supervised and unsupervised learning is the type of data used in each type of learning.

• Supervised learning uses labeled input and output data. Supervised learning learns from the data and adjusts itself to give the most accurate output given that the input and output are representative of the dataset and accurate themselves.
• Unsupervised learning uses input data that is not labeled. Unsupervised learning works on their own to discover the inherent structure of unlabeled data. Human intervention is still required for validating the output variables.

There are other differences between them too:

Goals:

• In supervised learning the goal is to predict the outcomes for new data. The type of output to expect is known upfront.
• In unsupervised learning the goal is to get insights from large volumes of data. Machine learning determines what is interesting and different from the dataset.

Applications:

• Supervised learning is great for spam detection, sentiment analysis, weather forecasting, among other things.
• Unsupervised learning is great for anomaly detection, recommendation systems, and medical imaging.

Short answer:

• The training set is used to fit the models;
• the validation set is used to estimate prediction error for model selection;
• the test set is used for assessment of the generalization error of the final chosen model.

When you have a large data set, it's recommended to split it into 3 parts:

• Training set (60% of the original data set): This is used to build up our prediction algorithm. Our algorithm tries to tune itself to the quirks of the training data sets. In this phase we usually create multiple algorithms in order to compare their performances during the Cross-Validation Phase.

• Cross-Validation set (20% of the original data set): This data set is used to compare the performances of the prediction algorithms that were created based on the training set. We choose the algorithm that has the best performance.

• Test set (20% of the original data set): Now we have chosen our preferred prediction algorithm but we don't know yet how it's going to perform on completely unseen real-world data. So, we apply our chosen prediction algorithm on our test set in order to see how it's going to perform in the wild so we can have an idea about our algorithm's performance on unseen data.

Both One-vs-Rest and One-vs-One are two techniques for splitting the multi-class dataset into multiple binary classification problems. It allows us to categorize the test data into multiple class labels present in trained data as a model prediction. One key difference in both techniques is the number of classifiers that are created in each one.

• In One-vs-Rest, for the N-class instances dataset, we have to generate the N binary classifier models. The predictions are then made using the model that is the most confident. For example, given a multi-class classification problem with examples for each class 'Red', 'Blue' and 'Green', the way to divide into three binary classifications could be as follows:

  • Classifier 1:- [Green] vs [Red, Blue].
  • Classifier 2:- [Blue] vs [Green, Red].
  • Classifier 3:- [Red] vs [Blue, Green].

• In One-vs-One, we split the primary dataset into one dataset for each class opposite to every other class, so for the N-class instances dataset, we have to generate N* (N-1)/2 binary classifier models. Taking the above example, we have a classification problem having three types: 'Green', 'Blue', and 'Red' (N=3). So we divide this problem into N* (N-1)/2 = 3 binary classifier problems:

  • Classifier 1: [Green] vs. [Blue].
  • Classifier 2: [Green] vs. [Red].
  • Classifier 3: [Blue] vs. [Red].

  Each binary classifier predicts one class label. When we input the test data to the classifier, then the model with the majority counts is concluded as a result.

All three are so-called "meta-algorithms": approaches to combine several machine learning techniques into one predictive model in order to decrease the variance (bagging), bias (boosting) or improving the predictive force (stacking alias ensemble).

• Bagging (stands for Bootstrap Aggregating) is a way to decrease the variance of your prediction by generating additional data for training from your original dataset using combinations with repetitions to produce multisets of the same cardinality/size as your original data. By increasing the size of your training set you can't improve the model predictive force, but just decrease the variance, narrowly tuning the prediction to expected outcome.

• Boosting is a two-step approach, where one first uses subsets of the original data to produce a series of averagely performing models and then "boosts" their performance by combining them together using a particular cost function (=majority vote). Unlike bagging, in the classical boosting the subset creation is not random and depends upon the performance of the previous models: every new subsets contains the elements that were (likely to be) misclassified by previous models.

• Stacking is a similar to boosting: you also apply several models to your original data. The difference here is, however, that you don't have just an empirical formula for your weight function, rather you introduce a meta-level and use another model/approach to estimate the input together with outputs of every model to estimate the weights or, in other words, to determine what models perform well and what badly given these input data.