17 Naive Bayes interview Questions (SOLVED) To Check Before Next ML Interview

• Naive Bayes Classifiers are a family of simple probabilistic classifiers based on applying Bayes' theorem with strong independence assumptions between the features.
• Bayes' theorem is given by the following equation:
  $P(A|B)=\frac{P(B|A)P(A)}{P(B)}$
• Using Bayes' theorem, the probability of A happening given that B has occurred can be found.
• An example of the way a Naive Bayes Classifier can be used is, given that it has rained, the probability of temperature being low is P(Temperature|Rain).

Statistical significance is a term used by researchers to state that it is unlikely their observations could have occurred under the null hypothesis of a statistical test. Significance is usually denoted by a p-value, or probability value.

Statistical significance is arbitrary – it depends on the threshold, or alpha value, chosen by the researcher. The most common threshold is p < 0.05, which means that the data is likely to occur less than 5% of the time under the null hypothesis.

When the p-value falls below the chosen alpha value, then we say the result of the test is statistically significant.

• Descriptive statistics, as its name suggests, focus on describing the characteristics or features of a dataset. Here we look for measures of distribution, central tendency and variability in order to draw conclusions based on known data.

• Inferential statistics focus on making generalizations about a larger population based on a representative sample of that population, It also allows us to make predictions so its results are usually in the form of a probability. Here, we perform hypothesis testing, compute confidence intervals, make regression and correlation analyses, in order to draw conclusions that go beyond the available data.

We call it naive because its assumptions (it assumes that all of the features in the dataset are equally important and independent) are really optimistic and rarely true in most real-world applications:

• we consider that these predictors are independent
• we consider that all the predictors have an equal effect on the outcome (like the day being windy does not have more importance in deciding to play golf or not)

• If the availability of data is a constraint, i.e. if the training data is smaller or if the dataset has a fewer number of observations and a higher number of features, we can choose algorithms with high bias/low variance like Naïve Bayes and Linear SVM.

• If the training data is sufficiently large and the number of observations is higher as compared to the number of features, one can go for low bias/high variance algorithms like K-Nearest Neighbors, Decision trees, Random forests, and kernel SVM.

We can then discover the probability of dangerous Fire when there is Smoke using the Bayes' Theorem:

P(Fire | Smoke) = P(Fire) * P(Smoke | Fire) / P(Smoke)
                = 0.01 * 0.9 / 0.1 
                = 0.09 (9%)

So probability of dangerous fire when there is a smoke is 9%.

Naive Bayes methods work by applying Bayes’ theorem with the “naive” assumption of conditional independence between every pair of features given the value of the class variable.

The Bayes’ theorem states that for a given class variable $y$ and dependent feature vector $x = x_1 + x_2 ... x_n$. The conditional probability of the class label $y$, given the observation $x$ is:

We can simplify the above expression using the naive assumption that features of measurement are independent of each other, i.e.

Which lead to:

Since, $P(x_1, \dots, x_n)$ is constant given the input, we use Maximum A Posteriori (MAP) estimation to estimate $P(y)$ and $P(x_i \mid y)$, which results in:

This calculation can be performed for each of the class labels, and the label with the largest probability can be selected as the classification for the given instance.

The different Naive Bayes classifiers differ mainly by the assumptions they make regarding the distribution of $P(x_i \mid y)$.

We can use any kind of predictor in a Naive Bayes classifier. All we need is the conditional probability of a feature given the class, i.e., P(F | Class).

• For the categorical features, we can estimate P(F | Class) using a distribution such as multinomial or Bernoulli.
• For the numerical features, we can estimate P(F | Class) using a distribution such as Normal or Gaussian.
• For a numerical feature that follows a different specific distribution, for example, an exponential, then that specific distribution may be used instead.
• For a numerical or categorical without a well-defined distribution, then a kernel density estimator can be used to estimate the probability distribution instead.

Since Naive Bayes assumes the conditional independence of the features, it can use different types of features together. We can calculate each feature’s conditional probability and multiply them to get the final prediction.

Often, we know how frequently some particular evidence is observed, given a known outcome. We have to use this known fact to compute the reverse, to compute the chance of that outcome happening, given the evidence. Conceptually, this is a way to go from P(Evidence | Known Outcome) to P(Outcome | Known Evidence). Bayes' theorem is a relatively simple, but fundamental result of probability theory that allows for the calculation of certain conditional probabilities. Conditional probabilities are just those probabilities that reflect the influence of one event on the probability of another.

The formula is:

Which tells us:

• how often A happens given that B happens, written $P(A|B)$,

When we know:

• how often B happens given that A happens, written $P(B|A)$
• and how likely A is on its own, written $P(A)$
• and how likely B is on its own, written $P(B)$

As an example let us say P(Fire) means how often there is fire, and P(Smoke) means how often we see smoke, then

• P(Fire | Smoke) means how often there is fire when we can see smoke
• P(Smoke | Fire) means how often we can see smoke when there is fire

So the formula kind of tells us "forwards" P(Fire | Smoke) when we know "backwards" P(Smoke | Fire)

Some disadvantages of using Naive Bayes Algorithm are:

• It relies on a very big assumption that the independent variables are not related to each other.
• It is generally not suitable for datasets with large numbers of numerical attributes.
• It has been observed that if a rare case is not in the training dataset but is in the testing dataset, then it will most definitely be wrong.

Some advantages of using Naive Bayes algorithm are:

• It is a simple, fast, and very robust method of classification.
• It does well with both clean and noisy data.
• It requires a few examples for training, but the underlying assumption is that the training dataset is a true representative of the population.
• It is easy to get the probability for a prediction.

In probability theory, the Central Limit Theorem (CLT) states that the distribution of a sample variable approximates a normal distribution (i.e., a “bell curve”) as the sample size becomes larger, assuming that all samples are identical in size, and regardless of the population's actual distribution shape.

Put another way, CLT is a statistical premise that, given a sufficiently large sample size from a population with a finite level of variance, the mean of all sampled variables from the same population will be approximately equal to the mean of the whole population.

Mathematically, the Central Limit Theorem (CLT) is a statement about the cumulative distribution function of the random variable

where the $X_i$ are independent identically distributed random variables with mean $\mu$ and standard deviation $\sigma$. The CLT asserts that for each $a$, $-\infty < a < \infty$,

as $n \to \infty$.

• A Generative Model ‌explicitly models the actual distribution of each class. It ‌learns the joint probability distribution p(x,y) = P(y) * P(x|y), where both P(y) and P(x|y) can be estimated from the dataset by computing class frequencies. Some examples of generative models are:

  • Naïve Bayes
  • Bayesian networks
  • Markov random fields

• A Discriminative Model models the decision boundary between the classes by ‌learning the conditional probability distribution P(y|x) from the dataset. Some examples of such kinds of classifiers are:

  • ‌Logistic regression
  • Scalar Vector Machine
  • Traditional neural networks
  • KNN.