35 Advanced Statistics Interview Questions For Data Scientists and ML Engineers

A Probability Distribution is a statistical function that describes all the possible values and likelihood that a random variable can take within a given range.

There are two main types of probability distribution:

• Discrete probability distributions: used for random variables with discrete outcomes, for example, the number of heads in five consecutive coin tosses, the number of rainy days in a given week, the number of goals scored by a player, and so on.
• Continuous probability distributions: used for random variables with continuous outcomes, for example, the height of male students, median house prices in San Francisco, claim amounts experienced by an insurance company, and so on.

Let’s say the first card we draw from each deck is a red Ace. This means that in the deck of the first box with 12 reds and 12 blacks, there’s now 11 reds and 12 blacks. Therefore the odds of drawing another red are equal to:

In the second box that has a deck with 24 reds and 24 blacks, there would then be 23 reds and 24 blacks. Therefore the odds of drawing another red are equal to:

Since 23/47 > 11/23, the second deck with more cards has a higher probability of getting the same two cards.

The mode is the value that appears most frequently in a data set. A set of data may have one mode, more than one mode, or no mode at all. A data set can often have no mode, one mode, or more than one mode – it all depends on how many different values repeat most frequently.

For example, in the following list of numbers, 16 is the mode since it appears more times in the set than any other number:

• 3, 3, 6, 9, 16, 16, 16, 27, 27, 37, 48

Your data can be:

• without any mode
• unimodal, with one mode,
• bimodal, with two modes,
• trimodal, with three modes, or
• multimodal, with four or more modes.

Given A = {1.3,5} and B = {1,2,3}, if we know B has occurred, the outcome must be among {1,2,3}. For A to also happen the outcome must be in A ∩ B = {1,3}. Since all die rolls are equally likely, then we calculate the conditional probability as:

A measure of central tendency is a single value that attempts to describe a set of data by identifying the central position within that set of data. In the simplest of terms, it attempts to find a single value that best represents an entire distribution of scores.

Mean, Median and Mode are average values or central tendency of a numerical data set.

The normal distribution is the most important probability distribution in statistics because it fits many natural phenomena. For example, heights, blood pressure, measurement error, and IQ scores follow the normal distribution. It is also known as the Gaussian distribution and the bell curve.

Normal distributions have the following features:

• symmetric bell shape
• mean and median and mode are equal; both located at the center of the distribution
• ≈68% of the data falls within 1 standard deviation of the mean
• ≈95% of the data falls within 2 standard deviations of the mean
• ≈99.7% of the data falls within 3 standard deviations of the mean

Statistical significance is a term used by researchers to state that it is unlikely their observations could have occurred under the null hypothesis of a statistical test. Significance is usually denoted by a p-value, or probability value.

Statistical significance is arbitrary – it depends on the threshold, or alpha value, chosen by the researcher. The most common threshold is p < 0.05, which means that the data is likely to occur less than 5% of the time under the null hypothesis.

When the p-value falls below the chosen alpha value, then we say the result of the test is statistically significant.

• The 68-95-99.7 rule, is also known as the Empirical Rule.
• It is a shorthand used to remember the percentage of values that lie within an interval estimate in a Normal Distribution.
• It is shown below:

The rule states that 68%, 95%, and 99.7% of the values lie within one, two, and three standard deviations of the mean, respectively.

• Descriptive statistics, as its name suggests, focus on describing the characteristics or features of a dataset. Here we look for measures of distribution, central tendency and variability in order to draw conclusions based on known data.

• Inferential statistics focus on making generalizations about a larger population based on a representative sample of that population, It also allows us to make predictions so its results are usually in the form of a probability. Here, we perform hypothesis testing, compute confidence intervals, make regression and correlation analyses, in order to draw conclusions that go beyond the available data.

• A Combination is the choice of r elements from a set of n elements without replacement and where order does not matter. Is most used to group data. For example, picking three team members from a group, picking two colors from a color brochure, etc. It is mathematically defined as:

  $C_r^n = \frac{n!}{(n-r)r!}$

• A Permutation is the choice of r elements from a set of n elements without replacement and where the order matters. Is used to list data, for example picking first, second and third place winners, picking two favorite colors -in order- from a color brochure, etc. It is mathematically defined as:

  $P_{n,r} = \frac{n!}{(n-r)!}$

• The Bernoulli distribution is the discrete probability distribution of a random variable which takes a binary output: 1 with probability p, and 0 with probability (1-p). The idea is that, whenever we are running an experiment that might lead either to success or to a failure, we can associate with our success (labeled with 1) a probability p, while our failure (labeled with 0) will have probability (1-p).

• In the Binomial distribution we keep the same idea as before: we count probability of a success or failure outcome in an experiment, but this time is is repeated multiple times.

• The confidence level is the percentage of times we expect to get close to the same estimate if we run our experiment again or resample the population in the same way.

• The confidence interval is the actual upper and lower bounds of the estimate we expect to find at a given level of confidence.

For example, if we are estimating a 95% confidence interval around the mean proportion of female babies born every year based on a random sample of babies, we might find an upper bound of 0.56 and a lower bound of 0.48. These are the upper and lower bounds of the confidence interval for a confidence level of 95%.

This means that 95% of the time, we can expect our estimate to fall between 0.56 and 0.48.

To answer this question let's say $X$ is the number of heads and let's assume that the coin is not biased. Since each individual flip is a Bernoulli random variable, we can assume it has a probability of showing up heads as p = 0.5, so this will lead to the following expected number of heads:

And the following standard deviation:

Given that we got a 1000 sample size, we can apply the Central Limit Theorem to approximate the total number of heads as normal distribution and calculate the corresponding z-score to test the hypothesis that the coin is fair.

This means that, if the coin were fair, the event of seeing 550 heads should occur with a < 1% chance under normality assumptions. Therefore, the coin is likely biased.

In statistics, if a data distribution is approximately normal then about 68% of the data values lie within one standard deviation of the mean and about 95% are within two standard deviations, and about 99.7% lie within three standard deviations:

Therefore, if you have any data point that is more than 3 times the standard deviation, then those points are very likely to be anomalous or outliers.

Given the probability formula:

where $n$ is the number of desired outcomes and $n_{total}$ is the number of possible outcomes. For one single fair dice, we have 6 possible outcomes, so for two fair dices, we have 6*6 = 36 possible outcomes.

We want that the two dices sums 8 so the possible combinations are:

• (2,6),(3,5),(4,4)

• (5,3),(6,2)

We got that the number of desired outcomes are 5. The probability is then

For the second part we use the Bayes Theorem and conditional probability formula:

Let's say:

• A is the event of getting an 8
• B is the event of getting a 3.

Now, from the previous combination, we got that (3,5) is the only event that satisfies both A and B so then P(A ∩ B) = 1/36. On other hand, the probability of getting a 3 in a fair dice is just 1/6, we now put these values on the previous formula and compute:

• Simple Random Sampling: Every element has an equal chance of getting selected to be the part sample. It is used when we don’t have any kind of prior information about the target population.
• Stratified Sampling: we divide the elements of the population into small subgroups (or strata) based on the similarity in such a way that the elements within the group are homogeneous and heterogeneous among the other subgroups formed. Then, the elements are randomly selected from each of these strata.
• Cluster Sampling: we divide the entire population into subgroups, wherein, each of those subgroups has similar characteristics to that of the population when considered in totality. Also, instead of sampling individuals, we randomly select the entire subgroups.
• Systematic Clustering: is about sampling items from the population at regular predefined intervals. For example, every 5th element, 21st element and so on. All the elements are put together in a sequence first where each element has an equal chance of being selected.

Descriptive statistics, in short, help describe and understand the features of a specific data set by giving short summaries about the sample and measures of the data.

For instance, consider a simple number used to summarize how well a batter is performing in baseball, the batting average. This single number is simply the number of hits divided by the number of times at bat (reported to three significant digits). A batter who is hitting .333 is getting a hit one time in every three at bats. One batting .250 is hitting one time in four. The single number describes a large number of discrete events.

• Frequency Distribution: Used for both quantitative and qualitative data, frequency distribution depicts the count of the different outcomes in a data set and summarizes it in tables or graphs. It allows for a more structured and organized way to present raw data.
• Central Tendency: refers to a dataset’s descriptive summary using a single value reflecting the center of the data distribution. The three most common measures of central tendency are the mode, median, and mean.
• Variability: it concerns how far apart the data points appear to fall from the center. It is most commonly measured with range, interquartile range, standard deviation, and variance.

• Range: is the difference between the largest and smallest values in that dataset. This measure is easy to calculate but it is very much affected by extreme values.
• The Interquartile Range (IQR): Is the difference between upper quartile (Q3) and lower quartile (Q1), we can see it as the middle half of the data.
• Variance: is the average squared difference of the values from the mean and includes all values in the calculation by comparing each value to the mean. The more spread the data, the larger the variance is in relation to the mean.

The alternate hypothesis is usually your initial hypothesis that predicts a relationship between variables. The null hypothesis is a prediction of no relationship between the variables you are interested in.

For this example:

• Ho (null hypothesis): Men are, on average, not taller than women.
• Ha (alternate hypothesis): Men are, on average, taller than women.

Each ant has two possible ways to go: the edge on its left L and the edge on its right R. Now the only way no ant will collide is if they all walk in the same direction along the triangle (assuming they all move at the same speed). Overall the ways how the ants can move are:

1. LLL
2. LLR
3. LRL
4. RLL
5. LRR
6. RLR
7. RRL
8. RRR

We have a total of 8 ways how the 3 Ants can move, out of these, only RRR and LLL are the ways by which the Ants will never meet. So the probability of it is 2/8 = 0.25

Let's denoted as A a person who has the disease, and let B a positive test. To calculate P(A|B) and we will use the Bayes' theorem:

Where P(B) is given by the law of total probability:

Now, we know that:

• Out of 1000 people, 1 person who has the disease will get true positive result, so P(A) = 0.001 and P(notA) = 1 - P(A) = 0.999.
• The test has a true positive rate of 100%, so P(B|A) = 1
• The test has a false positive rate of 5%, so P(B|notA) = 0.05.

Therefore,

Thus, there is only a 2% probability of one person having the disease even if the reports say that it has the disease.

Often, we know how frequently some particular evidence is observed, given a known outcome. We have to use this known fact to compute the reverse, to compute the chance of that outcome happening, given the evidence. Conceptually, this is a way to go from P(Evidence | Known Outcome) to P(Outcome | Known Evidence). Bayes' theorem is a relatively simple, but fundamental result of probability theory that allows for the calculation of certain conditional probabilities. Conditional probabilities are just those probabilities that reflect the influence of one event on the probability of another.

The formula is:

Which tells us:

• how often A happens given that B happens, written $P(A|B)$,

When we know:

• how often B happens given that A happens, written $P(B|A)$
• and how likely A is on its own, written $P(A)$
• and how likely B is on its own, written $P(B)$

As an example let us say P(Fire) means how often there is fire, and P(Smoke) means how often we see smoke, then

• P(Fire | Smoke) means how often there is fire when we can see smoke
• P(Smoke | Fire) means how often we can see smoke when there is fire

So the formula kind of tells us "forwards" P(Fire | Smoke) when we know "backwards" P(Smoke | Fire)

The Bayesian approach defines probability as the measure of believability one has about how a particular event occurs. It uses the Bayes theorem to help us update beliefs about random events once we've seen new evidence about the event, so we use Bayesian statistics to create different beliefs after new evidence is uncovered. It differs from frequentist statistics that rely only on data from repeated trials.

In probability theory, the Central Limit Theorem (CLT) states that the distribution of a sample variable approximates a normal distribution (i.e., a “bell curve”) as the sample size becomes larger, assuming that all samples are identical in size, and regardless of the population's actual distribution shape.

Put another way, CLT is a statistical premise that, given a sufficiently large sample size from a population with a finite level of variance, the mean of all sampled variables from the same population will be approximately equal to the mean of the whole population.

Mathematically, the Central Limit Theorem (CLT) is a statement about the cumulative distribution function of the random variable

where the $X_i$ are independent identically distributed random variables with mean $\mu$ and standard deviation $\sigma$. The CLT asserts that for each $a$, $-\infty < a < \infty$,

as $n \to \infty$.

• Homoskedasticity occurs when the variance of the error term in a regression model is constant.
• Heteroskedasticity happens when the standard errors of a variable, monitored over a specific amount of time, are non-constant.

We can identify each case by plotting the residual values vs the fitted values of a linear regression model.

The importance of each one case relies on the context of the ordinary least squares (OLS) estimator, which is a common way to perform linear regression. In OLS, we must satisfy the assumption of homoskedasticity in order to get an efficient estimation.

• Kurtosis is a measure of the tailedness of the probability distribution of a real-valued random variable, it can also be seen as the heaviness of the distribution tails. In normal distributions, the kurtosis value is 3, so for any other distribution we measure excess kurtosis, defined as kurtosis - 3. According to that value, we can define 3 types of excess kurtosis:

• Skewness is a distortion or asymmetry that deviates from the symmetrical bell curve, or normal distribution, in a set of data. It has two types:

• The Binomial distribution describes the probability of obtaining k successes in n Bernoulli experiments, i.e an experiment which has only two possible outcomes, often call them success and failure. Its probability function describes the probability of getting exactly k successes in n independent Bernoulli trials:

  $P(X = k) = {{n}\choose{k}} p^k (1-p)^{n-k}$

• The Geometric distribution describes the probability of experiencing a certain amount of failures before experiencing the first success in a series of Bernoulli experiments. This probability is given by:

  $P(X = k ) = p^k (1-p)^{n-k}$

  So as we can see, the key difference is that in a binomial distribution, there is a fixed number of trials meanwhile in a geometric distribution, we’re interested in the number of trials required until we obtain a success.

The arithmetic mean is denoted as $\bar{x}$

where each $x_i$ represents a unique observation. The arithmetic mean measures the average value for a given set of numbers.

In contrast to this, the median is the value that falls directly in the middle of your dataset. The median is especially useful when you are dealing with a wide range or when there is an outlier (a very high or low number compared to the rest) which would skew the mean.

For example, salaries are usually discussed using medians. This is due to the large disparity between the majority of people and a very few people with a lot of money (with the few people with a lot of money being the outliers). Thus, looking at the 50% percentile individual will give a more representative value than the mean in this circumstance.

Alternatively, grades are usually described using the mean (average) because most students should be near the average and few will be far below or far above.