23 Common Artificial Intelligence Interview Questions (SOLVED) For Your Next ML & DS Interview

Reinforcement learning (RL) is a subset of machine learning that allows an AI-driven system (sometimes referred to as an agent) to learn through trial and error using feedback from its actions. This feedback is either negative or positive, signaled as punishment or reward with, of course, the aim of maximizing the reward function.

In terms of learning methods, RL is similar to supervised learning only in that it uses mapping between input and output, but that is the only thing they have in common. Whereas in supervised learning, the feedback contains the correct set of actions for the agent to follow. In RL there is no such answer key. The agent decides what to do itself to perform the task correctly.

Compared with unsupervised learning, RL has different goals. The goal of unsupervised learning is to find similarities or differences between data points. RL's goal is to find the most suitable action model to maximize total cumulative reward for the RL agent. With no training dataset, the RL problem is solved by the agent's own actions with input from the environment.

Dimensionality Reduction is typically choosing a basis or mathematical representation within which you can describe most but not all of the variance within your data, thereby retaining the relevant information, while reducing the amount of information necessary to represent it.

There are a variety of techniques for doing this including but not limited to PCA, ICA, and Matrix Feature Factorization.

These will take existing data and reduce it to the most discriminative components. These all allow you to represent most of the information in your dataset with fewer, more discriminative features.

There are some of the basic terminologies related to genetic algorithms:

• Population: This is a subset of all the probable solutions that can solve the given problem.
• Chromosomes: A chromosome is one of the solutions in the population.
• Gene: This is an element in a chromosome.
• Allele: This is the value given to a gene in a specific chromosome.
• Fitness function: This is a function that uses a specific input to produce an improved output. The solution is used as the input while the output is in the form of solution suitability.
• Genetic operators: In genetic algorithms, the best individuals mate to reproduce an offspring that is better than the parents. Genetic operators are used for changing the genetic composition of this next generation.

The very basic idea is the following: they take n tokens as input, and produce one token as output.

A token is a chunk of text. In the context of OpenAI GPT models, common and short words typically correspond to a single token and long and less commonly used words are generally broken up into several tokens.

This basic idea is applied in an expanding-window pattern. You give it n tokens in, it produces one token out, then it incorporates that output token as part of the input of the next iteration, produces a new token out, and so on. This pattern keeps repeating until a stopping condition is reached, indicating that it finished generating all the text you need.

Now, behind the output is a probability distribution over all the possible tokens. What the model does is return a vector in which each entry expresses the probability of a particular token being chosen.

This probability distribution comes from the training phase. During training, the model is exposed to a lot of text, and its weights are tuned to predict good probability distributions, given a sequence of input tokens.

GPT generative models are trained with a large portion of the internet, so their predictions reflect a mix of the information they’ve seen.

• As we increase the number of dimensions, our data becomes more sparse; every new dimension increases the volume of the feature space, giving our data a higher differentiation chance and therefore, the possibility of it becoming more spread out in a higher dimensional space than in a lower one. This means that if we need more training samples of a kind for our model to be able to learn about them and be able to predict them well, generalizing, in the future.

• As we increase the number of dimensions, especially for parametric models, we increase the time it takes to train them.

• Introducing features that don’t add much value to our models and therefore increasing the number of dimensions, makes our model learn from these noisy or irrelevant features and can lead to a reduction in its performance.

• More features and thus dimensions lead to models that are more complex and harder to interpret than those with a low number of features.

1. Many features and low samples/features ratios will introduce noise into your dataset. In such a case your classification algorithm is likely to overfit and give you a false feeling of good performance.
2. Reducing the number of features will reduce the running time in the later stages. That in turn will enable you to use algorithms of higher complexity, search for more hyperparameters or do more evaluations.
3. A smaller set of features is more comprehendible to humans. That will enable you to focus on the main sources of predictability and do more exact feature engineering. If you will have to explain your model to a client, you are better at presenting a model with 5 features than a model with 200 features.

• Deep Learning gives a better performance compared to machine learning if the dataset is large enough.
• Deep Learning does not need the person designing the model to have a lot of domain understanding for feature introspection. Deep learning outshines other methods if there is no feature engineering done.
• Deep Learning really shines when it comes to complex problems such as image classification, natural language processing, and speech recognition.

• Activation Functions help in keeping the value of the output from the neuron restricted to a certain limit as per the requirement. If the limit is not set then the output will reach very high magnitudes. Most activation functions convert the output to -1 to 1 or to 0 to 1.
• The most important role of the activation function is the ability to add non-linearity to the neural network. Most of the models in real-life is non-linear so the activation functions help to create a non-linear model.
• The activation function is responsible for deciding whether a neuron should be activated or not.

• The Principal Component Analysis (PCA) is the process of computing principal components and using them to perform a change of basis on the data.
• The Principal Component of a collection of points in a real coordinate space are a sequence of p unit vectors, where the i-th vector is the direction of a line that best fits the data while being orthogonal to the i - 1 vectors. The best-fitting line is defined as the line that minimizes the average squared distance from the points to the line.
• PCA is commonly used in dimensionality reduction by projecting each data point onto only the first few principal components to obtain lower-dimensional data while preserving as much of the data's variation as possible.

• Overfitting refers to a model that models the training data too well.
• Overfitting happens when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data. This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. The problem is that these concepts do not apply to new data and negatively impact the model's ability to generalize.

• High Bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting).
• High Variance may result from an algorithm modeling random noise in the training data (overfitting).

• The Bias-Variance tradeoff is a central problem in supervised learning. Ideally, a model should be able to accurately capture the regularities in its training data, but also generalize well to unseen data.
• It is called a tradeoff because it is typically impossible to do both simultaneously:
  • Algorithms with high variance will be prone to overfitting the dataset, but
  • Algorithms with high bias will underfit the dataset.

• Forward propagation is the input data that is fed in the forward direction through the network. Each hidden layer accepts the input data, processes it as per the activation function, and passes it to the successive layer.
• Back propagation is the practice of fine-tuning the weights of the neural network based on the error rate obtained from the previous epoch. Proper tuning of the weights ensures low error rates, making the model more reliable.

• Supervised learning is when the data you feed your algorithm with is tagged or labelled, to help your logic make decisions.

  Example: a hypothetical non-machine learning algorithm for face detection in images would try to define what a face is (round skin-like-colored disk, with dark area where you expect the eyes etc). A machine learning algorithm would not have such coded definition, but would "learn-by-examples": you'll show several images of faces and not-faces and a good algorithm will eventually learn and be able to predict whether or not an unseen image is a face.

• Unsupervised learning are types of algorithms that try to find correlations without any external inputs other than the raw data (your examples are not labeled, i.e. you don't say anything). In such a case the algorithm itself cannot "invent" what a face is, but it can try to cluster the data into different groups, e.g. it can distinguish that faces are very different from landscapes, which are very different from horses.

• Random Forest is very versatile and can be used in both regression and classification tasks. It can also handle all binary, categorical, and numerical features.
• The process is parallelizable where the process can be split to run in different machines.
• It performs better in high dimensionality since the work is on subsets of data.
• The training speed is faster than decision trees because they are working only on a subset of features. Even if there are hundreds of features the training speed will be significantly faster.
• The Random Forest is good at balancing errors for class population unbalanced data sets.
• It has low bias, but moderate variance because when all the trees are averaged in random forest, all the variances are also averaged so it has low bias but a moderate variance.

• Convolutional neural networks (CNN) is mostly used in image classification but it can also be used for NLP.
• For NLP tasks the sentences are represented as matrices. The row of the matrix consists of a token (or a character).
• The filters of the CNN can be made to slide over the row of the matrix.
• The height may vary, but sliding the windows over 2-5 words is typical.

A model with too little capacity cannot learn the problem, but a model with too much capacity will overfit the data. To prevent overfitting the data some things that can be done are:

• Training the model in more data. Giving the model an infinite number of examples will plateau the model in terms of what the capacity of the network is capable of learning.
• Changing the network structure (number of weights). Pruning by removing the nodes in the network will counteract the overfitting.
• Changing the network parameters (values of weights). Larger weights cause sharp transitions in the activation functions and cause large changes in output for small changes in inputs. This method to keep weights small is called Regularization.

The building blocks of ChatGPT are:

1. The Transformer architecture: The Transformer architecture is the foundation of ChatGPT. It is a neural network architecture that uses self-attention mechanisms to process input sequences. The transformer architecture is able to handle input sequences of varying lengths and allows for parallel processing of the input.
2. The Encoder: The Encoder is composed of multiple layers of self-attention and feed-forward neural networks. It processes and understands the input text.
3. The Decoder: The decoder is also composed of multiple layers of self-attention and feed-forward neural networks. It generates the output text.
4. The Language Model Head: The language model head is a linear layer with weights that are learned during pre-training. It is used to predict the next token in the sequence, given the previous tokens.
5. The Dialogue Generation Head: The dialogue generation head is a linear layer with weights that are learned during fine-tuning the model on conversational data. It is used to generate the response to a given prompt in the context of a dialogue.
6. Pre-training: ChatGPT is pre-trained on a large dataset of text, which enables it to generate human-like text in response to a given prompt.
7. Fine-Tuning: The model is fine-tuned on conversational data to improve its ability to generate responses in the context of a dialogue.

Depending on the target variable, there are use various methods for feature selection.

• If we have labeled data, we can select features using filter or wrapper methods.

• In Filter methods we use statistical techniques to evaluate the relationship between each input variable and the target variable. They are suitable when we have to deal with high-dimensional data. Some of these techniques are:

  • Information Gain: calculates for each variable the reduction in entropy or surprise from transforming a dataset in some way, and then we select the variables that maximizes the information gain, which in turn minimizes the entropy.
  • Scoring algorithm or Fisher’s Score: which returns the ranks of the variables based on the gradient of the log-likelihood function in descending order.
  • Correlation coefficient: correlation tells us the linear relationship of 2 or more variables, so we can select those features that have a high correlation with the target, but keeping in mind that the features should be uncorrelated among themselves.

• In wrapper methods we create many models with different subsets of input features and select those features that result in the best performing model according to a performance metric. Some of these techniques include:

  • Forward Feature Selection: here we start with the best performing variable against the target. Next, we select another variable that gives the best performance in combination with the first selected variable and we'll continue this process until satisfying some criterion.
  • Backward Feature Elimination: this method works exactly opposite to the Forward Selection method. Here, we start with all the features available and build a model. Next, we remove the variable from the model which gives the worst evaluation measure value and continue this process until an arbitrary criterion is achieved.
  • Exhaustive Feature Selection: here we evaluate every possible combination of the variables and returns the best performing subset.

• If we have unlabeled data, we can use some of the following criteria:

  • Variance Threshold: here we assume that features with a higher variance may contain more information, so we will remove all features which variance doesn’t meet some threshold. By default, we remove all zero-variance features.
  • Dispersion ratio: is the arithmetic mean divided by the geometric mean. Higher dispersion corresponds to more relevant features.
  • Mean Absolute Difference (MAD): computes the absolute difference from the mean value, the higher the MAD, the higher the discriminatory power.

There are several Transfer Learning techniques that are commonly used in LLMs. Here are three of the most popular:

• Feature-based transfer learning: This technique involves using a pre-trained language model as a feature extractor, and then training a separate model on top of the extracted features for the target task.

• Fine-tuning: involves taking a pre-trained language model and training it on a specific task. Sometimes when fine-tuning, you can keep the model weights fixed and just add a new layer that you will train. Other times you can slowly unfreeze the layers one at a time. You can also use unlabelled data when pre-training, by masking words and trying to predict which word was masked.

• Multi-task learning: involves training a single model on multiple related tasks simultaneously. The idea is that the model will learn to share information across tasks and improve performance on each individual task as a result.

• Bagging:

  • Is a parallel ensemble method: the base learners are created independently for each other and are trained simultaneously.
  • Is based on the bootstrap aggregate: it aggregates several sampling subsets of the original dataset to train different learners chosen randomly with replacement.
  • Is focused on decreases variance, not bias, and solves over-fitting issues in a model.

• Boosting

  • Is a sequential ensemble method: the base learners depend on each other, so each learner influences the next one and general paternal behavior can be deduced.
  • Each base learner in the sequence is fitted giving more importance to observations in the dataset that were badly handled by the previous learners in the sequence. Intuitively, each new model focuses its efforts on the most difficult observations to fit up to now, so that we obtain, at the end of the process, a strong learner.
  • Is mainly focused on reducing bias, not variance, and can increase the overfitting in some cases.

• Backpropagation is an algorithm that computes the gradient of the loss function with respect to the weights of the network for a single input-output example.
• Backpropagation only refers to the algorithm used to compute the gradient, it does not include how the gradient is used.
• The formula for updating the weights and biases after calculating the gradient (the partial differentiation in the equation below) is shown below:

$\theta^{t+1} = \theta^t - \alpha \frac{\partial E(X, \theta^t)}{\partial \theta}$

Where:

• $E(X, \theta)$ is the error function.
• $w_{ij}^k$ is the weights.
• $b_i^k$ is the biases.
• $\alpha$ is the learning rate.
• $\theta^t$ is the parameters of the neural network (weights and biases) at iteration t in gradient descent.