23 Natural Language Processing (NLP) Interview Questions (ANSWERED) For ML Engineers

• Text preprocessing is done to transform a text into a more digestible form so that the machine learning algorithms can perform better. It is found that in tasks such as sentiment analysis, performing some preprocessing such as removing stop-words helps improve the accuracy of the machine learning model.
• Some common text preprocessing done are:
  • removing HTML tags,
  • removing stop-words,
  • removing numbers,
  • lower casing all letters,
  • Lemmatization.

• It identifies the occurrence of words in a document. It identifies the vocabulary and the presence of known words. Hence, it is very simple and flexible.
• It is intuitive that documents consisting of similar content will be similar in other ways such as meaning too. So, the BoW process will create a simple and quick group of features which can be used.
• The BoW model can be made as simple, and as complicated as possible. The main difference is how the vocabulary of words is maintained, and how the different words are scored.

Definition:

• TF-IDF: Term Frequency-Inverse Document Frequency
• TF: Term Frequency

Difference:

• TF-IDF is a numerical statistic that is intended to reflect how important a word is to the document in a collection of the corpus.
• TF is a count of the number of times a word occurs in a document.
• TF-IDF is given by:

• TF is given by the count of a word in the document by the number of words in d:
  $tf(t,d) = \frac{f_{t,d}}{\sum_{t'\in d} f_{t',d}}$

Steps of text preprocessing can be divided into 3 major types:

• Tokenization: It is a process where a group of texts are divided into smaller pieces, or tokens. Paragraphs are tokenized into sentences, and sentences are tokenized into words.
• Normalization: Database normalization is where the structure of the database is converted to a series of normal forms. What it achieves is the organization of the data to appear similar across all records and fields. Similarly, in the field of NLP, normalization can be the process of converting all the words to its lowercase. This makes all the sentences and tokens appear the same and does not complicate the machine learning algorithm.
• Noise Removal: It is a process of cleaning up the text. Doing things such as removing characters which are not required, such as white spaces, numbers, special characters, etc.

• A one-hot is a group of bits which only has one high 1 bit and all other bits are low 0.
• In Natural Language Processing, the one-hot vector can be used to represent a sentence in the form of a matrix of 1 x N size where N is the number of individual words in the corpus.
• For example, the sentence "Peter picked a piece of pickled pepper" can be transformed into a matrix of 1 x 7 where each word is represented by the 7 columns. Hence the output of the sentence is: [0000001, 0000010, 0000100, 0001000, 0010000, 0100000, 1000000]
• An understandable representation of a one-hot vector is shown by the diagram below:

• Stemming just removes the last few characters of a word, often leading to incorrect meanings and spellings.

  Consider:

  eating -> eat, Caring -> Car.

• Lemmatization considers the context and converts the word to its meaningful base form, which is called Lemma.

  Consider:

  Stripes -> Strip (verb) -or- Stripe (noun), better -> good

• PoS tagging is used to classify each word into its part of speech.
• Parts of speech can be used to find grammatical, or lexical patterns without specifying the word used.
• In English especially, the same word can be different parts of speech, so hence, PoS tagging can be helpful to differentiate between them.

• Stemming is not computationally expensive, so it should be used where the dataset is large and performance is an issue.
• Lemmatization is computationally expensive because it involves dictionary look-up or a rule-based system. It is recommended for smaller datasets where accuracy is more important.

• In the Traditional methods the words needed to be translated required to be converted using multiple statistical models, including a probabilistic formulation using the Bayesian rule. There would be a requirement of a lot of human feature engineering, consisting of many machine learning problems, which made the overall system very complicated.
• Neural Machine Translation is the process of modelling this all through a big artificial neural network. The network used is the Recurrent Neural Network which is a stateful neural network. The standard Neural Machine Translation is an end-to-end neural network where the source sentence is encoded by an RNN called encoder, and the target words are predicted using another RNN known as decoder.

• Convolutional neural networks (CNN) is mostly used in image classification but it can also be used for NLP.
• For NLP tasks the sentences are represented as matrices. The row of the matrix consists of a token (or a character).
• The filters of the CNN can be made to slide over the row of the matrix.
• The height may vary, but sliding the windows over 2-5 words is typical.

• Extractive Text Summarization: In this method, the algorithm finds out the most significant sentences in the document and adds them to the summary. In this method the summary contains the exact sentence from the original document.
• Abstractive Text Summarization: In this method, the algorithms get more complicated and it is a sector of active research. The approach is to identify the important sectors, interpret the context and reproduce it in a new way. Generally, the text input is summarized in the shortest way possible with the main information of the text intact without copying verbatim the sentences from the text.

While both are valid approaches to text summarization, it should not be difficult to convince you that abstractive techniques are far more difficult to implement. In fact, the majority of summarization processes today are extraction-based.

Some ambiguities faced are:

1. Lexical Ambiguity: Words which have more than one meaning.
2. Syntactic Ambiguity: Grammars used in sentences are ambiguous, and also more than one parsed tree is correct for a sentence given grammar.
3. Semantic Ambiguity: More than one semantic interpretation for a sentence.
4. Pragmatic Ambiguity: It arises when the statement is not specific, and the context does not provide the information needed to clarify the statement.

• When TF is used, the frequency of the words appearing in the corpus will be used, which will be mostly populated with stop words such as "the", "a", "and", etc. So, most of the output will be skewed towards the stop words.
• When TF-IDF is used, the numerical output is such that the output is not affected by the most common words in the corpus. The way that this is done is that the weights of the common words are diminished down, and the weights of the uncommon words are scaled up.

• Dimensionality of the one-hot vector will increase as the size of the corpus increases. The vector is mostly populated with zeros with only one useful data.
• Due to the dimensionality, an exponentially large memory will be used. For example, for a document of 50,000 vocabulary, we need 2,500,000,000 units of memory (50,000 * 50,000).
• It is hard to extract meaning from the one-hot vectors. The output contains mostly zeros and a single one, so the vector can not create relationships between the different words.

Long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used in the field of deep learning.

• LSTM is a type of RNN which is capable of learning order dependence in sequence prediction problems.
• LSTM can solve problems that were not solvable by previous learning algorithms for RNN. LSTMs are effective at capturing long-term dependencies without suffering from optimization hurdles.
• Some of the applications of LSTMs are:
  • handwriting recognition and generation,
  • language modeling and translation,
  • acoustic modeling of speech,
  • speech synthesis

• Syntactic Analysis is defined as an analysis that tells us the logical meaning of certain given sentences or parts of those sentences. For syntactic analysis the rules of grammar need to be considered to define the logical meaning as well as correctness of the sentence.
• Syntactic Analysis looks at the word order and their meanings, retaining stop-words, the morphology of words, and parts-of-speech of words in a sentence.
• For NLP some of the techniques used in the syntactic analysis are:
  • parsing,
  • word segmentation,
  • morphological segmentation,
  • stemming, and
  • lemmatization.

• BLEU is the abbreviation for bilingual evaluation understudy.
• BLEU is used for evaluating the quality of machine translated text.
• A text is considered higher quality the similar it is to a professional human translator.
• BLEU's is always between 0 and 1. The value of 1 is for the most similar text to the target text.
• A BLEU score of 0.5 is considered high quality, while a score of >0.6 is considered better than humans. Anything less than 0.2 is not understandable and does not give the gist of the translation well.

• NER is a subtask of information extraction that seeks to locate and classify named entity mentioned in unstructured text into pre-defined categories such as person name, organizations, locations, medical codes, monetary values, etc.

Consider:

• Unstructured Text: Neil Armstong of the US had landed on the moon in 1969.
• Converted as: Neil Armstong - name; The US – country; 1969 – time (temporal token).

Sentiment Analysis is the process of analyzing online pieces of writing to determine the emotional tone they carry, whether they’re positive, negative, or neutral.

• A sentiment analysis model can be made to output a simple positive/negative, or there can be a positive/neutral/negative output, or there can also be a 5 stage output represented by 0, 1, 2, 3, 4 where 0 is the most negative and 5 is the most positive.
• The table below shows the type of loss which can be used depending on the type of model to be designed:

• As shown, if the output is a simple positive/negative a Binary Crossentropy loss can be used. If the output is more than 2 classes, a Categorical Crossentropy can be used.