17 Linear Algebra Interview Questions To Brush Before Data Science Interview

The invert of a square diagonal matrix exists if all entries of the diagonal are non-zero. If it is the case, the invert is obtained by replacing each element in the diagonal with its reciprocal.

Given a matrix A, we find its eigenvalues λ by solving the equation:

For example, given the following matrix,

we determine its eigenvalues in the following way:

Now the characteristic polynomial is:

$\lambda ^2 + \lambda - 6 = 0$

The solutions of this equation and therefore the eigenvalues are then,

$\lambda_1 = 2, \lambda_2 = -3$

Ax = b is a system of linear equations expressed in matrix notation, in which:

• A is the coefficient matrix of order m x n.
• x is the incognite variables vector of order n x 1.
• b is the vector formed by the constants and its order is m x 1.

The system Ax = b has a unique solution if and only if

where the matrix A|b is matrix A with b appended as an extra column.

• A dot product is a fundamental way we can combine two vectors. Intuitively, it tells us something about how much two vectors point in the same direction or what is the projection of a vector on another vector and it returns a single number. It's defined as:
  $\vec{u} \cdot \vec{v} = \| \vec{u} \|\| \vec{v} \| \cos (\alpha)$
  where α is the angle between the vectors.
  In the picture, OA' is the projection of the vector of u on v.

• A cross product is another way to combine two vectors. The difference here is that the result is another vector that is perpendicular to the initial ones. The cross prodct it's defined as:
  $\vec{a} \times \vec{b} = \| \vec{a} \|\| \vec{b} \| \sin (\theta)$
  where θ is the angle between the vectors.

Two vectors are said to be orthogonal if the dot product of them is equal to zero,

This is because the definition of the dot product:

where θ is the angle between the two vectors, therefore if x and y are orthogonal, the angle between them is 90 and cos(90) = 0.

We say that a matrix A is normal if $AA^* = A^* A,$ where A* is the conjugate transpose of matrix A.

1. Prove equality for A:

   $AA^* = \begin{pmatrix} 1 & i \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ -i & 1 \end{pmatrix} = \begin{pmatrix} 2 & i \\ -i & 1 \end{pmatrix}$
   $A^*A = \begin{pmatrix} 1 & 0 \\ -i & 1 \end{pmatrix} \begin{pmatrix} 1 & i \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & i \\ -i & 2 \end{pmatrix}$

   $\therefore AA^* \not = A^* A \Rightarrow A \text{ is not normal.}$

2. Prove equality for B:

   $BB^* = \begin{pmatrix} 1 & i \\ 1 & 2+1 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ -i & 2-i \end{pmatrix} = \begin{pmatrix} 2 & 2+2i \\ 2-2i & 6 \end{pmatrix}$
   $B^*B = \begin{pmatrix} 1 & 1 \\ -i & 2-i \end{pmatrix} \begin{pmatrix} 1 & i \\ 1 & 2+i \end{pmatrix} = \begin{pmatrix} 2 & 2+2i \\ 2-2i & 6 \end{pmatrix}$

   $\therefore BB^* = B^* B \Rightarrow B \text{ is normal.}$

We will compute inverse using the following equation: $A^{-1} = \frac{1}{det(A)} adj(A)$ where adj(A) is the adjugate of matrix A. Now we follow the next steps:

1. Calculate the determinant of A:

   $\begin{aligned} det(A) =& 4\left|\begin{matrix}0&3\\2 & 6\end{matrix}\right| -(-2)\left|\begin{matrix}5&3\\-1 & 6\end{matrix}\right|+1\left|\begin{matrix}5&0\\-1& 2\end{matrix}\right| \\ \ \\ = \, & 4(0 - 6) + 2(30 + 3) + 1(10 - 0) \\ \ \\ =& \, 52 \not = 0 \therefore \text{the matrix is invertible} \end{aligned}$

2. Calculate the cofactor of each element:

   $\begin{aligned} C_{11} &= \left(-1\right)^{1 + 1} \left|\begin{array}{cc}0 & 3\\2 & 6\end{array}\right| = -6, \qquad C_{12} = \left(-1\right)^{1 + 2} \left|\begin{array}{cc}5 & 3\\-1 & 6\end{array}\right| = -33 , \\ \ \\ C_{13} &= \left(-1\right)^{1 + 3} \left|\begin{array}{cc}5 & 0\\-1 & 2\end{array}\right| = 10, \qquad C_{21} = \left(-1\right)^{2 + 1} \left|\begin{array}{cc}-2 & 1\\2 & 6\end{array}\right| = 14, \\ \ \\ C_{22} &= \left(-1\right)^{2 + 2} \left|\begin{array}{cc}4 & 1\\-1 & 6\end{array}\right| = 25, \qquad C_{23} = \left(-1\right)^{2 + 3} \left|\begin{array}{cc}4 & -2\\-1 & 2\end{array}\right| = -6, \\ \ \\ C_{31} &= \left(-1\right)^{3 + 1} \left|\begin{array}{cc}-2 & 1\\0 & 3\end{array}\right| = -6, \qquad C_{32} = \left(-1\right)^{3 + 2} \left|\begin{array}{cc}4 & 1\\5 & 3\end{array}\right| = -7, \\ \ \\ C_{33} &= \left(-1\right)^{3 + 3} \left|\begin{array}{cc}4 & -2\\5 & 0\end{array}\right| = 10. \end{aligned}$

   Thus, the cofactor matrix is: $C = \left[\begin{array}{ccc}-6 & -33 & 10\\14 & 25 & -6\\-6 & -7 & 10\end{array}\right]$

3. Obtain the adjugate matrix by transposing cofactor matrix
   $adj(A) = \left[\begin{array}{ccc}-6 & 14 & -6\\-33 & 25 & -7\\10 & -6 & 10\end{array}\right]$
4. Finally, the inverse matrix is the adjugate matrix divided by the determinant:
   $\begin{aligned} A^{-1} &= \dfrac{1}{52} \cdot \left[\begin{matrix}-6 & 14 & -6\\-33&25&-7\\10&-6 & 10\end{matrix}\right] \\ \ \\ A^{-1} &= \left[\begin{matrix}-3/26 & 7/26 & -3/26\\-33/52&25/52&-7/52\\5/26&-3/26 & 5/26\end{matrix}\right] \end{aligned}$

Given an nxn matrix A, to find its diagonal matrix D we must follow the next steps:

1. Find the characteristic polynomial of A.
2. Find the roots of the characteristic polynomial to obtain the eigenvalues λ of A.
3. For each eigenvalue λ of A, find their correspondent eigenvectors.
4. If the total number of eigenvectors m found in step 3 is not equal to n (the numbers of rows and columns of A), then the matrix is not diagonalizable, but if m = n then the diagonal matrix D is given by:
   $\bf D = P^{-1} A P,$
   where P is a matrix which columns are the eigenvectors of the matrix A.

For an arbitrary A matrix, we can derive it's inverse by following the next steps:

1. Check if the matrix if invertible by finding its determinant :
   $|A| = ad -bc.$
   If |A| ≠ 0 then the matrix is invertible.
2. Interchange the two elements on the diagonal.
3. Take the negatives of the other two elements out of the diagonal.
4. Divide each element of the matrix by |A|. The result of the inverse of the matrix A is then:

There are different ways to measure the magnitude of vectors, the most common are:

• L0 norm: Although it's not formally speaking a norm, it's often used as if was one. It corresponds to the total number of nonzero elements in a vector. For example, the L0 norm of the vector [0,2] is 1 because there is only one nonzero element.

• L1 norm: Also known as Manhattan Distance, is the sum of absolute values of the components of the vector. For example, for some vector:

  $X = [x_1, x_2]$

  The L1 norm is calculated by:

• L2 norm: Also known as the Euclidean norm, it is the shortest distance to go from one point to another. Using the same example as before, the L2 norm for the vector defined above is:

  $\| x\|_2 = \sqrt{ x_1^2 + x_2^2 }$

• L-infinity norm: Gives the largest magnitude among each element of a vector. For example, having the vector X= [-6, 4, 2], the L-infinity norm is 6.

• A Positive definite matrix is a symmetric matrix M such that the number zᵗMz is positive for every nonzero column vector z.
• A Positive semi-definite matrix is a symmetric matrix M such that the number zᵗMz is positive or zero for every nonzero column vector z.
• Negative-definite and negative semi-definite matrices are defined analogously.

Given that we can associate each matrix with the quadratic equation zᵗMz, these kinds of matrix helps us to solve optimization problems. For example, a positive definite matrix M will imply a convex function, which guarantees the existence of the global minimum. This allows us to use the Hessian matrix to solve the optimization problem. Similar arguments also hold for negative definite matrices.

Broadcasting is a mechanism for relaxing elementwise operations according to the dimension requirements. We say that two matrices are compatible for broadcasting if the corresponding dimensions in each matrix (rows vs rows, columns vs columns) meet the following criteria:

• The dimensions are equal, or
• One dimension is of size 1.

The way that broadcasting works are by duplicating the smaller array so that it is the dimensionality and size as the larger array.

Initially, this method was developed for NumPy, but it has also been adopted more broadly in other numerical computational libraries, such as Theano, TensorFlow, and Octave.

An orthogonal matrix is a type of square matrix whose columns and rows are orthonormal unit vectors, e.g. perpendicular, and have a length or magnitude of 1. Formally, it's defined as follows:

Where Q is the orthogonal matrix, Qᵗ indicates the transpose of Q, and I is the identity matrix. From the above definition, we can see that

Therefore, the orthogonal matrix is preferred because they are computationally cheap and stable to calculate their inverse as simply their transpose.

The determinant is a scalar value that is a function of the entries of a square matrix. The determinant of a matrix A is denoted det(A), det A, or |A|. Geometrically, it can be viewed as the volume scaling factor of the linear transformation described by the matrix.

In the case of a 2 × 2 matrix the determinant is calculated following the next diagram:

That is, the determinant is equal to the product of the elements along the plus-labeled arrow minus the product of the elements along the minus-labeled arrow.

Similarly, for a 3 × 3 matrix A, its determinant is

Each determinant of a 2 × 2 matrix in the equation above is called a minor of the matrix A.

For an n × n matrix, the previous procedure is extended and provides a recursive definition for the determinant, known as a Laplace expansion.

• In simple terms, a matrix is a grid of n × m (say, 3 × 3) numbers surrounded by brackets. We can add and subtract matrices of the same size, multiply one matrix with another as long as the sizes are compatible, and multiply an entire matrix by a constant.

• A tensor is a generalization of matrices to N-dimensional space. That is, it could be a 1-D matrix (a vector), a 3-D matrix (something like a cube of numbers), or even a 0-D matrix (a single number), etc. The dimension of the tensor is called rank.

• A tensor can be also seen as a mathematical entity that lives in a structure and interacts with other mathematical entities. If one transforms the other entities in the structure in a regular way, then the tensor obeys a related transformation rule. This dynamical property of a tensor is also a key to distinguish it from a mere matrix. For example, any rank-2 tensor can be represented as a matrix, but not every matrix is really a rank-2 tensor. The difference depends on the transformation rules that have been applied to the entire system.