8.1. Introduction to Eigenvalues and Eigenvectors

Video Resource

In this lecture, Professor Gilbert Strang of MIT gives a good introduction to eigenvalues and eigenvectors.

Matrix multiplication involves a lot of stretching and rotation for a vector. When we multiply a matrix and a vector together (\(\mathbf{A}\,\bm{x}\)), the result is usually a stretching and a rotation of the original vector \(\bm{x}\). However, that is not always the case. There are special matrices that rotate a vector, but do not change its length. See Rotation of a Point for more on rotation matrices. There are also vectors corresponding to square matrices that are stretched by the multiplication, but not rotated. This latter case is what we consider here.

Consider the following matrix and three vectors.

\[\begin{split}\mathbf{A} = \begin{bmatrix} 6 & 1 \\ 8 & 4 \end{bmatrix}, \quad \bm{x}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad \bm{x}_2 = \begin{bmatrix} 1 \\ -4 \end{bmatrix}, \quad \bm{x}_3 = \begin{bmatrix} 1 \\ -1 \end{bmatrix}\end{split}\]

\[\begin{split}\mathbf{A}\,\bm{x}_1 = \begin{bmatrix} 8 \\ 16 \end{bmatrix} = 8\;\bm{x}_1, \quad \mathbf{A}\,\bm{x}_2 = \begin{bmatrix} 2 \\ -8 \end{bmatrix} = 2\,\bm{x}_2, \quad \mathbf{A}\,\bm{x}_3 = \begin{bmatrix} 5 \\ 4 \end{bmatrix}\end{split}\]

Fig. 8.1 shows the three vectors and the product of each vector with the \(\bf{A}\) matrix. The product \(\mathbf{A}\,\bm{x}_3\) shows the typical matrix-to-vector multiplication result. The multiplication both stretches and rotates the vector. But in the cases of \(\mathbf{A}\,\bm{x}_1\) and \(\mathbf{A}\,\bm{x}_2\), the vectors are stretched, but not rotated. This is because vectors \(\bm{x}_1\) and \(\bm{x}_2\) are eigenvectors of matrix \(\bf{A}\). The eigenvalues tell us the stretching (scaling) factor for each eigenvector.

Fig. 8.1 Eigenvectors are only scaled by matrix multiplication. Other vectors are both scaled and rotated by multiplication.

When a vector \(\bm{x}_i\) is one of the eigenvectors of a matrix \(\bf{A}\), the following special relationship holds.

\[\boxed{\mathbf{A}\,\bm{x}_i = \lambda_i\,\bm{x}_i}\]

The power of this relationship is that when, and only when, we multiply a matrix by one of its eigenvectors, then, in a simplified equation, we can replace the matrix with a scalar.

For the \(n{\times}n\) square matrix, there are \(n\) eigenvectors, \(\bm{x_i}\), and \(n\) corresponding scalar eigenvalues, \(\lambda_i\). Each coupled eigenvalue and eigenvector pair is called an eigenpair.

The best kept secret of engineering

Whether you know it or not, you use eigenvalues and eigenvectors daily. Furthermore, knowing about them can greatly expand your ability to solve interesting and challenging problems. Engineers encounter eigenvalues and eigenvectors when studying mechanics, vibrations, or when working with big data. Google’s Internet search engine uses eigenvectors to rank web pages, and Netflix uses eigenvectors to predict your preference for a movie you have not yet watched. The algorithm for finding the eigenvalues of a matrix is also used to factor polynomials.

We use eigenvalues and eigenvectors to simplify expressions involving a matrix. Of course, we do not say that a matrix is equivalent to a single scalar value, but when multiplying a matrix by one of its eigenvectors, it may be replaced by a scalar (its eigenvalue). What an incredibly fantastic simplification! Making this replacement can reduce the computational complexity of a problem.

What is in the name?

In German, the word eigen means own, as in “my own opinion” or “my own family”. It is also used for the English word typical, as in “that is typical of him”. Perhaps the German word eigenschaft, meaning feature, property, or characteristic, is clearer as to the intent. For a matrix, the eigenvectors and eigenvalues represent the vector directions and magnitudes distinctly embodied by the matrix.

The first known writing about eigenvectors and eigenvalues is by Leonhard Euler in the 18th century. Various mathematicians studied and gave them different names in the next two centuries. German mathematician David Hilbert (1862 - 1943) named them eigenvalues and eigenvectors in 1904.