8.1. Introduction to Eigenvalues and Eigenvectors

Matrix multiplication is like yoga to a vector — mostly stretching and rotation. That is, when we multiply a matrix and a vector together (\mathbf{A}\,\bm{x}) the result is usually a stretching and a rotation of the original vector \bm{x}. However, that is not always the case. There are special matrices that rotate a vector, but do not change its length. (See Rotation of a Point for more on rotation matrices.) There are also vectors corresponding to each square matrix that will be stretched by the multiplication, but not rotated. This later case is what we consider here.

Consider the following matrix and three vectors.

\mathbf{A} = \mat{6 1; 8 4}, \; \bm{x}_1 = \vector{1; 2}, \; \bm{x}_2 = \vector{1; -4}, \; \bm{x}_3 = \vector{1; -1}

\mathbf{A}\,\bm{x}_1 = \vector{8; 16} = 8\,\bm{x}_1, \; \mathbf{A}\,\bm{x}_2 = \vector{2; -8} = 2\,\bm{x}_2, \, \mathbf{A}\,\bm{x}_3 = \vector{5; 4}

Fig. 8.1 shows the three vectors and the product of each vector with matrix \bf{A}. The product \mathbf{A}\,\bm{x}_3 shows the typical result of matrix to vector multiplication. The multiplication both stretches and rotates the vector. But in the cases of \mathbf{A}\,\bm{x}_1 and \mathbf{A}\,\bm{x}_2, the vectors are stretched, but not rotated. This is because vectors \bm{x}_1 and \bm{x}_2 are eigenvectors to matrix \bf{A}. The eigenvalues tell us the stretching (scaling) factor for each eigenvector.

../_images/eigVmultiply.png

Fig. 8.1 Eigenvectors are only scaled by matrix multiplication. Other vectors are both scaled and rotated by multiplication.

When a vector \bm{x}_i is one of the eigenvectors of a matrix \bf{A}, the following special relationship holds.

\boxed{\mathbf{A}\,\bm{x}_i = \lambda_i\,\bm{x}_i}

For the n{\times}n square matrix, there are n eigenvectors, \bm{x_i}, and n corresponding scalar eigenvalues, \lambda_i.

The power of this relationship is that when, and only when, we multiply a matrix by one of its eigenvectors, then in a simplified equation we can replace a matrix with just a simple scalar value. This simplification provides an elegant solution to several otherwise complex problems.