Appearance
4.5 — Eigenvalues and Eigenvectors
Apply a transformation to the plane and almost every vector both turns and changes length. But for most transformations there are a few special directions where the vector does not turn at all — it only gets longer or shorter, staying exactly on the line it started on.
Those directions are the eigenvectors, and the amount of stretching along each one is its eigenvalue. The German prefix eigen means "own" or "characteristic", so these are the transformation's own directions. Translations into English were attempted — "proper vector", "characteristic vector" — and none stuck, so we use the half-German word.
The whole definition is one short equation:
A\mathbf{v} = \lambda\mathbf{v}
A matrix times this particular vector gives back the same vector, merely scaled. The matrix, which does something complicated in general, acts on this direction like an ordinary number.
1. Finding them
Rearrange A\mathbf{v} = \lambda\mathbf{v} by moving everything to one side. Write \lambda\mathbf{v} as \lambda I \mathbf{v} so that both terms are a matrix times \mathbf{v}:
(A - \lambda I)\mathbf{v} = \mathbf{0}
We want a nonzero \mathbf{v} satisfying this. The zero vector always works and tells us nothing.
So we need a matrix (A - \lambda I) that sends some nonzero vector to the origin — which by Chapter 4.2 means it has a nonempty null space, which by Chapter 4.4 means it squashes space, which means:
\det(A - \lambda I) = 0
This is the characteristic equation. For an n \times n matrix it is a polynomial of degree n in \lambda, and its roots are the eigenvalues.
Worked example.
A = \begin{bmatrix} 4 & 1 \\ 2 & 3\end{bmatrix}
\det\begin{bmatrix} 4-\lambda & 1 \\ 2 & 3-\lambda\end{bmatrix} = (4-\lambda)(3-\lambda) - 2 = \lambda^2 - 7\lambda + 10 = 0
Factor: (\lambda - 5)(\lambda - 2) = 0, so \lambda_1 = 5 and \lambda_2 = 2.
Now the eigenvectors. For \lambda = 5, solve (A - 5I)\mathbf{v} = 0:
\begin{bmatrix} -1 & 1 \\ 2 & -2\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix} = \begin{bmatrix}0\\0\end{bmatrix}
Both rows say the same thing: -x + y = 0, so y = x. Any multiple of \begin{bmatrix}1\\1\end{bmatrix} works. Eigenvectors are directions, not specific vectors — the whole line qualifies, and by convention we pick a convenient or unit-length representative.
For \lambda = 2: \begin{bmatrix} 2&1\\2&1\end{bmatrix} gives 2x + y = 0, so y = -2x, and the eigenvector is \begin{bmatrix}1\\-2\end{bmatrix}.
Check: A\begin{bmatrix}1\\1\end{bmatrix} = \begin{bmatrix}5\\5\end{bmatrix} = 5\begin{bmatrix}1\\1\end{bmatrix} ✓
Two useful shortcuts, which also make an excellent arithmetic check:
\text{sum of eigenvalues} = \operatorname{tr}(A), \qquad \text{product of eigenvalues} = \det(A)
Here 5 + 2 = 7 = 4+3 ✓ and 5 \times 2 = 10 = 12 - 2 ✓. These are Vieta's relations from Chapter 2.3 applied to the characteristic polynomial.
2. What can go wrong, and what it means
Complex eigenvalues. A pure rotation has no real eigenvector — nothing keeps its direction when everything turns. The characteristic equation then has a negative discriminant and gives complex roots, exactly as Chapter 2.5 predicted. Those complex eigenvalues encode the rotation: their argument is the angle turned and their modulus is the scaling. This is not a failure; it is the algebra reporting a rotation.
Repeated eigenvalues. Sometimes a repeated root does not come with enough independent eigenvectors. The shear \begin{bmatrix}1&1\\0&1\end{bmatrix} has \lambda = 1 twice, and only one eigenvector direction — the horizontal one in the Mona Lisa picture. Such a matrix is called defective and cannot be diagonalised in the sense of Section 3.
Symmetric matrices are the well-behaved case. If A = A^\mathsf{T} then all eigenvalues are real, and eigenvectors for different eigenvalues are automatically perpendicular to each other. This is the spectral theorem, and it is why symmetric matrices are so pleasant. Covariance matrices in statistics, stiffness matrices in engineering, adjacency matrices of undirected graphs and the Hessian in optimisation are all symmetric, so the good case is also the common case.
3. Diagonalisation: the transformation in its own language
If an n\times n matrix has n independent eigenvectors, put them side by side as the columns of a matrix P, and put the eigenvalues down the diagonal of a matrix D. Then:
A = PDP^{-1}
Read the right-hand side from the right, remembering Chapter 4.2's rule that the rightmost acts first:
- P^{-1} rewrites the vector in eigenvector coordinates — describing it in terms of the transformation's own directions.
- D scales each of those coordinates by its eigenvalue. Just multiplication by numbers.
- P translates back to ordinary coordinates.
A complicated transformation is a simple scaling, seen through the wrong coordinate system. Change to the right coordinates and the complexity was never in the transformation — it was in the description.
This is one of the most reusable ideas in mathematics, and it is not confined to matrices. Chapter 9.3 does exactly the same thing with the Fourier transform: change to the frequency basis, do something trivial, change back. Logarithms in Chapter 1.4 do it too: change to log coordinates, addition replaces multiplication, change back.
Why anyone bothers: powers become easy
A^k = (PDP^{-1})(PDP^{-1})\cdots(PDP^{-1}) = PD^kP^{-1}
All the interior P^{-1}P pairs cancel to the identity. And D^k is trivial — just raise each diagonal entry to the $k$th power.
So computing A^{100}, which would be 99 matrix multiplications, becomes raising a few numbers to the hundredth power.
Worked example — Fibonacci in closed form. The Fibonacci recurrence F_{n+1} = F_n + F_{n-1} can be written as
\begin{bmatrix} F_{n+1}\\F_n\end{bmatrix} = \begin{bmatrix}1&1\\1&0\end{bmatrix}\begin{bmatrix}F_n\\F_{n-1}\end{bmatrix}
So F_n comes from raising that matrix to a power. Its characteristic equation is \lambda^2 - \lambda - 1 = 0, whose roots are
\lambda = \frac{1\pm\sqrt5}{2}
The larger is the golden ratio \varphi \approx 1.618 from Chapter 1.7. Diagonalising gives Binet's formula:
F_n = \frac{\varphi^n - (1-\varphi)^n}{\sqrt5}
Since |1-\varphi| \approx 0.618 is less than one, its powers shrink to nothing, so for any decent n the Fibonacci number is just \frac{\varphi^n}{\sqrt5} rounded. That is why consecutive Fibonacci numbers approach the golden ratio — the larger eigenvalue eventually dominates, which is the general principle of the next section.
4. The dominant eigenvalue governs the long run
Apply a matrix to a vector over and over. What happens?
Write the starting vector as a combination of eigenvectors. Each application multiplies each component by its own eigenvalue. After k steps the component along \mathbf{v}_i has been multiplied by \lambda_i^k.
The largest eigenvalue wins, because exponential growth with the bigger base overtakes everything (Chapter 1.4). Eventually the vector points almost exactly along the dominant eigenvector, whatever it started as.
This one fact explains an enormous amount.
Population models. A Leslie matrix tracks how many individuals are in each age group and moves them forward one year. The dominant eigenvalue is the long-run growth rate: above 1 the population grows, below 1 it dies out, and the dominant eigenvector is the stable age distribution that any starting population settles into.
Markov chains. A matrix of transition probabilities has a dominant eigenvalue of exactly 1, and its eigenvector is the steady state — the long-run fraction of time spent in each state, independent of where you started. This is why a shuffled deck forgets its initial order, and why a random walk on a network settles into a predictable distribution.
PageRank. Google's original algorithm is precisely this. Build a matrix of "if you are on page i, the chance you click through to page j", find its dominant eigenvector, and that vector's entries are the page importances. Larry Page and Sergey Brin's 1998 contribution was noticing that the importance of a web page is an eigenvector problem. Volume I, 4.7 covers the graph side.
Stability of anything. A physical system near equilibrium is described by a matrix, and the sign of the eigenvalues' real parts decides everything: all negative means disturbances die away and the system is stable; any positive means disturbances grow and it is not. This is how a control engineer decides whether an aircraft, a chemical reactor or a feedback loop will hold together, and Chapter 6.3 and Volume III, Part 6 develop it.
5. Computing them in practice
Do not solve the characteristic polynomial. For matrices of any size it is numerically catastrophic — the roots of a polynomial are extremely sensitive to its coefficients, so tiny rounding errors produce completely wrong eigenvalues. It also cannot work in principle beyond degree four, since Chapter 2.3 noted that no formula in radicals exists for degree five and above.
What is actually used:
Power iteration, when only the dominant eigenvalue is needed. Start with any vector, multiply by A repeatedly, normalise each time. By Section 4 it converges to the dominant eigenvector. This is how PageRank was computed over the entire web — a few dozen multiplications by an enormous sparse matrix.
The QR algorithm, for the full set. It repeatedly factors the matrix into an orthogonal and a triangular part and multiplies them back in the other order; the result converges to triangular form with the eigenvalues on the diagonal. Discovered around 1960, and named one of the top ten algorithms of the twentieth century. Every eig function in every numerical library is a refined version of it.
Every formula above, built from scratch
None of the results in this chapter are worth memorising, because each one can be rebuilt in under a minute from something simpler. What follows is that rebuilding, one result at a time, so the formula and the reason for it sit on the same page as the explanation that needed them.
Eigenvalues and eigenvectors
A\mathbf{v} = \lambda\mathbf{v}, \qquad \mathbf{v} \ne \mathbf{0}
Read it aloud. "A times v equals lambda times v" — the matrix does nothing to this vector except stretch it. Its direction survives the transformation.
The characteristic equation.
\det(A - \lambda I) = 0
Where it comes from. Rearrange the definition:
A\mathbf{v} - \lambda\mathbf{v} = \mathbf{0} \quad \Rightarrow \quad (A-\lambda I)\mathbf{v} = \mathbf{0}
The I has to be there because you cannot subtract a number from a matrix; \lambda I is the matrix version of the number \lambda.
Now, this says the matrix (A-\lambda I) sends the non-zero vector \mathbf{v} to zero. A matrix that squashes a non-zero vector to nothing must be crushing space, and §3 established that crushing space means a zero determinant. So the values of \lambda that permit an eigenvector are exactly the ones making \det(A-\lambda I) = 0.
For a 2\times2 matrix, that determinant works out to
\lambda^2 - (\text{trace})\lambda + \det = 0, \qquad \text{trace} = a+d
so
\lambda = \frac{\text{tr} \pm \sqrt{\text{tr}^2 - 4\det}}{2}
Two facts worth carrying, true for any size:
\lambda_1 + \lambda_2 + \cdots + \lambda_n = \operatorname{tr}(A), \qquad \lambda_1\lambda_2\cdots\lambda_n = \det(A)
These are Vieta's formulas from 2.3 — quadratics applied to the characteristic polynomial, and they are the fastest check on any eigenvalue calculation.
Finding the eigenvector. Once you have \lambda, solve (A-\lambda I)\mathbf{v} = \mathbf{0}. That system is guaranteed to have infinitely many solutions — every multiple of the eigenvector is also an eigenvector, since scaling both sides of A\mathbf{v}=\lambda\mathbf{v} changes nothing.
Diagonalisation.
A = PDP^{-1}
where D holds the eigenvalues on its diagonal and P holds the matching eigenvectors as its columns. Read right to left: P^{-1} rewrites a vector in terms of the eigenvector directions, D stretches each of those directions by its own factor, and P translates back. The messy matrix has been split into "change your point of view, stretch, change back".
What it is for.
A^k = PD^kP^{-1}
Raising a diagonal matrix to a power is just raising each diagonal entry to that power. So a hundredth power of a matrix costs one diagonalisation and a hundred scalar powers, rather than ninety-nine matrix multiplications. This is how a Markov chain's long-run behaviour, a Fibonacci closed form, and a population model's fate are all computed.
Symmetric matrices are the good case. If A = A^T, then all eigenvalues are real, and eigenvectors for different eigenvalues are perpendicular. The change of basis is then a rotation, so A = QDQ^T with Q^{-1} = Q^T. Every covariance matrix is symmetric, which is why principal component analysis always works.
6. Where this shows up in your life
Every search result you have ever seen. PageRank, or its descendants.
Every recommendation and every dimensionality reduction. Chapter 4.6's PCA is eigenvectors of the covariance matrix.
Every bridge, building and aircraft. Structures are analysed for their natural frequencies, which are the eigenvalues of the stiffness and mass matrices. Drive a structure at one of those frequencies and it resonates. The Tacoma Narrows Bridge collapse of 1940 and the wobble of London's Millennium Bridge in 2000 were both this, and Chapter 6.2 covers the mechanism.
Every quantum mechanical measurement. Observable quantities are represented by matrices, and the possible measured values are literally their eigenvalues. The discreteness of energy levels in an atom — the reason chemistry works — is a statement about a matrix's eigenvalues being separated rather than continuous. Volume IV, Chapter 7.
Every image compression and noise reduction. Keep the large eigenvalues, discard the small ones, and you have thrown away the least important part of the data. That is the next chapter.
Every stability question in economics, ecology and epidemiology. The reproduction number R_0 that dominated public conversation in 2020 is a dominant eigenvalue.
Eigenvalues need a square matrix and do not always exist in a usable form. There is a decomposition that works for every matrix, of any shape, and it is arguably the single most useful result in applied mathematics.