Eigenvalues and Eigenvectors: The Directions That Do Not Rotate

Most vectors get scrambled by matrix transformations. They rotate, stretch, and shear into something unrecognizable.

But some special vectors only stretch or shrink. They keep pointing in the same direction. The transformation scales them, nothing more.

These are eigenvectors. Their scaling factors are eigenvalues.

And they're the key to understanding what a matrix really does.

The Definition

An eigenvector of matrix A is a nonzero vector v such that:

Av = λv

Apply the transformation A to v, and you get a scalar multiple of v back. Same direction, different magnitude.

The scalar λ is the eigenvalue corresponding to that eigenvector.

Here's what this means geometrically: when you transform space by A, the eigenvector v doesn't rotate. It just stretches (if |λ| > 1), shrinks (if |λ| < 1), or flips (if λ < 0).

Why They Matter

Eigenvectors tell you the transformation's natural axes.

Most transformations are complicated—they rotate and stretch in tangled ways. But along eigenvector directions, the transformation is simple: just scaling.

If you can find enough eigenvectors, you can use them as a coordinate system. In those coordinates, the transformation becomes a simple diagonal matrix—pure scaling along each axis.

This is diagonalization. It turns complex transformations into simple ones.

Finding Eigenvalues and Eigenvectors

Start with Av = λv.

Rewrite as: Av - λv = 0 Factor: (A - λI)v = 0

This is a homogeneous system. It has a nonzero solution v only if the matrix (A - λI) is not invertible.

And a matrix is not invertible exactly when its determinant is zero:

det(A - λI) = 0

This is the characteristic equation. It's a polynomial in λ. Solve it to find eigenvalues.

For a 2×2 matrix:

| a-λ   b   |
|  c   d-λ  |

det = (a-λ)(d-λ) - bc = λ² - (a+d)λ + (ad-bc)

Set equal to zero and solve the quadratic.

Once you have eigenvalues, plug each λ back into (A - λI)v = 0 and solve for v. That gives you eigenvectors.

A Worked Example

Take the matrix:

| 2  1 |
| 1  2 |

Characteristic equation: det(A - λI) = (2-λ)(2-λ) - 1 = λ² - 4λ + 3 = (λ-1)(λ-3)

Eigenvalues: λ₁ = 1 and λ₂ = 3.

For λ = 1:

| 1  1 | | x |   | 0 |
| 1  1 | | y | = | 0 |

This gives x + y = 0, so y = -x. Eigenvector: (1, -1) (or any scalar multiple).

For λ = 3:

| -1  1 | | x |   | 0 |
|  1 -1 | | y | = | 0 |

This gives -x + y = 0, so y = x. Eigenvector: (1, 1) (or any scalar multiple).

Geometric interpretation: This matrix stretches along the diagonal (1,1) by factor 3, and leaves the anti-diagonal (1,-1) unchanged (factor 1). It's a stretch along one axis.

Diagonalization

If A has n linearly independent eigenvectors (for an n×n matrix), you can diagonalize it.

Form matrix P whose columns are the eigenvectors.

Then: P⁻¹AP = D, where D is diagonal with eigenvalues on the diagonal.

Equivalently: A = PDP⁻¹.

This is powerful because:

• Powers of A: Aⁿ = PDⁿP⁻¹, and powers of diagonal matrices are trivial (just power each diagonal entry)
• Understanding A: the action of A is "change to eigenbasis, scale, change back"

Real World Applications

Google PageRank: The importance of a webpage is the steady-state distribution of a random walk on the web graph. That steady state is an eigenvector of the transition matrix.

Principal Component Analysis (PCA): Find the eigenvectors of a covariance matrix. They point along the directions of maximum variance in the data. Used for dimensionality reduction in machine learning.

Quantum Mechanics: Observables are represented by matrices. Eigenvalues are possible measurement outcomes. Eigenvectors are states that give definite measurements.

Vibration Analysis: The natural frequencies of a vibrating system are eigenvalues of a matrix describing the system. Eigenvectors describe the mode shapes.

Population Dynamics: For Leslie matrices modeling age-structured populations, the dominant eigenvalue determines long-term growth rate. The corresponding eigenvector gives stable age distribution.

Facial Recognition: Eigenfaces are eigenvectors of the covariance matrix of face images. Any face can be approximated as a combination of eigenfaces.

Special Properties

Trace and Determinant

For an n×n matrix with eigenvalues λ₁, λ₂, ..., λₙ:

• Sum of eigenvalues = trace(A) = sum of diagonal entries
• Product of eigenvalues = det(A)

These are elegant relationships. The trace tells you the sum of scalings; the determinant tells you the total volume scaling.

Symmetric Matrices Are Nice

If A is symmetric (A = Aᵀ):

• All eigenvalues are real (not complex)
• Eigenvectors for different eigenvalues are orthogonal
• A can always be diagonalized

Symmetric matrices arise naturally in physics and statistics (covariance matrices, second derivatives). Their eigenstructure is particularly clean.

Complex Eigenvalues

For non-symmetric real matrices, eigenvalues can be complex (a + bi).

Complex eigenvalues come in conjugate pairs. Geometrically, they represent rotation combined with scaling.

A matrix like:

|  0  -1 |
|  1   0 |

has eigenvalues i and -i. It rotates by 90°—no real eigenvector exists because no direction survives rotation unchanged.

The Characteristic Polynomial

The equation det(A - λI) = 0 is a polynomial in λ of degree n.

By the Fundamental Theorem of Algebra, it has exactly n roots (counting multiplicity, including complex roots).

So every n×n matrix has exactly n eigenvalues (with possible repeats).

Some eigenvalues might be complex. Some might be repeated. But there are always exactly n of them.

When Diagonalization Fails

Not every matrix is diagonalizable.

If an eigenvalue is repeated but doesn't have enough independent eigenvectors, you can't form a full basis of eigenvectors.

Example:

| 1  1 |
| 0  1 |

This has eigenvalue λ = 1 (repeated). But the only eigenvector direction is (1, 0). You can't diagonalize.

Such matrices can be put in "Jordan normal form"—almost diagonal, with some 1s above the diagonal. But that's more advanced.

The Power of Eigenvectors

Here's why eigenvectors are so central: they simplify everything.

Matrix powers? Use Aⁿ = PDⁿP⁻¹.

Differential equations? Solutions are exponentials of eigenvalues.

Stability analysis? Eigenvalues determine if a system grows, decays, or oscillates.

Data analysis? Eigenvectors point toward the most important patterns.

Eigenvectors are the coordinate system in which the transformation becomes simple. Find them, and you understand the transformation.

The Insight

Every matrix transformation, no matter how complicated, has hidden simplicity.

The eigenvectors are the directions where that simplicity reveals itself. Along these directions, the transformation does nothing but scale.

Finding eigenvectors is finding the natural axes of the transformation. The axes that match its intrinsic behavior, not the arbitrary coordinates you started with.

This is why eigenvalues and eigenvectors appear in every application of linear algebra. They're how you decode what a matrix actually does.

This is Part 6 of the Linear Algebra series. Next: "Systems of Linear Equations: Matrices as Equation Solvers."

Part 6 of the Linear Algebra series.

Previous: Determinants: The Volume-Scaling Factor of Matrices Next: Systems of Linear Equations: Matrices as Equation Solvers