The singular value decomposition

Reading: Golub & Van Loan §2.4–2.5, pp. 76–87.

By the end of this week you should be able to

• State the SVD theorem and read rank, range, and null space off the factors.
• Compute the distance from a matrix to the nearest rank-deficient matrix.
• Use principal angles to measure the distance between two subspaces.

Algorithms introduced

• The SVD as an analytical tool (computation comes in Week 14)
• Numerical rank determination

Where this shows up in AI

PCA is an SVD of centred data. Numerical rank tells you how much of a weight matrix is actually being used, which is the premise behind low-rank adaptation (LoRA) and model compression.

Materials