Symmetric eigenproblems and computing the SVD

Reading: Golub & Van Loan §8.1–8.3, 8.6, pp. 440–467, 486–497.

By the end of this week you should be able to

• State the spectral theorem and the min-max characterisation of eigenvalues.
• Apply the symmetric QR algorithm to a tridiagonal matrix.
• Describe Golub-Kahan bidiagonalization and how the SVD is actually computed.

Algorithms introduced

• Tridiagonalization and the symmetric QR algorithm
• Rayleigh quotient iteration
• Golub-Kahan bidiagonalization for the SVD

Where this shows up in AI

Spectral clustering and Laplacian eigenmaps are symmetric eigenproblems. The Hessian spectrum is how the loss landscape of a network is studied.

Materials