The full-rank least squares problem

Reading: Golub & Van Loan §5.3–5.4, pp. 260–288.

By the end of this week you should be able to

• Derive the normal equations and state their conditioning drawback.
• Solve a least squares problem by QR and by SVD, and choose between them.
• Use a complete orthogonal decomposition.

Algorithms introduced

• Least squares via normal equations, QR, and SVD
• Complete orthogonal decomposition

Where this shows up in AI

Linear regression, done correctly. Every fitting procedure in machine learning descends from this problem.

Materials