Sensitivity and finite precision

Reading: Golub & Van Loan §2.6–2.7, pp. 87–105.

By the end of this week you should be able to

• Define the condition number and use it to bound the error in a computed solution.
• Distinguish forward error, backward error, and residual, and explain why a small residual is not enough.
• Apply the floating-point model to a simple computation.

Algorithms introduced

• Condition number estimation
• Backward error analysis of a basic kernel

Where this shows up in AI

An ill-conditioned Hessian is exactly why gradient descent crawls in some directions. Conditioning also explains when float32 training is safe and when it silently is not.

Materials