Optimization Introduction
Loss functions
Turn prediction errors into a quantity to minimize.
Before this
The intuition
A loss function measures how undesirable a prediction is. Training adjusts model parameters to reduce an aggregate of these losses.
A concrete example
If a prediction is 5 and the target is 3, squared error is (5 − 3)² = 4. Errors of 2 and −2 receive the same squared loss.
What to remember
The choice of loss defines what training rewards. Low training loss alone does not establish useful predictions on new data.
Check your understanding
What is squared error for a prediction of 1 and target of 4?
Reveal explanation
9, because (1 − 4)² = 9.