Neural Network Learning: The Engineering Problem

Learning shows up as a simple before and after. The same input goes into the network, but the output changes after training, and the change is measurable in an error score.

This lesson sets a concrete definition of learning and the training loop you will reason about in later lessons.

Same input, different output after training

At the start, a neural network usually produces unstable predictions because its parameters start as small random numbers. If you feed the same input $x$ into the same code twice without changing anything, the output stays the same, but it can be wrong in a consistent way.

After training runs for a while, the input $x$ still stays the same, but the prediction $\hat{y}$ often changes toward the correct label $y$ because the weights were updated in between. That is the core observable behavior you can check before worrying about any theory.

Use the interactive classification example to compare initial random guesses with accuracy after several update steps.

Learning means reducing measured error by updates

The change comes from an explicit measurement. The network outputs a prediction $\hat{y}$, you compare it to a target label $y$, and you compute an error metric that assigns a larger number to worse predictions and a smaller number to better ones.

Training performs an update step that nudges the parameters so the next prediction tends to produce a smaller error on similar examples. Learning is not a mood or a feature. It is a repeated process where parameter values change because the measured error pushed them to.

Explore how the terms relate in the mapping between parameters, predictions, labels, the error metric, and the update step.