Section 4.3: Real Networks, MLPs and Classification

One neuron fits a straight line, which Module 3 fitted in one solve. A network earns its cost when neurons are stacked into layers with a nonlinearity (a function such as ReLU, which keeps a positive number and turns a negative one into zero) between them, because two linear layers with nothing between them collapse into one linear layer, and the fit proves it: 641 parameters doing the work of 19. This section builds that network on the listings, scales the target as well as the inputs, and watches it beat Module 3's linear model on the price for the first time, 45,585 against 52,725 on the validation rows. Then the output becomes a class, a logit (the raw output of the last layer) that a sigmoid turns into a probability and a loss reads directly, the full batch becomes mini-batches handed out by a DataLoader, and the one step becomes a choice of optimizer and learning rate, swept on a grid and chosen on the validation rows.

Section 4.2 gave you a correct loop on a problem small enough for one step per pass. Real data gives you more rows than a step should see at once, a target whose scale sends a network's loss to inf in three epochs, a label whose probabilities after 300 epochs all sit under the 0.5 threshold, so the accuracy equals the majority baseline while the model has learned something, a loss that takes probabilities without complaint and returns the wrong number when it wanted logits, a batch's loss that jumps around the epoch's, and a learning rate that is right for one optimizer and useless for the next. None of these stops the program. On the sale label the network passes logistic regression by about a hundredth of AUC at its best, and the section says so.

Lessons

From Neuron to Network - the MLP - the table a network reads, why the target is scaled too, stacking layers and why the nonlinearity is not optional, the MLP fitted and compared with the linear model, a network's parameters read, and width previewed.
Classification - Logits and Cross-Entropy - a label in place of a price, the logit and the sigmoid, binary cross-entropy on logits, training the classifier and moving the threshold, and many classes with softmax and CrossEntropyLoss.
Dataset and DataLoader - Mini-Batches - epochs against updates, TensorDataset and DataLoader, the loop restructured, the noise in a batch's loss, and what mini-batches buy, measured.
SGD, Adam and the Learning-Rate Sweep - what an optimizer decides, momentum and Adam, the sweep on a log grid, reading its curves, choosing on validation and scoring the test once, and the cost of a fit in seconds and parameters.

By the end you will be able to:

Build the table a network reads from numeric and text columns with statistics and an encoder fitted on the fit rows, scale the target, and predict the shape of every tensor through a network's layers.
Say why a nonlinearity between layers is not optional, and prove it with the collapse of two linear layers into one.
Train a classifier on logits with binary cross-entropy, move its threshold and read the confusion matrix, and write the ten-class form with softmax and CrossEntropyLoss without applying either activation twice.
Restructure a full-batch loop into mini-batches with a seeded DataLoader, count its updates, and read the per-epoch curve rather than a batch's loss.
Sweep the learning rate for SGD and Adam on a log grid, name divergence, a crawl and a plateau from the curves, choose on the validation rows, score the test rows once, and report a fit's cost in seconds and parameters.