Chapter 6 · Measuring Models and Next Steps
Summary, Formula Sheet and What's Next
- Page 17 of 17
- 3 min read
What you can do now
- Represent data as vectors and matrices, measure distance and similarity, and read the shapes in numeric code.
- Explain how a model learns: derivatives, gradients, gradient descent, the learning rate and loss functions.
- Describe a dataset honestly with medians, spreads and percentiles, scale features, and tell correlation from cause.
- Judge whether a difference between two results is real, with standard errors and bootstrap intervals.
- Use probability rules and Bayes' theorem, and explain how temperature and sampling shape an LLM's output.
- Evaluate a classifier with a confusion matrix, precision, recall and F1, and choose a threshold.
Formula sheet
| Idea | Formula | NumPy |
|---|---|---|
| Vector length | ‖a‖ = √(Σ aᵢ²) | np.linalg.norm(a) |
| Dot product | a · b = Σ aᵢbᵢ | a @ b |
| Cosine similarity | a · b / (‖a‖ ‖b‖) | a @ b / (norm(a) * norm(b)) |
| A layer | z = W x + b | W @ x + b |
| Matrix shapes | (m × n)(n × p) → (m × p) | (A @ B).shape |
| Gradient descent | w ← w − η ∂L/∂w | w -= lr * grad |
| Sigmoid | 1 / (1 + e⁻ᶻ) | 1 / (1 + np.exp(-z)) |
| Softmax | eᶻⁱ / Σ eᶻʲ | np.exp(z) / np.exp(z).sum() |
| MSE | mean((ŷ − y)²) | np.mean((pred - y) ** 2) |
| Cross-entropy | −log p(correct) | -np.log(p) |
| Standard deviation | √ mean((x − x̄)²) | x.std() |
| Z-score | (x − x̄) / σ | (x - x.mean()) / x.std() |
| Standard error (accuracy) | √(p(1 − p) / n) | np.sqrt(p * (1 - p) / n) |
| Bayes | P(A|B) = P(B|A) P(A) / P(B) | page 14 |
| Precision / recall | TP/(TP+FP) / TP/(TP+FN) | sklearn.metrics |
Glossary
| Term | Meaning |
|---|---|
| Vector / matrix | A list of numbers / a table of numbers |
| Embedding | A vector that places similar meanings close together |
| Gradient | The list of slopes for every weight; it points uphill |
| Learning rate | How big each step of gradient descent is |
| Loss | One number saying how wrong a model is; training makes it smaller |
| Logits | A model's raw scores, before softmax |
| Standardisation | Rescaling a feature to mean 0 and standard deviation 1 |
| Confounder | A hidden variable that drives two others, creating a misleading correlation |
| Confidence interval | A range that probably contains the true value |
| Prior / posterior | A belief before / after seeing the evidence |
| Temperature | A setting that sharpens (low) or flattens (high) an LLM's next-token probabilities |
| Precision / recall | How often a "yes" is right / how many real "yes" cases are found |
Check yourself
Take the three quizzes in order: Easy (after chapter 2, vectors and matrices), Medium (after chapter 4, calculus and statistics) and the Hard final exam (after chapter 6, everything including probability and metrics). Read the explanation for every answer you miss, then revisit that page.
What's next
The next topic in the Level 0 path is SQL and data handling: getting data out of databases and into the shape your models need. After that come Git, APIs and basic back-end, Prompt Engineering and AI Ethics.