Overfitting vs Underfitting, Explained Simply
What overfitting and underfitting mean, how to spot each from training and test error, and how to fix them, with a small runnable example.
Short answer: Overfitting means a model has memorized its training data, noise included, so it does well on data it has seen and badly on new data. Underfitting is the opposite: the model is too simple to learn the real pattern, so it does badly on both. A good model sits between them and performs well on data it has never seen.
What is overfitting in simple words?
Imagine a student who memorizes last year's exam answers word for word. They score full marks on that exam, but fail when the questions change. That is overfitting: learning the examples instead of the idea behind them.
What is underfitting?
Now imagine a student who only learned "the answer is usually B". They never understood the subject, so they do badly on old and new exams alike. A model that is too simple, trained too little, or given unhelpful inputs behaves the same way.
How do you tell them apart?
Compare the error on the training data with the error on a separate test set the model never trained on.
Underfitting | Good fit | Overfitting | |
|---|---|---|---|
Training error | High | Low | Very low |
Test error | High | Low, close to training | Much higher than training |
Model | Too simple | About right | Too complex or trained too long |
Can you see it in code?
This small script needs no libraries. The true pattern is a straight line with some noise. We compare a model that ignores the input, one that memorizes the training points, and a fitted line.
import random
random.seed(1)
data = [(x, 2 * x + random.gauss(0, 3)) for x in range(20)]
random.shuffle(data)
train, test = data[:10], data[10:]
def mse(model, rows):
return sum((model(x) - y) ** 2 for x, y in rows) / len(rows)
# Underfit: ignores x and always predicts the training average
avg = sum(y for _, y in train) / len(train)
too_simple = lambda x: avg
# Overfit: memorises the training set and copies the closest training point
memoriser = lambda x: min(train, key=lambda r: abs(r[0] - x))[1]
# Good fit: a straight line found by least squares
mx = sum(x for x, _ in train) / len(train)
slope = sum((x - mx) * (y - avg) for x, y in train) / sum((x - mx) ** 2 for x, _ in train)
line = lambda x: avg + slope * (x - mx)
for name, m in [("too simple", too_simple), ("memoriser", memoriser), ("line", line)]:
print(f"{name:10} train {mse(m, train):7.1f} test {mse(m, test):7.1f}")A typical run prints something like this (exact values come from the random seed):
too simple train 91.1 test 222.5
memoriser train 0.0 test 18.9
line train 6.6 test 12.4The "too simple" model is bad everywhere. The memoriser has zero training error, but its test error is higher than the line's. The line has a little training error and the best test error, which is what a good fit looks like.
How do you reduce overfitting and underfitting?
Overfitting: get more training data, simplify the model, stop training earlier, use regularization, or use dropout in neural networks.
Underfitting: use a more capable model, add useful features, or train for longer.
Always: keep a test set, and use cross-validation when data is small, so you measure performance honestly.
For more on measuring a model fairly, see precision, recall, F1 and the confusion matrix, and the types of machine learning.
Common mistakes
Judging a model only by its training score.
Tuning the model again and again against the test set, which leaks the test data into your decisions. Keep a final set untouched.
Assuming a bigger model is always better.
Key Takeaway
Overfitting memorizes, underfitting never learns. Compare training and test error, and aim for a model that does well on new data.
Ready to start? Build the maths and statistics behind model evaluation, from averages and error to probability. Begin with the free course Maths and Statistics for AI: The Essentials.
Related articles
More in Machine Learning
Precision, Recall and F1 Score, Explained Simply
What precision, recall and F1 score mean, how to calculate them from a confusion matrix, and how to choose which one matters for your problem.
4 min readNov 16, 2026Training vs Validation vs Test Data
Why machine learning data is split into training, validation and test sets, what each is for, how to split it and how to avoid data leakage.
3 min readNov 13, 2026Classification vs Regression in Machine Learning
Classification predicts a category and regression predicts a number. See the differences, examples, metrics and a tiny Python demo of both.
4 min readNov 10, 202610 Real-World Uses of Deep Learning
Deep learning powers speech recognition, translation, recommendations, chatbots and more. See 10 real uses and the kind of model behind each.
4 min readNov 5, 2026How Much Maths Do You Need for Machine Learning?
An honest answer by goal: how much linear algebra, probability, statistics and calculus you need for machine learning, what to skip at first, and how to learn it alongside coding.
4 min readNov 3, 2026How Does a Neural Network Work? A Simple Explanation
Neurons, weights, bias, layers and how a network learns from its mistakes, explained in plain words with a handwritten-digit example and no heavy maths.
4 min readOct 30, 2026
Comments
Sign in to join the discussion.

