Training vs Validation vs Test Data
Why machine learning data is split into training, validation and test sets, what each is for, how to split it and how to avoid data leakage.
Short answer: the training set is the data a model learns from, the validation set is used to tune and compare models while you build, and the test set is kept untouched until the end to measure how well the final model works on new data. A train test split into three parts keeps your evaluation honest, because a model always looks good on data it has already seen.
Why Not Just Test on the Training Data?
A model can memorise its training examples. If you grade it on those same examples, you only learn how well it remembers, not how well it will do on data it has never seen. That gap is what machine learning tries to close, and it is the reason for splitting data.
What Are the Three Sets For?
Set | Used for | When |
|---|---|---|
Training | The model learns its parameters from it | During training |
Validation | Choose settings (hyperparameters), compare models, decide when to stop | During development, many times |
Test | One final, unbiased score | Once, at the end |
Think of a student. The training set is the textbook and homework, the validation set is the practice exam used to adjust how they study, and the test set is the real exam they have never seen.
How Should You Split the Data?
There is no single correct ratio. Splits such as 70/15/15 or 80/10/10 are common starting points. With very large datasets, even a small percentage gives plenty of validation and test examples. With small datasets, people often use cross-validation instead of one fixed validation set. Here is a split with no libraries:
import random
def split_data(items, train=0.7, val=0.15, seed=42):
items = list(items)
random.Random(seed).shuffle(items)
n = len(items)
n_train = int(n * train)
n_val = int(n * val)
return items[:n_train], items[n_train:n_train + n_val], items[n_train + n_val:]
data = list(range(100))
train, val, test = split_data(data)
print(len(train), len(val), len(test)) # 70 15 15
print(set(train) & set(val), set(train) & set(test), set(val) & set(test)) # set() set() set()
Shuffling with a fixed seed makes the split repeatable, and the last line proves the three sets do not overlap. In practice, scikit-learn's train_test_split does this for you, as shown in the first model with scikit-learn chapter.
What Is Data Leakage?
Data leakage happens when information from the validation or test data sneaks into training. Typical causes: scaling or cleaning the whole dataset before splitting, keeping duplicates in different sets, or shuffling time-ordered data so the model sees the future. Leakage makes scores look great and real-world results disappointing. For time series, split by time instead of randomly.
Common Mistakes
Tuning on the test set. Once you adjust the model after looking at test results, the test set becomes a validation set and is no longer unbiased.
Splitting after preprocessing. Split first, then fit any scaler or encoder on training data only.
Ignoring class balance. If one class is rare, use a stratified split so each set has a fair share.
The same idea applies to language-model apps: keep a held-out set of examples you never tweak prompts against. See how to evaluate AI and LLM system performance.
Key Takeaway
Train to learn, validate to tune, test once to judge. Keep the test set locked away, split before any preprocessing, and your reported score will mean something.
Ready to start? You will learn the probability and statistics that make evaluation, splitting and model scores make sense. Begin with the free course Maths and Statistics for AI: The Essentials.
Related articles
More in Machine Learning
Precision, Recall and F1 Score, Explained Simply
What precision, recall and F1 score mean, how to calculate them from a confusion matrix, and how to choose which one matters for your problem.
4 min readNov 16, 2026Classification vs Regression in Machine Learning
Classification predicts a category and regression predicts a number. See the differences, examples, metrics and a tiny Python demo of both.
4 min readNov 10, 2026Overfitting vs Underfitting, Explained Simply
What overfitting and underfitting mean, how to spot each from training and test error, and how to fix them, with a small runnable example.
4 min readNov 8, 202610 Real-World Uses of Deep Learning
Deep learning powers speech recognition, translation, recommendations, chatbots and more. See 10 real uses and the kind of model behind each.
4 min readNov 5, 2026How Much Maths Do You Need for Machine Learning?
An honest answer by goal: how much linear algebra, probability, statistics and calculus you need for machine learning, what to skip at first, and how to learn it alongside coding.
4 min readNov 3, 2026How Does a Neural Network Work? A Simple Explanation
Neurons, weights, bias, layers and how a network learns from its mistakes, explained in plain words with a handwritten-digit example and no heavy maths.
4 min readOct 30, 2026
Comments
Sign in to join the discussion.

