Just as code needs version control, data versioning tracks exactly which version of a dataset was used for a given training run — essential for reproducibility, since training data itself changes over time in a real, ongoing project.
Why Datasets Need Versioning Too
Unlike code, datasets are often large binary files poorly suited to standard git version control — but they still change: new data gets added, errors get corrected, labeling schemes get updated. Without tracking exactly which dataset version produced a given model, reproducing or debugging a past result becomes genuinely difficult, undermining the reproducibility goals covered in Reproducibility.
Common Data Versioning Approaches
| Approach | How It Works |
|---|---|
| DVC (Data Version Control) | Git-like versioning specifically designed for large data files and ML pipelines, storing data in cloud storage while tracking lightweight pointers in git |
| Dataset hashing/checksums | Computing and recording a hash of the exact dataset used for each run, verifying dataset identity without full versioning infrastructure |
| Immutable, timestamped dataset snapshots | Never modifying a dataset in place — instead creating a new, timestamped version whenever changes are made, preserving full history |
Code — Basic DVC Usage
# Initialize DVC in a project (alongside git)
# $ dvc init
# Track a large dataset file with DVC (git tracks only a small pointer file)
# $ dvc add data/train_images/
# $ git add data/train_images.dvc .gitignore
# $ git commit -m "Add v1 of training images"
# Later, after the dataset changes:
# $ dvc add data/train_images/
# $ git commit -am "Add v2 of training images -- added 5000 new labeled examples"
# Anyone can check out the exact dataset version matching a specific git commit:
# $ git checkout
# $ dvc checkout # pulls the matching data version from storage The Direct Connection to Experiment Tracking
Data versioning and experiment tracking (Experiment Tracking) work together — logging the exact dataset version (or hash) alongside each experiment run's other metadata means any past result can later be traced back to the precise data it was trained on, closing a reproducibility gap that tracking code version alone leaves open.
Common Mistakes
- Modifying a training dataset in place without any versioning — this makes it impossible to later reproduce or debug a model trained on a now-overwritten earlier version of the data.
- Tracking code version carefully while leaving dataset version completely untracked — a model's behavior depends on both, and omitting either one leaves a real reproducibility gap.
Interview Relevance
Q: "Why is versioning training data important for reproducibility, in addition to versioning code?" A model's behavior is a function of both the code that trained it and the exact data it was trained on — datasets in real projects change over time as new data is added or errors are corrected, so tracking code version alone leaves a genuine gap: without knowing exactly which data version produced a given model, past results can't be reliably reproduced or debugged. Tools like DVC extend familiar git-like versioning to large data files, closing this gap.
Practice Question
A model trained six months ago performed well, but retraining on "the same dataset" today produces meaningfully different results. What data versioning gap might explain this?
