With clean, split data ready, model selection is the process of choosing an appropriate architecture — guided by task type, data size, and compute constraints, rather than picking the newest or most impressive-sounding option by default.
A Practical Decision Framework
| Consideration | Guides Toward |
|---|---|
| Task type (classification, generation, sequence modeling) | The broad architecture family — CNN, Transformer, RNN, etc. |
| Available data volume | Small data strongly favors transfer learning from a pretrained model over training from scratch (see the Transfer Learning category) |
| Compute/latency budget | Model size and complexity — a resource-constrained deployment target rules out very large architectures regardless of their potential accuracy ceiling |
| Existing proven solutions for similar tasks | A strong, well-validated starting point — reinventing an architecture from scratch is rarely the efficient choice for well-studied problem types |
Starting Simple, Then Scaling Complexity
A common, efficient practical approach: start with the simplest reasonable baseline (sometimes even a non-deep-learning model), establish it as a performance floor, then incrementally move to more complex architectures only if the simpler baseline demonstrably falls short — rather than starting with the most sophisticated possible architecture and working backward. This mirrors the exact "start shallow, add depth incrementally" strategy from Network Depth Tuning, applied here to architecture choice more broadly.
Code — A Simple Baseline-First Comparison
results = {}
# Baseline: a simple, fast, non-deep model
from sklearn.linear_model import LogisticRegression
baseline = LogisticRegression()
baseline.fit(X_train_flat, y_train)
results['logistic_regression'] = evaluate(baseline, X_val_flat, y_val)
# A small CNN
small_cnn = build_small_cnn()
train(small_cnn, train_loader)
results['small_cnn'] = evaluate(small_cnn, val_loader)
# A pretrained, fine-tuned model
pretrained = build_pretrained_resnet()
train(pretrained, train_loader)
results['pretrained_resnet'] = evaluate(pretrained, val_loader)
for name, score in results.items():
print(f"{name}: {score:.4f}")Common Mistakes
- Choosing the most complex, state-of-the-art architecture available without first establishing whether a much simpler approach already performs adequately — added complexity should be justified by a genuine performance gain, not assumed.
- Ignoring deployment constraints (latency, memory, cost) during model selection, only to discover the chosen architecture is impractical to actually deploy after significant training investment.
Interview Relevance
Q: "Why would you start a new deep learning project with a simple baseline model, even if you strongly suspect a more complex architecture will ultimately be needed?" A simple baseline establishes a concrete performance floor, validates the overall pipeline (data, evaluation, infrastructure) works correctly before investing in more complex modeling, and gives a clear, honest reference point for measuring whether added architectural complexity actually earns its cost in improved performance — rather than assuming complexity helps without ever verifying it against something simpler.
Practice Question
A team has only 500 labeled images for a 10-class classification task. Would you recommend training a large CNN from scratch, or a different approach? Justify your answer.
