A trained deep learning system is the product of several distinct components working together in a loop. Naming each one precisely makes every later topic in this hub easier to place — every note you read is deepening exactly one of these pieces.
The Training Loop as a System
Data flows forward through the model to a loss score; the optimizer flows a gradient-based correction back into the model's weights.
Component Checklist
| Component | Role | Covered In Depth Later |
|---|---|---|
| Dataset & DataLoader | Supplies batches of (input, label) pairs to the model during training | PyTorch category |
| Model architecture | Defines the layers, weights and biases that transform an input into a prediction | Neural Network Fundamentals, CNN, RNN, Transformers |
| Loss function | Scores how wrong a prediction is compared to the true label — the quantity training minimizes | Loss Functions category |
| Optimizer | Uses gradients of the loss to update the model's weights | Optimization category |
| Evaluation metrics | Task-specific scores (accuracy, F1, BLEU, IoU) used to judge the model, separate from the loss used to train it | Evaluation Metrics category |
| Hardware (CPU/GPU/TPU) | Executes the matrix operations — GPUs make large-scale training practical | Deployment, Production DL & MLOps |
| Framework (PyTorch/TensorFlow) | Provides automatic differentiation, layer implementations, and hardware acceleration | PyTorch, TensorFlow & Keras categories |
| Experiment tracking | Records hyperparameters, metrics and checkpoints across training runs for comparison | Production DL & MLOps category |
Seeing the Components in Code
This is a minimal (untrained, illustrative) PyTorch skeleton — just to make each component from the table concrete. Full working training loops appear in the PyTorch category:
import torch
import torch.nn as nn
# Model — the architecture component
model = nn.Sequential(
nn.Linear(10, 32),
nn.ReLU(),
nn.Linear(32, 1)
)
# Loss function component
loss_fn = nn.MSELoss()
# Optimizer component — will update model.parameters()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
# Hardware component — move computation to GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
print(model)Common Mistakes
- Confusing the loss function with the evaluation metric — the loss must be differentiable (so gradients can flow); the evaluation metric (e.g. accuracy) usually doesn't need to be, and often measures something slightly different from what the loss optimizes.
- Treating the optimizer as interchangeable with "training" in general — the optimizer is specifically the algorithm that turns gradients into weight updates (see the Optimization category for how SGD, Adam and others differ).
Interview Relevance
Q: "What's the difference between a loss function and an evaluation metric?" The loss function is what training directly minimizes via gradients — it must be differentiable. The evaluation metric (e.g. accuracy, F1) is what you report to judge real-world usefulness, and doesn't need to be differentiable at all. They're often related but not identical — e.g. cross-entropy loss vs. accuracy.
Practice Question
List the eight components from the table above for a project you're familiar with (or imagine): a spam email classifier. Name a concrete choice for each (e.g. which loss function, which optimizer).
