Backpropagation Through Time (BPTT) is standard backpropagation applied to the unrolled RNN computational graph from Unrolling RNN — with one distinctive twist: because the same shared weights appear at every time step, gradient contributions from every step must be summed together.
The Key Formula — Summing Contributions Across Time
Since \(\mathbf{W}_{hh}\) is reused at every one of the \(T\) time steps, its total gradient is the sum of the gradient contributions computed at each individual step — exactly analogous to how a shared weight in a CNN (used at every spatial position) accumulates gradient contributions from every position it was applied at.
How Each Time Step's Contribution Is Computed
For a specific time step \(t\), the contribution to \(\frac{\partial L}{\partial \mathbf{W}_{hh}}\) depends on the chain of hidden states from \(t\) back to the start of the sequence — via the chain rule applied repeatedly:
This chain — \(\frac{\partial \mathbf{h}_t}{\partial \mathbf{h}_{t-1}} \cdot \frac{\partial \mathbf{h}_{t-1}}{\partial \mathbf{h}_{t-2}} \cdots\) — is exactly the same multiplicative structure from Chain Rule in Backpropagation, just unfolding across time steps instead of across network layers. This is precisely the mechanism that makes vanishing and exploding gradients especially severe for RNNs over long sequences, covered in the next two notes.
Diagram — Gradient Flowing Backward Through the Unrolled Chain
The total gradient for the shared weight matrix sums contributions from every time step — this summation is BPTT's defining feature.
Code — Autograd Handles BPTT Automatically
import torch
import torch.nn as nn
rnn = nn.RNN(input_size=5, hidden_size=8, batch_first=True)
sequence = torch.randn(1, 10, 5, requires_grad=True)
output, _ = rnn(sequence)
loss = output.sum() # a simple placeholder loss
loss.backward() # this single call performs full BPTT internally
print(rnn.weight_hh_l0.grad.shape) # (8, 8) -- ONE gradient tensor, already summed across all 10 time stepsTruncated BPTT — A Practical Compromise
For very long sequences, backpropagating all the way through every single time step can be computationally expensive and memory-intensive (recall the forward-pass caching requirement from Forward Pass, now multiplied across potentially thousands of time steps). Truncated BPTT limits gradient flow to only the most recent \(k\) time steps, trading some long-range gradient accuracy for dramatically reduced memory and compute cost — a common practical compromise for training RNNs on long sequences.
Common Mistakes
- Assuming each time step contributes an independent gradient for \(\mathbf{W}_{hh}\) — because the weight is shared, PyTorch (and any correct implementation) sums all time steps' contributions into one combined gradient before the optimizer sees it.
- Forgetting truncated BPTT exists as a practical necessity for very long sequences — full BPTT over thousands of time steps can be prohibitively expensive in memory.
Interview Relevance
Q: "Why does BPTT need to sum gradient contributions across time steps, rather than just computing one gradient?" Because \(\mathbf{W}_{hh}\) (and the other RNN weight matrices) are shared — reused identically at every time step — the total effect of a small change to \(\mathbf{W}_{hh}\) on the loss is the sum of its effect at each individual time step it was used. This mirrors how a shared weight in a CNN accumulates gradient contributions from every spatial position it's applied to.
Practice Question
Why might truncated BPTT be necessary in practice for a sequence with 5,000 time steps, even though standard (full) BPTT is mathematically well-defined for any sequence length?
