Padding adds extra border values (typically zeros) around an input before convolution — giving you direct control over the output size, and preventing edge pixels from being used in fewer convolution computations than pixels near the center.
Formula for Output Size, With Padding
\(P\) is the padding added to each side. Setting \(P=0\) recovers the unpadded formula from Stride.
"Valid" vs "Same" Padding
| Padding Mode | What It Does | Output Size |
|---|---|---|
| Valid (no padding) | The kernel only visits positions where it fully fits within the original input | Smaller than the input |
| Same | Padding is added specifically so the output size matches the input size (for stride 1) | Equal to the input size |
Numerical Example
Input width 5, kernel size 3, stride 1: without padding, output width \(=\lfloor\frac{5-3}{1}\rfloor+1=3\). To achieve "same" padding (output width = 5), solve for \(P\): \(5 = \lfloor\frac{5-3+2P}{1}\rfloor+1 \Rightarrow 4 = 2+2P \Rightarrow P=1\) — adding 1 pixel of padding on each side restores the original size.
Why Padding Matters Beyond Just Size Control
Without padding, a pixel at the very edge of the input participates in far fewer convolution computations than a pixel near the center — the corners and edges get systematically "under-processed" relative to the middle, especially across many stacked layers. Padding ensures every input position, including the edges, gets processed a comparable number of times, avoiding this edge-information bias.
Code
import torch
import torch.nn as nn
x = torch.randn(1, 3, 32, 32)
conv_valid = nn.Conv2d(3, 16, kernel_size=3, padding=0) # "valid" -- no padding
print(conv_valid(x).shape) # torch.Size([1, 16, 30, 30]) -- smaller than input
conv_same = nn.Conv2d(3, 16, kernel_size=3, padding=1) # "same" for a 3x3 kernel, stride 1
print(conv_same(x).shape) # torch.Size([1, 16, 32, 32]) -- matches input sizeCommon Mistakes
- Forgetting that "same" padding depends on both kernel size and stride — the specific padding amount needed to preserve size changes if either of those changes; it's not a single universal constant.
- Stacking many convolutional layers without any padding, causing the spatial size to shrink rapidly (and potentially reach zero or negative dimensions) after just a few layers — a common source of confusing shape errors deep into an architecture.
Interview Relevance
Q: "Why would you choose 'same' padding over 'valid' padding for a deep CNN?" Without padding, every convolutional layer shrinks the spatial dimensions, and this compounds quickly across many stacked layers, potentially shrinking feature maps to unusably small sizes (or even invalid, negative dimensions) before the network reaches its intended depth. "Same" padding keeps the spatial size constant at each layer (for stride 1), giving more direct, predictable control over the architecture's dimensions and avoiding runaway shrinkage.
Practice Question
For an input width of 8, kernel size 5, and stride 1, what padding value \(P\) achieves "same" padding (output width also 8)?
