OCR (Optical Character Recognition) converts images of text — scanned documents, photographed signs, screenshots — into actual machine-readable text, combining computer vision and sequence modeling into one pipeline.
The Two-Stage Pipeline
| Stage | Task | Typical Approach |
|---|---|---|
| Text detection | Locate where text regions are in the image (analogous to object detection) | Detection architectures adapted for text regions/lines |
| Text recognition | Convert each located text region into actual characters | A CNN feature extractor feeding into a sequence model (RNN/Transformer), predicting a sequence of characters |
Why Recognition Is a Sequence Problem
Unlike classifying a whole image into one label, reading a line of text means predicting a variable-length sequence of characters from an image — this is why OCR's recognition stage typically combines a CNN (for extracting visual features across the text line) with a sequence model (an RNN, or more modern Transformer-based approach) trained with a loss like CTC (Connectionist Temporal Classification), which handles the tricky alignment problem between variable-width image regions and variable-length output text without requiring pre-segmented individual characters.
Code — Using a Pretrained OCR Pipeline
# Using a common OCR library rather than building the pipeline from scratch
import easyocr
reader = easyocr.Reader(['en'])
# results = reader.readtext('scanned_document.jpg')
# each result: (bounding_box, recognized_text, confidence_score)Common Mistakes
- Assuming OCR is a single-stage task — the detection (where is text) and recognition (what does it say) stages address genuinely different sub-problems, and most practical OCR systems address both explicitly.
- Expecting an OCR system trained mostly on printed text to perform well on handwriting, or vice versa — these are meaningfully different visual distributions, often requiring separately trained or fine-tuned models.
Interview Relevance
Q: "Why is the text recognition stage of OCR typically framed as a sequence prediction problem, rather than a per-character classification problem?" Text regions vary in length, and pre-segmenting individual characters reliably (especially for cursive handwriting or tightly-spaced fonts) is itself a hard, error-prone sub-problem. Framing recognition as predicting a whole character sequence directly from the image region (often via CTC loss) sidesteps the need for perfect pre-segmentation, letting the model learn the alignment between image regions and output characters jointly.
Practice Question
Why might an OCR system trained primarily on clean, printed English text perform poorly on a photograph of handwritten Arabic script?
