An ML API exposes a trained model's predictions over HTTP — the standard way other applications (a website, a mobile app, another backend service) actually consume a deployed model's output.
The Core Design Pattern
| Component | Purpose |
|---|---|
| Endpoint | A URL a client sends a request to, e.g. POST /predict |
| Request schema | The expected input format — which fields, what types, what's required |
| Response schema | The prediction, plus useful metadata (confidence, model version) |
| Health check endpoint | A simple GET /health that confirms the service is running and the model loaded correctly |
A Well-Designed Request/Response Shape
// Example request body -- POST /predict
{
"features": {
"income": 45000,
"age": 34,
"credit_score": 680
}
}
// Example response body
{
"prediction": "approved",
"probability": 0.82,
"model_version": "1.2.0"
}Including model_version in every response is a small habit that pays off enormously later — it lets you trace exactly which model version produced any given historical prediction, essential for debugging and auditing.
Input Validation — Non-Negotiable
A production ML API must validate incoming requests before passing them to the model — missing fields, wrong types, or out-of-range values should return a clear error (see FastAPI ML API for how Pydantic automates this), rather than causing a confusing internal crash or, worse, a silently wrong prediction.
Versioning Your API
# Include the version in the URL path itself -- lets you run
# multiple model versions simultaneously during a gradual rollout
POST /v1/predict
POST /v2/predictPractical Use Cases
- Real-time prediction serving for web/mobile applications
- Exposing a model's predictions to other internal backend services
Common Mistakes
- Skipping input validation, letting malformed requests crash the service or silently produce nonsense predictions.
- Not versioning the API or the model, making it impossible to trace which model produced a specific past prediction.
- Loading the model fresh on every single request instead of once at startup — see ML Inference.
Interview Relevance
Q: "What should a production ML API's response always include, beyond the raw prediction?" At minimum, a confidence/probability score and the model version that produced it — the version lets you trace and debug any specific historical prediction, and the confidence score lets downstream systems make risk-aware decisions instead of treating every prediction as equally certain.
Practice Question
Design the JSON request and response shape for an API that predicts house prices from size, bedrooms, and location.
