Building on FastAPI Model Serving, this note covers the broader considerations of exposing a model as a production-grade REST API — beyond just a working /predict endpoint.
What "Production-Grade" Adds Beyond a Minimal Endpoint
| Concern | Why It Matters |
|---|---|
| Input validation | Reject malformed or out-of-range requests clearly, before they reach the model — prevents confusing errors or silent incorrect predictions |
| Error handling | Return meaningful HTTP status codes and error messages rather than raw stack traces, which can leak internal implementation details |
| Health check endpoint | Lets load balancers and orchestration systems (e.g. Kubernetes) know whether the service is actually ready to handle traffic |
| Authentication/authorization | Controls who is allowed to call the API — essential for any externally-exposed or cost-sensitive endpoint |
| Logging and request tracing | Essential for debugging issues in production, where you can't simply attach a debugger the way you could locally |
Code — A Health Check Endpoint
from fastapi import FastAPI
app = FastAPI()
@app.get("/health")
def health_check():
# A real health check often verifies the model is actually loaded and
# can run a trivial forward pass, not just that the web server is up
try:
_ = model(torch.zeros(1, *expected_input_shape))
return {"status": "healthy"}
except Exception as e:
return {"status": "unhealthy", "error": str(e)}, 503Code — Structured Error Handling
from fastapi import HTTPException
@app.post("/predict")
def predict(request: PredictionRequest):
if len(request.features) != EXPECTED_FEATURE_COUNT:
raise HTTPException(
status_code=400,
detail=f"Expected {EXPECTED_FEATURE_COUNT} features, got {len(request.features)}"
)
try:
return run_prediction(request)
except Exception as e:
logger.error(f"Prediction failed: {e}")
raise HTTPException(status_code=500, detail="Internal prediction error")
# Note: the CLIENT sees a generic message; the DETAILED error is logged
# server-side only -- avoids leaking internal implementation detailsCommon Mistakes
- Returning raw internal exception messages or stack traces directly to API clients — this can leak sensitive implementation details and is poor practice; log the detailed error server-side and return a generic message to the client instead.
- Omitting a health check endpoint — orchestration systems like Kubernetes rely on health checks to know when to route traffic to an instance and when to restart an unhealthy one; without one, a broken instance can keep receiving traffic indefinitely.
Interview Relevance
Q: "Why should a production model-serving API return generic error messages to clients while logging detailed errors server-side?" Detailed exception messages and stack traces can inadvertently expose internal implementation details (file paths, library versions, internal logic) that could aid an attacker, or simply confuse legitimate API consumers with information that isn't actionable for them. Logging the full detail server-side preserves it for debugging by the team, while returning a generic, safe message to the client follows sound security and API design practice.
Practice Question
Why does a health check endpoint that only confirms "the web server process is running" provide less real value than one that also verifies the model can perform a successful forward pass?
