As a concrete example of the cloud patterns from Cloud Deployment, this note covers AWS's specific model deployment services — among the most widely used in industry.
Common AWS Services for Model Deployment
| Service | Role |
|---|---|
| SageMaker | A managed, purpose-built ML platform — handles model hosting, endpoint creation, auto-scaling, and monitoring with less manual infrastructure setup |
| EC2 | General-purpose virtual machines — full control, used for self-managed serving setups (e.g. running a Docker container directly) |
| ECS / EKS | Container orchestration services (ECS is AWS-native; EKS is managed Kubernetes) — for deploying Dockerized serving applications at scale |
| Lambda | Serverless functions — for lightweight, low-traffic, or infrequent inference workloads |
| S3 | Object storage — commonly used to store model artifacts, datasets, and logs |
Code — Deploying a Model with SageMaker
import sagemaker
from sagemaker.pytorch import PyTorchModel
sagemaker_session = sagemaker.Session()
pytorch_model = PyTorchModel(
model_data="s3://my-bucket/model.tar.gz", # trained model artifact in S3
role=sagemaker_execution_role,
entry_point="inference.py", # defines how to load the model and run predictions
framework_version="2.1",
py_version="py311"
)
predictor = pytorch_model.deploy(
initial_instance_count=1,
instance_type="ml.g4dn.xlarge" # a GPU instance type
)
# Now callable directly
result = predictor.predict(input_data)Why a Managed Platform Like SageMaker Is Often Chosen
SageMaker (and equivalent managed platforms on other clouds) handles much of the operational complexity covered across this Deployment category automatically — auto-scaling, health checks, endpoint management, and monitoring integration — significantly reducing the engineering effort needed to get a model safely into production, at the cost of some flexibility and often at a premium cost compared to fully self-managed infrastructure.
Choosing Instance Types
AWS offers many EC2/SageMaker instance types with different CPU/GPU/memory configurations — matching the instance type to the model's actual resource needs (verified through the kind of profiling covered in Inference Latency and GPU Utilization) avoids both under-provisioning (poor performance) and over-provisioning (wasted cost).
Common Mistakes
- Defaulting to the largest, most powerful (and most expensive) instance type available without profiling the model's actual resource requirements first.
- Using a fully self-managed EC2 setup for a standard use case that a managed service like SageMaker would handle with substantially less ongoing operational effort.
Interview Relevance
Q: "What's the tradeoff between using a managed platform like SageMaker versus self-managing deployment on EC2 or Kubernetes?" A managed platform handles much of the deployment complexity automatically — auto-scaling, health checks, endpoint management — significantly reducing engineering effort, but at the cost of less fine-grained control and often a higher direct cost compared to self-managed infrastructure. Self-managed deployment (EC2, self-run Kubernetes) offers maximum flexibility and potentially lower direct infrastructure cost, but requires the team to build and maintain all of that operational tooling itself.
Practice Question
Why should instance type selection for a deployed model be based on actual profiling data, rather than choosing the largest available instance by default?
