All 7 Amazon SageMaker & Machine Learning on AWS questions
More in Amazon SageMaker & Machine Learning on AWS
What is Amazon SageMaker and what stages of the machine learning workflow does it support? How does SageMaker handle model training, and what compute resources does it use? What is SageMaker hyperparameter tuning, and how does it help improve model accuracy? What is SageMaker Studio, and how does it improve the machine learning development experience? How does SageMaker support MLOps practices such as model monitoring and automated retraining pipelines? How does SageMaker support multi model endpoints and model versioning for cost efficient deployment at scale?
Advertisement
AWS › Amazon SageMaker & Machine Learning on AWS

How do SageMaker endpoints support real time inference, and how do they differ from batch transform?

Intermediate

Answer

A SageMaker real time endpoint hosts a trained model behind a persistent, auto scaling HTTPS endpoint that your application can call to get predictions with low latency for individual requests as they arrive, while SageMaker Batch Transform is designed for processing large datasets all at once in a single job, generating predictions for an entire batch of data without needing a persistently running endpoint, which is more cost effective when you do not need instant, on demand predictions.
aws sagemaker create-endpoint --endpoint-name my-endpoint --endpoint-config-name my-endpoint-config
Real-world example A recommendation engine deploys its model to a real time SageMaker endpoint so the website can fetch personalized suggestions instantly as a customer browses, while a separate marketing model uses Batch Transform to score an entire customer list overnight for a weekly email campaign.

Common follow-ups: How does SageMaker endpoint auto scaling work under varying traffic?;What is the cost difference between a real time endpoint and Batch Transform?

Auto Scaling Groups;Lambda & Serverless
All 7 Amazon SageMaker & Machine Learning on AWS questions