Beyond GPUs: Production LLMs on AWS Trainium & Inferentia

Abstract

This session evaluates AWS Trainium and Inferentia through a reproducible production pipeline. It covers LoRA fine tuning, vLLM serving, multimodal workloads, quantization, speculative decoding, retrieval augmented generation, cost comparisons, and documented failures to help teams determine when AWS Neuron silicon fits their workloads.

Outline

1. Why and when to consider AWS silicon instead of GPUs
2. Provisioning Trainium and Inferentia with AWS CDK
3. Fine tuning an 8B model with LoRA and publishing it to Amazon S3
4. Serving models with vLLM and testing advanced inference techniques
5. Reviewing multimodal workloads, retrieval augmented generation, and documented failures
6. Comparing costs and applying a production workload decision framework

Key takeaways

• Identify which training and inference workloads fit Trainium and Inferentia.
• Compare Neuron instance costs with equivalent GPU deployments.
• Understand the practical limits of model support, compilation, quantization, and serving.
• Reuse an AWS CDK pipeline and runbooks for fine tuning, artifact storage, and deployment.