Frontiers of Modern AI: Multimodality and Accelerated Computing

Abstract

Two earlier talks combined into one session, reusing their slides: Foundations of Multimodal AI and De-mystifying PyTorch for ASICs. The first half covers multimodal fusion architectures that combine text, vision, and audio. The second covers the accelerators that train them, including H100, TPU v6e, and Trainium. The slides here are the multimodal half; the accelerator half has its own entry.

Outline

1. Multimodality and fusion techniques
2. Multimodal models, datasets, training, and evaluation
3. How ASICs and XLA work
4. Google TPU and AWS Trainium with PyTorch
5. Performance, the compiler tax, and common ASIC errors

Key takeaways

• Multimodal work is moving toward reasoning, multilingual, mobile-first, embedding, and RAG models
• PyTorch stays the standard framework, and ASICs are building on top of it