AlphaGet in touch
All talks

ViableView internal session, 9 October 2026

Self Hosting LM Models: Concepts and Practices

Running language models on your own hardware, from how inference works to choosing a stack.

Organizer
ViableView
Attendees
8
Type
Talk
  • LLM
  • self-hosting
  • inference
  • quantization
Title slide of Self Hosting LM Models: Concepts and Practices

Abstract

An internal ViableView session on running language models on your own hardware. It covers how inference works, why ASICs win over CPUs and GPUs for this work, the provider landscape of VPS, hyperscalers, and neoclouds, how accelerators compare, the software stack from Ollama and MLX to vLLM, and quantization.

Outline

  1. How inference works: tensors, matrix multiplication, prefill, and decode
  2. CPUs, GPUs, and ASICs in deep learning
  3. The provider landscape: VPS, hyperscalers, and neoclouds
  4. How accelerators compare on memory, bandwidth, and price
  5. The compute stack: Ollama, MLX, and vLLM
  6. Quantization and inference