Series / Agent Engineering / Applied Track
Inference Serving · vLLM
A focused path on why to serve with vLLM on internal GPUs, how to tune it, and how to diagnose slowdowns.
- 01
Why Serve Open-Source LLMs with vLLM on Internal GPUs?
Deciding internal open-model serving for security, data control, cost, and performance
When internal GPU serving and vLLM make sense: compare external APIs, adoption trade-offs, batching and prefix caching in plain terms, and a pre-flight checklist.
- 02
What is vLLM tuning?
Serving LLMs when GPUs are scarce
A practical guide to serving LLMs on limited on-premises GPUs: diagnose prefill, decode, KV cache, queues, and tail latency to tune vLLM for sustainable throughput.
- 03
Why Is vLLM Slow?
Diagnosing bottlenecks with TTFT, TPOT, and KV cache
A practical guide to diagnosing vLLM serving bottlenecks by reading TTFT, TPOT, queue, prefill, KV-cache, and preemption signals together.