The MetaRix guide library
Hosting & inference
Plan GPU capacity, latency and serving costs.
5 guides to explore
Platforms & infrastructure
Hosting & inference
Plan GPU capacity, latency and serving costs.
- Guide · 11 min
LLM Latency and Throughput Explained
This guide defines TTFT, time to first answer token, output tokens per second, and end-to-end latency the way major benches and provider docs use them. It then shows which clocks coding agents and batch jobs should optimize—and how to compare speed claims without mixing harnesses.
Read guide → - Guide · 10 min
Spot vs Reserved GPUs for LLM Inference: When to Commit
Spot GPUs are up to 90% cheaper but can be reclaimed with seconds to minutes of warning, while reservations and commitments trade flexibility for certainty. Here is how to split an LLM inference fleet across them.
Read guide → - Guide · 11 min
How to Benchmark LLM Inference: Tools and Method
A repeatable method for load-testing LLM serving, the metrics to report (TTFT, ITL, goodput), and vllm bench serve, SGLang bench_serving, GuideLLM and AIPerf compared.
Read guide → - Comparison · 12 min
Serverless GPU Inference Compared: Pricing and Cold Starts
Modal, Runpod Serverless, Replicate, Baseten and Cloud Run with GPUs compared on list price, cold starts and idle billing, with a worked cost example and a decision flow for picking one.
Read guide → - Comparison · 11 min
vLLM vs SGLang vs TensorRT-LLM: Which to Use
The three leading open-source LLM inference engines compared on hardware, prefix caching, disaggregated serving and release churn, with setup commands and a clear pick for each situation.
Read guide →