Skip to content

Guide library

The MetaRix guide library

Hosting & inference

Plan GPU capacity, latency and serving costs.

5 guides to explore

Platforms & infrastructure

Hosting & inference

Plan GPU capacity, latency and serving costs.

  1. Guide · 11 min

    LLM Latency and Throughput Explained

    This guide defines TTFT, time to first answer token, output tokens per second, and end-to-end latency the way major benches and provider docs use them. It then shows which clocks coding agents and batch jobs should optimize—and how to compare speed claims without mixing harnesses.

    Read guide →
  2. Guide · 10 min

    Spot vs Reserved GPUs for LLM Inference: When to Commit

    Spot GPUs are up to 90% cheaper but can be reclaimed with seconds to minutes of warning, while reservations and commitments trade flexibility for certainty. Here is how to split an LLM inference fleet across them.

    Read guide →
  3. Guide · 11 min

    How to Benchmark LLM Inference: Tools and Method

    A repeatable method for load-testing LLM serving, the metrics to report (TTFT, ITL, goodput), and vllm bench serve, SGLang bench_serving, GuideLLM and AIPerf compared.

    Read guide →
  4. Comparison · 12 min

    Serverless GPU Inference Compared: Pricing and Cold Starts

    Modal, Runpod Serverless, Replicate, Baseten and Cloud Run with GPUs compared on list price, cold starts and idle billing, with a worked cost example and a decision flow for picking one.

    Read guide →
  5. Comparison · 11 min

    vLLM vs SGLang vs TensorRT-LLM: Which to Use

    The three leading open-source LLM inference engines compared on hardware, prefix caching, disaggregated serving and release churn, with setup commands and a clear pick for each situation.

    Read guide →