Cloud & Architecture · Paid
Runpod Serverless
Runpod Serverless is the pay-per-second GPU endpoint product from Runpod, a GPU cloud that also rents dedicated GPU Pods and multi-node clusters. With Serverless, you package a model and a handler function in a Docker container, deploy it as an endpoint, and Runpod starts and stops GPU workers as requests arrive, scaling down to zero when the endpoint is quiet.
Who it is for: developers and startups serving image, audio or language models through an API, especially with spiky or unpredictable traffic, who want lower-level control over the container than a fully managed model API gives them.
How it works: endpoints are either queue-based, where requests wait in a queue and a handler function processes each job, or load-balancing endpoints, where you run your own HTTP server such as FastAPI. Runpod offers two worker types. Flex workers scale to zero and are billed at the standard per-second rate; active workers run around the clock to avoid cold starts and can be discounted through sales. Runpod's documentation says workers are billed from start until fully stopped, rounded up to the nearest second, across three phases: cold start (mostly loading the model into GPU memory), execution, and an idle timeout that defaults to 5 seconds. Pulling the container image and downloading cached models normally happen while the worker is scaled down and are not billed. FlashBoot and model caching reduce cold starts.
Pricing type: usage-based. As of November 11, 2026, Runpod's pricing page lists Serverless GPU rates from $0.58 per hour for 16 GB cards to $11.99 per hour for the B300, including $5.39 per hour for the H100, $2.72 per hour for the A100 and $0.75 per hour for the L4, A5000 and 3090 tier. Container disk costs about $0.10 per GB per month and network volumes $0.07 per GB per month under 1 TB. Accounts have a default spend limit of $80 per hour that support can raise.
Alternatives: Modal, Replicate, Baseten, Google Cloud Run with GPUs, and Runpod's own dedicated GPU Pods for steady traffic.
Official site: https://www.runpod.io/product/serverless
