Cold Start

Definition

A cold start is the added latency incurred when a serverless function, container, or newly-scaled instance must be initialized from scratch — provisioning resources, loading code, and warming a runtime — before it can serve its first request, as opposed to reusing an already-running ('warm') instance.

Overview

Cold starts are an inherent tradeoff of elastic and serverless architectures. When a platform like AWS Lambda, Google Cloud Run, or Azure Functions scales an application from zero (or adds new instances to handle a traffic spike), it has to allocate compute, pull or mount the application's code and dependencies, initialize the language runtime (JVM startup, Node.js module loading, Python interpreter and import resolution), run any application-level initialization (opening database connections, loading configuration, warming caches), and only then can it begin processing the request. This entire sequence adds latency on top of the request itself — ranging from tens of milliseconds for lightweight runtimes to several seconds for large containers or JVM-based applications with heavy dependency graphs. Once an instance has handled a request, most platforms keep it 'warm' for some period, reusing it for subsequent requests with no cold-start penalty — this is why cold starts are typically felt on the first request after a period of inactivity, or when traffic bursts faster than existing warm capacity can absorb. Cold starts are more pronounced in serverless/FaaS platforms that scale to zero (no cost, but every dormant period risks a cold start) than in traditional autoscaling groups that maintain a warm minimum fleet. Mitigation strategies include provisioned concurrency (pre-warming a fixed number of instances, at extra cost, common in AWS Lambda), keeping runtime images small and dependency graphs lean, choosing faster-starting languages/runtimes (Go and Rust generally cold-start faster than Java or .NET), using lighter base container images, and periodic 'keep-warm' pings that prevent instances from being fully torn down. Cold starts matter directly to user experience and SLOs — a latency-sensitive API that scales to zero can violate p99 latency targets on its very first request of the day, which is why architects weigh the cost savings of scale-to-zero against the latency risk it introduces.

Key Concepts

• Occurs when new compute (function, container, or instance) must initialize before serving a request
• Includes resource provisioning, code/dependency loading, and runtime + application initialization
• Most pronounced in scale-to-zero serverless platforms (Lambda, Cloud Run, Azure Functions)
• Runtime choice matters: interpreted/JIT-heavy runtimes (JVM, .NET) typically cold-start slower than Go or Rust
• Provisioned/reserved concurrency mitigates cold starts at extra always-on cost
• Warm instances avoid the penalty by handling subsequent requests without reinitializing
• Container image size and dependency count directly affect cold-start duration
• Directly impacts p99 latency and SLO compliance for bursty or infrequently-called services

Use Cases

Frequently Asked Questions

Related in Cloud