What Is the Cold Start Problem in Serverless Computing?
Learn what causes cold starts in serverless computing, why scale-to-zero creates them, and how provisioned concurrency helps.
Expected Interview Answer
A cold start is the extra latency incurred when a serverless platform must provision and initialize a brand-new execution environment (download the code, start the runtime, run initialization code) before it can handle an invocation, as opposed to a warm start that reuses an already-initialized environment sitting idle.
Serverless platforms like AWS Lambda scale down to zero running instances when there is no traffic, which saves cost but means the very next request after idle time (or any request beyond current concurrency) triggers a cold start: the platform allocates compute, pulls the deployment package or container image, boots the language runtime, and runs any global/init-time code (creating SDK clients, loading models, opening connections) before the handler even begins executing the request. This can add anywhere from tens of milliseconds (lightweight interpreted runtimes) to several seconds (JVM-based runtimes, large dependency trees, or VPC-attached functions needing an ENI). Mitigations include provisioned concurrency (keeping a pool of pre-initialized environments always warm, at a cost), minimizing package size and dependency count, choosing faster-starting runtimes, moving heavy initialization to a lazy or cached path, and periodic “warming” pings, though warming is an imperfect workaround compared to provisioned concurrency. The trade-off is fundamental: serverless’s scale-to-zero cost efficiency is exactly what creates the cold-start latency tax on the first request into a new environment.
- Explains why serverless bills near-zero at idle but can spike latency on the first request
- Provisioned concurrency directly trades cost for guaranteed warm environments
- Package size, runtime choice, and VPC attachment are concrete levers a team controls
- Framing cold start as inherent to scale-to-zero clarifies it is a trade-off, not a bug to eliminate entirely
AI Mentor Explanation
A cold start is like a substitute fielder who has been sitting in the pavilion suddenly being called onto the field mid-over. Before they can actually field a ball, they need to jog out, get their bearings, and settle into position, all of which takes real time compared to a fielder who was already warmed up and standing ready. A team that keeps a substitute stretched, warmed up, and standing at the boundary the whole match (provisioned concurrency) avoids that lag entirely, but it costs the team a body on standby the whole game even when unused. That trade-off between paying to keep someone ready versus accepting a slower first response is exactly the cold-start trade-off in serverless.
Step-by-Step Explanation
Step 1
Request arrives with no warm environment
No idle execution environment exists for the function, or existing ones are all busy handling other concurrent invocations.
Step 2
Platform provisions a new environment
Compute is allocated, the deployment package/container image is pulled, and the language runtime starts up.
Step 3
Init-time code executes
Global/init code runs once per environment (SDK client creation, config loading, model loading) before the handler is invoked.
Step 4
Handler executes the request
Only after initialization completes does the actual function handler run and return a response; the environment then stays warm for subsequent requests.
What Interviewer Expects
- Explains cold start as the cost of provisioning + initializing a fresh execution environment
- Distinguishes cold start from warm start (reusing an idle, already-initialized environment)
- Names concrete mitigations: provisioned concurrency, smaller packages, faster runtimes, avoiding VPC ENI overhead
- Frames cold start as an inherent trade-off of scale-to-zero cost efficiency, not simply a bug
Common Mistakes
- Claiming serverless has no cold starts or that they can be fully eliminated for free
- Not mentioning provisioned concurrency as the direct mitigation (at a cost)
- Ignoring init-time code (global scope work) as a major contributor to cold start duration
- Forgetting that VPC-attached functions historically add extra cold-start latency from ENI setup
Best Answer (HR Friendly)
“A cold start is the extra delay you get when a serverless function has not run in a while, because the platform has to spin up a brand-new environment and load everything before it can handle your request, instead of reusing one that is already warmed up. You can pay extra to keep environments always warm and ready, or you can shrink your code and dependencies so that when a cold start does happen, it is as fast as possible.”
Code Example
Resources:
OrdersFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: nodejs20.x
Handler: index.handler
MemorySize: 512
AutoPublishAlias: live
ProvisionedConcurrencyConfig:
ProvisionedConcurrentExecutions: 5
# keep init-time work minimal:
# - create SDK clients once at module scope, not per-invocation
# - avoid VPC attachment unless required
# - keep the deployment package small to speed cold bootsFollow-up Questions
- How does provisioned concurrency differ from on-demand concurrency in cost and behavior?
- Why do JVM-based runtimes typically have longer cold starts than interpreted languages?
- How does VPC attachment historically affect Lambda cold start latency?
- What init-time optimizations would you apply to reduce cold start duration for a given function?
MCQ Practice
1. What is a cold start in serverless computing?
Cold start latency comes from allocating compute, pulling code, booting the runtime, and running init-time code before the handler executes.
2. What does provisioned concurrency do?
Provisioned concurrency pays to keep environments initialized and idle-ready so invocations skip the cold-start path.
3. Why is cold start considered an inherent trade-off rather than simply a bug?
The same scale-to-zero behavior that avoids paying for idle capacity is what forces a fresh environment to be provisioned on the next request.
Flash Cards
What is a cold start? — The extra latency from provisioning and initializing a new serverless execution environment before a request can be handled.
Cold start vs warm start? — Cold start provisions a new environment; warm start reuses an already-initialized, idle environment.
Main mitigation? — Provisioned concurrency keeps a pool of pre-warmed environments ready, at extra cost.
Why is cold start fundamental to serverless? — It is the direct cost of scale-to-zero: no idle capacity means new capacity must be provisioned on demand.