How does gRPC load balancing work?
Learn how gRPC load balancing distributes RPCs over HTTP/2 with client-side, proxy and look-aside approaches, round_robin policies and DNS/xDS resolvers.
Expected Interview Answer
gRPC load balancing distributes RPCs across multiple backend servers, and because gRPC runs many requests over long-lived HTTP/2 connections, it balances at the request (L7) level rather than the connection (L4) level, using either client-side balancing or a proxy/look-aside balancer.
With client-side load balancing, the gRPC client resolves a list of backend addresses (via a resolver like DNS or xDS) and applies a policy such as round_robin or pick_first to spread individual RPCs across sub-channels. With proxy load balancing, an intermediary like Envoy terminates the connection and forwards each request. A look-aside design adds an external balancer service that tells clients which backends to use. Plain L4 balancers work poorly because one HTTP/2 connection can carry all a client's traffic to a single backend.
- Even request distribution over persistent HTTP/2 connections
- Choice of client-side, proxy, or look-aside architectures
- Pluggable policies like round_robin and pick_first
- Dynamic backend discovery via DNS or xDS resolvers
- Avoids hotspotting a single server that L4 balancing causes
AI Mentor Explanation
A captain rotating bowlers over an innings is L7 balancing: each delivery (RPC) can go to a different bowler (server) to spread the workload evenly. Naive L4 balancing is like handing one bowler the entire spell because they took the new ball first — they tire and leak runs while fresh bowlers stand idle in the outfield.
Step-by-Step Explanation
Step 1
Understand the HTTP/2 problem
gRPC multiplexes many RPCs on one long-lived connection, so connection-level (L4) balancing sends all traffic to one backend.
Step 2
Resolve backends
A name resolver (DNS, xDS, custom) produces the current list of server addresses.
Step 3
Apply a policy
A load-balancing policy like round_robin or pick_first assigns each RPC to a sub-channel.
Step 4
Choose an architecture
Pick client-side balancing, a proxy (e.g. Envoy), or a look-aside balancer service.
Step 5
Handle changes
React to backend health and address updates so traffic shifts away from failed servers.
What Interviewer Expects
- Why L4 balancing is inadequate for HTTP/2
- Difference between request-level and connection-level balancing
- Knowledge of round_robin vs pick_first policies
- Client-side vs proxy vs look-aside approaches
- Role of resolvers like DNS and xDS
Common Mistakes
- Assuming a standard TCP/L4 balancer works well for gRPC
- Confusing connection balancing with request balancing
- Ignoring backend discovery and health updates
- Thinking proxy balancing is the only option
Best Answer (HR Friendly)
“gRPC load balancing spreads incoming requests across several backend servers so no single one gets overwhelmed. Because gRPC keeps connections open for a long time, it shares out individual requests — either from the client itself or through a proxy — instead of just splitting up connections.”
Code Example
import (
"google.golang.org/grpc"
_ "google.golang.org/grpc/balancer/roundrobin"
)
conn, err := grpc.Dial(
"dns:///my-service.default.svc.cluster.local:50051",
grpc.WithDefaultServiceConfig(`{"loadBalancingConfig":[{"round_robin":{}}]}`),
grpc.WithTransportCredentials(insecure.NewCredentials()),
)Follow-up Questions
- Why doesn't a normal L4 load balancer work well for gRPC?
- What is the difference between round_robin and pick_first?
- How does xDS enable dynamic load balancing?
- What is look-aside (external) load balancing?
- How does gRPC detect and route around unhealthy backends?
MCQ Practice
1. Why is L4 (connection) load balancing poor for gRPC?
gRPC multiplexes many requests over one long-lived HTTP/2 connection, so L4 balancing pins all of a client's RPCs to one server.
2. Which is a client-side gRPC load-balancing policy?
round_robin is a built-in gRPC policy that cycles each RPC across the resolved backend sub-channels.
3. What does a look-aside load balancer do?
A look-aside balancer is an external service that informs clients of available backends, while clients still send RPCs directly.
Flash Cards
Why L7 balancing for gRPC? — Many RPCs share one HTTP/2 connection, so you must balance per request, not per connection.
Name two gRPC LB policies — round_robin and pick_first.
Three balancing architectures — Client-side, proxy (e.g. Envoy), and look-aside/external balancer.
What resolves backend addresses? — A name resolver such as DNS or xDS.