Why Load Balancing in gRPC Is Different
Like a single long over where all six deliveries go to the same batsman rather than rotating strike, a single HTTP/2 connection multiplexes many gRPC calls, so an L4 load balancer that just spreads TCP connections leaves one backend batting through everything. gRPC's persistent, multiplexed connections mean traditional connection-level load balancing under-distributes traffic across a fleet of backend replicas, because once a connection is established, every RPC sent over it goes to the same server until the connection closes. This is fundamentally different from HTTP/1.1, where each request commonly opens (or reuses from a small pool) a short-lived connection that a simple TCP load balancer can freely redistribute.
Cricket analogy: Like a single long over where all six deliveries go to the same batsman rather than rotating strike, a single HTTP/2 connection multiplexes many gRPC calls, so an L4 load balancer that just spreads TCP connections leaves one backend batting through everything.
Client-Side vs Proxy-Side Load Balancing
gRPC ships with two built-in client-side balancing policies: pick_first, which connects to the first address the resolver returns and sends every call there (falling back only on failure), and round_robin, which opens connections to every resolved address and cycles calls across them evenly. Beyond these built-ins, larger deployments often adopt xDS — the same control-plane API Envoy uses — either through a service mesh sidecar or via gRPC's own 'proxyless mesh' support, letting a central control plane push load balancing configuration, endpoint membership, and health status to clients dynamically instead of hardcoding a policy in application code.
Cricket analogy: Like a captain rotating the bowling attack ball by ball between Bumrah, Shami, and Siraj rather than letting one bowler send down every over, the round_robin policy rotates each RPC across all available backend subchannels.
conn, err := grpc.Dial(
"dns:///backend.svc.cluster.local:50051",
grpc.WithDefaultServiceConfig(`{"loadBalancingConfig": [{"round_robin":{}}]}`),
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
if err != nil {
log.Fatalf("failed to dial: %v", err)
}
client := pb.NewInventoryServiceClient(conn)Name Resolution and Service Discovery
Client-side load balancing depends entirely on the name resolver returning multiple backend addresses rather than a single virtual IP; the default DNS resolver does this by resolving a hostname to its A/AAAA records, which works reasonably well against a Kubernetes headless Service (one without a clusterIP) that returns every backing pod's IP directly. The catch is that DNS results are cached for a TTL and carry no live health information, so a gRPC client can keep sending calls to a pod that Kubernetes has already terminated, or miss a newly scaled-up pod until the next scheduled re-resolution — a limitation that pushes many teams toward a custom resolver.Builder or an xDS-based control plane instead.
Cricket analogy: Like a scorer relying on a stadium announcer's periodic team-sheet updates rather than live tracking of who's on the field, DNS-based resolution only refreshes backend addresses periodically via TTL, missing pods that scale up between refreshes.
DNS-based name resolution alone is a poor fit for Kubernetes: it only refreshes on TTL expiry and returns a flat list of IPs with no per-endpoint health signal, so most production gRPC deployments pair it with either a custom resolver.Builder, an xDS control plane like Istio, or a headless Service combined with client-side round_robin.
Load Balancing Policies and Health Checking
Beyond pick_first and round_robin, more advanced policies like weighted round robin and least_request take backend load into account: weighted round robin sends proportionally more traffic to backends reporting better metrics via the ORCA (Open Request Cost Aggregation) protocol, while least_request routes each new call to whichever subchannel currently has the fewest outstanding in-flight requests. All of these policies interact with gRPC's health checking protocol, defined in grpc.health.v1.Health, which lets a balancer periodically ask each backend 'are you serving?' and exclude any backend reporting NOT_SERVING from the pick, preventing calls from being routed to a server that's still connected but overloaded or shutting down.
Cricket analogy: Like a captain giving more overs to a bowler with a better economy rate in the current match rather than splitting evenly, weighted round robin sends proportionally more RPCs to backends reporting lower load via ORCA metrics.
Placing a traditional L4 (TCP-level) load balancer in front of gRPC servers is a common mistake: because HTTP/2 connections are long-lived and multiplexed, an L4 LB balances at connection-establishment time only, so once connections are made, all subsequent RPCs stick to whichever backend was picked first, concentrating load unevenly.
- gRPC's persistent, multiplexed HTTP/2 connections make simple L4 (connection-level) load balancing ineffective.
- pick_first sticks to one backend; round_robin cycles calls across all resolved addresses.
- Client-side load balancing requires a name resolver that returns a list of backend addresses, not a single VIP.
- DNS-based resolution is TTL-cached and lacks real-time health signals, making it a weak fit for fast-scaling Kubernetes environments.
- xDS-based balancing (Istio, proxyless mesh) offers dynamic, centrally-managed load balancing policies.
- Weighted round robin and least_request policies distribute load based on reported backend utilization.
- gRPC's health checking protocol lets a balancer avoid routing to backends that report NOT_SERVING.
Practice what you learned
1. Why is a traditional L4 load balancer a poor fit for gRPC by default?
2. What does the round_robin load balancing policy do?
3. What is a key limitation of DNS-based name resolution for gRPC in Kubernetes?
4. What does the least_request load balancing policy optimize for?
5. What role does gRPC's health checking protocol play in load balancing?
Was this page helpful?
You May Also Like
gRPC Interceptors
How to use gRPC interceptors to implement cross-cutting logic like logging, authentication, retries, and tracing without touching individual handlers.
gRPC Authentication with TLS
How gRPC uses TLS and mutual TLS to secure transport, and how call credentials layer token-based authorization on top for production services.
gRPC-Gateway for REST Compatibility
How grpc-gateway generates a REST/JSON reverse-proxy from annotated .proto files so browsers and legacy clients can reach a gRPC-only backend.
Related Reading
Related Study Notes in Web Development
Browse all study notesWebSockets Study Notes
Web Development · 30 topics
Web DevelopmentWebAssembly Study Notes
WebAssembly · 30 topics
Web DevelopmentSpring Boot Study Notes
Java · 30 topics
Web DevelopmentFlask Study Notes
Python · 30 topics
Web DevelopmentDjango Study Notes
Python · 30 topics
Web DevelopmentNext.js Study Notes
JavaScript · 30 topics