Designing a URL Shortener
A URL shortener (think bit.ly or TinyURL) takes a long URL and maps it to a short, unique alias that redirects users to the original destination. It looks trivial at first glance, but a production system must handle billions of stored mappings, extremely read-heavy traffic (redirects vastly outnumber creations, often 100:1 or more), collision-free key generation across many machines, and low-latency redirects — typically a single-digit-millisecond lookup plus an HTTP 301/302 response. The interesting design decisions are how short codes are generated without central bottlenecks, how the data is stored and partitioned, and how the read path is kept fast and highly available.
Cricket analogy: Like a stadium's turnstile system needing to handle a billion ticket scans with far more entries (reads) than ticket print-runs (writes), each scan needing a near-instant, collision-free match to the right seat.
Core requirements and capacity estimation
Functional requirements: given a long URL, generate a short alias (optionally custom); given a short alias, redirect to the long URL; support link expiration and basic analytics (click counts). Non-functional requirements: high availability, low read latency, and uniqueness of generated keys even with many app servers writing concurrently. A rough capacity estimate matters: if the service accepts 100 new URLs per second and reads are 100x that, you need to plan for roughly 10,000 redirects per second at peak, and with 7-character base62 codes you get 62^7 (~3.5 trillion) possible codes — comfortably enough for years of writes without reuse.
Cricket analogy: Like planning a stadium's capacity around ticket sales of 100/sec but gate entries of 10,000/sec at peak, and choosing a 7-digit seat code scheme that comfortably covers every seat for years without reuse.
Key generation strategies
Three common approaches exist. Hashing the long URL (e.g., MD5/SHA-256 truncated to 7 base62 characters) is simple but risks collisions, requiring a check-and-retry loop against the datastore. A counter-based approach assigns each URL a unique monotonically increasing ID (from a database sequence or a distributed ID generator like Snowflake) and encodes that ID in base62 — this guarantees uniqueness with no collision checks, at the cost of needing a coordinated ID source. A pre-generated key pool has a background service produce batches of unused random codes and store them in a 'key database'; app servers simply claim an unused key, avoiding any collision-check latency on the write path entirely. Most production designs favor the pre-generated pool or a Snowflake-style ID generator because both scale writes horizontally without a shared counter becoming a bottleneck.
Cricket analogy: Like assigning a bowler's jersey number by hashing their name (simple but risks two bowlers colliding, needing a recheck), versus a central board issuing sequential numbers (guaranteed unique but needs one coordinator), versus pre-printing a pool of spare jerseys any team can claim instantly.
Base62 encoding of an auto-increment ID:
id = 125214460 (unique integer from ID generator)
alphabet = [0-9a-zA-Z] (62 characters)
while id > 0:
remainder = id % 62
code = alphabet[remainder] + code
id = id // 62
# 125214460 -> "8xUL2" (5-char short code)
Request flow for a redirect:
1. Client requests GET /8xUL2
2. Edge/CDN cache miss -> app server
3. App server checks Redis cache for key "8xUL2"
4. Cache miss -> query sharded DB (shard chosen by hash of the code)
5. DB returns long URL -> populate cache -> respond 301 with Location header
6. Async: increment click-count counter for analyticsStorage, sharding, and caching
The core mapping (short_code -> long_url, created_at, expiry, owner) is simple key-value data, so it fits well in either a relational table with an index on short_code or a NoSQL key-value/wide-column store — the choice usually comes down to whether you need strong relational features elsewhere. As the dataset grows past what a single node can serve, shard by a hash of the short code so lookups route deterministically to the owning shard without a lookup service. Because reads dominate so heavily, an in-memory cache (Redis/Memcached) sitting in front of the database absorbs the vast majority of redirect traffic; popular links follow a Zipfian access pattern, so even a modest cache achieves a high hit ratio. A CDN or edge cache can serve extremely hot redirects without hitting the origin at all.
Cricket analogy: Like storing each player's stats as a simple key-value record, sharding the league's records across regional archives by player-name hash, and keeping the most-searched stars like Kohli in a fast-access cache since a few players get most of the lookups.
Real systems favor HTTP 302 (temporary redirect) over 301 (permanent) for short links, even though 301 lets browsers cache the redirect locally and reduces server load. The tradeoff is that 301 caching prevents the server from ever seeing subsequent clicks on that link, breaking click analytics — a concrete example of a non-functional requirement (analytics) overriding an obvious performance optimization.
A common mistake is designing the short-code generator around a single auto-increment counter in one SQL database. That counter becomes a single point of write contention and a single point of failure — every URL creation serializes through it. Prefer a distributed ID scheme (range-based ID allocation per app server, or a Snowflake-style generator) so writes scale horizontally.
- URL shorteners are read-heavy systems (often 100:1 read:write) so caching and fast redirect paths matter more than write throughput.
- Base62 encoding of a unique integer ID is a simple, collision-free way to generate short codes.
- Pre-generated key pools or distributed ID generators (Snowflake-style) avoid the write bottleneck of a single shared counter.
- Shard the mapping table by a hash of the short code so lookups route deterministically without a separate lookup service.
- Use 302 redirects when click analytics matter, since 301 responses get cached by browsers and hide subsequent visits from the server.
- A cache in front of the database (plus CDN/edge caching for hot links) is the single highest-leverage optimization for redirect latency.
Practice what you learned
1. Why do most production URL shorteners avoid using a single relational auto-increment column as the sole source of short-code IDs?
2. Why might a URL shortener choose HTTP 302 over HTTP 301 for redirects despite 301 reducing server load?
3. What is the main advantage of a pre-generated key pool over hashing the long URL to derive a short code?
4. In a sharded URL shortener, why is sharding by hash of the short code preferable to sharding by creation timestamp?
5. Given 7-character base62 short codes, roughly how many unique codes are available?
Was this page helpful?
You May Also Like
Designing a Rate Limiter
Walks through building a distributed rate limiter that protects APIs from abuse, comparing token bucket, sliding window, and fixed window algorithms and where state should live.
Database Sharding
Explains how sharding partitions a dataset horizontally across multiple database nodes to scale beyond what a single machine can hold, and the tradeoffs of common sharding strategies.
Consistent Hashing
Introduces the hashing technique that minimizes data movement when nodes are added or removed, forming the backbone of distributed caches, sharded databases, and load balancers.
CDN Caching and Edge Computing
Covers how Content Delivery Networks cache content close to users to cut latency, and how edge computing extends this to running logic near the request origin.
Related Reading
Related Study Notes in Software Engineering
Browse all study notesMicroservices Study Notes
Software Architecture · 30 topics
Software EngineeringTesting & TDD Study Notes
Software Testing · 30 topics
Software EngineeringDesign Patterns Study Notes
Software Design · 30 topics
Software EngineeringSoftware Engineering Study Notes
Python · 75 topics
Software EngineeringGit & Version Control Study Notes
Bash · 40 topics