What is Database Sharding?
Understand database sharding, how shard keys distribute data across servers, and the trade-offs of horizontal scaling in interviews.
Expected Interview Answer
Database sharding is a horizontal partitioning technique that splits a large dataset across multiple independent database servers (shards), where each shard holds a distinct subset of the rows based on a shard key.
Instead of one server storing every row of a table, sharding distributes rows — for example by user ID ranges or a hash of the ID — across many smaller databases that can each be scaled, backed up, and queried independently. This lets the system handle far more data and traffic than a single machine could, at the cost of added complexity: cross-shard joins, rebalancing when shards grow unevenly, and routing every query to the correct shard.
- Scales write and storage capacity horizontally
- Reduces load on any single server
- Improves query performance per shard
- Enables geographic or tenant-based data distribution
AI Mentor Explanation
Imagine a national cricket board that cannot let one office handle every player’s registration for the entire country, so it splits records by region: North Zone office holds North Zone players, South Zone office holds South Zone players. Each office serves its region fast because it only manages a fraction of the total players. Sharding splits a database the same way: each shard holds only the rows assigned to it, so no single server bears the full load.
How Database Sharding Distributes Rows
Shard 1 (user_id 1-1000)
- user_id
- name
Shard 2 (user_id 1001-2000)
- user_id
- name
Shard 3 (user_id 2001-3000)
- user_id
- name
Step-by-Step Explanation
Step 1
Choose a shard key
Pick a column (e.g. user_id) whose values distribute data and traffic evenly.
Step 2
Define the partitioning scheme
Use range-based, hash-based, or directory-based mapping from key to shard.
Step 3
Deploy independent shard servers
Each shard runs as its own database instance holding only its assigned rows.
Step 4
Route queries
An application layer or proxy computes the shard key and directs each query to the correct shard.
What Interviewer Expects
- Understanding of horizontal partitioning across servers
- Awareness of shard key selection and its impact on balance
- Knowledge of trade-offs like cross-shard joins and rebalancing
- Distinction between sharding and simple replication
Common Mistakes
- Confusing sharding with database replication
- Ignoring the difficulty of cross-shard queries and joins
- Picking a poor shard key that causes uneven "hot" shards
- Assuming sharding solves every scalability problem automatically
Best Answer (HR Friendly)
“Database sharding means splitting a large table across multiple servers, so each server only stores and serves a portion of the data based on a shard key like user ID. It lets a system scale well beyond what a single database server could handle, though it adds complexity around routing queries and joining data across shards.”
Code Example
-- Pseudocode: application layer picks the shard before querying
function getShardForUser(userId) {
const shardCount = 4;
const shardIndex = hash(userId) % shardCount;
return shardIndex; // e.g. 0, 1, 2, or 3
}
-- The query itself runs unmodified against the chosen shard:
SELECT * FROM Users WHERE user_id = 12345;
-- executed against shard db_shard_2 after routingFollow-up Questions
- What is the difference between sharding and partitioning?
- How do you handle a query that needs data from multiple shards?
- What problems arise when rebalancing shards after uneven growth?
- How does sharding differ from database replication?
MCQ Practice
1. Database sharding primarily achieves what?
Sharding horizontally partitions rows across multiple independent servers so no single machine holds all the data.
2. What determines which shard a given row is stored on?
A shard key (e.g. a hashed or ranged column value) deterministically maps each row to a specific shard.
3. Which is a common challenge introduced by sharding?
Splitting data across shards makes joins spanning multiple shards and rebalancing uneven shards notably harder.
Flash Cards
What is sharding? — Horizontally splitting a dataset across multiple independent database servers.
What is a shard key? — The column used to decide which shard a given row belongs to.
What is a downside of sharding? — Cross-shard joins and rebalancing become significantly more complex.
Sharding vs replication? — Sharding splits distinct data across servers; replication copies the same data to multiple servers.