Sharding
Sharding is a database scaling technique that splits a large dataset horizontally across multiple independent database instances (shards), each holding a subset of the total data, so that no single machine needs to store or serve the entire dataset.
26 resources across 3 libraries
Glossary Terms(6)
Couchbase
Couchbase is a distributed NoSQL Document Database that stores JSON documents and combines high-performance key-value access with a SQL-like query language (N1…
Google Cloud Spanner
Google Cloud Spanner is a fully managed, horizontally scalable relational database service that combines strong relational consistency (SQL schemas, ACID trans…
Vitess
Vitess is an open-source database clustering system that adds horizontal scaling, sharding, and connection management to MySQL, letting it operate reliably at…
CAP Theorem
The CAP theorem states that a distributed data system can provide at most two of three guarantees simultaneously during a network partition: Consistency, Avail…
Sharding
Sharding is a database scaling technique that splits a large dataset horizontally across multiple independent database instances (shards), each holding a subse…
Database Replication
Database replication is the process of copying and synchronizing data from a primary database to one or more secondary databases, used to improve read scalabil…
Study Notes(5)
Parallel Workers and Sharding
How Playwright Test runs suites faster using isolated worker processes locally and distributes them across multiple CI machines with sharding.
Sharding in MongoDB
How MongoDB distributes data across multiple shards using shard keys, chunks, and a query router to scale horizontally beyond a single server's capacity.
Redis Cluster Explained
How Redis Cluster shards data across multiple nodes using hash slots, and how it handles failover and multi-key operations.
Database Replication and Sharding
Learn the difference between replicating data for availability and sharding data for write scalability.
Database Sharding
Explains how sharding partitions a dataset horizontally across multiple database nodes to scale beyond what a single machine can hold, and the tradeoffs of com…
Interview Questions(15)
What is Consistent Hashing and Why is it Used?
Consistent hashing maps both keys and servers onto the same circular hash ring so that when a server is added or removed, only the keys between it and its neig…
Horizontal vs Vertical Scaling
Vertical scaling (scaling up) means adding more power — CPU, RAM or disk — to a single machine, while horizontal scaling (scaling out) means adding more machin…
SQL vs NoSQL for Scale: Which Should You Choose?
SQL databases scale best when data is relational and consistency matters, typically via vertical scaling or read replicas, while NoSQL databases are built to s…
What is Consistent Hashing?
Consistent hashing is a distribution technique that maps both data keys and servers onto the same hash ring, so that when a server is added or removed only a s…
Sharding vs. Partitioning: What is the Difference?
Partitioning is the general act of splitting a dataset into smaller pieces by some rule, while sharding is a specific form of horizontal partitioning where tho…
How Would You Design Instagram?
Instagram is designed around three separable concerns — durable object storage for photos and videos behind a CDN, a metadata service backed by a sharded datab…
How to Design a Web Crawler
A web crawler is designed as a distributed pipeline of a URL frontier, fetchers, parsers, and a dedup/storage layer, where politeness rules, priority schedulin…
How Would You Design a Real-Time Leaderboard?
A real-time leaderboard ranks a large, constantly updating set of scores and answers “what is this player’s rank” and “who are the top N” in near-constant time…
How Would You Design a Distributed Counter (e.g. Like Counts)?
A distributed counter tracks a fast-changing count (like a like button) across many concurrent writers by sharding the counter into multiple sub-counters that…
How to Design a Parking Garage System?
A parking garage system models floors, rows and spots as a hierarchy with real-time availability counters, uses atomic decrement/increment operations at entry…
How Do You Design for Read-Heavy vs Write-Heavy Systems?
Read-heavy systems are optimized by aggressively caching and replicating data close to readers so most requests never touch the primary store, while write-heav…
What is the Hot Partition Problem and How Do You Fix It?
The hot partition problem occurs when a partitioning scheme sends a disproportionate share of traffic to one shard — because the chosen partition key is skewed…
What Are the Main Data Partitioning Strategies?
Data partitioning splits a large dataset across multiple nodes using a strategy such as range partitioning (contiguous key ranges per node), hash partitioning…
How Do You Design a Good Partition Key?
A good partition key spreads data and traffic evenly across all partitions (avoiding hot spots) while still keeping any data that must be read or ordered toget…
How Do You Mitigate Hot Spots in a Sharded System?
A hot spot is a single shard, partition, or key that receives disproportionately more traffic than its peers, and it is mitigated by spreading that skewed load…