How does DynamoDB partitioning and data distribution work under the hood?
Learn how DynamoDB hashes partition keys to distribute data, the per-partition limits, hot partitions, write sharding and adaptive capacity with examples.
Expected Interview Answer
DynamoDB hashes each item's partition key to place the item on one of many physical storage partitions, spreading data and traffic across nodes so reads and writes scale horizontally.
Every table is split into partitions, each holding up to 10 GB and delivering up to 3,000 read capacity units and 1,000 write capacity units. DynamoDB applies an internal hash function to the partition key to decide which partition an item lives on; items sharing a partition key are stored together and ordered by sort key. When a partition fills or exceeds throughput, DynamoDB automatically splits it and redistributes data. Because throughput is divided across partitions, a poorly chosen partition key that concentrates traffic on one key creates a hot partition that throttles even when the table's total capacity is far from exhausted. Adaptive capacity now shifts throughput toward busy partitions to soften this.
- Horizontal scaling to virtually unlimited size and throughput
- Consistent single-digit millisecond latency at scale
- Automatic partition splitting with no downtime
- Even load distribution when keys have high cardinality
- Predictable capacity planning per partition
AI Mentor Explanation
Think of assigning fielding positions by hashing each batter's name: the captain uses a fixed rule to send every ball from a given batter to the same fielder, spreading work evenly across the field. If one superstar batter faces most deliveries, that single fielder gets swamped while others idle, exactly the hot partition problem when one key takes all the traffic.
Step-by-Step Explanation
Step 1
Hash the partition key
DynamoDB runs the item's partition key value through an internal hash function to derive a location.
Step 2
Map to a physical partition
The hash output maps to one storage partition; items with the same partition key always land together.
Step 3
Order by sort key
Within a partition, items sharing a partition key are stored sorted by their sort key for efficient range queries.
Step 4
Divide throughput
The table's provisioned capacity is split across partitions, so each partition has a fraction of the total.
Step 5
Split and redistribute
When a partition exceeds 10 GB or its throughput ceiling, DynamoDB splits it and moves data automatically.
Step 6
Adaptive capacity
DynamoDB shifts unused throughput toward busy partitions to reduce throttling from mild skew.
What Interviewer Expects
- Understanding that the partition key is hashed to choose storage location
- Awareness of the ~10 GB and 3,000 RCU / 1,000 WCU per-partition limits
- A clear explanation of hot partitions and how key design causes them
- Knowledge that throughput is divided across partitions
- Familiarity with automatic partition splitting and adaptive capacity
Common Mistakes
- Believing total table throughput is available to a single key
- Choosing a low-cardinality partition key like status or country
- Confusing the partition (hash) key with the sort (range) key
- Thinking DynamoDB never throttles if the table is under capacity
- Ignoring write sharding for high-volume single-key workloads
Best Answer (HR Friendly)
“DynamoDB spreads your data across many servers by running each item's partition key through a formula that decides where it goes. This lets it scale almost limitlessly, but if too much traffic targets one key, that server gets overloaded, so choosing a well-spread key matters.”
Code Example
import random
import boto3
from boto3.dynamodb.conditions import Key
table = boto3.resource('dynamodb').Table('Events')
SHARDS = 10
def put_event(event_type, payload):
# Spread a high-volume key across N synthetic sub-partitions
shard = random.randint(0, SHARDS - 1)
table.put_item(Item={
'pk': f'{event_type}#{shard}', # partition key with shard suffix
'sk': payload['timestamp'], # sort key
'data': payload,
})
def query_all(event_type):
# Read every shard and merge the results
items = []
for shard in range(SHARDS):
resp = table.query(
KeyConditionExpression=Key('pk').eq(f'{event_type}#{shard}')
)
items.extend(resp['Items'])
return itemsFollow-up Questions
- What is a hot partition and how do you design keys to avoid one?
- How does write sharding spread load across partitions?
- What are the per-partition throughput and storage limits?
- How does adaptive capacity mitigate uneven access patterns?
- How does on-demand mode change partition throughput behaviour?
MCQ Practice
1. How does DynamoDB decide which partition an item is stored on?
DynamoDB applies an internal hash function to the partition key to deterministically map each item to a physical partition.
2. What is the approximate storage limit of a single DynamoDB partition?
Each physical partition holds up to about 10 GB; exceeding it triggers an automatic split.
3. Which partition key choice is most likely to cause a hot partition?
Low-cardinality keys like status concentrate many items and requests onto few partitions, creating hotspots.
Flash Cards
How is an item's partition chosen? — DynamoDB hashes the partition key value with an internal hash function to pick a physical partition.
Per-partition limits? — About 10 GB storage, 3,000 RCU and 1,000 WCU per partition.
What is a hot partition? — A partition receiving disproportionate traffic, causing throttling even when the table has spare total capacity.
How to fix a hot key? — Increase key cardinality or use write sharding by appending a shard suffix to the partition key.