Partitioning

Partitioning and Shuffling

How Spark distributes data into partitions for parallelism, why shuffles are the costliest operation in a job, and how to control partition count and skew.

Performance Tuning

Practical techniques for tuning Spark jobs: partitioning, caching, join strategies, memory configuration, and avoiding data skew.

Partitioning Large Tables

Learn how PostgreSQL's declarative partitioning splits large tables into manageable pieces for faster queries, easier maintenance, and efficient data lifecycle…

Cosmos DB Basics

Azure's globally distributed, multi-model NoSQL database service, covering APIs, partitioning, consistency levels, and request units.

Message Keys and Partitioning

How Kafka uses a record's key to choose a partition, why that determines per-key ordering, and the risks of custom partitioners and changing partition counts.

How Do You Find the Median of Two Sorted Arrays?

The median of two sorted arrays can be found in O(log(min(m, n))) time by binary searching a partition point on the smaller array, choosing a matching partitio…

How Do You Find the Kth Largest Element in an Array?

The kth largest element can be found in average O(n) time using quickselect, a partition-based algorithm derived from quicksort that only recurses into the sid…

How Do You Solve the Palindrome Partitioning Problem?

Palindrome partitioning — splitting a string into the minimum number of cuts so every resulting piece is a palindrome — is solved with dynamic programming in O…

What is the Quickselect Algorithm?

Quickselect is a selection algorithm that finds the k-th smallest (or largest) element in an unsorted array in average O(n) time by reusing quicksort's partiti…

Sharding vs. Partitioning: What is the Difference?

Partitioning is the general act of splitting a dataset into smaller pieces by some rule, while sharding is a specific form of horizontal partitioning where tho…

How to Design a Distributed Message Queue

A distributed message queue is designed as partitioned, append-only logs replicated across brokers, where producers write to a partition by key, consumers in a…