Multithreading vs Multiprocessing in Python
Understand multithreading vs multiprocessing in Python, the GIL, when to use threads for I/O and processes for CPU-bound work, with code and interview tips.
Expected Interview Answer
Multithreading runs multiple threads inside one process sharing the same memory, while multiprocessing runs separate processes each with its own memory and Python interpreter. Because of the Global Interpreter Lock (GIL), threads cannot run Python bytecode truly in parallel, so threading suits I/O-bound work and multiprocessing suits CPU-bound work.
In CPython the GIL allows only one thread to execute Python bytecode at a time, so threads are excellent for overlapping I/O waits (network, disk, database) but give little speedup for pure computation. Multiprocessing sidesteps the GIL by spawning independent processes that run on separate cores, achieving real parallelism at the cost of higher memory use and inter-process communication overhead via pickling. You pick threading for concurrency on I/O, and multiprocessing for parallelism on CPU-heavy tasks.
- Threading gives lightweight concurrency for I/O-bound tasks
- Multiprocessing achieves true CPU parallelism across cores
- Threads share memory, making data sharing cheap
- Processes are isolated, so one crash does not take down the rest
- Both are exposed through similar high-level APIs (ThreadPoolExecutor / ProcessPoolExecutor)
AI Mentor Explanation
Multithreading is one batter facing many bowlers but only one ball allowed in play at a time — quick footwork while balls arrive, yet strictly one shot per moment. Multiprocessing is fielding eleven separate players who each chase their own ball across the ground at the same instant, so real simultaneous work happens across the whole outfield.
Step-by-Step Explanation
Step 1
Classify the workload
Decide whether the bottleneck is I/O (waiting on network, disk, DB) or CPU (heavy computation).
Step 2
Understand the GIL
In CPython only one thread runs bytecode at a time, so threads do not parallelize pure computation.
Step 3
Pick threading for I/O
Use threads or ThreadPoolExecutor when tasks spend most time waiting, since the GIL is released during I/O.
Step 4
Pick multiprocessing for CPU
Use Process or ProcessPoolExecutor to run on multiple cores for compute-heavy work that ignores the GIL.
Step 5
Account for overhead
Weigh process memory cost and pickling/IPC overhead against the parallel speedup you gain.
What Interviewer Expects
- Clear grasp of the GIL and why it limits thread parallelism
- Correct mapping of I/O-bound to threads and CPU-bound to processes
- Awareness of memory sharing vs process isolation
- Knowledge of concurrent.futures executors
- Understanding of IPC and pickling overhead in multiprocessing
Common Mistakes
- Claiming Python threads run CPU work in true parallel
- Using multiprocessing for lightweight I/O and paying needless overhead
- Forgetting the GIL exists in CPython
- Ignoring that data crossing processes must be picklable
- Assuming shared global state works the same across processes as threads
Best Answer (HR Friendly)
“Multithreading lets one program juggle many waiting tasks like downloads at once, while multiprocessing runs several full copies of the work on different CPU cores for genuinely parallel number-crunching. In Python you generally use threads for waiting-heavy work and processes for heavy calculations.”
Code Example
import math
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
def fetch(url):
# I/O-bound: mostly waiting on the network
import urllib.request
with urllib.request.urlopen(url) as r:
return len(r.read())
def crunch(n):
# CPU-bound: pure computation
return sum(math.sqrt(i) for i in range(n))
urls = ['https://example.com'] * 8
with ThreadPoolExecutor(max_workers=8) as pool:
sizes = list(pool.map(fetch, urls))
with ProcessPoolExecutor(max_workers=4) as pool:
totals = list(pool.map(crunch, [10_000_00] * 4))
print(sizes, totals)Follow-up Questions
- What exactly is the GIL and why does CPython have it?
- How does asyncio compare to threading for I/O-bound work?
- When does the GIL get released during a thread's execution?
- How do processes share data safely (Queue, Pipe, shared memory)?
- Does multiprocessing help on a single-core machine?
MCQ Practice
1. Which workload benefits most from multiprocessing in CPython?
Multiprocessing bypasses the GIL and uses multiple cores, which helps CPU-bound computation the most.
2. Why can't CPython threads run Python bytecode in true parallel?
The GIL permits only one thread to execute Python bytecode at any moment, preventing true bytecode parallelism.
3. A key cost of multiprocessing compared to threading is?
Processes have isolated memory, so objects passed between them must be serialized (pickled), adding overhead.
Flash Cards
What limits Python thread parallelism? — The Global Interpreter Lock (GIL) — only one thread runs Python bytecode at a time in CPython.
Threading is best for? — I/O-bound work where tasks spend most time waiting, since the GIL is released during I/O.
Multiprocessing is best for? — CPU-bound work — separate processes run on multiple cores and are not blocked by the GIL.
Main cost of multiprocessing? — Higher memory use and IPC/pickling overhead because processes have isolated memory.