Python Multithreading & Multiprocessing Cheat Sheet
Concurrent Python covering threading basics, locks, executor pools, multiprocessing, and when to choose threads versus processes.
2 PagesAdvancedApr 8, 2026
Basic Threading
Starting and joining threads with the threading module.
python
import threadingdef worker(n): print(f"Worker {n} running")threads = []for i in range(3): t = threading.Thread(target=worker, args=(i,)) t.start() threads.append(t)for t in threads: t.join() # wait for all threads to finish
Locks & Synchronization
Protecting shared state from race conditions.
python
lock = threading.Lock()counter = 0def increment(): global counter with lock: # only one thread at a time in this block counter += 1threads = [threading.Thread(target=increment) for _ in range(100)]for t in threads: t.start()for t in threads: t.join()
ThreadPoolExecutor / ProcessPoolExecutor
A higher-level API for managing worker pools.
python
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutordef square(n): return n * nwith ThreadPoolExecutor(max_workers=4) as executor: results = list(executor.map(square, range(10)))with ProcessPoolExecutor(max_workers=4) as executor: futures = [executor.submit(square, i) for i in range(10)] results = [f.result() for f in futures]
Multiprocessing with Pool
Running CPU-bound work in separate processes to bypass the GIL.
python
from multiprocessing import Pooldef square(n): return n * nif __name__ == "__main__": # required on Windows/macOS spawn start method with Pool(processes=4) as pool: results = pool.map(square, range(10)) print(results)
Threading vs Multiprocessing
Choosing the right concurrency model.
- GIL (Global Interpreter Lock)- allows only one thread to execute Python bytecode at a time in CPython
- threading- best for I/O-bound work (network, disk) where threads spend time waiting
- multiprocessing- best for CPU-bound work since each process has its own interpreter and GIL
- Shared memory- threads share memory directly; processes need explicit IPC (Queue, Pipe, Value)
- multiprocessing.Queue- process-safe queue for passing data between processes
- multiprocessing.Manager- provides shared, synchronized objects like dicts and lists across processes
Producer/Consumer with queue.Queue
The standard thread-safe way to hand work off between threads without manual locking.
python
import threading, queueq = queue.Queue(maxsize=10)def producer(): for i in range(20): q.put(i) # blocks if the queue is full q.put(None) # sentinel to signal completiondef consumer(): while True: item = q.get() if item is None: break print(f"processing {item}") q.task_done()threading.Thread(target=producer).start()threading.Thread(target=consumer).start()
as_completed() with Timeouts & Cancellation
Processing futures as they finish rather than waiting for all of them in submission order.
python
from concurrent.futures import ThreadPoolExecutor, as_completed, TimeoutErrordef fetch(url): ...urls = ["https://a", "https://b", "https://c"]with ThreadPoolExecutor(max_workers=3) as executor: futures = {executor.submit(fetch, u): u for u in urls} try: for future in as_completed(futures, timeout=5): url = futures[future] try: print(url, future.result()) except Exception as e: print(url, "failed:", e) except TimeoutError: for f in futures: f.cancel() # only cancels futures that haven't started
Coordinating Threads with Condition
threading.Condition lets one thread wait until another signals a state change, avoiding busy-polling.
python
import threadingcondition = threading.Condition()buffer = []def producer(): with condition: buffer.append("item") condition.notify() # wake one waiting consumerdef consumer(): with condition: while not buffer: condition.wait() # releases the lock while waiting item = buffer.pop() print(item)
Advanced Concurrency Primitives
Building blocks beyond basic locks and pools.
- RLock (reentrant lock)- can be acquired multiple times by the same thread, useful for recursive functions that need locking
- Semaphore- limits how many threads can access a resource concurrently, e.g. capping simultaneous DB connections
- Barrier- blocks a fixed number of threads until they all reach the barrier point, then releases them together
- multiprocessing.Manager()- provides proxy objects (dict, list) that multiple processes can share and mutate safely, at higher overhead than shared_memory
- daemon threads- threads with `daemon=True` are killed abruptly when the main program exits, without running cleanup code
- GIL / free-threaded builds- CPython 3.13+ offers an experimental free-threaded build (PEP 703) that removes the GIL, making true CPU-bound threading finally viable
- deadlock- occurs when two threads each hold a lock the other needs; avoid by always acquiring multiple locks in the same global order
Pro Tip
Don't reach for multiprocessing by default — process creation and inter-process communication have real overhead. Profile first: use threading or asyncio for I/O-bound bottlenecks, and reserve multiprocessing for genuinely CPU-bound work that threading can't speed up due to the GIL.
Was this cheat sheet helpful?
Explore Topics
#PythonMultithreadingMultiprocessing#PythonMultithreadingMultiprocessingCheatSheet#Programming#Advanced#BasicThreading#LocksSynchronization#ThreadPoolExecutorProcessPoolExecutor#MultiprocessingWithPool#Concurrency#CheatSheet#SkillVeris
