Webhooks and Event Delivery Design — API Design & Best Practices

A payment gateway knows the instant a charge succeeds; a merchant's order system finds out only if it asks. Polling for that answer every few seconds burns requests on every check that comes back "nothing yet" and still leaves a gap between the real event and the merchant noticing it. A webhook flips the direction: the merchant registers a URL in advance, and the gateway calls that URL the moment the charge succeeds, so the notification and the event happen within moments of each other instead of within one polling interval of each other.

That speed comes at a cost a function call inside one process never pays: an HTTP request over the open internet can be delivered but have its response lost, can arrive alongside a retry of itself, and can arrive out of sequence relative to another event fired moments earlier. None of that is a defect in a particular implementation — it is what sending events between two independent systems over an unreliable network actually guarantees, and a webhook design has to be built around those guarantees rather than around the guarantees a single in-process call would offer.

Analogy🏏Cricket

🏏 Think of it like cricket: A stadium's ground operations team does not make every stand's safety liaison radio in every two minutes to ask whether a weather alert has been issued. Instead, each liaison registers their radio channel with the control room in advance, and the control room calls out on that channel the moment there is something to report — rain approaching, a pitch invasion, a delayed restart — rather than waiting to be asked. This is faster for everyone: the liaison in the north stand hears about an approaching storm the second the control room knows, not whenever their next scheduled check-in happens to fall. But a radio call over a stadium's open channels cannot make the same promise a face-to-face handoff can: the call might go out while the liaison's radio is briefly out of range, and the control room, hearing no acknowledgment, has to call again — which means the liaison sometimes hears the same alert twice. Just as registering a radio channel in advance replaces repeated check-in calls with a single call made the moment something happens, a webhook replaces a consumer repeatedly polling an API with the producer calling the consumer's registered endpoint the moment an event occurs. Just as the control room's retry on a missed acknowledgment means a liaison can genuinely receive the same alert twice, a webhook producer's retry on a missed 200 response means a consumer must be built to handle a genuine duplicate delivery, not merely be surprised by one. The insight is that push-based notification over an unreliable channel trades the cost of polling for a different cost — handling duplicates and out-of-range moments — and a design that ignores that trade has not actually solved the polling problem, it has just hidden it.