Pub/Sub Architecture

Publish-subscribe (pub/sub) is a messaging pattern in which publishers send messages to a named channel or topic without any knowledge of who, if anyone, is listening, and subscribers express interest in a topic without any knowledge of who is publishing to it. This is a fundamentally different relationship than a direct queue between one producer and one consumer: pub/sub is built for fan-out, where a single event may need to reach zero, one, or many independent subscribers, and new subscribers can be added later without changing the publisher at all. This decoupling in both direction (who) and cardinality (how many) is what makes pub/sub the natural backbone for event-driven architectures, where many independently-deployed services all want to react to the same underlying business events.

In most pub/sub systems (Google Cloud Pub/Sub, AWS SNS, or Kafka topics with multiple consumer groups), a publisher writes a message to a topic, and the messaging system fans that single message out to every active subscription on that topic — each subscription gets its own independent copy or offset into the stream, so one subscriber being slow or behind does not affect any other subscriber. This is distinct from a plain queue's competing-consumers model, where multiple consumers reading the same queue split the work among themselves rather than each receiving every message. Pub/sub systems often layer filtering on top of fan-out, letting subscribers specify attributes or patterns (e.g., 'only orders over $100' or 'only events from region=EU') so they receive a relevant subset of a topic's traffic rather than the full firehose. A common design mistake is relying purely on this client-side filtering against one undifferentiated topic, since it wastes bandwidth and couples every subscriber to a single evolving schema; it's usually better to define separate topics around meaningful, versioned business events (e.g., 'order.placed' vs. 'order.cancelled') so subscribers can select at the topic level and schema changes don't silently break unrelated consumers.

Pub/sub subscribers can receive messages via push, where the messaging system actively calls a subscriber's endpoint (e.g., an HTTP webhook) as messages arrive, or pull, where the subscriber's own process actively polls the system for new messages when it's ready to handle them. Push delivery minimizes latency and simplifies the subscriber (no polling loop needed) but requires the subscriber to expose a reachable endpoint and to handle backpressure carefully, since the publisher-side system controls the delivery rate. Pull delivery puts the consumer in control of its own processing rate, which is usually preferable for high-throughput or batch-oriented consumers that want to control concurrency and avoid being overwhelmed by a push-based flood, at the cost of slightly higher latency and the operational overhead of running a polling consumer process.

Most pub/sub systems provide at-least-once delivery per subscription and make no strong ordering guarantee across an entire topic by default, especially once a topic is partitioned or sharded for scale — messages published close together in time may be delivered to a given subscriber out of order, and the same message may be redelivered if an acknowledgment is lost. Systems that need ordering (e.g., Kafka, or Google Cloud Pub/Sub's ordering keys) typically only guarantee order within a partition or ordering key, meaning you must deliberately route causally related messages (e.g., all events for the same order ID) to the same partition/key if you need them processed in order, while accepting no cross-key ordering guarantee.