GCP Pub/Sub Cheat Sheet
Topics, subscriptions, gcloud commands, and client library snippets for asynchronous messaging on Google Cloud Pub/Sub.
Create Topic & Subscription
Provision a topic and a pull subscription with gcloud.
gcloud pubsub topics create orders-topicgcloud pubsub subscriptions create orders-sub \ --topic=orders-topic \ --ack-deadline=30 \ --message-retention-duration=7d# Push subscription instead of pullgcloud pubsub subscriptions create orders-push-sub \ --topic=orders-topic \ --push-endpoint=https://api.example.com/pubsub/push
Publish a Message (Python)
Publish JSON data with attributes using the google-cloud-pubsub client.
from google.cloud import pubsub_v1import jsonpublisher = pubsub_v1.PublisherClient()topic_path = publisher.topic_path("my-project", "orders-topic")data = json.dumps({"orderId": "o-123"}).encode("utf-8")future = publisher.publish(topic_path, data, priority="high")print(future.result()) # message ID
Pull & Ack Messages (Python)
Synchronously pull messages and acknowledge them after processing.
from google.cloud import pubsub_v1subscriber = pubsub_v1.SubscriberClient()sub_path = subscriber.subscription_path("my-project", "orders-sub")def callback(message): print(message.data.decode("utf-8")) message.ack()future = subscriber.subscribe(sub_path, callback=callback)try: future.result(timeout=60)except TimeoutError: future.cancel()
Core Concepts
Key Pub/Sub terms and delivery guarantees.
- Topic- named resource publishers send messages to
- Subscription- pull or push delivery channel attached to a topic
- Ack deadline- window to acknowledge before Pub/Sub redelivers (default 10s)
- At-least-once delivery- consumers must be idempotent; duplicates are possible
- Dead-letter topic- routes messages after max delivery attempts are exceeded
- Ordering keys- guarantee per-key ordering when enabled on the subscription
Publish with Ordering Keys
Guarantee per-key delivery order by setting an ordering key and enabling message ordering on the subscription.
from google.cloud import pubsub_v1publisher = pubsub_v1.PublisherClient( publisher_options=pubsub_v1.types.PublisherOptions(enable_message_ordering=True))topic_path = publisher.topic_path("my-project", "orders-topic")for event in events: future = publisher.publish( topic_path, data=event.payload, ordering_key=event.customer_id, # same key => same order ) future.add_done_callback(lambda f: print(f.result()))# On the subscription side:# gcloud pubsub subscriptions update orders-sub --enable-message-ordering
Flow-Controlled Streaming Pull
Bound in-flight messages per client to avoid overwhelming downstream consumers under bursty load.
from google.cloud import pubsub_v1subscriber = pubsub_v1.SubscriberClient()sub_path = subscriber.subscription_path("my-project", "orders-sub")flow_control = pubsub_v1.types.FlowControl( max_messages=200, max_bytes=50 * 1024 * 1024, max_lease_duration=600,)def callback(message): try: process(message.data) message.ack() except Exception: message.nack() # redeliver instead of losing itstreaming_pull = subscriber.subscribe( sub_path, callback=callback, flow_control=flow_control)
Configure a Dead-Letter Topic
Route messages to a dead-letter topic after repeated nack/timeout, with IAM bindings the service agent needs.
gcloud pubsub topics create orders-dlqgcloud pubsub subscriptions update orders-sub \ --dead-letter-topic=orders-dlq \ --max-delivery-attempts=5# Pub/Sub's service account must be allowed to publish/ack on your behalfgcloud pubsub topics add-iam-policy-binding orders-dlq \ --member="serviceAccount:[email protected]" \ --role="roles/pubsub.publisher"gcloud pubsub subscriptions add-iam-policy-binding orders-sub \ --member="serviceAccount:[email protected]" \ --role="roles/pubsub.subscriber"
Exactly-Once Delivery Subscription
Create a subscription with exactly-once delivery so successful acks are guaranteed not to be redelivered.
gcloud pubsub subscriptions create orders-eos-sub \ --topic=orders-topic \ --enable-exactly-once-delivery# Client must check the ack result, since ack() can now fail:# ack_future = message.ack_with_response()# result = ack_future.result(timeout=20) # raises AcknowledgeError on failure
Schemas & Scaling Knobs
Options that matter once Pub/Sub moves from prototype to a production event backbone.
- Schema validation- attach an Avro/Protobuf schema to a topic to reject malformed publishes at the API
- BigQuery subscriptions- write messages directly into a BigQuery table with no consumer code
- Cloud Storage subscriptions- batch messages into GCS objects on a schedule, no consumer needed
- Filter on subscription- gcloud subscriptions create --filter='attributes.priority="high"' pushes filtering server-side
- Seek- rewind or fast-forward a subscription to a timestamp or snapshot for replay/testing
- Snapshots- capture a subscription's backlog position to seek back to later
- Regional endpoints- pin publish/subscribe traffic to a region for data residency requirements
Set `message-retention-duration` and enable `--enable-message-ordering` only on subscriptions that need replay or per-key ordering — both features add latency and cost that most fire-and-forget event flows don't need.