Producers and Consumers Basics
A producer is any application that publishes messages, using basic.publish on a channel, to an exchange with a routing key. A consumer is any application that subscribes to a queue, either by actively pulling with basic.get (rarely used, inefficient for steady traffic) or, far more commonly, by registering a subscription with basic.consume, after which RabbitMQ pushes messages to the consumer as they arrive, calling a callback function for each one.
Cricket analogy: Like a bowler (producer) delivering a ball down the pitch to wherever the field placement (routing key) sends it, and a fielder (consumer) subscribed to that zone reacting each time the ball arrives, rather than checking every few seconds if a ball landed.
Publishing a Message
A basic publish call takes an exchange name, a routing key, a message body (raw bytes), and optional properties such as delivery_mode, content_type, and correlation_id. Setting delivery_mode=2 marks the message persistent, meaning RabbitMQ writes it to disk so it survives a broker restart, but persistence alone is not enough: the queue itself must also be declared durable=True, and the exchange durable too, or the routing path disappears on restart even if individual messages were marked persistent.
Cricket analogy: Like marking a delivery for DRS review (delivery_mode=2) only mattering if the review system itself (durable queue) is actually switched on for that match; a marked delivery reviewed by a system that isn't running achieves nothing.
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
# Publisher side
channel.exchange_declare(exchange='orders', exchange_type='direct', durable=True)
channel.queue_declare(queue='order_processing', durable=True)
channel.queue_bind(exchange='orders', queue='order_processing', routing_key='order.new')
channel.basic_publish(
exchange='orders',
routing_key='order.new',
body=b'{"order_id": 501, "item": "widget"}',
properties=pika.BasicProperties(delivery_mode=2, content_type='application/json')
)
# Consumer side
def callback(ch, method, properties, body):
print(f"Received: {body}")
ch.basic_ack(delivery_tag=method.delivery_tag)
channel.basic_qos(prefetch_count=10)
channel.basic_consume(queue='order_processing', on_message_callback=callback)
channel.start_consuming()basic_qos(prefetch_count=N) limits how many unacknowledged messages RabbitMQ will push to a consumer at once. Without a prefetch limit, a fast producer and a slow consumer can result in RabbitMQ dumping thousands of unacked messages into one consumer's memory, defeating the purpose of load distribution across multiple consumer instances.
Acknowledgment and Prefetch in Practice
When multiple consumers subscribe to the same queue, RabbitMQ distributes messages between them round-robin by default, a pattern often called 'competing consumers,' which is how you scale processing horizontally: add more consumer instances and throughput increases without any code change. Setting prefetch_count=1 combined with manual acknowledgment ensures each consumer only gets a new message once it has acked the previous one, which is the standard way to get fair work distribution when tasks take varying amounts of time to process, rather than one consumer being handed a large batch upfront while another sits idle.
Cricket analogy: Like a captain rotating bowlers over-by-over based on who's ready to bowl next rather than assigning a whole fixed spell upfront, prefetch_count=1 hands each consumer one new task only once it's finished the last.
Forgetting to call basic_ack (or setting auto_ack=True carelessly) means a consumer that crashes mid-processing will have already told RabbitMQ the message was handled, so it is never redelivered and the work is silently lost. Prefer manual acknowledgment for anything where losing a message matters, and only ack after processing has genuinely succeeded (e.g., after a database write commits, not before).
- Producers publish via
basic.publishto an exchange with a routing key, never directly to a queue. - Consumers typically subscribe with
basic.consumefor push delivery rather than polling withbasic.get. - Persistent messages need
delivery_mode=2AND a durable queue/exchange to survive a broker restart. basic_qos(prefetch_count=N)caps how many unacked messages a consumer can hold at once.- Multiple consumers on one queue compete for messages, enabling horizontal scaling of processing.
prefetch_count=1with manual ack gives fair, load-based work distribution across consumers.- Only ack after processing has genuinely succeeded, or crashed work can be silently lost with auto_ack.
Practice what you learned
1. What method does a producer typically use to send a message in RabbitMQ?
2. What two things are BOTH required for a message to survive a broker restart?
3. What does `basic_qos(prefetch_count=N)` control?
4. What happens when multiple consumers subscribe to the same queue?
5. Why is careless use of auto_ack=True risky?
Was this page helpful?
You May Also Like
What Is RabbitMQ?
An introduction to RabbitMQ as a message broker that decouples producers and consumers using exchanges, queues, and bindings.
AMQP Protocol Explained
A deep dive into the Advanced Message Queuing Protocol (AMQP) and the model RabbitMQ implements: connections, channels, exchanges, queues, and bindings.
The RabbitMQ Management UI
A tour of the RabbitMQ management plugin's web UI for inspecting queues, exchanges, connections, and managing users and permissions.