rabbitmq-tutorials/python/tutorial-three.mdx

Learning RabbitMQ, part 3 (Broadcast)
=====================================

{% dot -Gsize="10,1.3" -Grankdir=LR %}
digraph G {
    P1 [label=<P>, {{ dotstyle.producer }}];
    X [label="X", {{ dotstyle.exchange }}];
    Q1 [label="{<s>||||<e>}", {{ dotstyle.queue }}];
    Q2 [label="{<s>||||<e>}", {{ dotstyle.queue }}];
    C1 [label=<C<font point-size="7">1</font>>, {{ dotstyle.consumer }}];
    C2 [label=<C<font point-size="7">2</font>>, {{ dotstyle.consumer }}];

    P1 -> X;
    X -> Q1;
    X -> Q2;
    Q1 -> C1;
    Q2 -> C2;
}
{% enddot %}

In [previous part]({{ python_two_url}}) of this tutorial we created
a task queue. The core assumption behind a task queue is that a task
is delivered to exactly one worker. In this part we'll do something completely
different - we'll try to deliver a message to multiple consumers. This
pattern is known as "publish-subscribe".

To illustrate this, we're going to build a simple
logging system. It will consist of two programs - the first will emit log
messages and the second will receive and print them.

In our logging system every running copy of the receiver program
will get the same messages. That way we'll be able to run one
receiver and direct the logs to disk; and at the same time we'll be able to run
another receiver and see the same logs on the screen.

Essentially, emitted log messages are going to be broadcasted to all
the receivers.


Exchanges
---------

In previous parts of the tutorial we've understood how to send and
receive messages to and from a queue. Now it's time to introduce
the full messaging model in Rabbit.

Let's quickly cover what we've learned:

 * A _producer_ is user application that sends messages.
 * A _queue_ is a buffer that stores messages.
 * A _consumer_ is user application that receives messages.


The core idea in the messaging model in Rabbit is that the
producer never sends any messages directly to the queue. Actually,
quite often the producer doesn't even know if a message will be
delivered to any queue at all!

Instead, the producer can only send messages to an _exchange_. An
exchange is a very simple thing. On one side it receives messages from
producers and the other side it pushes them to queues. The exchange
must know exactly what to do with a received message. Should it be
appended to a particular queue? Should it be appended to many queues?
Or should it get discarded. The exact rules for that are
defined by the _exchange type_.

{% dot -Gsize="10,1.3" -Grankdir=LR %}
digraph G {
    P1 [label=<P>, {{ dotstyle.producer }}];
    X [label="X", {{ dotstyle.exchange }}];
    Q1 [label="{<s>||||<e>}", {{ dotstyle.queue }}];
    Q2 [label="{<s>||||<e>}", {{ dotstyle.queue }}];

    P1 -> X;
    X -> Q1;
    X -> Q2;
}
{% enddot %}


There are a few exchange types available: `direct`, `topic`,
`headers` and `fanout`. We'll focus on the last one - the
fanout. Let's create an exchange of that type, and call it `logs`:

{% highlight python %}
    channel.exchange_declare(exchange='logs',
                             type='fanout')
{% endhighlight %}

The fanout exchange is very simple. As you can probably guess from the
name, it just broadcasts all the messages it receives to all the
queues it knows. And that's exactly what we need for our logger.


> #### Listing exchanges
>
> To list the exchanges on the server you can once again use the
> Swiss Army Knife - `rabbitmqctl`:
>
>     $ sudo rabbitmqctl list_exchanges
>     Listing exchanges ...
>     logs      fanout
>     amq.direct      direct
>     amq.topic       topic
>     amq.fanout      fanout
>     amq.headers     headers
>     ...done.
>
> You can see a few `amq.` exchanges. They're created by default, but
> chances are you'll never need to use them.

<div></div>

> #### Nameless exchange
>
> In previous parts of the tutorial we knew nothing about exchanges,
> but still were able to send messages to queues. That was possible
> because we were using a default `""` _empty string_ (nameless) exchange.
> Remember how publishing worked:
>
>     channel.basic_publish(exchange='',
>                           routing_key='test',
>                           body=message)
>
> The _empty string_ exchange is special: messages are
> routed to the queue with name specified by `routing_key`.



Temporary queues
----------------

As you may remember previously we were using queues which had a specified name
(remember `test` and `task_queue`?). Being able to name a
queue was crucial for us - we needed to point the workers to the same
queue.  Essentially, giving a queue a name is important when you
want to share the queue between multiple consumers.

But that's not true for our logger. We do want to hear about all
currently flowing log messages, we do not want to receive only a subset
of messages. We're also interested only in currently flowing messages
not in the old ones. To solve that we need two things.

First, whenever we connect to Rabbit we need a fresh, empty queue. To do it
we could create a queue with a random name, or, even better - let server
choose a random queue name for us. We can do it by not supplying the
`queue` parameter to `queue_declare`:

{% highlight python %}
    result = channel.queue_declare()
{% endhighlight %}

At that point `result.queue` contains a random queue name. For example it
may look like `amq.gen-U0srCoW8TsaXjNh73pnVAw==`.

Secondly, once we disconnect the client the queue should be
deleted. There's an `auto_delete` flag for that:

{% highlight python %}
    result = channel.queue_declare(auto_delete=True)
{% endhighlight %}


Bindings
--------

{% dot -Gsize="10,1.0" -Grankdir=LR %}
digraph G {
    P1 [label=<P>, {{ dotstyle.producer }}];
    X [label="X", {{ dotstyle.exchange }}];
    Q1 [label="{<s>||||<e>}", {{ dotstyle.queue }}];
    Q2 [label="{<s>||||<e>}", {{ dotstyle.queue }}];

    P1 -> X;
    X -> Q1 [label="binding"];
    X -> Q2 [label="binding"];
}
{% enddot %}


We've already created a fanout exchange and a queue. Now we need to
tell the exchange to send messages to our queue. That relationship
between exchange and a queue is called a _binding_.

{% highlight python %}
    channel.queue_bind(exchange='logs',
                       queue=result.queue)
{% endhighlight %}

From now on the `logs` exchange will broadcast all the messages also to
our queue.

> #### Listing bindings
>
> You can list existing bindings using, you guessed it,
> `rabbitmqctl list_bindings`.


Putting it all together
-----------------------

{% dot -Gsize="10,1.8" -Grankdir=LR %}
digraph G {
    P [label=<P>, {{ dotstyle.producer }}];
    X [label=<logs>, {{ dotstyle.exchange }}];
    subgraph cluster_Q1 {
        label="amq.gen-RQ6...";
	color=transparent;
        Q1 [label="{<s>||||<e>}", {{ dotstyle.queue }}];
    };
    subgraph cluster_Q2 {
        label="amq.gen-As8...";
	color=transparent;
        Q2 [label="{<s>||||<e>}", {{ dotstyle.queue }}];
    };
    C1 [label=<C<font point-size="7">1</font>>, {{ dotstyle.consumer }}];
    C2 [label=<C<font point-size="7">2</font>>, {{ dotstyle.consumer }}];

    P -> X;
    X -> Q1;
    X -> Q2;
    Q1 -> C1;
    Q2 -> C2;
}
{% enddot %}

The producer program, which emits log messages, doesn't look much
different than in previous tutorial. The most important change is
that, we now need to publish messages to the `logs` exchange instead of
the nameless one. We need to supply a `routing_key`, but
its value is ignored for `fanout` exchanges. Here goes the code for
`emit_log.py` script:

{% highlight python linenos=true %}
    #!/usr/bin/env python
    import pika
    import sys

    connection = pika.AsyncoreConnection(pika.ConnectionParameters(
            host='127.0.0.1',
            credentials=pika.PlainCredentials('guest', 'guest')))
    channel = connection.channel()

    message = ' '.join(sys.argv[1:]) or "info: Hello World!"
    channel.basic_publish(exchange='logs',
                          routing_key='',
                          body=message)
    print " [x] Sent %r" % (message,)
{% endhighlight %}
[(emit_log.py source)]({{ examples_url }}/python/emit_log.py)

As you see, we avoided declaring exchange here. If the `logs` exchange
isn't created at the time this code is executed the message will be
lost. That's okay for us - if no consumer is listening yet (ie:
the exchange hasn't been created) we can discard the message.

The code for `receive_logs.py`:

{% highlight python linenos=true %}
    #!/usr/bin/env python
    import pika

    connection = pika.AsyncoreConnection(pika.ConnectionParameters(
            host='127.0.0.1',
            credentials=pika.PlainCredentials('guest', 'guest')))
    channel = connection.channel()

    channel.exchange_declare(exchange='logs',
                             type='fanout')

    result = channel.queue_declare(auto_delete=True)
    queue_name = result.queue

    channel.queue_bind(exchange='logs',
                       queue=queue_name)

    print ' [*] Waiting for logs. To exit press CTRL+C'

    def callback(ch, method, header, body):
        print " [x] %r" % (body,)

    channel.basic_consume(callback,
                          queue=queue_name,
                          no_ack=True)

    pika.asyncore_loop()
{% endhighlight %}
[(receive_logs.py source)]({{ examples_url }}/python/receive_logs.py)


We're done. If you want to save logs to a file, just open a console and type:

    $ ./receive_logs.py > logs_from_rabbit.log

If you wish to see the logs on your screen, spawn a new terminal and run:

    $ ./receive_logs.py

And of course, to emit logs type:

    $ ./emit_log.py