Message Throughput in RabbitMQ Bigwig

There are times when RabbitMQ cannot cope with the rate or volume of incoming messages, for example, when the rate is faster than it can write to disk, or the volume exceeds the available memory. In these cases, the only method that RabbitMQ has to regulate the incoming flow is to exert TCP back-pressure on the connection.

While effective, it has a couple of drawbacks.

Firstly, we have noticed that a connection’s message rate might “jitter” as RabbitMQ exerts TCP back-pressure to alleviate a problem, and then a producer races to publish as quick as it can causing the broker to exert back pressure again, in a repeating cycle. Our experience has shown that consistent message throughput makes for a smoother running message broker, and a more reliable experience for the client.

Therefore, we apply flow control to connections to shape traffic and minimise the chances of RabbitMQ using flow control, giving you consistent throughput. This also ensures a fair use of resources amongst users on shared infrastructure.

Secondly, it cannot distinguish between connections publishing data to the broker (and therefore making the situation worse), and connections consuming data (and therefore potentially making the situation better). In AMQP, publishing a message usually consists of a client sending a basic.publish and the broker replying with a basic.ack. Conversely, a client consumes a message in one of two ways: either the broker delivers a message because of a subscription, or the client requests a message by sending a basic.get. In either case, you still have a pair of messages sent, one in each direction. As a result, there is no way at the TCP level to distinguish producers and consumers.

To alleviate this, we give you two URLs — RABBITMQ_BIGWIG_TX_URL and RABBITMQ_BIGWIG_RX_URL — which you should use for publishing and consuming respectively. We shape the ports associated with the two URLs differently. We shape the RABBITMQ_BIGWIG_TX_URL port to allow a high rate of inbound traffic (but not high enough to trigger RabbitMQ’s flow control). We tightly throttle the inbound rate on the RABBITMQ_BIGWIG_RX_URL port to allow you to consume messages as fast as you want, while allowing just enough traffic for your consumers to acknowledge message receipt.

Of course, there is nothing stopping you using these URLs any way you want, but you will not get optimum throughput.