STOMP/ActiveMQ communication for Ruby/Rails applications

Introduction

Using a messaging protocol to facilitate communication between different applications is nothing new, in fact it has been the defacto standard for fast and reliable communication.  It is important to note, this is a type of communication that you would usually use when communicating between different hosts. When working on a single host, there are simpler and more scalable methods that you can use (e.g. sockets).

What is needed

Let us take an example of three applications running on two different servers. To simplify this example, one application will be master and the other two will serve as slaves. Master application will provide work tasks, which will slave execute and report back progress. It seems simple enough, let us make it happen.

Message Bus, enter ActiveMQ

ActiveMQ is one of the most popular solutions for open source messaging, there are some other popular alternatives such as RabbitMQ, Zer0MQ, etc. The following examples will be less so oriented on the underlying technology, and more so on functional examples that will use STOMP. STOMP is a protocol, and as long as the messaging server supports that protocol you should be able to change it.

Installing ActiveMQ is really simple, just follow getting started guide. After ActiveMQ has been installed we will need to make some adjustments to its configuration. Communication between master and slave will be implemented via two channels, one for broadcasting messages to slaves and one for replies from mentioned slaves.

In messaging there are two major types of communication channels topics and queues. Rules of thumb would be if you would like to broadcast messages you would use topics, if you wanna process messages in ordered fashion you would use queues. For our example it would be good to broadcast messages to slaves via topic and to process their results via the queue.

Since we are clear on what to do, let us create a topic, and queue on ActiveMQ. After successfully installing ActiveMQ you can find it’s configuration in /opt/activemq/conf/activemq.xml.

 ...
  	<!-- We create user that we will use to connect to ActiveMQ -->
  	<simpleAuthenticationPlugin>
    	<users>
        	<authenticationUser username="foo" password="bar" groups="foo,everyone"/>
        </users>
    </simpleAuthenticationPlugin>
    ...
    <!-- We create topic(s) and queue(s) under namespace "myapp" -->
    <authorizationPlugin>
  		<map>
    		<authorizationMap>
      			<authorizationEntries>
        			<authorizationEntry topic="myapp.>" write="foo" read="foo" admin="foo" />
        			<authorizationEntry queue="myapp.>" write="foo" read="foo" admin="foo" />
      			</authorizationEntries>
    		</authorizationMap>
  		</map>
	</authorizationPlugin>

Change configuration by adding above snippets and restart ActiveMQ, sudo service ActiveMQ restart. The important thing to note is when we define topic/queue as myapp.> this allows us to create multiple topics/queues under that namespace e.g. myapp.reply or myapp.foo.bar.reply.

Connecting to message bus

For interaction, via STOMP we are going to use stomp gem, for a more straight forward and simple approach. As gem’s README points out this is really simple to do. I am going to assume that you are running ActiveMQ locally with default port localhost:61613 our example will reflect that.

client = Stomp::Client.new("foo", "bar", "localhost", 61613)

Pretty simple so far now let us publish a message from our master node to topic myapp.work.

	original_message   = { :foo => 'test', :bar => { :test => [1, 2, 3, 4]}}
	marshalled_message = Marshal.dump(original_message)
	client.publish('myapp.work', marshalled_message)

On the slave side we need to listen for these messages, or subscribe to them so we do that by initializing client and subscribing.

    client = Stomp::Client.new("foo", "bar", "localhost", 61613)
    client.subscribe('myapp.work') do |marshalled_message|
    	original_message = Marshal.load(marshalled_message)
    	# process message
   	end

That is all functional pieces that we need to create more complex applications. There is a convention that we in ABH use and that is replying to messages to queues with reply suffix. For this master and slaves relationship, or slaves would reply to queue /myapp/reply and master node would have the responsibility to handle these messages.

Note

Keep in mind that demonstrated solution had blocking properties, and as such might not be the best solution for Rails stack. For Rails I would recommend EventMachine based STOMP client. If enough interest has been shown I can write an article on how to implement this solution in existing Rails applications.

Keeping track of sent messages

One important detail that should be included is request_id for messages that will be going between master and slaves. That is the simple way to keep track of jobs that were dispatched to slaves and their results. You can create UUID by using:

	require 'securerandom'
	request_id = SecureRandom.uuid

That way when you receive a response on queue /myapp/reply you can match it to sent request.

Conclusion

Hopefully, this will be enough to get you started. This article can be easily extended by previously mentioned reactor based EM STOMP client or by adding security to message since if SSL is not used these will be easy to read if intercepted. If enough interest is shown, will expend this topic more.