Using a messaging protocol to facilitate communication between different applications is nothing new, in fact it has been the defacto standard for fast and reliable communication.  It is important to note, this is a type of communication that you would usually use when communicating between different hosts. When working on a single host, there are simpler and more scalable methods that you can use (e.g. sockets).

What is needed

Let us take an example of three applications running on two different servers. To simplify this example, one application will be master and the other two will serve as slaves. Master application will provide work tasks, which will slave execute and report back progress. It seems simple enough, let us make it happen.


Message Bus, enter ActiveMQ

ActiveMQ is one of the most popular solutions for open source messaging, there are some other popular alternatives such as RabbitMQ, Zer0MQ, etc. The following examples will be less so oriented on the underlying technology, and more so on functional examples that will use STOMP. STOMP is a protocol, and as long as the messaging server supports that protocol you should be able to change it.

Installing ActiveMQ is really simple, just follow getting started guide. After ActiveMQ has been installed we will need to make some adjustments to its configuration. Communication between master and slave will be implemented via two channels, one for broadcasting messages to slaves and one for replies from mentioned slaves.

In messaging there are two major types of communication channels topics and queues. Rules of thumb would be if you would like to broadcast messages you would use topics, if you wanna process messages in ordered fashion you would use queues. For our example it would be good to broadcast messages to slaves via topic and to process their results via the queue.

Since we are clear on what to do, let us create a topic, and queue on ActiveMQ. After successfully installing ActiveMQ you can find it’s configuration in /opt/activemq/conf/activemq.xml.

  	<!-- We create user that we will use to connect to ActiveMQ -->
        	<authenticationUser username="foo" password="bar" groups="foo,everyone"/>
    <!-- We create topic(s) and queue(s) under namespace "myapp" -->
        			<authorizationEntry topic="myapp.>" write="foo" read="foo" admin="foo" />
        			<authorizationEntry queue="myapp.>" write="foo" read="foo" admin="foo" />

Change configuration by adding above snippets and restart ActiveMQ, sudo service ActiveMQ restart. The important thing to note is when we define topic/queue as myapp.> this allows us to create multiple topics/queues under that namespace e.g. myapp.reply or

Connecting to message bus

For interaction, via STOMP we are going to use stomp gem, for a more straight forward and simple approach. As gem’s README points out this is really simple to do. I am going to assume that you are running ActiveMQ locally with default port localhost:61613 our example will reflect that.

client ="foo", "bar", "localhost", 61613)

Pretty simple so far now let us publish a message from our master node to topic

	original_message   = { :foo => 'test', :bar => { :test => [1, 2, 3, 4]}}
	marshalled_message = Marshal.dump(original_message)
	client.publish('', marshalled_message)

On the slave side we need to listen for these messages, or subscribe to them so we do that by initializing client and subscribing.

    client ="foo", "bar", "localhost", 61613)
    client.subscribe('') do |marshalled_message|
    	original_message = Marshal.load(marshalled_message)
    	# process message

That is all functional pieces that we need to create more complex applications. There is a convention that we in ABH use and that is replying to messages to queues with reply suffix. For this master and slaves relationship, or slaves would reply to queue /myapp/reply and master node would have the responsibility to handle these messages.


Keep in mind that demonstrated solution had blocking properties, and as such might not be the best solution for Rails stack. For Rails I would recommend EventMachine based STOMP client. If enough interest has been shown I can write an article on how to implement this solution in existing Rails applications.

Keeping track of sent messages

One important detail that should be included is request_id for messages that will be going between master and slaves. That is the simple way to keep track of jobs that were dispatched to slaves and their results. You can create UUID by using:

	require 'securerandom'
	request_id = SecureRandom.uuid

That way when you receive a response on queue /myapp/reply you can match it to sent request.


Hopefully, this will be enough to get you started. This article can be easily extended by previously mentioned reactor based EM STOMP client or by adding security to message since if SSL is not used these will be easy to read if intercepted. If enough interest is shown, will expend this topic more.

Software DevelopmentTech Bites
February 23, 2024

Background Jobs in Elixir – Oban

When and why do we need background jobs? Nowadays, background job processing is indispensable in the world of web development. The need for background jobs stems from the fact that synchronous execution of time-consuming and resource-intensive tasks would heavily impact an application's  performance and user experience.  Even though Elixir is…
Software DevelopmentTech Bites
December 22, 2023

In-memory Caching using Redis

The importance of computer memory utilization The CPU and memory are the main components of any computer system. Computer memory stores data and program instructions, temporarily or permanently, that the CPU processes. In CPU-intensive applications with large amounts of data being processed, memory usually becomes the bottleneck, resulting in a…
JavaScript Event Loop
Software DevelopmentTech Bites
December 21, 2023

JavaScript Event Loop

Often, when switching to JavaScript from any other language, developers face many frustrations. That’s because JavaScript has some unique features that may not be obvious or intuitive to those who have not used it before. When we first start learning JavaScript, we are told it executes the code line-by-line. console.log("Beginning");…

Want to discuss this in relation to your project? Get in touch:

One Comment

  • luqmaan s says:

    In the article, you mentioned that the demonstrated solution using STOMP has blocking properties, which might not be the best fit for Rails stack. Could you elaborate on why this blocking behavior could be problematic for Rails applications, and how an EventMachine-based STOMP client can help address this issue?

Leave a Reply