BlogQA/Test Automation

Setting up Owl – test results reporting tool

By December 11, 2017 No Comments

Getting started with Owl

This step by step guide will help you to successfully install and enable our Owl tool, used for test results reporting and presentation. In this particular case, we will be installing Owl for presenting tests, written in Ruby and Rspec (testing tool for Ruby), created for behaviour driven development.

For purposes of this tutorial, we used a clean Amazon EC2 instance running Ubuntu 16.04.

Prerequisites

The first Owl prerequisite is that you need to have PostgreSQL database installed. Owl currently supports only PostgreSQL databases. The database needs to be started on the same machine where Owl will be installed, further in the text referred to as “localhost”. So basically, we will only need to create our database and then later let the Owl create a schema that will be used. For these purposes, we created an additional role named ‘owluser’ and a database named ‘owldbtest’. You can do this using following commands:

Login to postgres with psql client:

Log in to prompt interface with:

(Or use following command to login with custom postgres role:

Create new database with:

 In order to use Owl with Rspec, you need to include ‘rspec2db’ gem in your tests. This gem is a database formatter which enables writing RSpec test results into our database. See Rspec2db GitHub repo for more information: https://github.com/ATLANTBH/rspec.

In case you already used rspec2db and now you want to integrate it with Owl, you will need to run alter migration that will alter the existing tables and add the needed columns for Owl. You can read more about this approach at the end of the article.

To run the Owl application, Java (JRE) needs to be installed (You’ll have to set JAVA_HOME environment variable too). And also, to build the application, there should be Maven (>= v3) installed.

Setup the Rspec project

Create new directory for rspec project, and initialise it with:

After project initialisation, create config directory in your project and add rspec2db.yml file with following content populated:

Open .rspec file in your project and make sure it contains following lines:

Add a Gemfile to your project and add following gems:

Run 

Run RSpec tests using the following command (from the location where .rspec file exist, to be able to pick up parameters defined in .rspec):

  (Of course, you’ll need to have some rspec tests in your spec folder to do this (See http://rspec.info/ to start writing tests in rspec).)

 

Install and start the Owl

Clone Owl repository:

Inside cloned directory, run:

This will create target/owl-VERSION.jar file, and we will copy it to different location on our ec2 host, along with /src/main/resources/application.properties file to start it locally.

Open the application.properties file where we will edit our database configuration and other properties. We need to change the database name, username and password in the database properties, to match our properties. In addition, you can edit project name, test suite names or toggle specific features to suit your test suite. You can enable git information on the dashboard to show the exact commit on which the test was executed. There is also a bug tracking feature, which adds additional column to test steps, so you can add error description and i.e. link to your JIRA ticket. At the end, application.properties file should look something like this:

Note: Server port from needs to be enabled for listening, so application could be reached. In our case we will be adding Custom TCP Rule in Security Groups on our Amazon EC2 instance. (Screenshot 1)

Start the application with your configuration by running:

You should start the process with nohup, so it won’t be killed when you logout:

To be sure that port is enabled and running our app, run    and you should get something like this:

So that’s it! At this point, the application should be running on port 8090, or on your custom server port. Every test execution will be written to the database, and you can easily access it, formatted and segmented.

Once you open the Owl, you can navigate to your particular test suite:

You can see  the graph for ‘Amazon Regression’ test suite with success percentage and complete duration of each test suite. The X axis represents build ID of the test suite (specified in the rspec2db.yml file), and Y axis represents the success rate:

Opening the particular build, you can see all specs executed in this particular test run:

From here you can open a single test and see all the test steps. You can see the description and the expected result, if it passed or not, as well as the duration of each step:

When the step fails, you can see the exception by clicking on ‘Failed’ execution result:

If you have enabled the option for bug tracking in the application.properties file, you can add a link to your bug tracking tool (i.e. JIRA) in ‘Bug Tracking’ column:

Note: If you already have a database schema generated with rspec2db, you can specify the flyway properties in the application.properties file. For example, if you have a database named ‘testdb’, simply add its properties to application.properties file:

And then run the flyway baseline migration with:

This way you will alter it for Owl usage.