Setting up Owl – test results reporting tool

Getting started with Owl

This step by step guide will help you to successfully install and enable our Owl tool, used for test results reporting and presentation. In this particular case, we will be installing Owl for presenting tests, written in Ruby and Rspec (a testing tool for Ruby), created for behavior-driven development.

For the purposes of this tutorial, we used a clean Amazon EC2 instance running Ubuntu 16.04.

Prerequisites

The first Owl prerequisite is that you need to have a PostgreSQL database installed. Owl currently supports only PostgreSQL databases. The database needs to be started on the same machine where Owl will be installed, further in the text referred to as “localhost”. So basically, we will only need to create our database and then later let the Owl create a schema that will be used. For these purposes, we created an additional role named ‘owluser’ and a database named ‘owldbtest’. You can do this using the following commands:

Login to postgres with psql client:

sudo -i -u postgres

Log in to prompt interface with:

 psql

(Or use following command to login with custom postgres role:

psql -U <your_postgres_role> -d postgres -h 127.0.0.1 -W)

Create a new database with:

CREATE DATABASE <database_name>;  In order to use Owl with Rspec, you need to include ‘rspec2db’ gem in your tests. This gem is a database formatter which enables writing RSpec test results into our database. See Rspec2db GitHub repo for more information: https://github.com/ATLANTBH/rspec.

In case you already used rspec2db and now you want to integrate it with Owl, you will need to run alter migration that will alter the existing tables and add the needed columns for Owl. You can read more about this approach at the end of the article.

To run the Owl application, Java (JRE) needs to be installed (You’ll have to set JAVA_HOME environment variable too). And also, to build the application, there should be Maven (>= v3) installed.

Setup the Rspec project

Create a new directory for the RSpec project, and initialize it with:

rspec —-init

After project initialization, create a config directory in your project and add rspec2db.yml file with following content populated:

options:

  suite: <TEST_SUITE_NAME>

  build: <BUILD_ID>

  backtrace: false #set backtrace to true for more details and metadata

dbconnection:

  adapter: postgresql

  database: <DATABASE_NAME>

  username: <DATABASE_USER>

  password: <DATABASE_PASSWORD>

  host: localhost

  port: 5432

Open .rspec file in your project and make sure it contains the following lines:

--require rspec2db

--format Rspec2db

--options ./config/rspec2db.yml #--options with location to rspec2db.yml relative to .rspec file

Add a Gemfile to your project and add the following gems:

gem ‘rspec'

gem 'rspec2db', :github => 'ATLANTBH/rspec’

Run bundle install.

Run RSpec tests using the following command (from the location where .rspec file exist, to be able to pick up parameters defined in .rspec):

bundle exec RSpec spec spec/<TEST_SCRIPT_NAME>  (Of course, you’ll need to have some RSpec tests in your spec folder to do this (See http://rspec.info/ to start writing tests in RSpec).)

Install and start the Owl app

Clone Owl repository:

git clone https://github.com/ATLANTBH/owl.git

Inside the cloned directory, run:

mvn clean package

This will create target/owl-VERSION.jar file, and we will copy it to a different location on our ec2 host, along with /src/main/resources/application.properties file to start it locally.

Open the application.properties file where we will edit our database configuration and other properties. We need to change the database name, username, and password in the database properties, to match our properties. In addition, you can edit project name, test suite names, or toggle specific features to suit your test suite. You can enable git information on the dashboard to show the exact commit on which the test was executed. There is also a bug tracking feature, which adds an additional column to test steps, so you can add error description and i.e. link to your JIRA ticket. In the end, application.properties file should look something like this:

# Sets the project name to be visible on frontend

project.name=Owl_Test_Project

# Custom server port

server.port=8090

# Comma separated list of test suite names that will be shown on dashboard

suite.statistics=smoke test,regression test

# Feature toggles

# Show git information on dashboard

project.features.git.info=true

# Show bug tracking feature

project.features.bug.tracking=true

# Github repo link

git.github.repo=http://github.com/org/repo

# Database Properties

spring.datasource.url=jdbc:postgresql://localhost:5432/<DATABASE_NAME>

spring.datasource.username=<DATABASE_USER>

spring.datasource.password=<DATABASE_PASSWORD>

# Logging Properties

logging.level.org.hibernate.SQL=debug

# Properties for flyway baseline migrations

#flyway.url=jdbc:postgresql://localhost:5432/owl-db

#flyway.user=owlusername

#flyway.password=owlpassword

Note: Server port from needs to be enabled for listening, so the application could be reached. In our case we will be adding Custom TCP Rule in Security Groups on our Amazon EC2 instance. (Screenshot 1)

Start the application with your configuration by running:

java -jar owl-VERSION.jar –spring.config.location=<application.properties>

You should start the process with nohup, so it won’t be killed when you logout:

nohup java -jar owl-VERSION.jar --spring.config.location=<application.properties> &

To be sure that port is enabled and running our app, run netstat -nltp  and you should get something like this:

So that’s it! At this point, the application should be running on port 8090, or on your custom server port. Every test execution will be written to the database, and you can easily access it, formatted, and segmented.

Once you open the Owl, you can navigate to your particular test suite:

You can see the graph for the ‘Amazon Regression’ test suite with a success percentage and complete duration of each test suite. The x-axis represents build ID of the test suite (specified in the rspec2db.yml file), and y-axis represents the success rate:

Opening the particular build, you can see all specs executed in this particular test run:

From here you can open a single test and see all the test steps. You can see the description and the expected result, if it passed or not, as well as the duration of each step:

When the step fails, you can see the exception by clicking on ‘Failed’ execution result:

If you have enabled the option for bug tracking in the application.properties file, you can add a link to your bug tracking tool (i.e. JIRA) in ‘Bug Tracking’ column:

Note: If you already have a database schema generated with rspec2db, you can specify the flyway properties in the application.properties file. For example, if you have a database named ‘testdb’, simply add its properties to application.properties file:

# Properties for flyway baseline migrations

flyway.url=jdbc:postgresql://localhost:5432/testdb

flyway.user=testdbusername

flyway.password=testdbpassword

And then run the flyway baseline migration with:

mvn flyway:baseline -Dflyway.configFile=<application.properties>

This way you will alter it for Owl usage.