Boiled down, the test execution flow is basically:
- Setup the test data
- Run the test
- Delete the test data
It seems simple enough. Deceptively, there are so many ways of doing this and there is no gold standard. All QA engineers try to make automation tests as simple as possible, as over-engineering will only give you a headache. However, headaches are one thing you don’t have time for because the next release will probably be big and it will probably have multiple functionalities. How will you even prepare all the test data and test cases? It looks like your best option is to switch careers, move to Iceland and look after Elks.
To help you be a little more efficient, and less error-prone, I have decided to share a few tips.
Prefer API calls to UI automation of certain processes
First of all your devs have probably created APIs that their web-app uses. Those same APIs are ready to be used and are at your disposal.
Did you know that an API call usually takes a couple of hundredths of a second?
So why not use it to your advantage?
There are situations wherein the “End to End” tests it is necessary for us to test the flow of the application and create UI tests (i.e. a registration process of the web-application we are testing). Yet, sometimes laziness gets the better of us and we continue to use it throughout all of our tests and to recycle that “register user UI test” for registering users in all subsequent tests. This is not a good idea. Why? Let me explain through code.
This code is an example of how a test engineer might register a User through the UI test automation of the web application:
goToWebApplicationWeAreTesting(link) clickTheRegisterButton() waitForPageToLoad(1000) populateRegistrationFields(userData) clickRegister() waitForPocessingToComplete() expectUserIsRegistered()
user={ name:”john”, surname: “Doe” birthdate: “09.09.1982” role:”user” } response =callRegisterUserApi(user) expect(response.status).equals(“OK”)
Wasn’t that nice, it’s simple, it’s very fast, it’s compact, easy to maintain and it directly tests the backend logic. Of course, the front end UI test should be done at least once in the test suite, however, anything more than testing the specific UI requirements is overkill.
Automate the creation of test data (as much as possible).
When going through tests, often at the beginning of test data, you usually find the initialization of certain objects which will be used during the test. This could be the initializations of users, products, etc. Let’s look at an example:
User testUser= new User(); testUser.name=“John” testUser.surname=“Doe” testUser.birthdate:”09.09.1982” testUser.role=“user”
Or an alternative:
User testUser = new User(“John”,”Doe”,”09.09.1982”,”user”)
Whilst this is not always bad practice, it inherently means that for every user you must provide data to the method. You have to remember the order of the parameters you have to provide and must create the constructor in such a way that, if certain data is purposefully omitted, the following combinations don’t yield an exception because of a forgotten overload:
User testUser = new User(“John”,”Doe”) User testUser = new User(“”,”Doe”,””,”user”)
I believe that creating a factory method pattern (where possible), that handles the creation of data used in the test, can simplify your life in the long run.
Example:
public class UserFactory { public static User createUserWithAllDetails(){ return new User(“John”,”Doe”,”09.09.1982”,”user”); } public static User createUserWithMissingName(){ return new User(“”,”Doe”,”09.09.1982”,”user”); } public static User createUserWithMissingSurname(){ return new User(“John”,””,”09.09.1982”,”user”); } }
If you use tools like IntelliJ, Eclipse, etc… you can use autoComplete which will help you find the method you need easier.
The benefit of this approach, whilst maybe not easy to spot in this block of code, is that you can have dedicated methods to create complex User objects. These have more data and once the method is created, everyone can use it.
Something not to shy away from are methods that create random names. While the drawback is you don’t create real names, the cool thing is that it can be used where the length of strings plays a role in testing. So, let’s create a method:
public static String generateRandomString(int len) { final int alphabetSize = 'z' - ‘a'; final Random random = new Random(); final byte[] result = new byte[len]; random.nextBytes(result); for (int i = 0; i < len; i++) { result[i] = (byte) ('a' + Math.abs(result[i] % alphabetSize)); } return new String(result, StandardCharsets.US_ASCII); }
The benefit would be randomizing characters used for testing. We provide the length of the string used in the test and that is something all developers test.
Imagine just adding the following methods in the User Factory:
public static User userWithNameTooLong(){ return new User(generateRandomString(100),”Doe”,”09.09.1982”,”user”) } public static User userWithTwoLetterName(){ return new User(generateRandomString(2),”Doe”,”09.09.1982”,”user”) } public static User userWithRandomNameAndSurname(){ return new User(generateRandomString(10),generateRandomString(10),”09.09.1982”,”user”) }
Now we have made something that all testers can use.
Now in the @before block, we can simply do:
response =callCreateUserApi(UserFactory.getUserWithMissingName) expect(response.status).equals(“OK”) response =callCreateUserApi(UserFactory.getUserWithVeryLongName) expect(response.status).equals(“OK”) response =callCreateUserApi(UserFactory.getUserWithVeryShortName) expect(response.status).equals(“OK”) response =callCreateUserApi(UserFactory.getUserWithMissingSurname) expect(response.status).equals(“OK”)
Teardowns
Ah those pesky teardowns, they are a specific creature which you can’t live without, while they in/turn often can’t remove all the data.
What are teardowns?
Teardowns are usually blocks of code which are executed at the end of tests, they are usually used for but not limited to, the removal of test data and data which was generated as a result of the executed tests.
Why are teardown important?
Simply put, every test which is triggered usually generates data that is stored in some database.
Tests should ideally never contaminate the database with test data.
Without a proper teardown mechanism, our tests are very limited on where they can run. A simple example would be running tests on production environments. Imagine triggering a test 1000 times, only to discover that RandomUser123 is inserted 1000 times into a production DB. If we decide to delete or alter that data via SQL queries we risk breaking something (DB relations, indexes, counters, foreign keys )
From my experiences with working on different teams, I have learned that not all projects are made equal when it comes to testing data removal. Sometimes you test on systems that are initialized and seeded with databases that take hours to get up and running, and the nonchalant emptying of the entire database can only cause you to lose a lot of time.
Finding out how to delete generated data can often be a complex task in itself, and if it is not properly deleted it will serve as a bitter reminder every time we sift through the DB (or at least until the DB is truncated or we dare to manually drop it from the DB).
In the end, like every writer let me leave you with a bit of yearning for more…
See you in the next blog 🙂