Data-Driven Testing with Cucumber

Cucumber is a testing framework that supports Data-Driven Testing. 

What is Data-driven testing?

Data-Driven Testing (DDT) is a software testing approach that uses data sets to drive the testing process. This approach involves testing an application with a range of input values, each designed to test a specific feature or functionality of the application.

Why do we use Data-Driven Testing?

Testers often have multiple data sets used for a single test, and creating individual tests with these different data sets can be time-consuming. This type of testing allows the test case to be executed multiple times with different data sets (inputs) and validation values. 

Here are some reasons why we use Data-Driven Testing:

1. Increased tests coverage: using various data sets as input, this type of testing helps cover a wide range of scenarios and combinations of inputs.
2. Reusability of test scripts: the test scripts are designed to be reusable, allowing them to be executed with multiple sets of input data.
3. Faster bug identification: by testing various input data combinations, Data-Driven Testing can quickly identify bugs and anomalies in the system during the testing phase. 

While Data-Driven Testing gives us many benefits, like any approach, there are some potential pitfalls and challenges that should be considered:

1. Input data set management: managing a large set of input data files can be time-consuming and challenging.
2. Test data validity: perhaps we can say that the most important item is to have valid test data sets, if not, it can lead to inaccurate test results and false positive or false negative.
3. Maintenance and updates: as the system evolves, test scripts need to be updated accordingly, and managing the test scripts and test data can be challenging. 

Data-Driven Testing in Cucumber

Cucumber inherently supports Data-Driven testing using the Scenario Outline and Examples section. Scenario Outline is a powerful feature of Cucumber that allows you to run the same scenario with different inputs or data sets. It enables you to write a single scenario with placeholders for inputs, and Cucumber will automatically generate multiple scenarios by replacing these placeholders with actual data.

Using a scenario outline to write the testing scenario

Below is an example of a testing scenario for the comparison of two CSV files using a scenario outline.

Feature: Compare data in csv files


   Scenario Outline: Comparison two CSV files
     Given two input files "<file1>" and "<file2>"
     When we compare those two files
     Then the content from first input should "<matchOrNot>" the content from the second one


     Examples:
       | file1      | file2      | matchOrNot   |
       | input1.csv | input2.csv | match        |
       | input3.csv | input4.csv | not match    |

In this example, the scenario outline is defined in the feature file with placeholders for the part of CSV file paths and the expected match results. The Scenario Outline keyword is used to indicate that we have a scenario with multiple examples. The examples table is defined using the Examples keyword, containing the different data input sets we want to compare.

import io.cucumber.java.en.Given;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;


import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;


import static org.junit.Assert.assertEquals;
 
public class Steps {
    private static final String INPUT_DATA_PATH = "./src/test/resources/inputData/";
    private String path1;
    private String path2;
    private boolean filesMatch;


    @Given("two input files {string} and {string}")
    public void getPathToCsvFiles(String fileName1, String fileName2) {
        this.path1 = INPUT_DATA_PATH + fileName1;
        this.path2 = INPUT_DATA_PATH + fileName2;
    }






   
    @When("we compare those two files")
    public void compareFiles() {
        filesMatch = compareCSVFiles(path1, path2);
    }


    @Then("the content from first input should {string} the content from the second one")
    public void filesShouldMatchOrNot(String matchOrNot) {
        boolean expectedMatch = matchOrNot.equals("match");
        assertEquals(expectedMatch, filesMatch);
    }


    private boolean compareCSVFiles(String file1Path, String file2Path) {


        try (BufferedReader reader1 = new BufferedReader(new FileReader(file1Path));
             BufferedReader reader2 = new BufferedReader(new FileReader(file2Path))) {


            String line1, line2;
            int lineCount = 1;


            while ((line1 = reader1.readLine()) != null && (line2 = reader2.readLine()) != null) {
                if (!line1.equals(line2)) {
                    return false;                }
                lineCount++;
            }


            // Verify if there are any unprocessed lines remaining in the fields
            if (reader1.readLine() != null || reader2.readLine() != null) {
                return false;
            }


            return true;
        } catch (IOException e) {
            e.printStackTrace();
            return false;
        }
    }
}

The step definition file (“Steps.java”) contains an implementation of step definitions that correspond to the scenario outline steps. The “getPathToCsvFiles” step sets the file paths, the “compareFiles” step compares files, and the “filesShouldMatch” step verifies the expected match result. 

In the “compareFiles” step we call the “compareCsvFiles” method, which compares the input data files line by line. It returns a boolean value: “true” if all lines in the input files are the same and a “false” value otherwise.  

We can use the Cucumber test runner to execute the comparison between two input files based on provided examples.

import io.cucumber.junit.CucumberOptions;
import net.serenitybdd.cucumber.CucumberWithSerenity;
import org.junit.runner.RunWith;


@RunWith(CucumberWithSerenity.class)
@CucumberOptions(
        features = "src/test/resources/compareCsvFiles.feature",
        plugin = {"pretty"},
        glue = {"steps"}
)
public class Runner {
}

Conclusion

Data-Driven Testing can help reduce the amount of manual effort required for testing and can make it easier to manage and maintain test cases over time. Overall, the use of Data-Driven Testing can improve the quality of software and help ensure that it meets the requirements and expectations of its users.