How To Integrate Cucumber With Jenkins?

Getting Started with Cucumber Jenkins Setup

Cucumber is well-known for agile development of web applications and reporting the results. With an easy-to-understand language, powerful plugins, and a straightforward integration with Selenium, starting Cucumber with Jenkins integration has never been easier.

Before we begin, let’s understand why you need Cucumber with Jenkins integration in the first place. Here’s why- Cucumber acts as a bridge between:

• Manual Tester and Developers.
• Software Engineer and Business Analyst.
• Manual Tester and Automation Tester.

And as you already know, Jenkins is an open-source, accessible, easy to use CI/CD tool. With Jenkins, you can schedule a run on scheduled time or trigger builds after an event, and it also comes with a plethora of plugins. You can refer to our comprehensive Jenkins tutorial to learn more about this tool.

Now, let us see how to integrate Cucumber with Jenkins.

Prerequisites for Cucumber with Jenkins Integration

We need the installation of the below plugins to start Cucumber with Jenkins integration. Let us see an overview and utilities of the below plugins. This section will see how to make use of some Jenkins plugins that work together with Cucumber formatting outputs.

• Cucumber Test Results plugin
• Cucumber Reports
• Cucumber perf plugin

A. Cucumber Test Results plugin

This plugin takes Cucumber JSON output as input and publishes the results in polychrome. The results will appear in GREEN when they have passed and RED when they fail.

Here’s how to install the Cucumber Test Results plugin. You can also refer to our blog on generating TestNG reports in Jenkins for more information.

• Login to your Jenkins server as an administrator.
• Navigate to “Manage Jenkins.”
• Go ahead and click “Manage Plugins.”

• Go to the “Available” tab under the manage plugins page.
• Search for “cucumber” in the filter at the right top corner.
• Go ahead and check the box against the Cucumber reports.
• Choose “Install without restart” to start the installation.

• Click “Back to dashboard” in the left menu
• Navigate to the job already created. ‘Cucumber’ in our case.
• Click “Configure” against the job.

• Scroll down to Post-Build Actions. Under “Add post-build action,” select “ Cucumber reports.”

• Click on Advanced. Go to Post-build Actions. Add below report file in the text box as shown. You can Ignore the message at the bottom. It appears since we don’t have any files yet, once we run the build, they will be created.

• The next step is to generate Cucumber JSON reports.
• Open the cucumber.yml file in your page-object_framework and add the below-highlighted string.
• This is to make sure we have reports.json output in addition to the HTML reports. The cucumber plugin will parse these on Jenkins.

Execution

To execute, click the “Build Now” button from the left menu for the job. The build should be successful. Instead of watching console output, select “Cucumber Test Result” at the end of the build.

Results

Next, go to Feature –> Scenario, where you can see all the steps’ status. As shown below, they are all GREEN.

B. Cucumber Reports

Cucumber Reports plugin can read the JSON output and produce pie charts for feature or scenario results & bar graphs for tags. This is possible if we had options “-f json -out cucumber.json,” where you output the results to JSON format, i.e., in cucumber.yml. This plugin has advanced options to help us make vital decisions on the test results.

• Add a post-build step and select the “Publish cucumber results as a report” option, as shown below.

• Referring to the job, the below menu option will be visible to us as below.

• Next, run the job and click the reports. The reports will be in the form of SWF charts, as shown below.

Results

We get results as below.

C. Cucumber perf plugin

This plugin helps us demonstrate the history of how Cucumber results were performing as a trend. This is helpful when you keep running the suite multiple times.

There are additional features to it, like “worst-performing features” over time. This mainly helps us focus on those bottlenecks and allows us to debug them further.

Running Cucumber Script with Jenkins

Automation scripts built with the Cucumber framework benefit from being easy to understand, easy to communicate, and good maintenance, so it is popular among test automation teams. An essential aspect of integrating DevOps is incorporating automated testing into a continuous integration (CI) process.

Now that you have completed the Cucumber Jenkins setup, in this section, we will see how to configure Jenkins to run an automated Cucumber test script and show the Jenkins UI report. This can be done using the following methods-

• Cucumber’s Command Line Mode
• CukeTest’s Command Line Mode
• Jenkins Continuous Integration Configuration
• Jenkins Test Report Plugin Configuration

Cucumber’s Command Line Mode

To illustrate the Cucumber Command Line, we use Cucumber.js. Many who already know this tool know that it means running the “Cucumber-js” command directly in the directory of the automation script. Or if you don’t have Cucumber installed globally, run “node modules/.bin/cucumber-js” (or, if you’re on Windows, the .cmd extension), and the Cucumber installed locally will be used. After execution, the output of the text on the command line is written. You can refer to our Cucumber.js tutorial for more information.

The command line is identical if you are running it as a job under Jenkins. Just add the “format” parameter, indicating the output file in the “JSON” format, to make Jenkins render a visual report. The following is the command line:

node_modules.bincucumber-js --format json:report.json

A report is output to report.json, and then this file is used by the Jenkins job to view the report.

CukeTest’s Command Line Mode

CukeTest is Cucumber.js’s development tool. These editions support command line mode and are available on Windows Desktop, Windows Store, MacOS, etc. It is available for free at CukeTest’s official website. CukeTest is used to build scripts for Cucumber.js and can also be used to execute scripts for Cucumber.js from the command line.

The benefit of using CukeTest is that with each project, you don’t have to install Cucumber.js. CukeTest already has Cucumber.js built-in, which saves your storage space and time. Another advantage is that it can run Windows automation with the library built-in, capture video, etc.

The Windows Store version of CukeTest has a different command line process, so the command line varies slightly from the desktop version. Luckily, you don’t need to recall all the specifics of the command line.

• In CukeTest, you can open a run profile when your project is opened to get the command line by clicking the drop-down arrow next to the “Run Project” button and then clicking “Edit Profiles.”

• In the “Run Profiles” dialogue, the corresponding command line is automatically created for this profile in the command line:

• Click the “Copy to clipboard” option to copy the commands to the clipboard.
• Then click the “Open console window” option, and paste the command line to this newly opened window.

Below are the command lines for various platforms-

• Command line for Windows Store version

chcp 65001 && start cuketest://?profile=Profile1

CukeTest’s command line output is encoded in UFT8. So using the command “chcp 65001” helps change the command line encoding from UFT8 to UTF8.

• CukeTest desktop version

chcp 65001 && cuke --run --format html

CukeTest’s command line output is encoded in UFT8. So using the command “chcp 65001” helps change the command line encoding from UFT8 to UTF8.

Jenkins Continuous Integration Configuration

Once we are aware of the steps to run the project in the command line, we can configure Jenkins to run it. Below are the steps to configure the Jenkins job.

For the demo, the source code is kept in https://github.com/CnodejsTest. This project is used to automate test cnodejs.org community site. Alternatively, we can use our Cucumber project to configure and run the job.

• Create a new job in Jenkins. Enter any arbitrary name.

• In the item bar, select “Windows Batch Command.” Here we have to copy & paste the corresponding command line from CukeTest to the Jenkins batch command box. In case you have a CukeTest desktop version installed natively, below is your command line.

npm install
chcp 65001 && cuke --run --format html

The ‘npm install’ command installs npm dependency packages that are consumed by the automation script.

• Perform the below job configuration UI with the commands as entered below.

• Save this project configuration by clicking ‘Save.’ Now click the “Build Now” button, and the project will then start executing.

  You can observe the execution progress upon build now of the job in the build history.

  We can also find the running job log under the Build History. At the exact moment, the report file is also generated in the Jenkins Workspace directory.

• The next step is to configure the Jenkins job to show the Jenkins interface test report. This can be done quickly using the Plugin we just saw in the previous section to format and display the test report.

Jenkins Test Report Plugin Configuration

In Jenkins, we can use the Cucumber reports plugin to show the report. We have already seen the introduction to this plugin in the Cucumber Jenkins setup above. Additionally, you can also read the information about this Plugin available at the official Jenkins website:

The following are the steps to configure Jenkins to generate reports.

• Install the Jenkins reports pluginOpen Jenkins –> Manage Plugin –> Filter “Cucumber reports” in Optional Plugin.


We need to restart Jenkins after the installation of this plugin.

• Reconfigure the Jenkins task

  We need JSON log data files to generate reports because the Cucumber reports Plugin generates reports by parsing the JSON report file created by Cucumber or CukeTest. Therefore, it is necessary that we first configure a running project which will create JSON log data files.

  In the run configuration file, we have to select “JSON” in the report format.



When we would run this command, the command line execution output is changed to json.

• Display the Cucumber reportsIn the ‘Add post-build operation’ action, select the ‘Cucumber reports’ option.


Go to the ‘Advanced’ option under the ‘Cucumber reports.’ In the ‘JSON Reports Path,’ enter “reports.”



Click Save to finish the project configuration.

• Generating test report
  To generate the test report, we have to click the “Build Now” button. After the job is completed, a test report will be generated automatically. Besides, the test report provides us with a link to each scenario, where each link, when clicked, shows the details of the scenarios. The report also indicates the screenshot captured in each scenario step or hooks along with each step’s execution result.

So till now, what we have seen is how to integrate Cucumber with Jenkins and how to run a Cucumber script on Jenkins along with the generation of graphical reports.

It is also possible to leverage cloud-based platforms’ automation capability to make remote testing more seamless than ever. You can configure your Cucumber scripts to run on TestMu AI’s Selenium Grid cloud and perform testing on 3000+ browsers, devices & operating systems. In the next section, we will learn how you can make the most of Cucumber with Jenkins integration & TestMu AI.