XML And HTML Report In PyUnit For Test Automation

PyUnit HTML Report Generation Using HTMLTestRunner

To generate PyUnit HTML reports that have in-depth information about the tests in the HTML format, execution results, etc.; you can make use of HtmlTestRunner module in Python.

There are different ways in which reports can be generated in the HTML format; however, HtmlTestRunner is widely used by the developer community. To install HtmlTestRunner module, you need to execute the following command in your terminal:

pip install html-testRunner

The entire purpose of HtmlTest runner is to save the output of your automation test execution in html file to make it much easier to interpret.

Below is a snapshot of the HtmlTestRunner module installation.

We make use of the Eclipse IDE for development purpose and the same can be downloaded from here. You also have the option of using the Community version of the PyCharm IDE which can be downloaded from here

Now that you have installed HtmlTestRunner, let’s have a look at the test suite used for generating PyUnit HTML reports. The testsuite contains two test cases.

a. Google Search where the search term is ‘TestMu AI’.

b. Wikipedia Search where the search term is ‘Steve Jobs’.

Test Case A – Google Search where the search term is ‘TestMu AI’

The filename is GoogleTest.py and the compiled Python file (GoogleTest) would be imported in the file where the TestSuite is created. For more information on the setUp() and tearDown() methods, please have a look at the earlier articles that have covered PyUnit/unittest in more detail.

Filename – GoogleTest.py (Compiled Python File – GoogleTest)

import unittest
from selenium import webdriver
import time
from time import sleep

class GoogleSeachTest(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()

    def test_GoogleSearch(self):
        driver_firefox = self.driver
        driver_firefox.maximize_window()
        driver_firefox.get('http://www.google.com')

        # Perform search operation
        elem = driver_firefox.find_element_by_name("q")
        elem.send_keys("Lambdatest")
        elem.submit()

        sleep(10)

    def tearDown(self):
        # Close the browser.
        self.driver.close()

if __name__ == '__main__':
    unittest.main()

Test Case B – Wikipedia Search where the search term is ‘Steve Jobs’

The filename is WikiTest.py and the compiled Python file (WikiTest) would be imported in the file where the TestSuite is created.

Filename – WikiTest.py (Compiled Python File – WikiTest)

import unittest
from selenium import webdriver
import time
from time import sleep

class WikipediaSeachTest(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Firefox()

    def test_WikipediaSearch(self):
        driver_firefox = self.driver
        driver_firefox.maximize_window()


        # Perform search operation
        driver_firefox.get('http://en.wikipedia.org')

        driver_firefox.find_element_by_id('searchInput').clear()
        driver_firefox.find_element_by_id('searchInput').send_keys('Steve Jobs')

        sleep(10)
        driver_firefox.find_element_by_id('searchButton').click()

        sleep(10)

    def tearDown(self):
        # Close the browser.
        self.driver.close()

if __name__ == '__main__':
    unittest.main()

Compiling Test Case A & B For Generating PyUnit HTML Reports

For compiling the files, you can make use of the command:

python GoogleTest.py
python WikiTest.py

We make use of the compiled Python code (GoogleTest and WikiTest) to create a testsuite that has these two test cases. We make use of the HtmlTestRunner module to create PyUnit HTML report that contains details about the tests along with the execution results.

Filename – test_html_runner_search.py

import unittest
import GoogleTest
import WikiTest
import os

# Import the HTMLTestRunner Module
import HtmlTestRunner

# Get the Present Working Directory since that is the place where the report
# would be stored

current_directory = os.getcwd()

class HTML_TestRunner_TestSuite(unittest.TestCase):
    def test_GoogleWiki_Search(self):

        # Create a TestSuite comprising the two test cases
        consolidated_test = unittest.TestSuite()

        # Add the test cases to the Test Suite
        consolidated_test.addTests([
            unittest.defaultTestLoader.loadTestsFromTestCase(GoogleTest.GoogleSeachTest),
            unittest.defaultTestLoader.loadTestsFromTestCase(WikiTest.WikipediaSeachTest)
        ])

        output_file = open(current_directory + "HTML_Test_Runner_ReportTest.html", "w")

        html_runner = HtmlTestRunner.HTMLTestRunner(
            stream=output_file,
            report_title='HTML Reporting using PyUnit',
            descriptions='HTML Reporting using PyUnit & HTMLTestRunner'
        )

        html_runner.run(consolidated_test)

if __name__ == '__main__':
    unittest.main()

As seen in the implementation above, we make use of the HTMLTestRunner method which is implemented in the HtmlTestRunner module (HtmlTestRunner.HTMLTestRunner). For the purpose of testing, we have passed three arguments to the HTMLTestRunner method:

For more details about the HTMLTestRunner method, you can refer to runner.py that can be found in the location where HtmlTestRunner module is installed (< Installation Path >\…\venv\Lib\site-packages\runner.py).

class HTMLTestRunner(TextTestRunner):
    """" A test runner class that output the results. """

    time_format = "%Y-%m-%d_%H-%M-%S"

    def __init__(self, output="./reports/", verbosity=2, stream=sys.stderr,
                 descriptions=True, failfast=False, buffer=False,
                 report_title=None, report_name=None, template=None, resultclass=None,
                 add_timestamp=True, open_in_browser=False,
                 combine_reports=False, template_args=None):
        self.verbosity = verbosity
        self.output = output
        self.encoding = UTF8

        TextTestRunner.__init__(self, stream, descriptions, verbosity,
                                failfast=failfast, buffer=buffer)

Below is the execution snapshot of test_html_runner_search.py

The report location & report name is passed as an argument to the HTMLTestRunner() method. Below is the content for the PyUnit HTML report (HTML_Test_Runner_ReportTest.html). The report contains information about the two tests that were executed as a part of the testsuite, their execution results and time taken for completion.

Running tests... ----------------------------------------------------------------------
test_GoogleSearch (GoogleTest.GoogleSeachTest) ... OK (18.393839)s
test_WikipediaSearch (WikiTest.WikipediaSeachTest) ... OK (32.298229)s
Ran 2 tests in 0:00:50 OK Generating HTML reports... reportsTestResults_GoogleTest.GoogleSeachTest_2019-05-31_17-48-20.html reportsTestResults_WikiTest.WikipediaSeachTest_2019-05-31_17-48-20.html

Apart from the consolidated report shown above, there would be individual reports that would be located in the < output-folder >\reports\ folder

TestResults_GoogleTest.GoogleSeachTest_2019-05-31_xxxx.html

TestResults_WikiTest.WikipediaSeachTest_2019-05-31_xxxx.html

For more details about formatting the report, you can visit the following link.

Scope Of PyUnit HTML Reports/ XML Reports For Automated Cross Browser Testing

Now that we know how to generate a PyUnit HTML report, or PyUnit XML report, we need to understand the scope of implementation based on the automated cross browser testing needs of your project. How practical is this approach and is it going to hamper your productivity as you perform Selenium? Let’s find out.

Not So Scalable Approach

HtmlTestRunner in conjunction with Selenium & PyUnit (unittest) can be used to a good extent to test the features of a web product. However, the approach may not be scalable if the testing is performed on a project/product at a large scale.

Maintenance Is Going To Be A Hassle

Maintenance of the test reports would be a strenuous task and its complexity will multiply with the growing scale of the project.

TestMu AI Build & Session API Demonstration

API that can be used for detailed reporting:

https://api.lambdatest.com/automation/api/v1/

You need to append the necessary end-point to the end of the API and issue a GET/POST to get the results.

Endpoints can be builds, sessions, tunnels, platforms.

For example, you can make use of /tunnels i.e.

https://api.lambdatest.com/automation/api/v1/tunnels

to fetch the running tunnels used for testing locally hosted web pages through your account.

When you initiate a test request, a session-id is assigned to that test session. Using TestMu AI Selenium API, you can get detailed information at the test session level using the

https://api.lambdatest.com/automation/api/v1/sessions

Extracting Build Details Using TestMu AI Selenium API

For demonstration, we extract details of the tests that have status – completed, error, and timeout and we limit the search for first 20 tests. To get started, you need to authorize those APIs to fetch build, session, tunnel & other related information from your account. Login to https://www.lambdatest.com/support/docs/api-doc/ with your user-name & session-key.

Since you require information about the builds that match the execution state (completed, error, timeout), you have to fill those requirements in the Builds section in https://www.lambdatest.com/support/docs/api-doc/#/Build/builds Below is the screenshot of the command in execution. You also get the CURL API which can be used in your code for fetching the same information programmatically.

The response code can be 200 (OK), 400 (Session Invalid), and 401 (Authentication Failed). Since we are using Python, we convert the CURL response into Python code using the converter located in https://curl.trillworks.com/. You can use other tools or websites that can offer similar functionality (for free), for our demonstration we are making use of the above-mentioned website.

We extract details of the first 20 builds that matches status-completed, timeout, error. Once we have extracted the details, we update the build headline for the buildID = 12024. We make use of:

https://api.lambdatest.com/automation/api/v1/builds/{Build_ID}

In our example, the build ID would be 12024. Below is the code which demonstrates the TestMu AI API integration:

# Requirements

# Fetch first 20 builds which have status as completed, error, timeout
# Once the details are out, change the Build Title of the build_id 12024

# Refer https://www.lambdatest.com/support/docs/api-doc/#/Build/builds for more information
import requests
import json

# Equivalent Python code from https://curl.trillworks.com/
headers = {
    'accept': 'application/json',
    'Authorization': 'Basic aGltYW5zaHUuc2hldGhAZ21haWGI0T2x1OVI4bHdCc1hXVFNhSU9lYlhuNHg5',
}

params = (
    ('limit', '20'),
    ('status', 'completed,error,timeout'),
)

# Updated build information for build 12024
headers_updated_build = {
    'accept': 'application/json',
    'Authorization': 'Basic aGltYW5zaHUuc2hldGhAZ21haWGI0T2x1OVI4bHdCc1hXVFNhSU9lYlhuNHg5',
    'Content-Type': 'application/json',
}

data_updated_build = '{"name":"Updated build details for PyUnit test from prompt"}'

response = requests.get('https://api.lambdatest.com/automation/api/v1/builds', headers=headers, params=params)

print(response)

json_arr = response.json()

# Print the build_id matching our requirements and Change build title of build_id 12024

for loop_var in range(20):
    build_id = ((json_arr['data'][loop_var])['build_id'])
    test_status = ((json_arr['data'][loop_var])['status_ind'])

    if build_id == 12024:
        response_updated_build = requests.patch('https://api.lambdatest.com/automation/api/v1/builds/12024', headers=headers_updated_build, data=data_updated_build)
        print(response_updated_build)

    print ((build_id), (test_status))

As seen in the screenshot, the Build headline for BuildID = 12024 is updated.

We execute the code using the normal Python command, the response of the execution is 200 (OK).