How to Generate Test Reports in NUnit
In this tutorial, you will learn about generating reports in NUnit along with some new features and advantages of Extent Reports.
Join the DZone community and get the full member experience.Join For Free
Test reports are an integral part of any activity related to testing, whether it is automation testing or manual testing. Test reports help track how the activities related to automation testing have evolved over a period of time. The same principle also applies when using NUnit/xUNit/MSTest for automated browser testing. Of the lot, NUnit is the most-used test automation framework for all .Net languages. NUnit reports can serve as a considerable value addition to the tests performed using the said framework.
Let’s look at NUnit report generation and how seamlessly you can integrate the NUnit reporting tool in the test implementation. As far as Selenium C# is concerned, Extent and Allure are the preferred NUnit reporting tools. Both are third-party tools that have to be installed from the NuGet gallery, either from the Visual Studio UI or using the Package Manager Console commands.
Extent Report is a more popular NUnit report generator. By the end of this blog, you would be comfortable integrating Extent Reports in the Selenium C# test implementation. So, let’s get started.
Introduction to Extent Reports
ExtentReport is a popular multi-language test reporting tool that gives in-depth information about the status of the tests. The Extent framework lets you create interactive and detailed test reports. Apart from the .NET framework, ExtentReport is also available for the Java language. ExtentReports (or Extent Reports) is an open-source reporting library, and version 4.0 of ExtentReports is licensed under Apache 2.0.
The Extent Framework source code is available on GitHub; the implementation of ExtentReports 4 .NET Core is available here. As of writing this article, ExtentReports 4.0 was the latest version of ExtentReports for the .NET framework. Since ExtentReports 3.x is no longer maintained, it is recommended to migrate from ExtentReports 3.x to ExtentReports 4.x.
In case you are using SpecFlow for automation testing with Selenium C#, you have the flexibility to integrate ExtentReports 4.x in the SpecFlow BDD tests for HTML test report generation. Extent Reports follows a freemium pricing approach, which essentially means that it is available in Community (or Free) and Pro (or Paid) Editions. As per the official documentation of ExtentReports, there are no differences in the underlying ExtentReports API. The community & pro users can leverage the advantage of features offered by full-featured APIs.
For the demonstration, we have used the Community Edition of ExtentReports 4.x.
Advantages of Using Extent Reports
There are several advantages of using Extent Reports; the major ones are below:
Easy to setup and integrate ExtentReports with Selenium C# frameworks like NUnit, MSTest, and xUnit and Selenium Java frameworks like TestNG, JUnit, etc.
Helps generate super-customizable HTML test reports that help visualize the status of the tests executed on the Selenium Grid.
Stepwise and Pie-chart representations in the NUnit test report provide top-level information on how the tests have fared (i.e., how many passed/failed) on the execution front.
It lets you customize the report’s look & feel through an XML-based configuration file where you can input details such as report theme (Standard/Dark), report title, document title, etc.
It makes it easy to track multiple test cases that are a part of a single test suite.
It can be used with ease even when the test scenarios are executed parallel on a Selenium Grid.
Each test case is accompanied by vital information like logs, results, and the overall time to execute the test.
It lets you capture the ScreenShot with every test step. The captured ScreenShots can help identify the exact step at which the test has failed.
Methods OnNodeAdded, OnTestStarted, OnScreenCaptureAdded, etc., let you perform actions (e.g., adding relevant logs) when that particular ‘event’ (i.e., test has started, screenshot is captured, the test is removed, etc.) has occurred in the tests.
Extent Reports 4 is a significant upgrade over Extent Reports 3, as a lot of new features are introduced in version 4.x of ExtentReports.
New Features Introduced in Extent Reports 4
In case you are currently using Extent Reports 3 as the NUnit reporting tool, you might want to upgrade to Extent Reports 4. In Extent Reports, vital related information like tests, nodes, events, and assignment of tags, devices, environment values, etc., can be printed to ‘multiple destinations.’ These destinations are referred to as ‘reporter.’
Here are the major feature upgrades in Extent Reports 4, most of which are also available for Community Edition of this NUnit report generator:
Extent Reports 4 provides reporters for BDD/Gherkin, non-BDD, and both. Here are some of the reporters available with Extent Reports 4:
ExtentTabularReporter, and many more.
LoggerReporter that eases the navigation from Category (or Bug) view to the Test view.
Provision for adding base64 screenshots to logs through the CreateScreenCaptureFromBase64String method of MediaEntityBuilder class.
Numerous layout improvements for the BDD view.
Support for adding a Gherkin dialect other than ‘en’ or English keywords. The API for setting the dialects is inline with the dialect defined in the Gherkin spec. For example, you can set the Gherkin dialect to German (or de) by using the command extent.GherkinDialect = “de”; in the test code.
The ExtentHtmlReporter method, which is newly introduced in Extent Reports 4, is used for creating HTML reports. ExtentV3HtmlReporter method was earlier used for creating HTML reports with Extent Reports 3.
For migrating from ExtentReports 3 to ExtentReports 4, it is recommended to remove ChartLocation from the setup code as it is no longer available in Extent Reports 4.
How To Use Extent Reports With NUnit And Selenium WebDriver?
The Extent framework follows an Observer pattern where the reporter (e.g., ExtentHTMLReporter) attached to the framework becomes an ‘observer’ and is notified of changes like the creation of logs and logs, etc. Some of the notified changes result in content updation in the report. Though the Extent framework offers numerous capabilities, we would require only certain classes for report generation in NUnit.
Sneak Peek Of The Extent Framework For Report Generation
Here are the two major classes that are frequently used when creating NUnit report using the Extent framework:
ExtentHtmlReporter (for version 4.x) or ExtentV3HtmlReporter (for version 3.x)
We would be using the ExtentHtmlReporter method in the code as our demo will be with ExtentReports 4.
The ‘ExtentHtmlReporter’ class
The ExtentHtmlReporter class is used for generating Extent Report in the HTML format in the path specified as the argument.
It supports two reporter configurations – Default and SPA (Single-Page Applications).
It also provides the flexibility to load the report configurations from a configuration XML file. The config XML file lets you set the theme, document encoding format (e.g., UTF-8), document title, report name, report headline, and more.
Syntax (or sample usage)
The first step is creating an object of the class ExtentHtmlReporter. The optional step is loading the custom report configurations from the configuration XML file. Now that we have started the reporter (i.e., Extent HTML Reporter), the next step is creating the ExtentReports and attaching the reporter(s).
The ‘ExtentReports’ class
The ExtentReports class is primarily used for logging the test steps in the generated HTML report, the path of which was configured using the instance of the ExtentHtmlReporter class.
The AttachReporter method of the ExtentReports class is used for attaching the specified reporter.
The AddSystemInfo method is used for adding system or environment-related information to the report. This information is automatically added to the started reporters (i.e., ExtentHtmlReporter).
The Flush method writes (or updates) the specified reporter’s test information (i.e., Extent HTML Reporter) to the destination type. In most scenarios, the Flush method should be called only ‘once’ in the [OneTimeTearDown] attribute available in the NUnit framework.
Syntax (or sample usage)
The ‘ExtentTest’ Class
The ExtentTest class contains methods for the following tasks:
- Creating tests
- Creating nodes
- Log test steps in the HTML report
- Assigning Device Category
- Adding Screen Capture from a Base64 String
- Adding Screen Capture from a local Path
The CreateTest method is an overloaded method that is used for BDD and non-BDD tests. For non-BDD tests, the CreateTest method is used for setting the test name and test description (optional). It returns an ExtentTest object. Sample usage of the CreateTest method is below:
_test = _extent.CreateTest("Testing Google Search for LambdaTest");
When creating tests using the Gherkin instance, the Gherkin keyword can be directly passed in the CreateTest method.
var feature = extent.CreateTest(new GherkinKeyword("Feature"), "Browser version");
The Log method of the ExtentTest class is used for logging the test status along with a description that needs to be printed on the resultant report. The commonly used statuses are Pass, Fail, and Skip. In totality, the following statuses are available:
In terms of hierarchy, Fatal has the highest level in the hierarchy, and Pass has the lowest (and default) status in the hierarchy. Shown below is the hierarchy from Lowest to Highest:
Pass < Skip < Warning < Error < Fail < Fatal
The RemoveTest method is used to remove a test created using the CreateTest method or CreateNode method.
Screenshots serve as huge value addition in the NUnit test reports. Before ExtentReports 4, the traditional methodology for capturing page screenshots had to be used. With ExtentReports 4, you can add files and base64 snapshots to the tests.
The AddScreenCaptureFromPath method lets you add screenshots to the tests. The image is saved on the disk and referenced using the
< img > attribute in the report. Screenshots can be added to the logs, irrespective of whether the test has passed or failed.
Syntax (or sample usage)
The ‘MediaEntityBuilder’ Class
The MediaEntityBuilder class in the Extent framework lets you add screenshots to logs, as the logs do not accept image paths directly. The CreateScreenCaptureFromPath method of MediaEntityBuilder class should be used for adding a base64 encoded screenshot to the report.
The Build method is used along with CreateScreenCaptureFromPath so that it returns a MediaEntityModelProvider.
We would be covering both the ways (traditional screenshot capturing mechanism and CreateScreenCaptureFromBase64String method in ExtentReports 4) for capturing page screenshots in the demonstration of this NUnit reporting tool.
Syntax (or sample usage)
Now that we have covered the capabilities of ExtentReports 4 as an NUnit report generator let’s look at how we can use Extent Reports with NUnit and Selenium.
Generate Reports In NUnit And Selenium WebDriver
For demonstrating generating Extent Reports in NUnit and Selenium WebDriver, we use the example shown in the Page Object Model Tutorial with Selenium C#.
Here is the test scenario and browser/OS combination on which the tests are executed:
- Go to ‘Google.’
- Search for ‘LambdaTest.’
- Click on the first search result.
- Assert if the title of the newly opened webpage does not match with the expected title.
|BROWSER||BROWSER VERSION||PLATFORM/OPERATING SYSTEM|
|Internet Explorer||11.0||Windows 10|
|Safari||11.0||macOS High Sierra|
To get started, we create a new NUnit test project by navigating to New -> Project -> NUnit Test Project (.Net Core).
Once the project is created, install the following packages using the Package Manager (PM) Console. Run the following commands on the PM console to install Selenium WebDriver, PageObjects, and other packages.
With the basic setup complete, install packages – ExtentReports.Core and ExtentReports from the PM console. At the time of this article, the latest version of Extent Reports (Core) and ExtentReports in the NuGet gallery were 1.0.3 and 4.1.0.
Run the following commands on the PM Console:
Here are the installation screenshots from the PM console:
The status of the installed packages is obtained using the Get-Package PM command:
The project structure should look as shown below:
Page Classes created in < project_folder >/Src/PageObjects/Pages
Test Code created in < project_folder >/Src/Test/Scripts
Extent Report configuration :< project_folder >/report-config.xml
This is how the final project structure in Visual Studio 2019 looks like:
Code WalkThrough [< project_folder >/Src/Test/Scripts/test_POM.cs]
For a code walkthrough of the POM-related implementation, you can refer to the Implementation section in the Page Object Model tutorial for Selenium C#. We would limit this code walkthrough for the NUnit report or Extent Report-related implementation in the test code.
Lines (11) – (12):
Import or add the ExtentReports related namespaces to the test code.
Lines (20)- (22):
The TestFixture annotation (or attribute) is used for passing the browser and OS combinations to the test case.
The test scenario is tested in parallel on different browser and OS combinations. The Parallelizable annotation is used with the ParallelScope.All options for executing the tests and their descendants in parallel.
Lines (35) – (36):
Variables of type ExtentReports and ExtentTest are created. These would be further used in creating the NUnit report.
Lines (46) – (54):
The basic prerequisites for Extent Report creation are added under the [OneTimeSetUp] attribute. The method under this attribute will be called before executing any test in the fixture.
The GetCallingAssembly method returns the Assembly object of the method that has invoked ExtentStart (i.e., the currently executing method). The Uri .LocalPath converts the file path into a URI-style path.
The generated NUnit report will be stored in the Reports folder, and the report name is set to Index.html.
Start the ExtentHTMLReporter file in the location (i.e. reportPath) specified in the earlier step.
var htmlReporter = new ExtentHtmlReporter(reportPath);
Line(61) – (62):
Create an instance of the ExtentReports class and enable (or start) the Extent HTML reporter using the AttachReporter method.
Lines (63) – (65):
Necessary system or environment-related information is added using the AddSystemInfo method to the started HTML reporter.
The custom report configuration is loaded using the report configuration XML file (i.e., report-config.xml).
The Remote WebDriver object is instantiated using LambdaTest’s cloud-based Selenium Grid remote address, desired browser capabilities, and command timeout.
Lines (89) – (98):
The TestContext class in NUnit lets tests access information about the execution context. The CurrentContext property of TestContext class is used for getting the context of the currently executing test (i.e., SearchLT_Google). The test case’s name is appended with the browser name, browser version, and platform to differentiate the tests executing on different browser & OS combinations.
The test is created using the CreateTest method of the Extent framework. The Test case name is passed as the argument to the CreateTest method.
_test = _extent.CreateTest(context_name);
The remaining part of the implementation under the [Test] attribute is POM related, for which you can refer to the POM in Selenium C# blog.
The execution status of the test scenario is obtained using the Result property in the current context.
For demonstration, we capture the page screenshot even for the PASS scenario. The filename will be a combination of the test case name, browser & OS combination, and timestamp when the snapshot was generated.
Lines (137) – (172):
Based on the execution status (i.e., Passed, Failed, Skipped, etc.) of the test scenario, the next set of actions are performed. For demonstration, we capture screenshots in the Extent report for TestStatus.Passed and TestStatus.Failed statuses.
Log events are added to the Extent Report using the appropriate status (i.e., Pass, Fail, etc.).
The screenshots of AUT are captured using two different methods (only for demonstration):
- Capture screenshot using GetScreenshot method of ITakesScreenshot interface (in OpenQA.Selenium Namespace).
- Capture screenshot in base64 encoded string format using CreateScreenCaptureFromBase64String method of the Extent framework. The CreateScreenCaptureFromBase64String method is only available with Extent Reports 4.
We’ll be covering how to capture screenshots in NUnit using Extent Reports in more detail in the subsequent section.
Lines (115) – (120):
The Flush method of ExtentReports writes everything to the log file (or HTML report). It is called only ‘once’ in the [OneTimeTearDown] method. The report will not be generated if Flush is not called.
How to Capture Screenshots in NUnit Extent Report?
Screenshots of AUT during the test execution helps in identifying the potential issues in the application code. It is recommended to capture screenshots only when the test fails since screenshots might consume a lot of memory. However, for demonstration, we capture screenshots even when the test has passed.
Here are the two ways to capture screenshots in Extent Report:
Method – 1
Here we capture a screenshot of the application under test using the GetScreenshot method of the ITakesScreenshot interface. Here is the complete implementation for capturing screenshot:
a. A variable (i.e., ts) of type ITakesScreenshot is created, and the type-casted WebDriver instance is assigned to it.
ITakesScreenshot ts = (ITakesScreenshot)driver;
b. The GetScreenshot method in ITakesScreenshot returns the screenshot object that represents the image of the page on the screen.
Screenshot screenshot = ts.GetScreenshot();
c. The next set of steps create a sub-directory named Screenshots in the Reports directory where the final HTML report will be stored.
d. The SaveAsFile method in Screenshot class (i.e., Screenshot.SaveAsFile) saves the screenshot to a file. It overwrites if the file already exists in that particular location. The method takes two arguments – the location where the image will be stored and ScreenshotImageFormat (which is png in our case).
In our case, the same test is executed in parallel across different browser and OS combinations. Hence, the image name includes the test name, browser(and OS combination), & timestamp when the corresponding test was executed.
Usage: This is how the Capture method is invoked in the test code for saving screenshot on the local disk and adding the same to the HTML report.
Method – 2 (Only in Extent Reports 4)
Here we capture a screenshot of the application under test in the base64 string format using the CreateScreenCaptureFromBase64String method in Extent Reports 4. Here is the complete implementation for capturing screenshot:
a. A variable (i.e., ts) of type ITakesScreenshot is created, and the type-casted WebDriver instance is assigned to it.
b. The screenshot image is captured using the GetScreenshot method of the ITakesScreenshot interface. The value of the screenshot image is fetched as a Base64-encoded string.
var screenshot = ts.GetScreenshot().AsBase64EncodedString;
c. The screenshot image (in Base64-encoded string format) is added to the logs using CreateScreenCaptureFromBase64String method of MediaEntityBuilder class in Extent Reports 4. The Build method returns the screenshot image as MediaEntityModelProvider type.
ExtentTest.Pass, ExtentTest.Fail, ExtentTest.Skip, etc. used for logging respective events in the HTML report have the second argument as MediaEntityModelProvider.
Usage: This is how the CaptureScreenShot method is invoked in the test code for adding base64 screenshots to logs.
As shown below, the test scenario (SearchLT_Google) is run against different browser and OS combinations in parallel.
As seen below, the tests have executed successfully.
The following screenshots are generated in the Reports\Screenshots folder. These screenshots were generated using Method – 1 described in the Capture Screenshots in the NUnit Extent Report section.
Shown below is the Extent Report snapshot (taken from the report file) generated for the tests:
Test 1 (Chrome 86.0 on Windows 10):
Test 2 (IE 11.0 on Windows 10):
Test 3 (Safari 11.0 on macOS High Sierra):
As seen in the report, the base 64 screenshot is shown as “base64 img” with a link to the captured screenshot. The screenshot captured using the GetScreenshot is shown at the bottom of the corresponding test logs.
Reports play an integral part in keeping track of the test execution. Extent Report is used as a preferred NUnit reporting tool since the Extent Framework lets you generate customizable HTML test reports. The NUnit report generated using the Extent Framework provides insightful information about the tests, environment values, devices against which tests were conducted, and more. It also lets you create Gherkin-style tests, due to which Extent Reports can be used as an NUnit report generator for BDD tests. In particular, Extent Reports 4 has several new features like the simplified generation of base64 screenshots to be added to the logs, selecting a Gherkin dialect, etc.
Which NUnit reporting tool do you use for your Selenium tests? Do leave your preference in the comments section below.
Happy Automation Testing.
Published at DZone with permission of Himanshu Sheth. See the original article here.
Opinions expressed by DZone contributors are their own.