Languages

Regular expressions are effective tools for pattern matching and text processing. A regex, or regular expression, is a group of characters that forms a search pattern. To determine whether a string contains a particular search pattern, use RegEx. They are supported by many programming languages, including Python, which provides a powerful and flexible regular expression engine that can handle a wide range of text-matching tasks. This article will provide a brief guide to using regular expressions in Python. What Are Regular Expressions? Regular Expressions (RegEx) are unique character combinations that use a search pattern to locate a string or group of strings, such as finding all email addresses in a document or validating the format of a phone number. It can distinguish between the presence and absence of a text by comparing it to a specific pattern. It may also divide a pattern into one or more sub-patterns. The use of regex in Python is supported via the re-module, which is provided by Python. Its main purpose is to provide a search; to do this, a regular expression and a string are required. In this case, it either returns the first match or none at all. Regular expressions are used to match patterns in text. Regular expressions are often used in text editors, command-line utilities, and programming languages. Regular expressions consist of two types of characters: Literals: These are characters that match themselves. For example, the letter "a" will match the letter "a" in a text string. Metacharacters: These are special characters that have a special meaning. For example, the dot (.) metacharacter matches any single character. Using Regular Expressions in Python Python provides a built-in module called "re" that provides regular expression support. This module provides several functions for working with regular expressions, including searching for matches, replacing matches, and splitting a string into a list of substrings based on a pattern. The "re" module also provides several special characters that can be used to create complex regular expressions. Here are some of the most commonly used special characters in regular expressions: Character Description . any single character, excluding the newline (\n), is matched. For example, the regular expression for he.. will match for “hell”,” help,” etc. * compares to 0 or more occurrences of the preceding character. For example, the regular expression a* will match zero or more occurrences of the letter "a". + matches to one or more occurrences of the preceding character. For example, the regular expression a+ will match one or more occurrences of the letter "a". ? matches either zero or one instance of the preceding character. For example, the regular expression colo?r will match both "color" and "colour". {m,n} Matches the previous character between m and n times. For example, the regular expression a{2,3} will match either "aa" or "aaa". [] Matches any single character within the brackets. For instance, the regular expression [aeiou] will match any kind of vowel. \ Used to drop the special meaning of the character following it. For example, the regular expression \. will match a period character. ^ The string should start with the characters following ^. For example, the regular expression ^hello will match only if the sentence starts with hello. $ The string should end with the characters following $. For example, the regular expression hello$ will match only if the sentence ends with hello. | Either or. For example, the regular expression suman|ritik Check if the string contains either "suman" or "ritik" Let’s discuss some important of these metacharacters in detail: . – Dot Except for the newline character (\n), the dot (.) symbol only recognizes one character. For instance: a.b will look for any character other than a dot in the string, including acb, acbd, abbb, etc. .. will determine whether the string has at least two characters. * – Star Star (*) symbol matches zero or more instances of the regex that comes before the star symbol. For instance: Because b is not followed by c, ab*c will be matched for the strings ac, abc, abbbc, dabc, etc., but not for abdc. + - Plus One or more instances of the regex that comes before the + symbol are matched by the Plus (+) symbol. For instance : Because there is no b in ac and b is not followed by c in abdc, ab+c will match for the string abc, abbc, dabc but not for ac, abdc. ? - Question The question mark (?) determines whether the string in the regex appears at least once or not at all. For instance: As there are two b’s in the string abbc, it will not be matched. However, ab?c will be matched for the strings ac, acb, and dabc. Because b is not followed by c, it will also not match for abdc. Braces {m, n} All repetitions from m to n, inclusive, before the regex are matched by the braces. Example- The strings aaab, baaac, and gaad will be matched for regular expression a{2, 4}, but it won't be matched for strings like abc, bc because there is either just one an or none at all in both situations. Square brackets []. A character class made up of a group of characters that we want to match is represented by square brackets ([]). The character class [abc] will, for instance, match any single a, b, or c. With the - symbol between the square brackets, we can also specify a range of characters. For instance: The sample for [0123] is [0,3]. The sample for [abc] is [a-c]. The caret(^) sign can be used to reverse the character class as well. For instance: [^0-3] denotes any number other than 0 and 1 or 3. [^a-c]Any character that is not an a, b, or c. \ Backslash To ensure that the character is not given special treatment, use the backslash (/). This could be thought of as a metacharacter escape. As an illustration, the dot (.) will be treated as a special character and one of the metacharacters if you want to search for it in the string (as shown in the above table). In order to prevent it from losing its specialization, we will employ the backslash (/) before the dot (.) in this instance. The example below will help you understand. Code: Python import re s = 'suman.singh' # without using \ match = re.search(r'.', s) print(match) # using \ match = re.search(r'\.', s) print(match) Output: Python <re.Match object; span=(0, 1), match='s'> <re.Match object; span=(6, 7), match='.'> | - Or Symbol Determines whether the pattern before or after the or symbol is present in the string. For instance: Any string that contains either an or b, such as acd, bcd, abcd, etc., will be matched by a|b. Special Sequences Special sequences provide the precise position in the search string where the match must take place rather than matching for the actual character in the string. It makes it simpler to write patterns that are used frequently. Special Sequences List Special Sequence Description Examples \A matches if the specified character appears at the start of the string. \Afor -> for suman \b Matches if the provided character either starts or finishes the word. \b(string) will look for the word's beginning, and \b(string) will look for the word's ending. \bsh -> suman \B In contrast to the \b, the string shall not begin or end with the specified pattern. \Bge -> together \d This is similar to the set class [0-9] because it matches any decimal digit. \d -> 1526 \D matches any character that is not a digit; this is the same as the set class [0-9]. \D -> suman \s each whitespace character is a match. \s -> sum an \S any non-whitespace character is a match. \S -> s uman \w This is comparable to the class [a-zA-Z0-9_] and matches any alphanumeric character. \w -> 3425 \W any non-alphanumeric character is matched. \W -> >$ \Z matches if the string contains the specified regex at the end. an\Z -> suman Basic Regular Expression Operations 1. Searching for Matches The most basic operation in regular expressions is searching for a match in a string. The "re" module provides the "search" function for this purpose. Here is an example of how to use the "search" function to find a pattern in a string: Code: Python import re text = "Suman Raghav and Ron are friends" pattern = "friends" result = re.search(pattern, text) if result: print("String Pattern Found") else: print("String Pattern not Found") This code will output "String Pattern Found" because the pattern "friends" is found in the text. 2. Replacing Matches Another common operation in regular expressions is replacing matches in a string. The "re" module provides the "sub" function for this purpose. Here is an example of how to use the "sub" function to replace a pattern in a string: Code: Python import re text = "Suman Raghav and Ron are friends" pattern = "friends" replacement = "students" result = re.sub(pattern, replacement, text) print(result) This code will output "Suman Raghav and Ron are students" because the pattern "friends" is replaced with "students" in the original text. 3. Splitting a String Based on a Pattern The "re" module can also be used to split a string into a list of substrings based on a pattern. The split function is used for this purpose. Here is an example of how to use the "split" function to split a string based on whitespace characters: Code: Python import re text = "Suman Raghav and Ron are friends" result = re.split("\s", text) print(result) This code will output ["Suman", "Raghav", "and", "Ron", "are", "friends"] because the string is split based on whitespace characters. 4. Regular Expression Flags Regular expressions in Python support flags that modify the behavior of the regular expression engine. Flags are specified as an optional second argument to the regular expression function. Some of the most widely used flags are listed below: re.IGNORECASE or re.I: Makes the regular expression case-insensitive. re.MULTILINE or re.M: Allows the ^ and $ metacharacters to match the beginning and end of each line in a multiline string rather than just the beginning and end of the entire string. re.DOTALL or re.S: Makes the dot (.) metacharacter match any character, including a newline character (\n). re.ASCII or re.A: Limits the regular expression engine to ASCII characters only. Here is an example of how to use the IGNORECASE flag to make a regular expression case-insensitive: Code: Python import re text = "Suman has a brown coloured bag." pattern = "BROWN" result = re.search(pattern, text, re.IGNORECASE) if result: print("String Pattern Found") else: print("String Pattern not Found") This code will output "String Pattern Found" because the pattern "BROWN" is found in the text, even though it is in uppercase and the search was performed with the IGNORECASE flag. 5. Grouping and Capturing Regular expressions in Python also support the grouping and capturing of substrings within a match. Grouping is achieved using parentheses (()). The contents of the first group are captured and can be accessed using the "group" method of the match object. Here is an example of how to use grouping and capturing in regular expressions: Code: Python import re text = "Suman Singh (sumansingh@example.com) wrote an email" pattern = "(\w+@\w+\.\w+)" result = re.search(pattern, text) if result: print("Email address validated: " + result.group(1)) else: print("Email address not validated") This code will output "Email address validated: sumansingh@example.com" because the regular expression matches the email address in the text and captures it using a group. Conclusion For text processing and pattern matching, regular expressions are an effective tool. The "re" module in Python provides a flexible and powerful regular expression engine. Special characters such as ., *, +, ?, ^, $, [], (), and | are used to define patterns in regular expressions. The most commonly used regular expression functions in Python are "search", "match", "findall", "sub", and "split". Regular expression flags such as re.IGNORECASE, re.MULTILINE, re.DOTALL, and re.ASCII can modify the behavior of the regular expression engine. Grouping and capturing of substrings within a match can be achieved using parentheses (()) and the "group" method of the match object.

The real name of the CSS variables is CSS Custom Properties is a draft standard (yes, when I wrote these lines, it is on Candidate Recommendation Snapshot), but it is widely supported by modern browsers. CSS variables allow us, like another kind of variable in another programming language, to store a value we can reuse across our document. For example, if we define a CSS variable for the primary color doing the following: --primary-color: #f00;, then we can use it in any component like: .my-component { color: var(--primary-color); } Usually, you “attach” your variable to :root, which means the variable will be available in all the document :root { color: var(--primary-color); } In this example :root is the variable scope. Using Together SCSS If you want to assign values from SCSS variables to CSS variables, you can not do the “normal” notation: // ❌ This doesn't work $scss-var: #f00; --my-var: $scss-var; In the example, the value of --my-var is literally $scss-var, not the value of $scss-var, this behavior was done to provide maximum compatibility with the plain CSS. To make it work, you need to use the Sass interpolation syntax: #{my scss script code}: // ✅ This works $scss-var: #f00; --my-var: #{$scss-var}; Scope The variables are only available in the element where it is defined and its children; that is the scope of the variable. Outside there, the variable doesn’t exist. If you try to access to use a variable that is not in the scope, you will not get an error, but the property that is using the not existing variable will be ignored. Hoisting Like the JS variables, the CSS variables are moved to the top, so you can use them before defining them. .my-element { color: var(--primary-color); } :root { --primary-color: #f00; } Override As I mentioned before, the variables have a scope where the variable exists, but: what happens if a variable with the same name is defined in two scopes: It happens the same as in a JS variable; the near local scope overrides other values: :root { --color: #0f0; } .my-element { --color: #0ff; color: var(--color); } This behavior is very convenient when we work with UI components with different styles depending on modifiers. CSS Variables in UI components Imagine we have a simple button component like that. <button class="ui-button"> Button content </button> .ui-button { background: #333; color: #fff; font-size: 12px; padding: 4px 10px; } This button has different variants by color (default, red and green) and size (default, small and big); using BEM, we can add a modifier class like .ui-button--green or .ui-button--big and use that to overwrite the styles, for example: .ui-button { background: #333; color: #fff; font-size: 12px; padding: 4px 10px; &--green { background: #1F715F; } &--big { font-size: 16px; padding: 6px 20px; } } This way works perfectly, but we need to know which properties to overwrite, and need to do it explicitly for each modifier, so it’s easy to forget something, or if we need to add a new property affected by the modifiers, add it in all of them Suppose we rewrite the styles using CSS variables, parameterizing the component styles. In that case, we can override the CSS variable values for each modifier without changing the CSS styles itself for the modifiers, only changing the value of the variables: .ui-button { --bg-color: #333; --text-color: #fff; --font-size: 12px; --padding: 4px 10px; background: var(--bg-color); color: var(--text-color); font-size: var(--font-size); padding: var(--padding); &--green { --bg-color: #1F715F; } &--red { --bg-color: #0ff; } &--big { --font-size: 16px; --padding: 6px 20px; } &--small { --font-size: 10px; --padding: 3px 5px; } } Variable Scope Priority In CSS, the elements can use more than a class, so that means the element’s CSS variables have multiple scopes at the same level; for example, if we apply the green and red modifiers at the same time <button class="ui-button ui-button--green ui-button--red"> Green + red </button> Both ui-button--green and ui-button--red define the same --bg-color variable, What value will be applied to the element? In cases like that, the class order is the priority, so the last class used overrides the value last, and its value is applied; in the example, the button will be red, but for <button class="ui-button ui-button--red ui-button--green"> the button will be green. Summarizing The use of CSS variables and scopes is a powerful tool when you are developing components in general. Still, if your components have modifiers, it requires extra work in the beginning to parameterize the component, but after that makes it simpler to create variants and modifiers.

We will go over Apache Kafka basics, installation, and operation, as well as a step-by-step implementation using a .NET Core 6 web application. Prerequisites Visual Studio 2022 .NET Core 6 SDK SQL Server Java JDK 11 Apache Kafka Agenda Overview of Event Streaming Introduction to Apache Kafka. Main concepts and foundation of Kafka. Different Kafka APIs. Use cases of Apache Kafka. Installation of Kafka on Windows 10. Step-by-step implementation Overview of Event Streaming Events are the things that happen within our application when we navigate something. For example, we sign up on any website and order something, so, these are the events. The event streaming platform records different types of data like transaction, historical, and real-time data. This platform is also used to process events and allow different consumers to process results immediately and in a timely manner. An event-driven platform allows us to monitor our business and real-time data from different types of devices like IoT and many more. After analyzing, it provides a good customer experience based on different types of events and needs. Introduction to Apache Kafka Below, are a few bullet points that describe Apache Kafka: Kafka is a distributed event store and stream-processing platform. Kafka is open source and is written in Java and Scala. The primary purpose to designed Kafka by Apache foundation is to handle real-time data feeds and provide high throughput and low latency platforms. Kafka is an event streaming platform that has many capabilities to publish (write) and subscribe to (read) streams of events from a different system. Also, to store and process events durably as long as we want, by default, Kafka stores events from seven days of the time period, but we can increase that as per need and requirement. Kafka has distributed system, which has servers and clients that can communicate via TCP protocol. It can be deployed on different virtual machines and containers in on-premise and cloud environments as per requirements. In the Kafka world, a producer sends messages to the Kafka broker. The messages will get stored inside the topics and the consumer subscribes to that topic to consume messages sent by the producer. ZooKeeper is used to manage the metadata of Kafka-related things, it tracks which brokers are part of the Kafka cluster and partitions of different topics. Lastly, it manages the status of Kafka nodes and maintains a list of Kafka topics and messages. Main Concepts and Foundation of Kafka 1. Event An event or record is the message that we read and write to the Kafka server; we do this in the form of events in our business world, and it contains a key, a value, a timestamp, and other metadata headers. The key, value, and time stamp, in this case, are as follows: Key: “Jaydeep” Value: “Booked BMW” Event Timestamp: “Dec. 11, 2022, at 12:00 p.m.” 2. Producer The producer is a client application that sends messages to the Kafka node or broker. 3. Consumer The consumer is an application that receives data from Kafka. 4. Kafka Cluster The Kafka cluster is the set of computers that share the workload with each other with varying purposes. 5. Broker The broker is a Kafka server that acts as an agent between the producer and consumer, who communicate via the broker. 6. Topic The events are stored inside the “topic,” it’s similar to our folder in which we store multiple files. Each topic has one or more producers and consumers, which write and reads data from the topic. Events in “topic” can be read as often as needed because it persists events and it’s not like another messaging system that removes messages after consuming. 7. Partitions Topics are partitions, meaning the topic is spread over multiple partitions that we created inside the topic. When the producer sends some event to the topic, it will store it inside the particular partitions, and then, the consumer can read the event from the corresponding topic partition in sequence. 8. Offset Kafka assigns one unique ID to the message stored inside the topic partition when the message arrives from the producer. 9. Consumer Groups In the Kafka world, the consumer group acts as a single logical unit. 10. Replica In Kafka, to make data fault-tolerant and highly available, we can replicate topics in different regions and brokers. So, in case something wrong happens with data in one topic, we can easily get that from another to replicate the same. Different Kafka APIs Kafka has five core APIs that serve different purposes: Admin API: This API manages different topics, brokers, and Kafka objects. Producer API: This API is used to write/publish events to different Kafka topics. Consumer API: This API is used to receive the different messages corresponding to the topics that are subscribed by the consumer. Kafka Stream API: This API is used to perform different types of operations like windowing, joins, aggregation, and many others. Basically, its use is to transform objects. Kafka Connect API: This API works as a connector to Kafka, which helps different systems connect with Kafka easily. It has different types of ready-to-use connectors related to Kafka. Use Cases of Apache Kafka Messaging User activity tracking Log aggregation Stream processing Realtime data analytics Installation of Kafka on Windows 10 Step 1 Download and install the Java SDK of version 8 or more. Note: I have Java 11, that’s why I put the same path in all commands that I used here. Step 2 Open and install EXE. Step 3 Set the environment variable for Java using the command prompt as admin. Command: setx -m JAVA_HOME “C:\Program Files\Java\jdk-11.0.16.1” setx -m PATH “%JAVA_HOME%\bin;%PATH%” Step 4 After that, download and install Apache Kafka. Step 5 Extract the downloaded Kafka file and rename it “Kafka.” Step 6 Open D:\Kafka\config\ and create a “zookeeper-data” and “kafka-logs” folder inside that. Step 7 Next, open D:\Kafka\config\zookeeper.properties file and add the folder path inside that: D:\Kafka\config\zookeeper.properties dataDir=D:/Kafka/zookeeper-data Step 8 After that, open D:\Kafka\config\server.properties file and change the log path over there: D:\Kafka\config\server.properties log.dirs=D:/Kafka/kafka-logs Step 9 Saves and close both files. Step 10 Run ZooKeeper: D:\Kafka> .\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties Step 11 Start Kafka: D:\Kafka> .\bin\windows\kafka-server-start.bat .\config\server.properties Step 12 Create Kafka topic: D:\Kafka\bin\windows>kafka-topics.bat — create — bootstrap-server localhost:9092 — replication-factor 1 — partitions 1 — topic testdata Step 13 Create a producer and send some messages after you’ve started a producer and consumer: D:\Kafka\bin\windows>kafka-console-producer.bat — broker-list localhost:9092 — topic testdata Step 14 Next, create a consumer. After, you will see the message the producer sent: D:\Kafka\bin\windows>kafka-console-consumer.bat — bootstrap-server localhost:9092 — topic testdata Step-by-Step Implementation Let’s start with practical implementation. Step 1 Create a new .NET Core Producer Web API: Step 2 Configure your application: Step 3 Provide additional details: Step 4 Install the following two NuGet packages: Step 5 Add configuration details inside the appsettings.json file: JSON { "Logging": { "LogLevel": { "Default": "Information", "Microsoft.AspNetCore": "Warning" } }, "AllowedHosts": "*", "producerconfiguration": { "bootstrapservers": "localhost:9092" }, "TopicName": "testdata" } Step 6 Register a few services inside the “Program” class: C# using Confluent.Kafka; var builder = WebApplication.CreateBuilder(args); // Add services to the container. var producerConfiguration = new ProducerConfig(); builder.Configuration.Bind("producerconfiguration", producerConfiguration); builder.Services.AddSingleton<ProducerConfig>(producerConfiguration); builder.Services.AddControllers(); // Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle builder.Services.AddEndpointsApiExplorer(); builder.Services.AddSwaggerGen(); var app = builder.Build(); // Configure the HTTP request pipeline. if (app.Environment.IsDevelopment()) { app.UseSwagger(); app.UseSwaggerUI(); } app.UseHttpsRedirection(); app.UseAuthorization(); app.MapControllers(); app.Run(); Step 7 Next, create the CarDetails model class: C# using Microsoft.AspNetCore.Authentication; namespace ProducerApplication.Models { public class CarDetails { public int CarId { get; set; } public string CarName { get; set; } public string BookingStatus { get; set; } } } Step 8 Now, create the CarsController class: C# using Confluent.Kafka; using Microsoft.AspNetCore.Mvc; using Microsoft.Extensions.Configuration; using Newtonsoft.Json; using ProducerApplication.Models; namespace ProducerApplication.Controllers { [Route("api/[controller]")] [ApiController] public class CarsController : ControllerBase { private ProducerConfig _configuration; private readonly IConfiguration _config; public CarsController(ProducerConfig configuration, IConfiguration config) { _configuration = configuration; _config = config; } [HttpPost("sendBookingDetails")] public async Task<ActionResult> Get([FromBody] CarDetails employee) { string serializedData = JsonConvert.SerializeObject(employee); var topic = _config.GetSection("TopicName").Value; using (var producer = new ProducerBuilder<Null, string>(_configuration).Build()) { await producer.ProduceAsync(topic, new Message<Null, string> { Value = serializedData }); producer.Flush(TimeSpan.FromSeconds(10)); return Ok(true); } } } } Step 9 Finally, run the application and send a message: Step 10 Now, create a “consumer” application: For that, create a new .NET Core console application: Step 11 Configure your application: Step 12 Provide additional information: Step 13 Install the NuGet below: Step 14 Add the following code, which consumes messages sent by the consumer: C# using Confluent.Kafka; var config = new ConsumerConfig { GroupId = "gid-consumers", BootstrapServers = "localhost:9092" }; using (var consumer = new ConsumerBuilder<Null, string>(config).Build()) { consumer.Subscribe("testdata"); while (true) { var bookingDetails = consumer.Consume(); Console.WriteLine(bookingDetails.Message.Value); } } Step 15 Finally, run the producer and consumer, send a message using the producer app, and you will see the message immediately inside the consumer console sent by the producer: Here is the GitHub URL I used in this article. Conclusion Here, we discussed Apache Kafka introduction, working, benefits, and step-by-step implementation using .NET Core 6. Happy Coding!

Retrieving the page source of a website under scrutiny is a day-to-day task for most test automation engineers. Analysis of the page source helps eliminate bugs identified during regular website testing, functional testing, or security testing drills. In an extensively complex application testing process, automation test scripts can be written in a way that if errors are detected in the program, then it automatically: Saves that particular page’s source code. Notifies the person responsible for the URL of the page. Extracts the HTML source of a specific element or code-block and delegates it to the responsible authorities if the error has occurred in one particular independent HTML WebElement or code block. This is an easy way to trace and fix logical and syntactical errors in the front-end code. In this article, we first understand the terminologies involved and explore how to get the page source in Selenium WebDriver using Python. What Is an HTML Page Source? In non-technical terminology, it’s a set of instructions for browsers to display info on the screen in an aesthetic fashion. Browsers interpret these instructions in their own ways to create browser screens for the client-side. These are usually written using HyperText Markup Language (HTML), Cascading Style Sheets (CSS), and Javascript. This entire set of HTML instructions that make a web page is called page source, HTML source, or simply source code. Website source code is a collection of source code from individual web pages. Here’s an example of a source code for a basic page with a title, form, image, and submit button. <!DOCTYPE html> <html> <head> <title>Page Source Example - LambdaTest</title> </head> <body> <h2>Debug selenium testing results : LambdaTest</h2> <img loading="lazy" data-fr-src="https://cdn.lambdatest.com/assetsnew/images/debug-selenium-testing-results.jpg" alt="debug selenium testing" width="550" height="500"><br><br> <form action="/"> <label for="debug">Do you debug test results using LambdaTest?</label><br> <input type="text" id="debug" name="debug" value="Of-course!"><br> <br> <input type="submit" value="Submit"> </form> <br><br> <button type="button" onclick="alert('Page Source Example : LambdaTest!')">Click Me!</button> </body> </html> What Is an HTML Web Element? The easiest way to describe an HTML web element would be, “any HTML tag that constitutes the HTML page source code is a web element.” It could be an HTML code block, an independent HTML tag like </br>, a media object on the web page—image, audio, video, a JS function, or a JSON object wrapped within <script> </script> tags. In the above example, <title> is an HTML web element, and the children of body tags are HTML web elements too, i.e., <img>, <button>, etc. How To Get Page Source in Selenium WebDriver Using Python Selenium WebDriver is a robust automation testing tool and provides automation test engineers with a diverse set of ready-to-use APIs. To make Selenium WebDriver get page source, Selenium Python bindings provide us with a driver function called page_source to get the HTML source of the currently active URL in the browser. Alternatively, we can also use the GET function of Python’s request library to load the page source. Another way is to execute JavaScript using the driver function execute_script and make Selenium WebDriver get page source in Python. A unrecommended way of getting page source is using XPath in tandem with the “view-source:” URL. Let’s explore examples for these four ways of how to get page source in Selenium WebDriver using Python. We’ll be using a sample small web page hosted on GitHub for all four examples. This page was created to demonstrate drag and drop testing in Selenium Python using LambdaTest. Get HTML Page Source Using driver.page_source We’ll fetch pynishant.github.io in the ChromeDriver and save its content to a file named page_source.html. This file name could be anything of your choice. Next, we read the file’s content and print it on the terminal before closing the driver: from selenium import webdriver driver = webdriver.Chrome() driver.maximize_window() driver.get("https://pynishant.github.io/") pageSource = driver.page_source fileToWrite = open("page_source.html", "w") fileToWrite.write(pageSource) fileToWrite.close() fileToRead = open("page_source.html", "r") print(fileToRead.read()) fileToRead.close() driver.quit() On successful execution of the above script, your terminal output will show the following page source: Get HTML Page Source Using driver.execute_javascript In the previous example, we have to comment out (or replace) the driver.page_source line and add the following line: driver.execute_script is a Selenium Python WebDriver API to execute JS in a Selenium environment. Here, we execute a JS script that returns an HTML body element. # pageSource = driver.page_source pageSource = driver.execute_script("return document.body.innerHTML;") The output code looks like this: As you can observe, it only returns the innerHTML of the body element. Like the last output, we do not get the whole page source. To get the entire document, we execute document.documentElement.outerHTML. The execute_script line now looks like this: pageSource = driver.execute_script("return document.documentElement.outerHTML;") This gives us precisely the output we got using the driver.page_source. Fetch Page Source Using Python’s Request Library in Selenium WebDriver This method has nothing to do with Selenium but you can check the “What Is Selenium?” article, it’s a purely Pythonic way to get a webpage source. Here, we use Python’s request library to make a get request to the URL and save the request’s response, i.e., page source to an HTML file and print on the terminal. Here is the script: import requests url = 'https://pynishant.github.io/' pythonResponse = requests.get(url) fileToWrite = open("py_source.html", "w") fileToWrite.write(pythonResponse.text) fileToWrite.close() fileToRead = open("py_source.html", "r") print(fileToRead.read()) fileToRead.close() This method can be used to quickly store a webpage source code without loading the page in the Selenium-controlled browser. Similarly, we can use the urllib Python library to fetch the HTML page source. Get HTML Page Source Using the “view-source” URL This is rarely required, but you can append the target URL with view-source and load it in the browser window to load the source code and save it in manual testing: Programmatically, to take source code of screenshots in Python Selenium (if required), you can load the page using: driver.get("view-source:https://pynishant.github.io/") Get HTML Page Source in Selenium Python WebDriver Using XPath The fourth method to make Selenium WebDriver get a page source is to use XPath for saving it. Here, instead of page_source or executing JavaScript, we identify the source element, i.e., <html> and extract it. Comment out the previous page source fetching logic and replace it with the following: # pageSource = driver.page_source pageSource = driver.find_element_by_xpath("//*").get_attribute("outerHTML") In the above script, we are using a driver method, find_element_by_xpath, to locate the web page’s HTML element. We enter the document using source nod:"//*" and get its “outer HTML,” which is the document itself. The output looks the same as we got earlier using driver.page_source. How To Retrieve HTML Source of WebElement in Selenium To get the HTML source of a WebElement in Selenium WebDriver, we can use the get_attribute method of the Selenium Python WebDriver. First, we grab the HTML WebElement using driver element locator methods like (find_element_by_xpath or find_element_by_css_selector). Next, we apply the get_attribute() method on this grabbed element to get it’s HTML source. Suppose, from pynishant.github.io, and we want to grab and print the source code of the div with id “div1.” The code for this looks like this: from selenium import webdriver driver = webdriver.Chrome() driver.maximize_window() driver.get("https://pynishant.github.io/") elementSource = driver.find_element_by_id("div1").get_attribute("outerHTML") print(elementSource) driver.quit() Here’s the output: Similarly, to get the children or innerHTML of a WebElement: driver.find_element_by_id("some_id_or_selector").get_attribute("innerHTML") There is an alternative way of doing this and achieving same result: elementSource = driver.find_element_by_id("id_selector_as_per_requirement") driver.execute_script("return arguments[0].innerHTML;", elementSource) How To Retrieve JSON Data from an HTML Page Source in Python Selenium WebDriver Modern applications are built with multiple APIs at play. And often, these API dynamically change the content of HTML elements. JSON objects have emerged as an alternative to XML response types. So, it has become essential for a pro Selenium Python tester to handle JSON objects, especially those embedded in <script> HTML tags. Python provides us with an in-built JSON library to experiment with JSON objects. To demonstrate with an example, we load “https://www.cntraveller.in/” in Selenium driver and look-out for SEO schema contained in <script type=”application/ld+json”> </script> to verify that logo URL is included in the “JSON” schema. By the way, if you feel confused, this “SEO schema” is useful to get web pages ranked on google. It has nothing to do with code-logic or testing. We’re using it just for demonstration. We’ll be using LambdaTest for this demo: from selenium import webdriver import json import re username = "hustlewiz247" accessToken = "1BtTGpkzkYeOKJiUdivkWxvmHQppbahpev3DpcSfV460bXq0GC" gridUrl = "hub.lambdatest.com/wd/hub" desired_cap = { 'platform' : "win10", 'browserName' : "chrome", 'version' : "71.0", "resolution": "1024x768", "name": "LambdaTest json object test ", "build": "LambdaTest json object test", "network": True, "video": True, "visual": True, "console": True, } url = "https://"+username+":"+accessToken+"@"+gridUrl print("Initiating remote driver on platform: "+desired_cap["platform"]+" browser: "+desired_cap["browserName"]+" version: "+desired_cap["version"]) driver = webdriver.Remote( desired_capabilities=desired_cap, command_executor= url ) # driver = webdriver.Chrome() driver.maximize_window() driver.get("https://www.cntraveller.in/") jsonSource = driver.find_element_by_xpath("//script[contains(text(),'logo') and contains(@type, 'json')]").get_attribute('text') jsonSource = re.sub(";","",jsonSource) jsonSource = json.loads(jsonSource) if "logo" in jsonSource: print("\n logoURL : " + str(jsonSource["logo"])) else: print("JSON Schema has no logo url.") try: if "telephone" in jsonSource: print(jsonSource["telephone"]) else: print("No Telephone - here is the source code :\n") print(driver.find_element_by_xpath("//script[contains(text(),'logo') and contains(@type, 'json')]").get_attribute('outerHTML')) except Exception as e: print(e) driver.quit() The output contains logoURL and webElement source: Code Breakdown The following three lines import required libraries: Selenium WebDriver, Python’s JSON, and re library to handle JSON objects and use regular expressions: from selenium import webdriver import json import re Next, we configure our script for running it successfully on LambdaTest’s cloud. It took me less than thirty seconds to get started (maybe because I had prior experience with the platform). But even if you are a first-timer, it would take less than one minute. Register on LambdaTest’s official website, login using Google, and click on “Profile” to copy your username and access token: username = "your_username_on_lambdaTest" accessToken = "your lambdaTest access token" gridUrl = "hub.lambdatest.com/wd/hub" desired_cap = { 'platform' : "win10", 'browserName' : "chrome", 'version' : "71.0", "resolution": "1024x768", "name": "LambdaTest json object test ", "build": "LambdaTest json object test", "network": True, "video": True, "visual": True, "console": True, } url = "https://"+username+":"+accessToken+"@"+gridUrl We launch the driver in full-screen mode and load the cntraveller home page with the following line of code: driver = webdriver.Remote( desired_capabilities=desired_cap, command_executor= url ) # driver = webdriver.Chrome() driver.maximize_window() driver.get("https://www.cntraveller.in/") Now, we locate JSON objects containing script using the XPath locator and delete the unnecessary semicolons to load the string in JSON format properly: jsonSource = driver.find_element_by_xpath("//script[contains(text(),'logo') and contains(@type, 'json')]").get_attribute('text') jsonSource = re.sub(";","",jsonSource) jsonSource = json.loads(jsonSource) And then, we check if the logo URL is present. If present, we print it: if "logo" in jsonSource: print("\n logoURL : " + str(jsonSource["logo"])) else: print("JSON Schema has no logo url.") Also, we check if the telephone detail is present. If not, we print the source code of the WebElement: try: if "telephone" in jsonSource: print(jsonSource["telephone"]) else: print("No Telephone - here is the source code :\n") print(driver.find_element_by_xpath("//script[contains(text(),'logo') and contains(@type, 'json')]").get_attribute('outerHTML')) except Exception as e: print(e) Lastly, we quit the driver: driver.quit() How To Get Page Source as XML in Selenium WebDriver If you’re loading an XML-rendered website, you may want to save the XML response. Here’s a working solution for making Selenium get XML page source: drive.execute_script(‘return document.getElementById(“webkit-xml-viewer-source-xml”).innerHTML’) Conclusion You can use any of the above-demonstrated methods and leverage the agility and scalability of LambdaTest Selenium Grid cloud to automate your test processes. It lets you execute your test cases on 3000+ browsers, operating systems, and their versions. Also, you can integrate the automation testing flow with modern CI/CD tools and adhere to the best continuous testing practices. Happy Testing!

When I started working on this post, I had another idea in mind: I wanted to compare the developer experience and performance of Spring Boot and GraalVM with Rust on a demo HTTP API application. Unfortunately, the M1 processor of my MacBook Pro had other ideas. Hence, I changed my initial plan. I'll write about the developer experience of developing the above application in Rust, compared to what I'm used to with Spring Boot. The Sample Application Like every pet project, the application is limited in scope. I designed a simple CRUD HTTP API. Data are stored in PostgreSQL. When one designs an app on the JVM, the first and only design decision is to choose the framework: a couple of years ago, it was Spring Boot. Nowadays, the choice is mostly between Spring Boot, Quarkus, and Micronaut. In many cases, they all rely on the same underlying libraries, e.g., logging or connection pools. Rust is much younger; hence the ecosystem has yet to mature. For every feature, one needs to choose precisely which library to use - or to implement it. Worse, one needs to understand there's such a feature. Here are the ones that I searched for: Reactive database access Database connection pooling Mapping rows to structures Web endpoints JSON serialization Configuration from different sources, e.g., YAML, environment variables, etc. Web Framework The choice of the web framework is the most critical. I've to admit I had no prior clue about such libraries. I looked around and stumbled upon Which Rust web framework to choose in 2022. After reading the post, I decided to follow the conclusion and chose axum: Route requests to handlers with a macro-free API Declaratively parse requests using extractors Simple and predictable error handling model Generate responses with minimal boilerplate Take full advantage of the tower and tower-http ecosystem of middleware, services, and utilities. In particular, the last point is what sets axum apart from other frameworks. axum doesn’t have its own middleware system but instead uses tower::Service. This means axum gets timeouts, tracing, compression, authorization, and more, for free. It also enables you to share middleware with applications written using hyper or tonic. - axum crate documentation axum uses the Tokio asynchronous library underneath. For basic usage, it requires two crates: TOML [dependencies] axum = "0.6" tokio = { version = "1.23", features = ["full"] } axum's router looks very similar to Spring's Kotlin Routes DSL: Rust let app = Router::new() .route("/persons", get(get_all)) //1 .route("/persons/:id", get(get_by_id)) //1//2 async fn get_all() -> Response { ... } async fn get_by_id(Path(id): Path<Uuid>) -> Response { ... } A route is defined by the path and a function reference. A route can have path parameters. axum can infer parameters and bind them. Shared Objects An issue commonly found in software projects is sharing an "object" with others. We established long ago that there were better ideas than sharing global variables. Spring Boot (and similar JVM frameworks) solves it with runtime dependency injection. Objects are created by the framework, stored in a context, and injected into other objects when the application starts. Other frameworks do dependency injection at compile-time, e.g., Dagger 2. Rust has neither runtime nor objects. Configurable dependency injection is not "a thing." But we can create a variable and inject it manually where needed. In Rust, it's a problem because of ownership: Ownership is a set of rules that govern how a Rust program manages memory. All programs have to manage the way they use a computer’s memory while running. Some languages have garbage collection that regularly looks for no-longer-used memory as the program runs; in other languages, the programmer must explicitly allocate and free the memory. Rust uses a third approach: memory is managed through a system of ownership with a set of rules that the compiler checks. If any of the rules are violated, the program won’t compile. None of the features of ownership will slow down your program while it’s running. - "What Is Ownership?" axum provides a dedicated wrapper, the State extractor, to reuse variables across different scopes. Rust struct AppState { //1 ... } impl AppState { fn create() -> Arc<AppState> { //2 Arc::new(AppState { ... }) } } let app_state = AppState::create(); let app = Router::new() .route("/persons", get(get_all)) .with_state(Arc::clone(&app_state)); //3 async fn get_all(State(state): State<Arc<AppState>>) -> Response { //4 ... //5 } Create the struct to be shared. Create a new struct wrapped in an Atomically Reference Counted. Share the reference with all routing functions, e.g., get_all. Pass the state. Use it! Automated JSON Serialization Modern JVM web frameworks automatically serialize objects in JSON before sending. The good thing is that axum does the same. It relies on Serde. First, we add the serde and serde_json crate dependencies: TOML [dependencies] serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" Then, we annotate our struct with the derive(Serialize) macro: Rust #[derive(Serialize)] struct Person { first_name: String, last_name: String, } Finally, we return the struct wrapped in a Json and the HTTP status code in an axum Response. Rust async fn get_test() -> impl IntoResponse { //1 let person = Person { //2 first_name: "John".to_string(), last_name: "Doe".to_string() }; (StatusCode::OK, Json(person)) //3 } The tuple (StatusCode, Json) is automatically converted into a Response. Create the Person. Return the tuple. At runtime, axum automatically serializes the struct in JSON: JSON {"first_name":"Jane","last_name":"Doe"} Database Access For a long time, I used the MySQL database for my demos, but I started to read a lot of good stuff about PostgreSQL and decided to switch. I needed an asynchronous library compatible with Tokio: it's exactly what the tokio_postgres crate does. The problem with the crate is that it creates direct connections to the database. I searched for a connection pool crate and stumbled upon deadpool (sic): Deadpool is a dead simple async pool for connections and objects of any type. - Deadpool Deadpool provides two distinct implementations: An unmanaged pool: The developer has complete control - and responsibility - over the pooled objects' lifecycle. A managed pool: The crate creates and recycles objects as needed. More specialized implementations of the latter cater to different databases or "drivers", e.g., Redis and... tokio-postgres. One can configure Deadpool directly or defer to the config crate it supports. The latter crate allows several alternatives for configuration: Config organizes hierarchical or layered configurations for Rust applications. Config lets you set a set of default parameters and then extend them via merging in configuration from a variety of sources: Environment variables String literals in well-known formats Another Config instance Files: TOML, JSON, YAML, INI, RON, JSON5, and custom ones defined with Format trait Manual, programmatic override (via a .set method on the Config instance) Additionally, Config supports: Live watching and re-reading of configuration files Deep access into the merged configuration via a path syntax Deserialization via serde of the configuration or any subset defined via a path - Crate config To create the base configuration, one needs to create a dedicated structure and use the crate: Rust #[derive(Deserialize)] //1 struct ConfigBuilder { postgres: deadpool_postgres::Config, //2 } impl ConfigBuilder { async fn from_env() -> Result<Self, ConfigError> { //3 Config::builder() .add_source( Environment::with_prefix("POSTGRES") //4 .separator("_") //4 .keep_prefix(true) //5 .try_parsing(true), ) .build()? .try_deserialize() } } let cfg_builder = ConfigBuilder::from_env().await.unwrap(); //6 The Deserialize macro is mandatory. The field must match the environment prefix (see below). The function is async and returns a Result. Read from environment variables whose name starts with POSTGRES_. Keep the prefix in the configuration map. Enjoy! Note that environment variables should conform to what Deadpool's Config expects. Here's my configuration in Docker Compose: Env variable Value POSTGRES_HOST "postgres" POSTGRES_PORT 5432 POSTGRES_USER "postgres" POSTGRES_PASSWORD "root" POSTGRES_DBNAME "app" Once we have initialized the configuration, we can create the pool: Rust struct AppState { pool: Pool, //1 } impl AppState { async fn create() -> Arc<AppState> { //2 let cfg_builder = ConfigBuilder::from_env().await.unwrap(); //3 let pool = cfg_builder //4 .postgres .create_pool( Some(deadpool_postgres::Runtime::Tokio1), tokio_postgres::NoTls, ) .unwrap(); Arc::new(AppState { pool }) //2 } } Wrap the pool in a custom struct. Wrap the struct in an Arc to pass it within an axumState (see above). Get the configuration. Create the pool. Then, we can pass the pool to the routing functions: Rust let app_state = AppState::create().await; //1 let app = Router::new() .route("/persons", get(get_all)) .with_state(Arc::clone(&app_state)); //2 async fn get_all(State(state): State<Arc<AppState>>) -> Response { let client = state.pool.get().await.unwrap(); //3 let rows = client .query("SELECT id, first_name, last_name FROM person", &[]) //4 .await //5 .unwrap(); // //6 } Create the state. Pass the state to the routing functions. Get the pool out of the state, and get the client out of the pool. Create the query. Execute it. Read the row to populate the Response. The last step is to implement the transformation from a Row to a Person. We can do it with the From trait. Rust impl From<&Row> for Person { fn from(row: &Row) -> Self { let first_name: String = row.get("first_name"); let last_name: String = row.get("last_name"); Person { first_name, last_name, } } } let person = row.into(); Docker Build The last step is the building of the application. I want everybody to be able to build, so I used Docker. Here's the Dockerfile: Dockerfile FROM --platform=x86_64 rust:1-slim AS build //1 RUN rustup target add x86_64-unknown-linux-musl //2 RUN apt update && apt install -y musl-tools musl-dev //3 WORKDIR /home COPY Cargo.toml . COPY Cargo.lock . COPY src src RUN --mount=type=cache,target=/home/.cargo \ //4 && cargo build --target x86_64-unknown-linux-musl --release //5 FROM scratch //6 COPY --from=build /home/target/x86_64-unknown-linux-musl/release/rust /app //7 CMD ["/app"] Start from a standard Rust image. Add musl target so we can compile to Alpine Linux. Install the required Alpine dependencies. Cache the dependencies. Build for Alpine Linux. Start from scratch. Add the previously built binary. The final image is 7.56MB. My experience has shown that an equivalent GraalVM native compiled image would be more than 100MB. Conclusion Though it was not my initial plan, I learned about quite a few libraries with this demo app and how they work. More importantly, I've experienced what it is like to develop an app without a framework like Spring Boot. You need to know the following: Available crates for each capability Crate compatibility Version compatibility Last but not least, the documentation of most above crates ranges from average to good. I found axum's to be good; on the other hand, I didn't manage to use Deadpool correctly from the start and had to go through several iterations. The documentation quality of Rust crates is different from crate to crate. All in all, they have room for the potential to reach the level of modern JVM frameworks. Also, the demo app was quite simple. I assume that more advanced features could be more painful. The complete source code for this post can be found on GitHub. To go further: Create an Optimized Rust Alpine Docker Image How to create small Docker images for Rust Using Axum Framework To Create Rest API

A Software Bill of Materials (SBOM) is getting more and more important in the software supply chain. In this blog, you will learn what an SBOM is and how to build the SBOM in an automated way. Enjoy! 1. Introduction An SBOM is a list of software components that makes up a software product. This way, it becomes transparent which software libraries, components, etc., and their versions are used in the software product. As a consequence, you will be able to react more adequately when a security vulnerability is reported. You only need to check the SBOMs for the vulnerable library, and you will know immediately which applications are impacted by the vulnerability. The SBOM of a library or application you want to use can also help you in your decision-making. It will become more common ground that software suppliers will be forced to deliver an up-to-date SBOM with each software delivery. Based on this information, you can make a risk assessment of whether you want to use the library or application. When you are a software supplier, you need to ensure that you deliver an SBOM with each software release. This actually means that you need to create the SBOM in an automated way, preferably in your build pipeline. As written before, the SBOM can help you to check whether a library used in your application contains security vulnerabilities. With the proper tooling, this check can be done in an automated way in your build pipeline. When security vulnerabilities are found, you can fail the build pipeline. One of these tools is grype, which can take an SBOM as input and check whether any components are used with known security vulnerabilities. In a previous post, it is explained how grype can be used. In the post, a Docker image is input to grype, but it is even better to create an SBOM first and then provide it to grype. How to create the SBOM will be explained in this post. When you start reading about SBOMs, you will notice that two standards are commonly used: CycloneDX: an open-source project originated within the OWASP community; SPDX (The Software Package Data Exchange): an international open standard format, also open source and hosted by the Linux foundation. So, which standard to use? This is a difficult question to answer. Generally, it is stated that both standards will continue to exist next to each other, and tools are advised to support both standards. SPDX is initially set up for license management, whereas CycloneDX had its primary focus on security. Reading several resources, the preferred format is CycloneDX when your focus is set on security. Interesting reads are SBOM formats SPDX and CycloneDX compared and the publication Using the Software Bill of Materials for Enhancing Cybersecurity from the National Cyber Security Centre of the Ministry of Justice and Security of the Netherlands. The latter is a must-read. In the remainder of this blog, you will learn how to use syft for building an SBOM. Syft is also a product of Anchore, just like grype is, and therefore integrates well with grype, the vulnerability scanning tool. Syft supports many ecosystems and has several export formats. It definitely supports CycloneDX and SPDX. CycloneDX also has tools for building SBOMs, but Syft is one tool that supports many ecosystems, which is an advantage compared to multiple tools. Sources being used in this post are available at GitHub. 2. Prerequisites The prerequisites needed for this blog are: Basic Linux knowledge; Basic Java knowledge; Basic JavaScript knowledge; Basic Spring Boot knowledge. 3. Application Under Test Before continuing, you need an application to build the SBOM. This is a basic application that consists of a Spring Boot backend and a Vue.js frontend. The application can be built with Maven and contains two Maven modules, one for the backend and one for the front end. More information about the setup can be read in a previous post. It is important, however, to build the application first. This can be done with the following command: Shell $ mvn clean verify 4. Installation Installation of syft can be done by executing the following script: Shell $ curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sudo sh -s -- -b /usr/local/bin Verify the installation by executing the following command: Shell $ syft --version syft 0.64.0 5. Build Backend SBOM Navigate to the backend directory and execute the following command: Shell $ syft dir:. --exclude ./**/sbom.*.json --output cyclonedx-json=sbom.cyclonedx.build-complete-dir.json The parameters will do the following: dir:.: Scan the entire directory in order to find dependencies; –exclude: Exclude already present SBOM files because you want to generate the SBOM file every time anew based on the current state of the repository; –output: Here, you define the output format to use, and you define the file name of the SBOM file. The SBOM file sbom.cyclonedx.build-complete-dir.json is created in the backend directory. Take a closer look at the SBOM format. JSON { "bomFormat": "CycloneDX", "specVersion": "1.4", "serialNumber": "urn:uuid:afbe7b48-b376-40fb-a0d4-6a16fda38a0f", "version": 1, "metadata": { "timestamp": "2023-01-14T16:35:35+01:00", "tools": [ { "vendor": "anchore", "name": "syft", "version": "0.64.0" } ], "component": { "bom-ref": "af63bd4c8601b7f1", "type": "file", "name": "." } }, "components": [ ... ] } The top part consists of metadata: the format of the SBOM, versions used, which tool is being used, etc. The component part consists of a list of all the components syft has found. The complete specification of CycloneDX can be found here. The component list is the following and corresponds to the list of libraries that can be found in the target/backend-0.0.1-SNAPSHOT.jar file. The libraries are located in the directory /BOOT-INF/lib/ in the jar file (the jar file is just a zip file and can be opened with any archive tool). Plain Text backend jackson-annotations jackson-core jackson-databind jackson-datatype-jdk8 jackson-datatype-jsr310 jackson-module-parameter-names jakarta.annotation-api jul-to-slf4j log4j-api log4j-to-slf4j logback-classic logback-core micrometer-commons micrometer-observation slf4j-api snakeyaml spring-aop spring-beans spring-boot spring-boot-autoconfigure spring-boot-jarmode-layertools spring-boot-starter-test spring-boot-starter-web spring-context spring-core spring-expression spring-jcl spring-web spring-webmvc tomcat-embed-core tomcat-embed-el tomcat-embed-websocket Now take a closer look at the jackson-annotations component in the SBOM file. In the properties section, you can see that this component has a property syft:package:foundBy with the value java-cataloger. This means that this component was found in the jar file. JSON { "bom-ref": "pkg:maven/com.fasterxml.jackson.core/jackson-annotations@2.14.1?package-id=9cdc3a1e17ebbb68", "type": "library", "group": "com.fasterxml.jackson.core", "name": "jackson-annotations", "version": "2.14.1", "cpe": "cpe:2.3:a:jackson-annotations:jackson-annotations:2.14.1:*:*:*:*:*:*:*", "purl": "pkg:maven/com.fasterxml.jackson.core/jackson-annotations@2.14.1", "externalReferences": [ { "url": "", "hashes": [ { "alg": "SHA-1", "content": "2a6ad504d591a7903ffdec76b5b7252819a2d162" } ], "type": "build-meta" } ], "properties": [ { "name": "syft:package:foundBy", "value": "java-cataloger" }, { "name": "syft:package:language", "value": "java" }, { "name": "syft:package:metadataType", "value": "JavaMetadata" }, { "name": "syft:package:type", "value": "java-archive" }, ... ] } When you take a look at component spring-boot-starter-web, it mentions that this component was found by java-pom-cataloger. This means that this component was found in the pom file. This is quite interesting because this would mean that syft cannot find transitive dependencies based on the sources only. Execute the following command where the target directory is excluded from the analysis. Shell $ syft dir:. --exclude ./**/sbom.*.json --exclude ./**/target --output cyclonedx-json=sbom.cyclonedx.build-sources.json The result can be found in the file sbom.cyclonedx.build-sources.json and the previously made assumption seems to be right. Only the spring-boot-starter-web and spring-boot-starter-test dependencies are found. This is, after all, not a big issue, but you have to be aware of this. 6. Build Frontend SBOM Navigate to the frontend directory and execute the following command: Shell $ syft dir:. --exclude ./**/sbom.*.json --output cyclonedx-json=sbom.cyclonedx.build-complete-dir.json This analysis takes a bit longer than the backend analysis, but after a few seconds, the sbom.cyclonedx.build-complete-dir.json file is created. Again, similar information can be found in the SBOM. The information is now available from the javascript-lock-cataloger. This means that it originates from the package-lock.json file. Another difference is that the components also contain license information. JSON "components": [ { "bom-ref": "pkg:npm/%40babel/parser@7.20.7?package-id=ca6a526d8a318088", "type": "library", "name": "@babel/parser", "version": "7.20.7", "licenses": [ { "license": { "id": "MIT" } } ], ... License information is included and can be used to check regarding allowed company policies. This information is, however, not yet available for Java packages. 7. Conclusion SBOMs will become more and more important in the software development lifecycle. More and more customers will demand an SBOM, and therefore it is important to automatically generate the SBOM. Syft can help you with that. Besides that, the SBOM can be fed to grype in order to perform a security vulnerability analysis.

As we continue in this series of simulating and troubleshooting performance problems in Scala, now let’s discuss how to simulate the java.lang.OutOfMemoryError: Java Heap space problem. java.lang.OutOfMemoryError: Java Heap space will be thrown by the Scala application when it generates more objects than the maximum configured heap size. Scala OutOfMemoryError Program Here is a sample Scala program, which generates the java.lang.OutOfMemoryError: Java Heap space problem: package com.yc import java.util class OOMApp { } object OOMApp { var myMap = new util.HashMap[String,String]() def main(args: Array[String]): Unit = { var counter = 0; while (true) { System.out.println("Inserting large String") myMap.put("key"+counter, "Large stringgggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + "ggggggggggggggggggggggggggggggggggggggggggggggggggggg" + counter) counter = counter+(1); } } } This sample Scala program contains an OOMApp class. Since this class contains the while(true) loop, it keeps on inserting records into the HashMap infinitely. When HashMap grows beyond the maximum heap size (i.e., -Xmx), the java.lang.OutOfMemoryError: Java Heap space will be thrown. The diagram below illustrates the records present in the HashMap. HashMap causing OutOfMemoryError When we executed the program above, as expected, java.lang.OutOfMemoryError: Java heap space was thrown within a few seconds. Even though this is a hypothetical example that simulates java.lang.OutOfMemoryError, this is how a typical memory leak happens in enterprise applications. When records get inserted into a data structure (like HashMap, ArrayList, Set, etc.) and never get removed, java.lang.OutOfMemoryError will be thrown. How To Troubleshoot OutOfMemoryError You can diagnose OutOfMemoryError either through a manual or automated approach. Manual Approach In the manual approach, you will need to capture a heap dump as the first step. A heap dump is a snapshot of memory that will show all the objects in memory, the values contained by those objects, and their references. You can capture a heap dump using one of the 7 approaches given here, but an important criterion is that you need to capture the heap dump right before OutOfMemoryError is thrown. If you are going to capture a heap dump after OutOfMemoryError has occurred, then leaking objects can get garbage collected, and it will become hard (or even impossible) to diagnose the problem. Once heap dumps are captured, you need to import the heap dumps from your production servers to your local machine. From your local machine, you can use heap dump analysis tools like jHat and HeapHero to analyze the heap dumps. Automated Approach On the other hand, you can also use the yCrash open source script, which would capture 360-degree data (GC log, 3 snapshots of thread dump, heap dump, netstat, iostat, vmstat, top, top -H, etc.) from your application stack within a minute and generate a bundle zip file. You can then either manually analyze these artifacts or upload them to the yCrash server for automated analysis. We used the automated approach. Once the captured artifacts were uploaded to the yCrash server, it instantly generated the below root cause analysis report highlighting the source of the problem. Heap dump analysis report generated by the yCrash tool yCrash tool pointing out the root cause of OutOfMemoryError The heap dump analysis report above is from the tool which precisely points out that the HashMap (i.e., myMap) data structure present in the com.yc.OOMApp$.myMap to be the root cause of the memory leak. Also, the tool is reporting that this HashMap is holding 99% of memory. Equipped with this information one can easily go ahead and fix the problematic code. Video To see the visual walk-through of this post, click below:

This article is for developers and managers who have heard about Rust but do not know much about it. It is not a tutorial. I write what I see and give some unsolicited—or assuming you read it not by accident, then solicited—advice. Introduction This article is a few words about Rust. Rust is a ten-year-old programming language. In my opinion, this language will shape the industry in the future comparable to the impact of Java. Time will tell. New programming languages arise all the time, and then they get forgotten. I remember when Roger Lang told me in 1997 that Java was a paper tiger. We worked for Digital Equipment Corporation in the Hungarian subsidiary, and Java was relatively new. The expression “paper tiger” references the Chinese proverb, “The tiger that is paper is not a tiger.” He failed.Java did not.Java was a tiger, though an infant.Some believed it even then, but nobody could “know.”The future of Java was not sure. The same is with Rust now.Although it is a ten-year-old language, it is still an infant. Technically, it is not. There are many solutions in the language and in the whole infrastructure architecture that are mature and brilliant. What is still missing is community and industry acceptance. My prediction is: Rust will step over the industry acceptance threshold in 2023, becoming a player in the industry; you can be confident it will not fade and be forgotten soon. Before going on with the rest of the article, I must make a statement: I am not a Rust expert. I have recently started learning the language and am still a beginner. I have written more than a simple “Hello,World!” but that is it. On the other hand, I have more than forty years of programming experience if we start counting from my first program line, which was executed by a processor (TI-59 calculator) in 1980. I programmed in FORTRAN, BASIC, Pascal, C, C++, Java, JavaScript, Python, and many other languages. Java, as you can see from the topic of the article, is my main charter for now. May Rust be the future. In this article, I will write a few words about Rust. I will not teach you Rust. (After the disclaimer, this should be evident.) I will write down some of the features I felt were important and interesting. I try to focus on the essential features from the commercial point of view. The reason is that they are the features that will make Rust a success or failure. You, as a developer, may like some features. You may say, “how cool Rust is because of this or that feature.” But, if the feature is not essential for the industry, or the managers, then it is not crucial for Rust. A language is not a hobby project. This article is about what I have seen so far and what I imply from it for the future of the language. Rust Is a Systems Programming Language Being a Java developer, I do not feel threatened by Rust. Rust is a system programming language like C or C++. It is a robust language, and I believe writing enterprise applications in it is possible, but not now. Rust compiles to machine code or WebAssembly. There is no virtual machine and no garbage collection. It means the performance is comparable to C and C. I would not write it is faster than Java because even C and C are slower than Java in some particular cases. The important thing is that there are no hiccups in the performance during garbage collection. I acknowledge that Java has dramatically improved and has a highly optimized garbage collector. Still, it collects the garbage and then releases it in batches, affecting performance. Google’s Golang garbage collector is also fast. However, comparing it to any other GC, you should remember that it does not compact the heap. It simply can not because the language uses pointers, and as such, it is prone to memory fragmentation. By the way: using pointers also makes Rust prone to this problem. Memory Safety This is a technical section in this article. If you are a manager and impatient, here it is in one sentence: Rust uses a memory management approach, which delivers the same level of safety as Java without the garbage collection overhead. Note: Managers can skip to the next section. Memory safety is the feature that makes Rust outstanding and unique. C trusts the developer not to make mistakes. It is a wrong choice. Developers make mistakes, Rust does not. Developers have to know when the code is not using an allocated memory chunk anymore, and they have to free it. This is something a compiler could do. In principle, a compiler could identify the memory chunks not used anymore and generate code to free them. They don’t do that because the task is computationally expensive. In short, the compiler cannot identify when to free the memory. On the other end of the scale, Java safeguards everything using the garbage collector. In this case, the run-time keeps track of all the references and frees the memory when it is not referenced anymore. Rust language developers did not accept the statement that the compiler could not do it. They approached the problem again, and found a solution. The solution is called “ownership.” The reference to a memory location is either owning the resource or borrowing it. There can be only one resource owner, just like in real life. At first, it is a bit hard to grasp the concept and needs a bit of different thinking than C or Java development. However, it is a brilliant solution. Application Management Rust does not have built-in application management. The reason for that is Rust is a systems programming language. When you have a JVM process, you can attach a debugger, query the memory and thread parameters, and make a memory dump. With Rust, all you can do is what the operating system lets you do. When you run an enterprise application that runs for several months between two maintenance stops for the ubiquitous version update, you need information about the running system to know how well it performs. Rust may have and certainly will have libraries and tools in the future addressing this issue, but it is not part of the ecosystem and the default run-time. Ease of Learning This is a critical point. Rust is not easy to learn. Writing the first “Hello, World!” and having it up and running took a few days. I feel the weight of the language and ecosystem. This is good news for the developers. If you know Rust, you have little competition. Rust developers are expensive. For managers, it is bad news. Rust developers are expensive. Language Features There are many great features in Rust that developers can like. The code, after you learn the language, can be very readable. However, it can also be very unreadable. There are Rust quizzes on the net that I do not understand after a month of learning the language. Not being able to solve it would be natural, but I often do not understand the explanation. It is partially my age; forty years of experience does not come young, but it is also the complexity of the language. I usually say that the important question is not: BAD QUESTION: How readable code can someone write in a language? but rather: GOOD QUESTION: How unreadable code can a junior developer, who thinks they mastered it all, write? In this regard, Rust is guilty. The language is open for the developers to be extended. The developers can write macros, which are part of the language. It is similar to the C preprocessor, but macros run during, instead of before, the compilation. The input for the macros is the lexical tokens, and the output is the lexical tokens. Note: To be precise, the macros work on a lexical token tree, not a stream. It simply means the paired symbols, like ( and ), should be in pairs, or else the compiler will complain before passing the lexical tree to the macro. Java, for example, says that such a “preprocessor” or “macro” is counterintuitive. The language is powerful enough. If you need any macro to write your code readable, you have done something wrong; you must learn to use the language better. Developers, on the other hand, welcome such a possibility. We love to use the tool and not to “learn it more.” This is the main reason why projects, like Lombok, are so hated and loved at the same time. (In addition to the fact that Lombok is using undocumented API.) Rust is on the other end of the scale. It provides not only macro processing but also allows the developers to interpret the token tree in Rust. A code written in Rust will start during the compilation and transform part of the lexical tree to another. Using this possibility, you can significantly alter the language by giving different meanings and semantics to some constructs. Eventually, you should not do that; it was never intended to introduce procedural macros. A “clever” junior; however, could. Backward Compatibility Backward compatibility is a critical issue. Pulling an old 8mm magnetic tape zip archive with some Java 1.0 code will compile and run with the newest Java version. You can compile with the latest C compiler code written in 1970. (Most of the cases.) However, as we can see in the case of Java, this hinders language development. My favorite example is final. By the time Java was introduced, the obvious and reasonable choice was to make everything non final by default. Hence, we have the final keyword in Java and do not have something like nonfinal. Today, most of us feel that final would be a better default, but it is too late to change it. Rust cuts this Gordian knot. Rust introduces the edition mechanism. There are different editions of Rust, which are essentially language versions. Inside an edition, a new version is always backward compatible. Some old conventions can be thrown away when a new edition comes out. The new compiler; however, can still work with the old code. It just has to know which edition it is, and the edition is stated in the project’s Cargo.toml file. Overall, the use of editions in Rust is an innovative approach to address backward compatibility. Its effectiveness and impact depends on how it is adopted and used by the Rust developers’ community. Libraries Libraries are also vital parts of the ecosystem. There are libraries for Rust, but not as vast as for Java. It clearly shows that mainly enthusiasts support Rust. Most of the libraries look like well-designed open-source hobby projects. Documented, but not excessively, covers most use cases, but sometimes not all of them. There are rooms for library development. Tooling Rust tooling is excellent. Eventually, not as good as Java, but it is getting there. I tried to develop a Rust application using IntelliJ IDEA and CLion. It works; you can also debug the language interactively. I compiled my sample codes on an Apple M1 machine and coded to generate code for Intel Linux, Intel Windows, and WebAssembly. Cross-compilation is a strong point of Rust. There are two comments, however, that I got from Kristof Szabados and Irine Kokilashvili. One is that you should avoid cross-compilation if possible. It is not Rust specific. Cross compilation, setting up the environment contains many pitfalls that target architecture compilation does not have. Cross compilation is usually a practice for embedded systems. The other comment is that Rust compilation, using LLVM, targets only those provided by LLVM. C language using GCC has a wider breath. Implications Rust is an important language we have to pay attention to. Use Rust? The big question is: should you use Rust? When should you use Rust? If you are a developer, allocate time to learn it. You will need devotion because it is not trivial. Find some hobby projects or libraries that you will develop in Rust. Keep Rust in your toolset; it will pay back sooner or later. If you are a manager, know about it. Encourage your developers to learn it. If your actual project is low-level, more or less a systems programming project, then Rust must be considered. I do not advocate selecting Rust, but it must be considered. The decision has to be made based on non-technical requirements. Rust is mature enough to do anything that C can do. If there is something prohibited by compile time memory safety, you can still use unsafe code as a last resort. Technical requirements make no difference in this case. The questions you have to ask are: How long will the project run? How long will you need to maintain it? How many developers will work on the project? Can you afford to have a Rust developer? Can you educate? Using Rust at the moment is a significant investment. Rust developers are not available in the market significant amount. It is a good choice if you can afford to educate your developers to learn Rust. What Will Rust do ? My opinion is that Rust will be used in many areas. Systems Programming For low-level systems programming, it will definitely be a significant player. The promise to create a memory-safe, “unhackable” program is strong. There is no general guarantee for the resulting code 100% secure, but a significant share of the security issues use exploits based on memory safety problems. It is also something I genuinely believe Rust delivers. Application Programming However, this is not the whole story. Rust has the potential to be a significant player in enterprise application development. It has to evolve to have more commercial-grade libraries and a non-niche developer base to get there. Web UI Development I also see the possibility of it being a player in Web UI development. The reason for the possibility is WebAssembly. Compiling the code to run in the browser is a prerequisite, but not enough. The main reason, strangely, is the complexity of Rust. The needed and delivered code quality of UI projects reflecting the experience of the average UI developer is usually low. UI developers are typically young and less experienced. This also means an issue when you have a project that needs more experience. Finding a senior UI developer may be difficult in the swarm of young developers. On the other hand, when you find a developer who can program in Rust, you are more likely to find someone who is a senior developer. Summary and Conclusion Rust is a language we have to pay attention to. It is technically mature, and will become commercially mature in the future. 2023 is the year (in my opinion) when it will pass the tipping point. As a developer, you should learn it. As a manager, you should know about it.

In this article, we cover keyset pagination and infinite scroll via jOOQ. The schema used in the examples is available here. You may also like "We need tool support for keyset pagination." jOOQ Keyset Pagination Keyset (or seek) pagination doesn't have a default implementation in Spring Boot, but this shouldn't stop you from using it. Simply start by choosing a table's column that should act as the latest visited record/row (for instance, the id column), and use this column in the WHERE and ORDER BY clauses. The idioms relying on the ID column are as follows (sorting by multiple columns follows this same idea): SQL SELECT ... FROM ... WHERE id < {last_seen_id} ORDER BY id DESC LIMIT {how_many_rows_to_fetch} SELECT ... FROM ... WHERE id > {last_seen_id} ORDER BY id ASC LIMIT {how_many_rows_to_fetch} Or, like this: SQL SELECT ... FROM ... WHERE ... AND id < {last_seen_id} ORDER BY id DESC LIMIT {how_many_rows_to_fetch} SELECT ... FROM ... WHERE ... AND id > {last_seen_id} ORDER BY id ASC LIMIT {how_many_rows_to_fetch} Expressing these queries in jOOQ should be a piece of cake. For instance, let's apply the first idiom to the PRODUCT table via PRODUCT_ID: Java List<Product> result = ctx.selectFrom(PRODUCT) .where(PRODUCT.PRODUCT_ID.lt(productId)) .orderBy(PRODUCT.PRODUCT_ID.desc()) .limit(size) .fetchInto(Product.class); In MySQL, the rendered SQL is (where productId = 20 and size = 5) as follows: SQL SELECT `classicmodels`.`product`.`product_id`, `classicmodels`.`product`.`product_name`, ... FROM `classicmodels`.`product` WHERE `classicmodels`.`product`.`product_id` < 20 ORDER BY `classicmodels`.`product`.`product_id` DESC LIMIT 5 This was easy! You can practice this case in KeysetPagination for MySQL. In the same place, you can find the approach for PostgreSQL, SQL Server, and Oracle. However, keyset pagination becomes a little bit trickier if the WHERE clause becomes more complicated. Fortunately, jOOQ saves us from this scenario via a synthetic clause named SEEK. Let's dive into it! The jOOQ SEEK Clause The jOOQ synthetic SEEK clause simplifies the implementation of keyset pagination. Among its major advantages, the SEEK clause is type-safe and is capable of generating/emulating the correct/expected WHERE clause (including the emulation of row value expressions). For instance, the previous keyset pagination example can be expressed using the SEEK clause, as shown here (productId is provided by the client): Java List<Product> result = ctx.selectFrom(PRODUCT) .orderBy(PRODUCT.PRODUCT_ID) .seek(productId) .limit(size) .fetchInto(Product.class); Note that there is no explicit WHERE clause. jOOQ will generate it on our behalf based on the seek() arguments. While this example may not look so impressive, let's consider another one. This time, let's paginate EMPLOYEE using the employee's office code and salary: Java List<Employee> result = ctx.selectFrom(EMPLOYEE) .orderBy(EMPLOYEE.OFFICE_CODE, EMPLOYEE.SALARY.desc()) .seek(officeCode, salary) .limit(size) .fetchInto(Employee.class); Both officeCode and salary are provided by the client, and they land into the following generated SQL sample (where officeCode = 1, salary = 75000, and size = 10): SQL SELECT `classicmodels`.`employee`.`employee_number`, ... FROM `classicmodels`.`employee` WHERE (`classicmodels`.`employee`.`office_code` > '1' OR (`classicmodels`.`employee`.`office_code` = '1' AND `classicmodels`.`employee`.`salary` < 75000)) ORDER BY `classicmodels`.`employee`.`office_code`, `classicmodels`.`employee`.`salary` DESC LIMIT 10 Check out the generated WHERE clause! I am pretty sure that you don't want to get your hands dirty and explicitly write this clause. How about the following example? Java List<Orderdetail> result = ctx.selectFrom(ORDERDETAIL) .orderBy(ORDERDETAIL.ORDER_ID, ORDERDETAIL.PRODUCT_ID.desc(), ORDERDETAIL.QUANTITY_ORDERED.desc()) .seek(orderId, productId, quantityOrdered) .limit(size) .fetchInto(Orderdetail.class); And the following code is a sample of the generated SQL (where orderId = 10100, productId = 23, quantityOrdered = 30, and size = 10): SQL SELECT `classicmodels`.`orderdetail`.`orderdetail_id`, ... FROM `classicmodels`.`orderdetail` WHERE (`classicmodels`.`orderdetail`.`order_id` > 10100 OR (`classicmodels`.`orderdetail`.`order_id` = 10100 AND `classicmodels`.`orderdetail`.`product_id` < 23) OR (`classicmodels`.`orderdetail`.`order_id` = 10100 AND `classicmodels`.`orderdetail`.`product_id` = 23 AND `classicmodels`.`orderdetail`.`quantity_ordered` < 30)) ORDER BY `classicmodels`.`orderdetail`.`order_id`, `classicmodels`.`orderdetail`.`product_id` DESC, `classicmodels`.`orderdetail`.`quantity_ordered` DESC LIMIT 10 After this example, I think it is obvious that you should opt for the SEEK clause and let jOOQ do its job! Look, you can even do this: Java List<Product> result = ctx.selectFrom(PRODUCT) .orderBy(PRODUCT.BUY_PRICE, PRODUCT.PRODUCT_ID) .seek(PRODUCT.MSRP.minus(PRODUCT.MSRP.mul(0.35)), val(productId)) .limit(size) .fetchInto(Product.class); You can practice these examples in SeekClausePagination, next to the other examples, including using jOOQ-embedded keys as arguments of the SEEK clause. Implementing Infinite Scroll Infinite scroll is a classical usage of keyset pagination and is gaining popularity these days. For instance, let's assume that we plan to obtain something, as shown in this figure: So, we want an infinite scroll over the ORDERDETAIL table. At each scroll, we fetch the next n records via the SEEK clause: Java public List<Orderdetail> fetchOrderdetailPageAsc(long orderdetailId, int size) { List<Orderdetail> result = ctx.selectFrom(ORDERDETAIL) .orderBy(ORDERDETAIL.ORDERDETAIL_ID) .seek(orderdetailId) .limit(size) .fetchInto(Orderdetail.class); return result; } The fetchOrderdetailPageAsc() method gets the last visited ORDERDETAIL_ID and the number of records to fetch (size), and it returns a list of jooq.generated.tables.pojos.Orderdetail, which will be serialized in JSON format via a Spring Boot REST controller endpoint defined as @GetMapping("/orderdetail/{orderdetailId}/{size}"). On the client side, we rely on the JavaScript Fetch API (of course, you can use XMLHttpRequest, jQuery, AngularJS, Vue, React, and so on) to execute an HTTP GET request, as shown here: JavaScript const postResponse = await fetch('/orderdetail/${start}/${size}'); const data = await postResponse.json(); For fetching exactly three records, we replace ${size} with 3. Moreover, the ${start} placeholder should be replaced by the last visited ORDERDETAIL_ID, so the start variable can be computed as the following: Java start = data[size-1].orderdetailId; While scrolling, your browser will execute an HTTP request at every three records, as shown here: http://localhost:8080/orderdetail/0/3 http://localhost:8080/orderdetail/3/3 http://localhost:8080/orderdetail/6/3 … You can check out this example in SeekInfiniteScroll. Infinite Scrolling and Dynamic Filters Now, let's add some filters for ORDERDETAIL that allows a client to choose the price and quantity ordered range, as shown in this figure: We can easily implement this behavior by fusing the powers of SEEK and SelectQuery: Java public List<Orderdetail> fetchOrderdetailPageAsc( long orderdetailId, int size, BigDecimal priceEach, Integer quantityOrdered) { SelectQuery sq = ctx.selectFrom(ORDERDETAIL) .orderBy(ORDERDETAIL.ORDERDETAIL_ID) .seek(orderdetailId) .limit(size) .getQuery(); if (priceEach != null) { sq.addConditions(ORDERDETAIL.PRICE_EACH.between( priceEach.subtract(BigDecimal.valueOf(50)), priceEach)); } if (quantityOrdered != null) { sq.addConditions(ORDERDETAIL.QUANTITY_ORDERED.between( quantityOrdered - 25, quantityOrdered)); } return sq.fetchInto(Orderdetail.class); } The following example URL involves loading the first page of three records that have prices between 50 and 100 and an order quantity between 50 and 75: http://localhost:8080/orderdetail/0/3?priceEach=100&quantityOrdered=75 You can find the complete example in SeekInfiniteScrollFilter for MySQL, SQL Server, PostgreSQL, and Oracle.

Code coverage is a software quality metric commonly used during the development process that let’s you determine the degree of code that has been tested (or executed). To achieve optimal code coverage, it is essential that the test implementation (or test suites) tests a majority percent of the implemented code. There are a number of code coverage tools for languages like Java, C#, JavaScript, etc. Using the best-suited code coverage tool is important to understand the percentage of code tested and take appropriate actions to ensure that you achieve the ideal code coverage. For optimal code testing, many companies use the JaCoCo-Maven plugin that helps generate detailed code coverage reports. The JaCoCo-Maven plugin is a free code coverage library for Java projects. It is based on the study of existing integration libraries that were created by the EclEmma team. At a larger extent, code coverage does give a brief overview of the product quality because the higher the coverage, lesser are the chances of untested code getting into the release cycle. In this article, we will learn more about the JaCoCo-Maven plugin and how this plugin is implemented using Maven. We will also integrate a Maven project and generate a detailed code coverage report for the tests by performing Selenium. What Is Code Coverage? In software development, code coverage is a measure used to describe the degree to which the source code of an application is executed when a test suite is executed. A report is generated to view and analyze the code coverage of a software application. This code coverage report could then be used for ensuring code quality. The JaCoCo-Maven plugin is used to generate code coverage reports. Source code with high code coverage has more of its code executed during testing. For example, if the software you are testing contains 100 lines of code and the number of code lines validated in the software is 90, then the code coverage percentage of that software application will be 90%. Benefits of Code Coverage Code coverage is a very useful metric for developers, testers, and QA engineers. Here are some of the salient benefits of code coverage: Code coverage reports provide useful insights in detection and elimination of dead code. This can be avoided by following implementation best-practices, which, in turn, results in improved code maintainability and better product quality. Quality assurance can help detect code that has not been covered using the test implementation. Developers can finish the software development process faster, increasing their productivity, scalability, and efficiency. This results in reduced Time to Market (TTM). As we know, code coverage is very important for every software product. Now that we have done a quick recap about the integral aspects of code coverage, let’s deep dive into our core topic, i.e., generating code coverage reports using the JaCoCo-Maven plugin. What is the JaCoCo-Maven Plugin? The JaCoCo-Maven (abbreviation for Java Code Coverage) plugin is an open-source code coverage tool for Java. It creates code coverage reports and integrates well with IDEs (Integrated development environments), like Eclipse IDE. It also integrates smoothly with CI/CD tools (e.g. Jenkins, Circle CI, etc.) and project management tools (e.g., SonarQube, etc.). It is a part of the Eclipse Foundation and has replaced the EclEmma code coverage tool in Eclipse. How Does the JaCoCo-Maven Plugin Work? The JaCoCo-Maven plugin runs the coverage by instrumenting Java code through a runtime agent. In simple terms, you attach this agent to a JVM (Java Virtual Machine) when it starts. This agent is termed as JaCoCo agent. The first execution start-agent starts this JaCoCo runtime agent. Whenever a class is loaded, JaCoCo can instrument the class so it can view when the class is called and what lines (of code) are called during the testing process. By keeping this track, it builds up the code coverage statistics, which is done on the fly during the second execution (i.e., generate-report). By default, the file is created as soon as the JVM terminates, but it is also possible to run the agent in server mode. This triggers a dump of the results and the report is created before the termination. Shown below are the internals of the JaCoCo plugin: You can define goals and rules in the configuration of the JaCoCo-Maven plugin. This offers you the flexibility to set limits and helps in checking the amount of code coverage. The Maven-surefire plugin is the default Maven plugin. This runs the tests in JVM and provides coverage reports. While the JaCoCo plugin instruments the code already executed by a plugin (e.g., Surefire plugin). Thus, it is a good practice to check for the dependency of the maven-surefire plugin. Why Is the JaCoCo-Maven Plugin Good for Code Coverage? The JaCoCo-Maven plugin is appropriate for code coverage because of the following reasons: While working on any project, developers mostly prefer IDEs because it simplifies the coding and testing experience. JaCoCo can be installed on Eclipse IDE in the name of EclEmma, by downloading EclEmma from its marketplace. It is easy to add the JaCoCo plugin to all types of builds, including ANT, Maven, and Gradle. It can also be integrated with CI/CD tools like Jenkins, Circle CI, etc. This makes it versatile for a lot of use cases. The code coverage report generated by JaCoCo is a simple and informative HTML file that can be viewed in any browser or IDE. JaCoCo also provides offline instrumentation (i.e., all the classes are instrumented before running any tests). Analysis of a report is also quite easy, as it is color-based and provides the exact percentage of code coverage. How To Set up the JaCoCo Plugin With Maven To get code coverage reports in a Maven project, we first need to set up the JaCoCo Maven plugin for that project. By integrating the JaCoCo plugin, the results of the code coverage analysis can be reviewed as an HTML report. The current version of the JaCoCo-Maven plugin can be downloaded from the MVN Repository. Here are the steps to integrate the JaCoCo Maven plugin with a Maven project: 1. Every Maven project has a pom.xml file, used to declare all the dependencies and plugins. The JaCoCo-Maven plugin is declared in the same POM.xml file. The XML code for the same is: <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.6</version> </plugin> This is the basic XML code added under the build tag for specifying the JaCoCo plugin in a Maven-based project. We can enhance the functionality (like mentioning when a report should be generated, etc.) by specifying goals and rules in the execution tag. 2. After the version tag, we add the execution tag. This tag prepares the properties or execution to point to the JaCoCo agent and is passed as a VM (in this case, JVM) argument. 3. For running simple unit tests, two goals set in execution tags will work fine. The bare minimum is to set up a prepare-agent and report goals. <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.6</version> <executions> <execution> <id>prepare-agent</id> <goals> <goal>prepare-agent</goal> </goals> </execution> <execution> <id>report</id> <phase>test</phase> <goals> <goal>report</goal> </goals> </execution> </executions> </plugin> Prepare-agent goal: The prepare-agent goal prepares the JaCoCo runtime agent to record the execution data. It records the number of lines executed, backtraced, etc. By default, the execution data is written to the file target/jacoco-ut.exec. Report goal: The report goal creates code coverage reports from the execution data recorded by the JaCoCo runtime agent. Since we have specified the phase property, the reports will be created after the compilation of the test phase. By default, the execution data is read from the file target/jacoco-ut.exec, and the code coverage report is written to the directory target/site/jacoco/index.html. 4. For running simple unit tests, the above configuration works fine. However, we would need constraints to be put on the code coverage reports (e.g., specify the destination directory, etc). This can be done via a configuration tag: <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.6</version> <executions> <execution> <id>prepare-agent</id> <goals> <goal>prepare-agent</goal> </goals> <configuration> <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile> <propertyName>surefireArgLine</propertyName> </configuration> </execution> <execution> <id>report</id> <phase>test</phase> <goals> <goal>report</goal> </goals> <configuration> <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile> <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory> </configuration> </execution> </executions> </plugin> Configuration of First Execution As per the above code, you can see some tags, like destFile, etc., are specified. Here is a brief description of the tags: destFile tag: The destFile tag is used for setting the path to the file containing the execution data. propertyName-surefireArgLine tag: This tag sets the name of the property that contains the settings for the JaCoCo runtime agent. This also sets the VM argument line when the unit tests are run. Configuration of Second Execution As per the above code, you can see tags like dataFile, etc., are specified. Here is a brief description of the tags: dataFile tag: The dataFile tag is used to set the path to the file containing execution data. outputDirectory tag: This tag sets the output directory for the code coverage reports. 5. We can also add rules to our configuration tag to keep a check on the code coverage percentage. This can be done as shown below: <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.6</version> <executions> <execution> <id>prepare-agent</id> <goals> <goal>prepare-agent</goal> </goals> <configuration> <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile> <propertyName>surefireArgLine</propertyName> </configuration> </execution> <execution> <id>report</id> <phase>test</phase> <goals> <goal>report</goal> </goals> <configuration> <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile> <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory> </configuration> </execution> <execution> <id>jacoco-check</id> <goals> <goal>check</goal> </goals> <configuration> <rules> <rule> <element>PACKAGE</element> <limits> <limit> <counter>LINE</counter> <value>COVEREDRATIO</value> <minimum>0.50</minimum> </limit> </limits> </rule> </rules> </configuration> </execution> </executions> </plugin> Configuration of Third Execution Here, a new goal check is defined. The jacoco:check goal is bound to verify the rule specified. The rule is given in the rule tag. You have the flexibility to specify more than one rule. Element tag: This tag specifies the element on which the rule has to be applied. Limit tag: Tags like counter, value, minimum, etc., are used to limit the code coverage percentage. A command like mvn clean verify can be used for ensuring whether the rule is followed or not. 6. There are multiple goals and rules that can be defined in the JaCoCo-Maven plugin configuration. Code Coverage Reports Using Maven and the JaCoCo Plugin In this section, we will demonstrate the steps to generate code coverage report using the JaCoCo Maven plugin. The demonstration is done by taking a very simple test scenario. So, let’s get started. Prerequisites Maven 3.0 or higher: Maven is a project development management and comprehension tool. You can install the latest version from the official Apache Maven website. Java 1.5 or higher: You need to ensure that Java is installed on your machine. In case you do not have Java installed, please download the latest version of Java from the official Java website. Eclipse IDE for Java Developers: Though you can use the IDE of your choice, for this demo we have used the Eclipse IDE. Steps To Create a Simple Maven Project In Eclipse IDE, Go to File > New > Maven Project. 2. A new dialog box appears. Make sure that the “Use default Workspace location” checkbox is ticked and click “Next.” 3. For selecting the archetype in the project, type org.apache.maven in the textbox that is located next to the filter. Select maven-archetype-quickstart and click “Next.” 4. Now, specify the “Group Id” as com.example and the “Artifact Id” as jacoco-example. The “Artifact Id” is our project name. Finally, click on the “Finish” button. 5. You can see the project file and folder hierarchy in “Project Explorer.” How To Specify the JaCoCo Maven Plugin in POM.xml 1. Open POM.xml, scroll to tag. We will specify the JaCoCo-Maven plugin in the section that lists the Maven plugins. Just copy the below code and paste it in the above tag: <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.6</version> <executions> <!--first execution : for preparing JaCoCo runtime agent--> <execution> <id>prepare-agent</id> <goals> <goal>prepare-agent</goal> </goals> </execution> <!--second execution : for creating code coverage reports--> <execution> <id>report</id> <phase>test</phase> <goals> <goal>report</goal> </goals> </execution> </executions> </plugin> 2. Since we are demonstrating a report generation for automated web testing with JUnit, we will also declare the JUnit dependency in POM.xml. Here is the entire content of POM.xml: <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.example</groupId> <artifactId>jacoco-example</artifactId> <version>0.0.1-SNAPSHOT</version> <name>jacoco-example</name> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <maven.compiler.source>1.7</maven.compiler.source> <maven.compiler.target>1.7</maven.compiler.target> </properties> <!-- JUnit dependencies added to run test cases --> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <!-- Maven plugin for Project Management --> <plugin> <artifactId>maven-clean-plugin</artifactId> <version>3.0.0</version> </plugin> <!-- see http://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging --> <plugin> <artifactId>maven-resources-plugin</artifactId> <version>3.0.2</version> </plugin> <plugin> <artifactId>maven-compiler-plugin</artifactId> <version>3.7.0</version> </plugin> <plugin> <artifactId>maven-surefire-plugin</artifactId> <version>2.19.1</version> </plugin> <plugin> <artifactId>maven-jar-plugin</artifactId> <version>3.0.2</version> </plugin> <plugin> <artifactId>maven-install-plugin</artifactId> <version>2.5.2</version> </plugin> <plugin> <artifactId>maven-deploy-plugin</artifactId> <version>2.8.2</version> </plugin> <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.6</version> <executions> <execution> <id>prepare-agent</id> <goals><goal>prepare-agent</goal></goals> </execution> <execution> <id>report</id> <phase>test</phase> <goals><goal>report</goal></goals> </execution> </executions> </plugin> </plugins> </build> </project> 3. From the project directory, traverse to the com.example.jacoco_lambdatest package existing in src/main/java. Create a new Java class named LambdaTest.java. We will write a simple setUp() function in it that provides the desired capabilities of Selenium Grid. package com.example.Jacoco_lambdatest; import java.net.MalformedURLException; import java.net.URL; import org.junit.Before; import org.junit.Test; import org.openqa.selenium.By; import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.remote.RemoteWebDriver; public class LambdaTest { public static String username = "<LambdaTest_Username>"; public static String accessKey = "<LambdaTest_AccessKey>"; public static DesiredCapabilities setUp() throws Exception { DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability("platform", "Windows 10"); capabilities.setCapability("browserName", "Chrome"); capabilities.setCapability("version", "87.0"); // If this cap isn't specified, it will just get the any available one capabilities.setCapability("resolution","1024x768"); capabilities.setCapability("build", "First Test"); capabilities.setCapability("name", "Sample Test"); capabilities.setCapability("network", true); // To enable network logs capabilities.setCapability("visual", true); // To enable step by step screenshot capabilities.setCapability("video", true); // To enable video recording capabilities.setCapability("console", true); // To capture console logs return capabilities; } } Adding JUnit Test Cases in the Project 1. We will create a simple JUnit test case in AppTest.java. This is provided by default, in the src/test/java under the package name com.example.jacoco_lambdatest. package com.example.Jacoco_lambdatest; import java.net.MalformedURLException; import java.net.URL; import org.junit.Before; import org.junit.Test; import org.openqa.selenium.By; import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.remote.RemoteWebDriver; import com.example.Jacoco_lambdatest.*; public class AppTest { public static RemoteWebDriver driver; @Test public void testScript1() throws Exception { try { DesiredCapabilities capabilities = LambdaTest.setUp(); String username =LambdaTest.username; String accessKey = LambdaTest.accessKey; RemoteWebDriver driver = new RemoteWebDriver(new URL("https://"+username+":"+accessKey+"@hub.lambdatest.com/wd/hub"),capabilities); driver.get("https://lambdatest.github.io/sample-todo-app/"); driver.findElement(By.name("li1")).click(); driver.findElement(By.name("li2")).click(); driver.findElement(By.id("sampletodotext")).clear(); driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); driver.findElement(By.id("addbutton")).click(); driver.quit(); } catch (Exception e) { System.out.println(e.getMessage()); } } } Generating Code Coverage Reports 1. Click on the “Run As” button and set the configuration as Maven Test. 2. Instead, you can open the cmd (Command Line), traverse to the project folder, and run the maven command, “mvn test.” 3. Running the JUnit tests will automatically set the JaCoCo agent in motion. It will create a report in binary format in the target directory, with the path target/jacoco.exec. The output of jacoco.exec cannot be interpreted single-handedly but other tools like SonarQube and plugins can interpret it. As we have earlier specified, the jacoco:report goal will generate readable code coverage reports in popular formats like HTML, CSV, and XML. 4. As the build is successful, go to the target folder, then to the site > jacoco folder. The code coverage report (i.e., index.html) is located in target/site/jacoco/index.html. The report looks like this: 5. You can drill down at a micro level by clicking on com.example.jacoco_lambdatest>LambdaTest in the report. 6. By clicking on specific functions, you will have a more detailed view in LambdaTest.java. 7. Here, you will see a lot of diamonds of different colors like green, yellow, and red. These are the specifications used in the report to symbolize which line of code was executed and when it was executed. We will learn more about it in the next section of the report analysis. With this, you have successfully generated a code coverage report via the Jacoco Maven plugin. Analysis of the Code Coverage Report Our code coverage report shows 94% instruction coverage and 100% branch coverage, which is a great code coverage score. Later, we will try to achieve a 100% code coverage score by adding more test cases. The 38 instructions shown by JaCoCo in the report refer to the bytecode instructions instead of Java code instructions. The JaCoCo reports help you visually analyze code coverage by using diamonds with colors for branches and background highlight colors for lines. A brief explanation of the diamonds seen in the code coverage report is below: Red diamond; indicates that no branches have been exercised during the testing phase. Yellow diamond: indicates that the code is partially covered (i.e., some branches are not exercised). Green diamond: indicates that all branches are exercised during the test The same color code applies to the background highlight color for the line coverage. The report mainly provides three crucial metrics: Line coverage: This reflects the amount of code exercised based on the number of Java byte code instructions called by the tests. Branch coverage: This shows the percentage of exercised branches in the source code. These are typical if/else or switch statements. Cyclomatic complexity: This reflects code complexity via the number of paths needed to cover all possible paths in a code. It also refers to the number of test cases needed to implement to cover the entire code. As there is no switch or statement in the code, the cyclomatic complexity will be one; only one execution path is sufficient to cover the entire code. Introducing More Test Cases for Improving Code Coverage 1. To achieve better code coverage, more tests need to be introduced that test the code that was not covered earlier via the test implementation. 2. Go to AppTest.java in src/test/java to add more test cases. 3. The new test cases added to AppTest.java will look as follows: @Test public void testScript2() throws Exception { try { DesiredCapabilities capabilities = LambdaTest.setUp(); String username = LambdaTest.username; String accessKey = LambdaTest.accessKey; RemoteWebDriver driver = new RemoteWebDriver(new URL("https://"+username+":"+accessKey+"@hub.lambdatest.com/wd/hub"),capabilities); driver.get("https://lambdatest.github.io/sample-todo-app/"); driver.findElement(By.name("li2")).click(); driver.findElement(By.name("li3")).click(); driver.findElement(By.id("sampletodotext")).clear(); driver.findElement(By.id("sampletodotext")).sendKeys("Yes, Let's add it to list"); driver.findElement(By.id("addbutton")).click(); driver.quit(); } catch (Exception e) { System.out.println(e.getMessage()); } } @Test public void testScript3() throws Exception { try { DesiredCapabilities capabilities = LambdaTest.setUp(); String username = LambdaTest.username; String accessKey = LambdaTest.accessKey; RemoteWebDriver driver = new RemoteWebDriver(new URL("https://"+username+":"+accessKey+"@hub.lambdatest.com/wd/hub"),capabilities); driver.get("https://lambdatest.github.io/sample-todo-app/"); driver.findElement(By.name("li4")).click(); driver.findElement(By.id("sampletodotext")).clear(); driver.findElement(By.id("sampletodotext")).sendKeys("Yes, Let's add it!"); driver.findElement(By.id("addbutton")).click(); driver.quit(); } catch (Exception e) { System.out.println(e.getMessage()); } } 4. Let’s run the Maven JaCoCo report to publish a new coverage report. 5. JaCoCo offers a simple and easy way to track the code coverage score by declaring minimum requirements. The build fails if these requirements are not met, otherwise, the build is successful. 6. These requirements can be specified as rules in POM.xml. Just specify the new execution by specifying ‘check’ goal in POM.xml. Add the below code after the second <execution> tag in POM.xml. <!--Third execution : used to put a check on the entire package--> <execution> <id>jacoco-check</id> <goals> <goal>check</goal> </goals> <configuration> <rules> <rule> <element>PACKAGE</element> <limits> <limit> <counter>LINE</counter> <value>COVEREDRATIO</value> <minimum>0.50</minimum> </limit> </limits> </rule> </rules> </configuration> </execution> 7. With this, we are limiting our coverage ratio to 50%. This signifies a minimum of 50% of the code should be covered during the test phase. 8. You can run Maven clean verify to check whether the rules set in jacoco:check goal are met or not. 9. The log shows “All coverage checks have been met” as our code coverage score is 94%, which is greater than our minimum 50%. Automation Testing On LambdaTest Selenium Grid Using Maven Project With the JaCoCo Plugin Selenium testing on the cloud helps you attain better browser coverage, increased test coverage, and accelerated time to market. Parallel testing in Selenium helps you achieve the above mentioned requirements. LambdaTest Cloud Selenium Grid is a cloud-based scalable Selenium testing platform that enables you to run your automation scripts on 2000+ different browsers and operating systems. Prerequisites To run the test script using JUnit with Selenium, we need to set up an environment. You would first need to create an account on LambdaTest. Do make a note of the username and access-key that is available in the LambdaTest profile section. We will use this sample project for Java Selenium testing. Importing the Project to Eclipse IDE After downloading a zip file of the project: junit-selenium-sample from GitHub, we will import it to Eclipse IDE by following the below mentioned steps: 1. Go to your Eclipse IDE, click on the “File” menu and select “Import.” A new dialog box appears. 2. Type Maven in the textbox below and select “Existing Maven Projects,” and then click “Next.” 3. In the next dialog box, click on “Browse” and traverse to the project folder. Also, tick the checkbox giving the path to the POM.xml file, and click “Finish.” 4. Your project will be loaded in Eclipse IDE successfully. Adding Dependencies in the POM.xml File 1. Open the POM.xml, now add the dependencies of JUnit, Selenium, and JaCoCo Maven Plugin. After adding the dependencies to the code of POM.xml should look like this: <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.lambdatest</groupId> <artifactId>lambdatest-junit-sample</artifactId> <version>1.0-SNAPSHOT</version> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <surefire.version>2.19.1</surefire.version> <config.file>default</config.file> </properties> <dependencies> <!--JUnit dependency--> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>1.3.2</version> <scope>test</scope> </dependency> <!--Selenium dependency--> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-java</artifactId> <version>2.52.0</version> <scope>test</scope> </dependency> <dependency> <groupId>com.googlecode.json-simple</groupId> <artifactId>json-simple</artifactId> <version>1.1.1</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <!--Apache Maven Plugins--> <plugin> <artifactId>maven-compiler-plugin</artifactId> <version>3.0</version> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <artifactId>maven-surefire-plugin</artifactId> <version>2.19.1</version> </plugin> <!--JaCoCo Maven Plugin--> <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.6</version> <executions> <execution> <id>prepare-agent</id> <goals><goal>prepare-agent</goal></goals> </execution> <execution> <id>report</id> <phase>test</phase> <goals><goal>report</goal></goals> </execution> </executions> </plugin> </plugins> </build> </project> <em><a href="https://github.com/rachnaagrawal/junit-selenium-sample/blob/master/pom.xml" target="_blank" rel="nofollow">Github</a></em> Configuring the Desired Capabilities for JUnit Automation Testing 1. To connect to LambdaTest Selenium Automation Grid, the first thing done is invoking a remote webdriver. This remote driver requires some capabilities like browser, browser versions, operating system, etc., to build an environment. The code to it looks as follows: WebDriver driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + "@hub.lambdatest.com/wd/hub"), DesiredCapabilities.firefox()); //A class named DesiredCapabilities is used to create an environment as a Firefox browser. 2. In the JUnit automation testing code, the capabilities like browser, browser versions, operating system information, etc., can be customized and are passed via capabilities object. 3. LambdaTest has made this process very easy by providing an inbuilt capabilities generator. The capabilities generator will automatically generate the code for desired capabilities based on your input. Such as, our configurations are: Fields Selected Values Operating Systems Windows 10 Browser Chrome Browser Version 62.0 Resolution 1024×768 Selenium Version 3.13.0 4. Selecting the above-specified configuration in the capabilities generator and paste it to LambdaTestBseTest.java. Specifying LambdaTest Username and Access Key in the Required Java Class 1. In the Project Explorer, you will see three Java classes: LambdaTestBaseTest.java (contains the setup required for Java Selenium testing). Parallelized.java (contains Junit tests for parallel testing on LambdaTest Selenium grid). SimpleTest.java (contains simple unit tests). 2. LambdaTestBaseTest.java fetches the required data like desired capabilities, username, and access key from a config file. This is provided in src/test/resources as config.json. 3. Specify the desired capabilities, username, and access key in config.json. This JSON file is used as you can provide multiple configurations in it for realizing parallel testing with Selenium to specify multiple configurations in config.json and then fetch them later. [ { "tunnelName":"LambdaTest tunnel", "buildName":"running Java Selenium Tests", "testName":"Jacoco JUnit Test", "username": "user-name", "access_key":"access-key", "operatingSystem" : "win10", "browserName" : "chrome", "browserVersion" : "62.0", "resolution" : "1024x768" }] Unit Testing Using JUnit With Selenium: 1. SimpleTest.java is the Java class for specifying a single unit test case for testing and performing code coverage using the JaCoCo Maven plugin. package com.lambdatest; import com.lambdatest.LambdaTestBaseTest; import org.junit.Test; import org.openqa.selenium.By; import static org.junit.Assert.assertEquals; public class SimpleTest extends LambdaTestBaseTest { /** * Simple Test case annotation for JUnit Test * @throws Exception */ @Test public void validateUser() throws Exception { driver.get("https://lambdatest.github.io/sample-todo-app/"); driver.findElement(By.name("li1")).click(); driver.findElement(By.name("li2")).click(); driver.findElement(By.id("sampletodotext")).clear(); driver.findElement(By.id("sampletodotext")).sendKeys("Yey, Let's add it to list"); driver.findElement(By.id("addbutton")).click(); } } 2. This is a simple Selenium WebDriver test that will open a sample to-do application that will do the following tasks: Mark the first two items as “Done.” Add a new item to the list. Return the added item. 3. Trigger the command mvn test on the terminal to build and run the test case. 4. Now, log into your LambdaTest account and go to “Automation.” You will find the tests you ran under the build name “JUnit JaCoCo Tests.” 5. Click on “JUnit JaCoCo Tests” and review them in detail. LambdaTest has recorded the video. So you can see the visuals too. Generating Code Coverage Report via the JaCoCo Maven Plugin: Now, as we have run the JUnit test cases on LambdaTest Selenium Grid, the code coverage report should be generated via the JaCoCo Maven plugin. Just go to the target folder, and you will find the binary format of the report as jacoco.exec. You can view the HTML, CSV, and XML form of the code coverage report in the target/site/jacoco folder as index.html, jacoco.csv, and jacoco.xml, respectively. Now you can also try to improve the code coverage score and analyze the code coverage report generated. Conclusion In this article, we have seen how to use the JaCoCo-Maven plugin to generate code coverage reports for Java projects. We have also leveraged the agility and scalability of LambdaTest Selenium Grid cloud to automate the test processes. Remember, though, 100% code coverage is not responsible for reflecting effective testing, as it only shows the amount of code exercised during tests. Yet, it helps to reduce the number of bugs and improves the software release quality. Also, it adds minimal overhead to the build process and allows it to keep a certain threshold as the development team adds edge cases or implements defensive programming.