Improve Your Robot Framework Tests With Robocop
Verify the quality of your Robot Framework test cases with the static analyser Robocop. This blog will give you a good first impression of what Robocop can do for you!
Join the DZone community and get the full member experience.Join For Free
Testing software will tell you something about the quality of your software. But how do you ensure that your tests have a high quality? A static analysis tool can help you in order to improve the quality. Robocop is a such a static analysis tool for Robot Framework tests. In this blog, you will learn how to use and configure Robocop for your purposes.
Static code analyzers are commonly used during software development. SonarQube is a popular tool used for this purposes. The intention is to scan the code and verify whether it complies to a certain set of rules. The rules to check against should be agreed upon within your company and serve as best practices. The purpose of the scan is to learn, to standardize, to improve readability, etc.
- Automated Acceptance Testing With Robot Framework
- How to Write Data Driven Tests With Robot Framework
- Create Custom Robot Framework Libraries
The Robocop documentation gives you a good overview of all capabilities. In the sections below, you will learn how to use the tool from a practical point of view. This way, it should be quite easy for you to apply it to your own Robot Framework test cases. As an example, you can use the GitHub repository from the previous blogs which contain a set of test cases and resource files. The
master branch contains non-analyzed tests, the
feature/robocop branch contains fixes for the Robocop warnings.
First of all, you will need to install Robocop. At the time of writing, Robocop v1.5 is the latest release.
2. Run a Scan
Let’s see what happens when you run a Robocop scan from within the root of the repository. Robocop will search for test cases and resource files in the current directory and in the subdirectories and will run with default settings. The output shows you all the files where violations against the rules have been found. The output format gives you the following information:
- The file name where the violation has been found;
- The line number where the violation occured;
- The column where the violation has been detected;
- The severity of the violation (E for error, W for warning, I for information);
- The ID of the rule;
- The description of the rule.
In order to limit the number of violations, it is possible to limit the scan to one specific file. This way, only the results for this particular file will be shown.
Robocop also provides the option to generate a report of the issues found. It will give you a summary which provides a nice overview.
3. More About the Rules
You now have an overview of the issues being found, but what are the rules being checked against? This is well documented in the official documentation. Another way of showing this list, is by means of the following command:
It might be the case that you do not agree with the severity of a certain rule. Let us assume that you do not agree with the severity of the
line-too-long rule. This rule has a severity Warning, but you want to change it to severity Information. With Robocop it is possible to change this behaviour by means of the configuration argument. You just need to use the ID or message name of the rule, followed by argument
severity, followed by the new value.
In the output you can now see that the issue is still reported, but now with severity Information.
Not only the severity can be changed. A lot of rules have configurable parameters which can be changed according to your preferences. In order to know what these parameters are, you can take a look at the documentation. But it is also possible to show this information by means of the
list-configurables command followed by the ID or message name of the rule.
Besides the severity, it is also possible to change the line length of the
line-too-long rule. Just like the severity, you change this parameter to e.g. 160. Running the scan with this setting, no issue is raised now because the line length is within the limits you specified.
As you can see, one issue less is shown in the total number of issues and in your output, no
line-too-long warning is shown anymore.
It might be that you do not agree with a certain rule at all. In this case, you can disable a rule from being checked. You can do so by means of the
exclude argument. Again, this will result in 19 warnings for the
employee.robot test case.
Listing the rules, you notice that rule
0508 is now indicated as being disabled.
In another case, you might want the rule to be enabled by default, but you want to disable an issue. You have noticed the issue and you are ok with it and do not want to be triggered about this part again in the future. You can disable a certain part of your test case for a specific rule. It is not advised to use this regularly, it will not improve the readability of your test case. It is better to fix the issue or to disable the rule or to reconfigure the rule for your needs.
4. Use a Configuration File
Instead of tweaking the configuration of rules in the command line, you can add the configuration in a configuration file. This way you can ensure that everyone is using the same configuration, certainly when you put it under version control. The contents of this file is for example:
Now we run robocop with the
argumentfile argument followed by the file containing the arguments,
robocop_args.txt in this case.
Robocop also offers the possibility to read the configuration file by default. This is restricted under the following conditions:
- The argument file must be named
- Robocop must be run without any other arguments. Note that it may not contain any other argument, also no file to analyze for example. It only works when only the
robocopcommand is being used.
.robocop argument file has the following content when you follow the above example.
Now it is sufficient to use the
robocop command. In the output you can see that the default configuration file has been loaded.
5. Fix Issues
In this section, some experiences are shared when fixing the issues in the test cases in the repository.
employee.robot file, all lines containing a
Documentation tag raise the over-indented warning.
Tried a lot of things in order to fix this, but this was not successful. Raised an issue for this and received a very quick response from the Robocop maintainers. It seems that the rule is ok, but when fixing it, another warning is shown (under-indented warning). This last one is not correct, but is already fixed in a Pull Request and will be available in a next release.
data_driver.robot file, a warning is raised about a missing documentation in the test case. In this case, it is ok to disable the rule, because this is the way the Data Driver library works. You cannot fix this.
Before starting using Robocop, take a good look at the rules you want to enable or disable. Not every rule will be suitable for you and it is better to only enable those rules you really want to check upon. When too many rules are enabled which do not increase your overall quality, but are reported every time, people will tend to not using it anymore.
Robocop is a great initiative which can be of good help when you want to increase the quality of your Robot Framework scripts. Beware of which rules you enable, you do not want to be overflooded by warnings you do not care about.
Published at DZone with permission of Gunter Rotsaert, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.