Migrating From Commons CLI to Picocli

Apache Commons CLI, initially released in 2002, is perhaps the most widely-used Java command line parser, but its API shows its age. Applications looking for a modern approach with a minimum of boilerplate code may be interested in Picocli. Why is it worth the trouble to migrate, and how do you migrate your Commons CLI-based application to Picocli? Picocli offers a fluent API with strong typing, usage help with ANSI colors, autocompletion, and a host of other features. Let’s take a look using Checkstyle as an example. (Note: as the author of Picocli, it is fair to say I am biased.)

Why Migrate?

Is migrating from Commons CLI to Picocli worth the trouble? What is the benefit of moving from one command line parser to another? Is this more than just redecorating the living room of our application?

End-User Experience

What are the benefits for end users?

Command line completion. Picocli-based applications can have command line completion in bash and zsh shells, as well as in JLine-based interactive shell applications.

Beautiful, highly-readable usage help messages. The usage help generated by Commons CLI is a bit minimalistic. Out of the box, Picocli generates help that uses ANSI styles and colors for contrast to emphasize important information, like commands, options, and parameters. The help message layout is easy to customize using the annotations. Additionally, there is a Help API in case you want something different. See the Picocli README for some example screenshots.

Support for very large command lines via @-files, or "argument files." Sometimes, users need to specify command lines that are longer than supported by the operating system or the shell. When Picocli encounters an argument beginning with the character @, it expands the contents of that file into the argument list. This allows applications to handle command lines of arbitrary length.

Developer Experience

What are the benefits for you as developer?

Generally, a Picocli application will have a lot less code than the Commons CLI equivalent. The Picocli annotations allow applications to define options and positional parameters in a declarative way where all information is in one place. Also, Picocli offers a number of conveniences like type conversion and automatic help that take care of some mechanics, so the application can focus more on the business logic. The rest of this article will show this in more detail.

Documentation: Picocli has an extensive user manual and detailed javadoc.

Troubleshooting. Picocli has a built-in tracing facility to facilitate troubleshooting. End users can use system property picocli.trace to control the trace level. Supported levels are OFF, WARN, INFO, and DEBUG. The default trace level is WARN.

Future Expansion

Finally, other than the immediate pay-off, are there any future benefits to be gained by migrating from Commons CLI to Picocli?

Picocli has a lot of advanced features. Your application may not use these features yet, but if you want to expand your application in the future, Picocli has support for nested subcommands (and sub-subcommands to any depth), mixins for reuse, can easily integrate with Dependency Injection containers, and has a growing tool framework to generate source code, documentation and configuration files from a Picocli CommandSpec model.

Finally, Picocli is actively maintained, whereas Commons CLI seems to be near-dormant with 6 releases in 16 years.

Example Migration With CheckStyle

A command line application needs to do three things:

1. Define the supported options
2. Parse the command line arguments
3. Process the results

Let’s compare how this is done in Commons CLI and in Picocli, using CheckStyle’s com.puppycrawl.tools.checkstyle.Main command line utility as an example.

The full source code before and after the migration is on GitHub.

Defining Options and Positional Parameters

Defining Options with Commons CLI

Commons CLI has multiple ways to define options: Options.addOption, constructing a new Options(…) and invoking methods on this object, the deprecated OptionBuilder class, and the recommended Option.Builder class.

The Checkstyle Main class uses the Options.addOption method. It starts by defining a number of constants for the option names:

/** Name for the option 's'. */
private static final String OPTION_S_NAME = "s";

/** Name for the option 't'. */
private static final String OPTION_T_NAME = "t";

/** Name for the option '--tree'. */
private static final String OPTION_TREE_NAME = "tree";

... // and more. Checkstyle Main has 26 options in total.

The Main.buildOptions method uses these constants to construct and return a Commons CLI Options object that defines the supported options:

private static Options buildOptions() {
    final Options options = new Options();
    options.addOption(OPTION_C_NAME, true, "Sets the check configuration file to use.");
    options.addOption(OPTION_O_NAME, true, "Sets the output file. Defaults to stdout");
    ...
    options.addOption(OPTION_V_NAME, false, "Print product version and exit");
    options.addOption(OPTION_T_NAME, OPTION_TREE_NAME, false,
            "Print Abstract Syntax Tree(AST) of the file");
    ...
    return options;
}

Defining Options with Picocli

In Picocli, you can define supported options either programmatically with builders, similar to the Commons CLI approach, or declaratively with annotations.

Picocli’s programmatic API may be useful for dynamic applications where not all options are known in advance. If you’re interested in the programmatic approach, take a look at the CommandSpec, OptionSpec and PositionalParamSpecclasses. See also the Programmatic API for more detail.

In this article, we will use the Picocli annotations. For the CheckStyle example, this would look something like the below:

@Option(names = "-c", description = "Sets the check configuration file to use.")
private File configurationFile;

@Option(names = "-o", description = "Sets the output file. Defaults to stdout")
private File outputFile;

@Option(names = "-v", versionHelp = true, description = "Print product version and exit")
private boolean versionHelpRequested;

@Option(names = {"-t", "--tree"}, description = "Print Abstract Syntax Tree(AST) of the file")
private boolean printAST;

Comparison

Declarative

With Commons CLI, you build a specification by calling a method with String values. One drawback of an API like this is that good style compels client code to define constants to avoid "magic values," like the Checkstyle Main class dutifully does.

With Picocli, all information is in one place. Annotations only accept String literals, so definition and usage are automatically placed together without the need to declare constants. This results in cleaner and less code.

Strongly Typed

Commons CLI uses a boolean flag to denote whether the option takes an argument or not.

Picocli lets you use types directly. Based on the type, Picocli "knows" how many arguments the option needs: booleanfields don’t have an argument, Collection, Map and array fields can have zero to any number of arguments, and any other type means the options takes a single argument. This can be customized (see arity) but most of the time the default is good enough.

Picocli encourages you to use enum types for options or positional parameters with a limited set of valid values. Not only will Picocli validate the input for you, you can also show all values in the usage help message with @Option(description = "Valid values: ${COMPLETION-CANDIDATES}"). Enums also allow command line completion to suggest completion candidates for the values of the option.

Less Code

Picocli converts the option parameter String value to the field type. Not only does it save the application from doing this work, it also provides some minimal validation on the user input. If the conversion fails, a ParameterException is thrown with a user-friendly error message.

Let’s look at an example to see how useful this is. The Checkstyle Main class defines a -x, --exclude-regexp option that allows uses to specify a number of regular expressions for directories to exclude.

With Commons CLI, you need to convert the String values that were matched on the command line to java.util.regex.Pattern objects in the application:

/**
 * Gets the list of exclusions from the parse results.
 * @param commandLine object representing the result of parsing the command line
 * @return List of exclusion patterns.
 */
private static List<Pattern> getExclusions(CommandLine commandLine) {
    final List<Pattern> result = new ArrayList<>();

    if (commandLine.hasOption(OPTION_X_NAME)) {
        for (String value : commandLine.getOptionValues(OPTION_X_NAME)) {
            result.add(Pattern.compile(value));
        }
    }
    return result;
}

By contract, in Picocli, you would simply declare the option on a List<Pattern> (or a Pattern[] array) field. Since Picocli has a built-in converter for java.util.regex.Pattern, all that is needed is to declare the option. The conversion code goes away completely. Picocli will instantiate and populate the list if one or more -x options are specified on the command line.

/** Option that allows users to specify a regex of paths to exclude. */
@Option(names = {"-x", "--exclude-regexp"},
        description = "Regular expression of directory to exclude from CheckStyle")
private List<Pattern> excludeRegex;

Option Names

Commons CLI supports "short" and "long" options, like -t and --tree. This is not always what you want.

Picocli lets an option have any number of names, with any prefix. For example, this would be perfectly fine in Picocli:

@Option(names = {"-cp", "-classpath", "--class-path"})

Positional Parameters

In Commons CLI, you cannot define positional parameters up front. Instead, its CommandLine parse result class has a method getArgs that returns the positional parameters as an array of Strings. The Checkstyle Main class uses this to create the list of File objects to process.

In Picocli, positional parameters are first-class citizens, like named options. Not only can they be strongly typed, but parameters at different positions can also have different types, and each will have a separate entry and description listed in the usage help message.

For example, the Checkstyle Main class needs a list of files to process, so we declare a field and annotate it with @Parameters. The arity = "1..*" attribute means that at least one file must be specified, or Picocli will show an error message about the missing argument.

Help Options

It is surprisingly difficult in Commons CLI to create an application with a required option that also has a --help option. Commons CLI has no special treatment for help options and will complain about the missing required option when the user specifies <command> --help.

Picocli has built-in support for common (and custom) help options.

Parsing the Command Line Arguments

Commons CLI has a CommandLineParser interface with a parse method that returns a CommandLine representing the parse result. The application then calls CommandLine.hasOption(String) to see if a flag was set, or CommandLine.getOptionValue(String) to get the option value.

Picocli populates the annotated fields as it parses the command line arguments. Picocli’s parse… methods also return a ParseResult that can be queried on what options were specified and what value they had, but most applications don’t actually need to use the ParseResult class since they can simply inspect the value that was injected into the annotated fields during parsing.

Processing the Results

When the parser is done, the application needs to run its business logic, but first, there are some things to check:

• Was version info or usage help requested? If so, print out the requested information and quit.
• Was the user input invalid? Print out an error message with the details, print the usage help message and quit.
• Finally, run the business logic and deal with errors thrown by the business logic.

With Commons CLI, this looks something like this:

int exitStatus;
try {
    CommandLine commandLine = new DefaultParser().parse(buildOptions(), args);

    if (commandLine.hasOption(OPTION_VERSION)) { // --version
        System.out.println("Checkstyle version: " + version());
        exitStatus = 0;
    } else if (commandLine.hasOption(OPTION_HELP)) { // --help
        printUsage(System.out);
        exitStatus = 0;
    } else {
        exitStatus = runBusinessLogic(); // business logic
    }
} catch (ParseException pex) { // invalid input
    exitStatus = EXIT_WITH_CLI_VIOLATION;
    System.err.println(pex.getMessage());
    printUsage(System.err);
} catch (CheckstyleException ex) { // business logic exception
    exitStatus = EXIT_WITH_CHECKSTYLE_EXCEPTION_CODE;
    ex.printStackTrace();
}
System.exit(exitStatus);

Picocli offers some convenience methods that take care of most of the above. By making your command, implement Runnable or Callable. The application can focus on business logic. At its simplest, this can look something like this:

public class Main implements Callable<Integer> {
    public static void main(String[] args) {
        CommandLine.call(new Main(), args);
    }

    public Integer call() throws CheckstyleException {
        // business logic here
    }
}

The Checkstyle Main class needs to control the exit code and has some strict internal requirements for error handling, so we ended up not using the convenience methods and kept the parse result processing very similar to what it was with Commons CLI. Improving this area is on the Picocli todo list.

Usage Help Message

Picocli uses ANSI colors and styles in the usage help message on supported platforms. This doesn’t just look good, it also reduces the cognitive load on the user: the contrast make the important information like commands, options, and parameters stand out from the surrounding text.

Applications can also use ANSI colors and styles in the description or other sections of the usage help message with a simple markup like @|bg(red) text with red background|@. See the relevant section of the user manual.

For CheckStyle, we kept it to the bare minimum, and the resulting output for CheckStyle looks like this:

Wrapping Up: a Final Tip

Be aware that the Commons CLI default parser will recognize both single hyphen (-) and double hyphen (--) long options, even though the usage help message will only show long options with double hyphens. You need to decide whether to continue to support this.

In Picocli, you can use @Option(names = "-xxx", hidden = true) to declare long options with a single hyphen if you want to mimic the exact same behavior as Commons CLI: hidden options in Picocli are not shown in the usage help message.

Conclusion

Migrating from Commons CLI to Picocli can give end users a better user experience and can give developers significant benefits in increased maintainability and potential for future expansion. Migration is a manual process but is relatively straightforward.

Update: the CheckStyle project accepted a pull request with the changes in this article. From CheckStyle 8.15, its command line tools will use Picocli. It looks like the CheckStyle maintainers were happy with the result:

Checkstyle migrated from Apache CLI to @picocli (will be released in 8.15), finally documentation of CLI arguments is now well organized in declarative way in code, and checkstyle’s CLI is following CLI best practices.

— CheckStyle maintainer Roman Ivanov

If you like what you see, please star Picocli on GitHub and tell your friends!