Extracting Javadoc Documentation From Source Files Using JavaParser

A lot of people are using JavaParser for a lot of different goals. One of these is extracting documentation. In this short post, we will see how you can print all the Javadoc comments associated to classes or interfaces.

If you want to jump in, the code is available on GitHub.

Getting All the Javadoc Comments for Classes

We are reusing DirExplorer, a supporting class presented in the introduction to JavaParser. This class permits us to process a directory, recursively, parsing all the Java files contained there.

We can start by iterating over all the classes and finding the associated Javadoc comments.

/**
 * Iterate over the classes and print their Javadoc.
 */
public class ClassesJavadocExtractor {

    public static void main(String[] args) {
        File projectDir = new File("source_to_parse/");
        new DirExplorer((level, path, file) -> path.endsWith(".java"), (level, path, file) -> {
            try {
                new VoidVisitorAdapter<Object>() {
                    @Override
                    public void visit(ClassOrInterfaceDeclaration n, Object arg) {
                        super.visit(n, arg);
                        if (n.getComment() != null && n.getComment() instanceof JavadocComment) {
                            String title = String.format("%s (%s)", n.getName(), path);
                            System.out.println(title);
                            System.out.println(Strings.repeat("=", title.length()));
                            System.out.println(n.getComment());
                        }
                    }
                }.visit(JavaParser.parse(file), null);
            } catch (IOException e) {
                new RuntimeException(e);
            }
        }).explore(projectDir);
    }

}

As you can see, getting the Javadoc comments is fairly easy. It produces this result:

ASTParserConstants (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ASTParserConstants.java)
==============================================================================================================================
/**
 * Token literal values and constants.
 * Generated by org.javacc.parser.OtherFilesGen#start()
 */

ParseException (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ParseException.java)
======================================================================================================================
/**
 * This exception is thrown when parse errors are encountered.
 * You can explicitly create objects of this exception type by
 * calling the method generateParseException in the generated
 * parser.
 *
 * You can modify this class to customize your error reporting
 * mechanisms so long as you retain the public fields.
 */

ASTParser (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ASTParser.java)
============================================================================================================
/**
 * <p>This class was generated automatically by javacc, do not edit.</p>
 */

ASTParserTokenManager (/javaparser/javaparser-core/target/generated-sources/javacc/com/github/javaparser/ASTParserTokenManager.java)
====================================================================================================================================
/** Token Manager. */

Getting All the Javadoc Comments and Finding the Documented Elements

In other cases, we may want to start collecting all the JavaDoc comments and then finding the element that is commented. We can also do that easily with JavaParser:

/**
 * Iterate over all the Javadoc comments and print them together with a description of the commented element.
 */
public class AllJavadocExtractor {

    public static void main(String[] args) {
        File projectDir = new File("source_to_parse/");
        new DirExplorer((level, path, file) -> path.endsWith(".java"), (level, path, file) -> {
            try {
                new VoidVisitorAdapter<Object>() {
                    @Override
                    public void visit(JavadocComment comment, Object arg) {
                        super.visit(comment, arg);
                        String title = null;
                        if (comment.getCommentedNode().isPresent()) {
                            title = String.format("%s (%s)", describe(comment.getCommentedNode().get()), path);
                        } else {
                            title = String.format("No element associated (%s)", path);
                        }
                        System.out.println(title);
                        System.out.println(Strings.repeat("=", title.length()));
                        System.out.println(comment);
                    }
                }.visit(JavaParser.parse(file), null);
            } catch (IOException e) {
                new RuntimeException(e);
            }
        }).explore(projectDir);
    }

    private static String describe(Node node) {
        if (node instanceof MethodDeclaration) {
            MethodDeclaration methodDeclaration = (MethodDeclaration)node;
            return "Method " + methodDeclaration.getDeclarationAsString();
        }
        if (node instanceof ConstructorDeclaration) {
            ConstructorDeclaration constructorDeclaration = (ConstructorDeclaration)node;
            return "Constructor " + constructorDeclaration.getDeclarationAsString();
        }
        if (node instanceof ClassOrInterfaceDeclaration) {
            ClassOrInterfaceDeclaration classOrInterfaceDeclaration = (ClassOrInterfaceDeclaration)node;
            if (classOrInterfaceDeclaration.isInterface()) {
                return "Interface " + classOrInterfaceDeclaration.getName();
            } else {
                return "Class " + classOrInterfaceDeclaration.getName();
            }
        }
        if (node instanceof EnumDeclaration) {
            EnumDeclaration enumDeclaration = (EnumDeclaration)node;
            return "Enum " + enumDeclaration.getName();
        }
        if (node instanceof FieldDeclaration) {
            FieldDeclaration fieldDeclaration = (FieldDeclaration)node;
            List<String> varNames = fieldDeclaration.getVariables().stream().map(v -> v.getName().getId()).collect(Collectors.toList());
            return "Field " + String.join(", ", varNames);
        }
        return node.toString();
    }

}

Here, most of the code is about providing a description for the commented node (method describe).

Conclusions

Manipulating the AST and finding the Javadoc comments is quite easy. However, one missing feature is the possibility to extract the information contained in the Javadoc in a structured form. For example, you may want to get only the part of the Javadoc with a certain parameter or to the return value. Javaparser currently does not have this feature, but I am working on it, and it should be merged in the next weeks or two. If you want to follow the development take a look at issue 433.

Thanks for reading and happy parsing!