Make Your Xtext-based Editor 300 Times Faster*

My sincere apologies for the sensational title. Here is the fine print: With a 3-line code change, we were able to make the Protocol Buffer editor ~300 times faster when opening files in a Java project. Speed improvements may vary depending on the size of a project’s classpath.

Some background

At Google, our Eclipse projects are bigger than anything I’ve seen before: a project’s classpath can easily have 4,000 to 8,000 jar files! As a result, any performance issues, even minor ones, are greatly amplified in our environment. Frequent delays and freezes make using Eclipse a frustrating experience.

One of my team’s goals is to make Eclipse capable of handling our gigantic projects. In a effort to find performance issues (and later on fix them) we log delays in the Eclipse UI thread. In our logs we found that one of the top 5 offenders was, to my surprise, Xtext.

The problem

Before I continue, I’d like to clarify that I don’t mean to bash Xtext. IMHO, Xtext is awesome and it delivers on its promise: create a full-blown editor from a grammar definition. I consider Xtext as one of the top-3 best Eclipse projects of all time.

OK, back to our problem. According to our logs, Xtext is taking several seconds to open a document (a .proto file in our case.) We were using Xtext 2.0.3 M6 and Eclipse 3.8 M6, BTW.

The logged stack trace is the following:

java.util.zip.ZipFile.(ZipFile.java:131)
java.util.jar.JarFile.(JarFile.java:150)
java.util.jar.JarFile.(JarFile.java:114)
org.eclipse.xtext.ui.resource.XtextResourceSetProvider.computePlatformURIMap(XtextResourceSetProvider.java:68)
org.eclipse.xtext.ui.resource.XtextResourceSetProvider.get(XtextResourceSetProvider.java:49)
org.eclipse.xtext.ui.editor.model.ResourceForIEditorInputFactory.getResourceSet(ResourceForIEditorInputFactory.java:89)
org.eclipse.xtext.ui.editor.model.JavaClassPathResourceForIEditorInputFactory.getResourceSet(JavaClassPathResourceForIEditorInputFactory.java:59)
org.eclipse.xtext.ui.editor.model.ResourceForIEditorInputFactory.createResourceFor(ResourceForIEditorInputFactory.java:68)
org.eclipse.xtext.ui.editor.model.ResourceForIEditorInputFactory.createResource(ResourceForIEditorInputFactory.java:64)
org.eclipse.xtext.ui.editor.model.JavaClassPathResourceForIEditorInputFactory.createResource(JavaClassPathResourceForIEditorInputFactory.java:37)
org.eclipse.xtext.ui.editor.model.ResourceForIEditorInputFactory.createResource(ResourceForIEditorInputFactory.java:53)
org.eclipse.xtext.ui.editor.model.XtextDocumentProvider.setDocumentContent(XtextDocumentProvider.java:118)

According to XtextResourceSetProvider‘s code, when creating a Xtext document (e.g. opening a .proto file,) Xtext is reading the manifest in every single jar in the classpath. Given our projects’ huge classpaths, this scanning can take a considerable amount of time.

Since we never got a clear answer why Xtext is doing this, we had two options:

1. Read Xtext and EMF code to understand the issue. Reading layers of undocumented code is not as bad as it sounds. Understanding the code and fixing it requires time though. Unfortunately, I don’t have it. Other projects require my immediate attention.
2. Take an educated guess to solve the problem. Keep reading, please.

The solution

We went for the “educated guess” alternative. Given that:

1. we found out that the proto editor works well in non-Java projects (e.g. CDT-based ones,)
2. we suspected that this classpath scanning was necessary for custom JVM-based languages (like Xtend,) and
3. the Protocol Buffer language does not depend on JVM types

we were pretty confident that scanning the project’s classpath was not adding any value and we could safely remove it.

The change literally required adding 3 lines of code. The beauty of Xtext is that is uses Guice under the covers. We only needed to replace XtextResourceSetProvider with another implementation of IResourceSetProvider in the UI project’s module:

@Override public Class<? extends IResourceSetProvider> bindIResourceSetProvider() {
    return SimpleResourceSetProvider.class;
  }

And that’s it!

The following table shows the dramatic improvement in performance. Please note the ridiculous amount of time it used to take to open small files.

Please note that lines of code is not the only metric that affects performance. There are other factors (e.g. scoping,) that need to be considered when measuring performance.

We didn’t lose any functionality in our editors after making this change. Syntax highlighting, content assist, scoping, hyperlinking and validation kept working normally.

Conclusion

Xtext is a great framework for creating editors for custom languages. Although it is a mature project, there is still room from improvement. In this post, I showed you how we dramatically improved the performance of our Xtext-based editor with only a few lines of code. We were able to remove the problem in record time by using a combination of data, facts and intuition.

Feedback is always welcome :)