Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Observers for AST Nodes in JavaParser

DZone's Guide to

Observers for AST Nodes in JavaParser

We are getting closer to the first Release Candidate for JavaParser 3.0. One of the last features added was support for observing changes to all nodes of the Abstract Syntax Tree. Learn more here!

· Java Zone
Free Resource

Microservices! They are everywhere, or at least, the term is. When should you use a microservice architecture? What factors should be considered when making that decision? Do the benefits outweigh the costs? Why is everyone so excited about them, anyway?  Brought to you in partnership with IBM.

We are getting closer to the first Release Candidate for JavaParser 3.0. One of the last features we added was support for observing changes to all nodes of the Abstract Syntax Tree. While I wrote the code for this feature I received precious feedback from Danny van Bruggen (a.k.a. Matozoid) and Cruz Maximilien. So I use “we” to refer to the JavaParser team.

What Observers on AST Nodes Could Be Used For

I think this is a very important feature for the ecosystem of JavaParser because it makes easier to integrate with JavaParser by reacting to the changes made on the AST. Possible changes that can be observed are setting a new name for a class or add a new field. Different tools could react to those changes in different ways. For example:

  • an editor could update its list of symbols, which could be used for things like auto-completion
  • some frameworks could regenerate source code to reflect the changes
  • validation could be performed to verify if the new change lead to an invalid AST
  • libraries like JavaSymbolSolver could recalculate the types for expressions

These are just a few ideas that come to mind but I think that most scenarios in which JavaParser is used could benefit from the possibility to react to changes.

The AstObserver

The JavaParser 3.0 AST is based on Nodes and NodeLists. A Node, like a TypeDeclaration for instance, can have different groups of children. When these groups can contain more than one node we use NodeLists. For example a TypeDeclarations can have multiple members (fields, methods, inner classes). So each TypeDeclaration has a NodeList to contain fields, one to contain methods, etc. Other children, like the name of a TypeDeclaration, are instead directly contain in a node.

We introduced a new interface named AstObserver. An AstObserver receive changes on the Nodes and NodeLists.

/**

* An Observer for an AST element (either a Node or a NodeList).

publicinterfaceAstObserver{

/**

     * Type of change occurring on a List

publicenumListChangeType{

ADDITION,

REMOVAL

/**

     * The value of a property is changed

     * @param observedNode owner of the property

     * @param property property changed

     * @param oldValue value of the property before the change

     * @param newValue value of the property after the change

voidpropertyChange(Node observedNode,ObservableProperty property,ObjectoldValue,ObjectnewValue);

/**

     * The parent of a node is changed

     * @param observedNode node of which the parent is changed

     * @param previousParent previous parent

     * @param newParent new parent

voidparentChange(Node observedNode,Node previousParent,Node newParent);

/**

     * A list is changed

     * @param observedNode list changed

     * @param type type of change

     * @param index position at which the changed occurred

     * @param nodeAddedOrRemoved element added or removed

voidlistChange(NodeList observedNode,ListChangeType type,intindex,Node nodeAddedOrRemoved);

What to Observe

Now we have an AstObserver and we need to decide which changes it should be received. We thought of three possible scenarios:

  1. Observing just one node, for example, a ClassDeclaration. The observer would receive notifications for changes on that node (e.g., if the class change name) but not for any of its descendants. For example, if a field of the class change name the observer would not be notified
  2. For a node and all its descendants at the moment of registration of the observer. In this case, if I register an observer for the ClassDeclaration I would be notified for changes to the class and all its fields and methods. If a new field is added and later modified I would not receive notifications for those changes
  3. For a node and all its descendants, both the ones existing at the moment of registration of the observer and the ones added later.

So a Node has now this method:

/**

     * Register a new observer for the given node. Depending on the mode specified also descendants, existing

     * and new, could be observed. For more details see <i>ObserverRegistrationMode</i>.

publicvoidregister(AstObserver observer,ObserverRegistrationMode mode){

if(mode==null){

thrownewIllegalArgumentException("Mode should be not null");

switch(mode){

caseJUST_THIS_NODE:

register(observer);

break;

caseTHIS_NODE_AND_EXISTING_DESCENDANTS:

registerForSubtree(observer);

break;

caseSELF_PROPAGATING:

registerForSubtree(PropagatingAstObserver.transformInPropagatingObserver(observer));

break;

default:

thrownewUnsupportedOperationException("This mode is not supported: "+mode);

To distinguish these three cases we simply use an enum (ObserverRegistrationMode). Later you can see how we implemented the PropagatingAstObserver.

Implementing Support for Observers

If JavaParser was based on some meta-modeling framework like EMF this would be extremely simple to do. Given this is not the case I needed to add a notification call in all the setters of the AST classes (there are around 90 of those).

So when a setter is invoke on a certain node it notifies all the observers. Simple. Take for example setName in TypeDeclaration<T>:


@Override

publicTsetName(SimpleName name){

notifyPropertyChange(ObservableProperty.NAME,this.name,name);

this.name=assertNotNull(name);

setAsParentNodeOf(name);

return(T)this;

Given we do not have a proper metamodel we have no definitions for properties. Therefore we added a list of properties in an enum, named ObservableProperty. In this way an Observer can check which property was changed and decide how to react.

Internal Hierarchy of Observers

For performance reasons, each node has its own list of observers. When we want to observe all descendants of a node we simply add the same observer to all nodes and nodelists in that subtree.

However, this is not enough, because in some cases you may want to observe also all nodes which are added to the subtree after you have placed your observers. We do that by using a PropagatingAstObserver. It is an AstObserver that when see a new node been attached to a node it is observing start to observe the new node as well. Simple, eh?

/**

* This AstObserver attach itself to all new nodes added to the nodes already observed.

publicabstractclassPropagatingAstObserverimplementsAstObserver{

/**

     * Wrap a given observer to make it self-propagating. If the given observer is an instance of PropagatingAstObserver

     * the observer is returned without changes.

publicstaticPropagatingAstObserver transformInPropagatingObserver(finalAstObserver observer){

if(observer instanceofPropagatingAstObserver){

return(PropagatingAstObserver)observer;

returnnewPropagatingAstObserver(){

@Override

publicvoidconcretePropertyChange(Node observedNode,ObservableProperty property,ObjectoldValue,ObjectnewValue){

observer.propertyChange(observedNode,property,oldValue,newValue);

@Override

publicvoidconcreteListChange(NodeList observedNode,ListChangeType type,intindex,Node nodeAddedOrRemoved){

observer.listChange(observedNode,type,index,nodeAddedOrRemoved);

@Override

publicvoidparentChange(Node observedNode,Node previousParent,Node newParent){

observer.parentChange(observedNode,previousParent,newParent);

@Override

publicfinalvoidpropertyChange(Node observedNode,ObservableProperty property,ObjectoldValue,ObjectnewValue){

considerRemoving(oldValue);

considerAdding(newValue);

concretePropertyChange(observedNode,property,oldValue,newValue);

@Override

publicfinalvoidlistChange(NodeList observedNode,ListChangeType type,intindex,Node nodeAddedOrRemoved){

if(type==ListChangeType.REMOVAL){

considerRemoving(nodeAddedOrRemoved);

}elseif(type==ListChangeType.ADDITION){

considerAdding(nodeAddedOrRemoved);

concreteListChange(observedNode,type,index,nodeAddedOrRemoved);

publicvoidconcretePropertyChange(Node observedNode,ObservableProperty property,ObjectoldValue,ObjectnewValue){

// do nothing

publicvoidconcreteListChange(NodeList observedNode,ListChangeType type,intindex,Node nodeAddedOrRemoved){

// do nothing

@Override

publicvoidparentChange(Node observedNode,Node previousParent,Node newParent){

// do nothing

privatevoidconsiderRemoving(Objectelement){

if(element instanceofObservable){

if(((Observable)element).isRegistered(this)){

((Observable)element).unregister(this);

privatevoidconsiderAdding(Objectelement){

if(element instanceofNode){

((Node)element).registerForSubtree(this);

}elseif(element instanceofObservable){

((Observable)element).register(this);

Observers in Action

Let’s see how this works in practice:

// write some code and parse it

Stringcode="class A { int f; void foo(int p) { return 'z'; }}";

CompilationUnit cu=JavaParser.parse(code);

// set up our observer

List<String>changes=newArrayList<>();

AstObserver observer=newAstObserverAdapter(){

@Override

publicvoidpropertyChange(Node observedNode,ObservableProperty property,ObjectoldValue,ObjectnewValue){

changes.add(String.format("%s.%s changed from %s to %s",observedNode.getClass().getSimpleName(),property.name().toLowerCase(),oldValue,newValue));

cu.getClassByName("A").register(observer,/* Here we could use different modes */);

// Doing some changes

cu.getClassByName("A").setName("MyCoolClass");

cu.getClassByName("MyCoolClass").getFieldByName("f").setElementType(newPrimitiveType(PrimitiveType.Primitive.Boolean));

cu.getClassByName("MyCoolClass").getMethodsByName("foo").get(0).getParamByName("p").setName("myParam");

// Here we are adding a new field and immediately changing it

cu.getClassByName("MyCoolClass").addField("int","bar").getVariables().get(0).setInit("0");

// If we registered our observer with mode JUST_THIS_NODE

assertEquals(Arrays.asList("ClassOrInterfaceDeclaration.name changed from A to MyCoolClass"),changes);

// If we registered our observer with mode THIS_NODE_AND_EXISTING_DESCENDANTS

assertEquals(Arrays.asList("ClassOrInterfaceDeclaration.name changed from A to MyCoolClass",

"FieldDeclaration.element_type changed from int to boolean",

"VariableDeclaratorId.name changed from p to myParam"),changes);

// If we registered our observer with mode SELF_PROPAGATING

assertEquals(Arrays.asList("ClassOrInterfaceDeclaration.name changed from A to MyCoolClass",

"FieldDeclaration.element_type changed from int to boolean",

"VariableDeclaratorId.name changed from p to myParam",

"FieldDeclaration.modifiers changed from [] to []",

"FieldDeclaration.element_type changed from empty to int",

"VariableDeclaratorId.array_bracket_pairs_after_id changed from com.github.javaparser.ast.NodeList@1 to com.github.javaparser.ast.NodeList@1",

"VariableDeclarator.init changed from null to 0"),changes);

Conclusions

I am quite excited about this new feature because I think it enables more cool stuff to be done with JavaParser. I think our work as committers is to enable other people to do things we are not foreseeing right now. We should just act as enablers and then get out of the way.

I am really curious to see what people will build. By the way, do you know any project using JavaParser that you want to make known to us? Leave a comment or open an issue on GitHub, we are looking forward to hearing from you!

Discover how the Watson team is further developing SDKs in Java, Node.js, Python, iOS, and Android to access these services and make programming easy. Brought to you in partnership with IBM.

Topics:
machine learning

Published at DZone with permission of Federico Tomassetti, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}