New Ways Around The Generation Gap Pattern

This post describes some new ways around "Generation Gap Pattern (John Vlissides)" which deals with how to  modify or extend generated code just once no matter how many times it is regenerated. 

ModelDrivenSoftwareDevelopment (MDSD) - Best Practices

Some best practises are valid at all MDSD projects:

• generated code should be clearly separated from manually written code
• generated code should not checked into a VersionControlSystem (CVS, SVN, GiT)

Manually written code

Generated unchanged code

Generated but editable code

This is code where it's useful or expected that the generated code will be changed or extended. If files exist, they will not be overwritten - new files will be added and generated first time. (generate-once)

Generation Gap Pattern

The main problem with generated code is always how to combine, or separate, generated code and manually changed / extended code. That's the point where in most cases the Generation Gap Pattern will be used:

Generated classes will be generated into a directory which is cleared before generation. Subclasses will only generated first time into another directory where they can be changed.

In practice some problems happen - for example, "Ghost Classes" after refactorings / renaming. Also you cannot differ between generated and changed Subclasses - so all will be checked into the version control system.

To make this more practical, Heiko Behrens describes „Conditional Generation“ where generated and changed classes are clearly separated. oAW Workflow can use a "Veto Strategy": Before generating an artifact it will test if there's already a changed artifact into another directory - and if found no new generation happens.

This is described in detail in Heiko Behrens article „Generation Gap Pattern“. If you haven't read this article, please do now. I expect that you know the Veto Strategies described by Heiko.

Thanks to Heiko Behrens, I got the information about oAW „Veto Strategy“ which was new for me - I never read about it in the docs or the forum. Now I was inspired to find some solutions for my project better then previous workarounds.

At first I must say, that I had some problems using the „Generation Gap Pattern“ as generic pattern if generated code also has to be modified in MDSD projects. Working as a software architect and designing an entity model (esp. in the domain of Enterprise Business Applications), I have to carefully think about inheritance strategies and hierarchies. (esp. if all must be mapped from Hibernate or EclipseLink to a database)

A really simple inheritance hierarchy:

... then the inheritance hierarchy becomes "dirty" from my point of view. In big projects suddenly hundreds of classes are "twice" - and the only reason is technically because its a model driven project.

I only want to use abstract / concrete classes or super- / subclasses if I need them from my use-cases or software architecture, but not because I'm using a generator - framework.

As always, when using a pattern you should ask: is it the right pattern or are there other ways to reach the goal?

In my opinion: the "Generation Gap Pattern" helps in many cases, but not always. Heiko Behrens shows how the use of "Veto Strategies" in oAW workflows will help handling the files and folders, but its no solution to avoid getting many classes twice which bloats the architecture.

Protected Areas

So we again have to find an answer how to separate generated and manually written code without breaking the rules of best practises in MDSD projects.

There is another way: „ProtectedAreas“. STOP crying ;-) please read ahead - I will try to use protected areas in a different way, and then normally

What are „Protected Areas“ ?

If there are some parts of the code where its allowed to modify or extend, then the Begin and the End are marked as special comments identified by an unique ID. Inside these protected areas the default code will be generated and the area will be marked as DISABLED, so if regenerating it will be overwritten.

If a developer modifies / extends the content af a protected region, then this region has to be ENABLED and the changes are protected after regeneration. Using „Protected Areas“ the structure and inheritance of your classes remain unchanged - exact the same as using it without a generator.

How are „Protected Areas“ normally used ?

Whats the drawback ?

Now classes have protected areas, can be overwritten if regenerated but then they will be checked into the Version Control System. Thats bad because there are many classes without changes: only generated content. That breaks the rule "never check-in generated code".

Also you don't see which classes have changes and which are untouched after generation.

Protected Regions using VetoRedirectStrategie

Here's my solution

• at start of generator-workflow clear „src-gen“ and „veto“
• generator scannes „src-gen-pr“ and caches ENABLED Protected Areas
• „only-generated“ classes will be generated into „src-gen“, also classes with „Protected Regions“ without a veto
• classes with ENABLED „Protected Regions“ are stored in „src-gen-pr“
• while generating the generator checks using "VetoStrategy" if a class is already in „src-gen-pr“. If found, then the generated output is redirected into „veto“ directory where the new generated class together with ENABLED Protected Regions (from cache) is written to
• as last step of generation the files from „veto“ are compared with corresponding files from „src-gen-pr“. Only if they differ, the new ones will be copied into „src-gen-pr“. Now we are sure only really changed files will get a new modified flag and checked into VCS.

The Workflow

Generated-only and manually changed code now is clearly separated. Only changed artifacts will be commited into VCS and the Inheritance structure is unchanged.

Of course I have to watch, that a developer changing „Protected Regions“ doesnt forget to activate them (ENABLED) and to move the file into „src-gen-pr“ if not already there

If you want to check if a developer ENABLED a protected region, but forgets to move into „src-gen-pr“ you  can add a step at start of your workflow: scan „src-gen-pr“ for protected regions - if some enabled are found, stop workflow with warning.

If you want to check if a developer forgot to enable a protected region after making some changes, you can include something like a checksum into the region and write a component to test these before starting to generate.

As you can see, everything is possible, it's just course a matter of time: each extra step will slow down the performance of the generation.

Tip: sometimes the Local History of Eclipse helps, but there's currently a bug: oAW generator overwrites the last Local History entry instead of inserting a new one.

Inside the default code of a „ProtectedRegion“ I always insert a comment like:

Some Technical Information on how to implement a Redirect Veto Strategy

Tested using:

• Eclipse 3.5M6
• openArchitectureWare 4.3.1 (If you already use the plugins from Eclipse Modeling Project you have to change the namespace)
• Apache Commons IO 1.4 (SpringSourceEnterpriseBundleRepository - (Eclipse Orbit only has version 1.3.2 from 2007)

Workflow Components

Workflow Component Directory Cleaner (oAW Component)

outlet.src.dir points to src-gen, outlet.veto.pr.dir points to veto.

Workflow Component Outlet with Veto Strategy (Redirect of enabled Protected Regions)

This Outlet uses a new Veto strategy - also the Outlet Class was extended

outlet.src.dir points to src-gen, outlet.src.pr.dir points to src-gen-pr and outlet.veto.pr.dir points to veto.

Workflow Component „fileReplacer“ (Replace files from src-gen-pr ?)

A new Workflow Component compares corresponding files in veto directory and src-gen-pr directory and replaces modified ones.

A short view inside the new Classes and Interfaces

Class OutletWithVetoRedirection (extends oAW Outlet)

Interface VetoRedirectStrategy (extends oAW VetoStrategy)

hasVeto always returns false because we don't give a veto like from other Veto Strategies described by Heiko Behrens - we only want to redirect the Ouput Path into veto Directory.

Class ReplaceVetoRedirectedFiles (extends oAW AbstractWorkflowComponent2)

Using Apache Commons IO its very easy to compare file-content and to copy files from one directory to another.

outputFile represents a file from veto,

sourceFile represents a file from src-gen-pr

Summary

From my point-of-view the use of „Protected Areas“ combined with „Conditional Generation“ can be very useful in some projects. Best Practice rules are not broken.