Roundtrip Engineering: Remixed
If you've been around the programming block a few times, you've almost certainly come across the concept of Roundtrip Engineering: usually this means drawing a system diagram in a UML editor and then turning this diagram into some executable code in a language of your choice. You may also have found this wanting.
This article explores some of the reasons for that, and looks at how it might be possible to move ahead with the concept.
First things first, for the purposes of this article, some definitions:
Forward Engineering (FE)
This is where you have a diagram, or set of diagrams (perhaps put together by a business analyst) and you turn them into some executable code. Compare with...
Reverse Engineering (RE)
Which is where you take some code and turn it into a diagram.
Put them both together, and what have you got? Roundtrip engineering! The idea is that you can change the diagram, and the code will change as a result. Then, change some code, and the diagram stays up-to-date with it. Except, it doesn't always turn out like that, as we shall see later.
Ok, so that's the theory. But does it work? In my experience - not brilliantly. Here are some of the problems I've come across:
Problem 1: UML / Your Language Impedance Mismatch
You've probably come across Object Relational Impedance Mismatch, right? This is where the difference in philosophy underlying object oriented programming languages impedes with the philosophy of relational databases. Turning objects into relations in a complex problem, and there's a whole ecosystem of tools (like Hibernate) around to help deal with this.
Problem 2: The Code Looks Shoddy
Code generated by forward engineering often looks a bit shoddy. There are usually obtuse comments liberally scattered around in it, strange formatting and a general feel of foreign-ness. Code written by people (in the same way as anything else written by people) tends to have a "humanness" about it that can't be captured by something created by a machine. Because...
Code Is Poetry
The Wordpress motto is that "Code is Poetry": there's definitely some truth in this. The artisan programmer balances concerns such as readability, performance, structure, reuse and robustness as she responds to the problem at hand. But if you are building a forward engineering system, you face...
But no matter how good the programmer designing the forward engineering system, the trade-offs of the actual system being designed can't be considered, and have to be guessed at up-front. Even if the designer incorporates lots of different options, these too would need to be guessed at up-front, and the more options you provide, the more complex the forward engineering software will be. Also, another problem with these guesses is...
Best Practices Change Fast
A further problem is that time marches on. New libraries are released continually on the Internet, there are always new ways of doing things. What counts as the best approach one day is soon superceded later on. Will your forward engineering system keep up with this rate of change? Even with the best will in the world, that looks unlikely.
Problem 3: The Diagrams Look Shoddy
Here's the scene: you spend a lot of time creating a beautiful looking diagram, turn it into some code, edit the code and then move back into the diagram view. Problem is, the diagram has now changed - the boxes are different sizes, some have disappeared, new ones have been added. It's going to take a load more effort to make this diagram presentable again. Why does this happen?
The reason is that laying out diagrams properly involves a lot of graph drawing theory. This is really hard to get right, and lots of the aesthetics that humans value in a diagram are hard to codify.
Problem 4: Diagrams Have Different Purposes
What do I mean by this? Simply that the code for the software captures everything about the software that has been defined. But, the diagrams do not: they are deliberately a simplification of the world, in order that other people can easily understand.
For example, UML is pretty good at modelling entity relationships but not so good at synchronization, threading and so on. So you have to understand that while the functionality of your system is entirely captured in the code, it can't entirely be captured in the diagrams: UML is not Turing Complete, and nor does it want to be.
Where is the problem here? Basically, this means that when you are forward engineering, you need to make assumptions about the software from the diagram. And when you are reverse engineering, you need to make decisions about what to leave off the diagram.
But, not all diagrams are the same. You could plausibly have a UML diagram which is a representation of what the code is doing (a Software System Diagram, if you will) and also a UML diagram which is a representation of how the business analyst sees the world (a Business System Diagram, say). These diagrams may not be the same. They may show different concerns, they may be created by different people.
But roundtrip engineering says that they are the same. It invites you to take the Business System Diagram and use it as a base for the Software System Diagram, and in turn use that as a base for the code.
You can't do this automatically and well at the same time - after all, this is what programmers are paid for.
What is the Purpose Of Roundtrip Engineering?
Ok, let's stop for a second. What problems are we trying to solve here anyway? Or alternatively, why are we putting so much effort into roundtripping? And by effort, I mean not just effort by the team on the project building the diagrams but the effort by the people building the roundtrip engineering systems.
Purpose 1: To Audit the Software As It Is
One answer is that the diagrams from reverse engineering should be Software System Diagrams, which would otherwise be hidden amidst the code artifacts, only discernable to the developers. You could say that this forms some of the understanding about "what the software is actually doing", in much the same way as test results and documentation generators (like Java's Javadoc) are used to give documentation about a system. We need these to help new programmers and laypeople understand the system, and also so that people can see the difference between what the software currently does, and what they want it to do.
Purpose 2: To Help Automate Building the Software
The purpose of the forward engineering is therefore to help turn the requirements of the system into the actual system itself.
Purpose 3: Another Feedback Loop In the Software Process
A final answer is that by having the software be self-documenting in some respect, we are able to create a new feedback loop in the software building system.
A common, well-understood feedback loop is the one for automated tests: tests are defined by requirements, the code changes, the tests pass. If the tests don't pass, you know the software is broken, and therefore not likely to be meeting requirements.
The documentation feedback loop works like this: By exposing how the software is actually working as documentation, you invite people to compare this with how it should work. Thus you can track how closely the system does what it says it should (see Figure 2).
A Way Forward
Now, there may well be other big issues than these that I have glossed over: I am very interested to hear what people think they might be. However, I am going to suggest one possible way forward for roundtrip engineering which will hopefully sort out these problems. So here we go:
- You don't have problem 1 if you don't bother with UML
- You won't have problem 2 if you don't auto-generate code
- You won't have problem 3 if you have a robust diagramming solution
- You won't have problem 4 if you keep the business system diagrams and software systems diagrams separate, and leave the translation part to the programmers.
Let's dig into each of those assertions and see how this could turn out:
Don't Auto-Generate Code
If we don't auto-generate code then we are missing out on the ability to do forward engineering (which above I describe as purpose 2 of round-tripping). This seems like a sad loss, but is it really? There are some domains in which forward generation works reasonably well. For example, Eclipse EMF allows you to generate Java classes from XML entities, and roundtrip that to UML. But there are plenty of alternative ways of handling XML in Java without code generation. And creating Java classes to match up to business entities is pretty simple work anyway, and tends not to be where much development effort is spent.
Also by avoiding code generation, you're not forced down the path of using specific libraries, or having ugly code. So there are some advantages.
Don't Bother With UML
If you're not trying to generate code from a UML diagram, then do you really need UML?
If we lose UML, we also lose the semantics of the UML symbols, and instead we are forced to represent the meaning of our project in English (or another human language).
Now obviously, this means that the diagram is no longer machine-understandable: the semantics of English are (currently) beyond the comprehension of our machines. But, since we're not trying to generate code from the diagram, is that such a problem? I would argue that in many cases it may not be.
Do we get something else from UML? Yes: The UML provides us with a language of shapes and symbols for specific purposes within software modelling, which are a handy shortcut to English language equivalents. Now, that's great if everyone understands that language, but UML has so many extensions and types of diagram, that what you end up with is a fairly-well-understood core and a large periphery of poorly-understood extensions. So, perhaps using English across the board is a more consistent way of communicating the semantics of the system.
A Robust Diagramming Solution
This is actually a really hard part, and this is where Kite9 comes in. The aim of Kite9 is to provide a library for creating box-and-line diagrams which are as good as, if not better than the same diagram you could put together in Visio (or whatever). Are we there yet? Pretty much, although there are a few rough edges. For example, the diagrams in this article were created very quickly and easily using the Kite9 Designer, which uses Kite9's layout library. This is designed for business analysts to put together diagrams really easily in a short space of time.
Keep the Diagrams Separate
So, we allow the business analysts to contribute their own documentation and diagrams (the "Specification" box in Figure 2). They can do this in the Kite9 designer, or another tool of their choice.
Also, we generate some from the system being built (the "Audit" box in Figure 2). This is done by inspecting the code, extracting various features from it, describing that as a diagram, written in English, and then using Kite9 to render the results.
There is value in both of these approaches and they are not the same thing. One is showing what we would like to build (or one person's conception of it) the other shows a view on what we have built so far.
A Working Example
As an example of this, I am going to include three diagrams. And, I am going to eat my own dog-food. That is, I am going to use Kite9 (as diagramming software) to describe Kite9 (as a software system being built).
One is a Business System Diagram, of how Kite9's diagramming service should work. The other two are Software System Diagrams, generated from the Kite9 source code itself.
My Business System Diagram
Ok, here it is:
Ok, so I hope you can basically see that what I am trying to show here is that you can use Kite9 with the designer or with the build tool. Who is the intended audience for this diagram? It could be anyone - and with the right explanation I could show what's going on here to my manager, perhaps.
This took under 5 minutes to draw using the Kite9 Designer and insert into this post, which I think is good going. I would be happy to present this diagram in a meeting, as I think it is well laid out and captures what I want to say. But your mileage may vary, and yes, there is a bit of a learning curve to get up.
My Software Diagrams
Ok, next up, here my two Software diagrams. I've generated these from some of the code whcih handles planarization in Kite9. What is planarization? Put simply, this is the process of choosing the layout of the elements of the diagram to show them flat in a picture (i.e. on a 2D plane, hence the name). Planarizations are graphs, so they have vertices connected together by edges, and these form faces.
Here is the Kite9 Class Diagram showing the components of the planarization.
If you have any experience with Object Oriented programming, the idea of interfaces, classes, extending and implementing shouldn't be too onerous. As with most of these types of diagram, abstraction is shown going up the page (Kite9 manages that automatically).
Ok, so that's pretty simple, but how do you generate a planarization? Well, you use an Planarizer. What's involved in the planarizer?
Looks a bit complicated, right? So this diagram is showing the planarizer on the left. Over on the right are things that the planarizer requires in order to work. There's a few bits and pieces going on there... don't spend too much time trying to figure out what it's all about.
Who is the audience for this diagram? Clearly, someone working on the project who has a fairly intimate understanding of the planarization process.
How hard was it to generate these diagrams? Not that hard: Kite9 has a java library which helps you with extracting the details to include in each diagram. All I had to do was add a few Java annotations to my classes to indicate which items I wanted to add, and then add a 5-line method to say I wanted a class diagram. So hopefully, this reduces the cost of creating software system diagrams, and means that programmers can get on with doing the programming rather than the drawing.
So, in conclusion: software diagrams are generally a good thing. They do help communication on projects between the different stakeholders. But, they're a pain to create and a pain to keep updated. Kite9 can help you with that. Forget the roundtrip engineering tools, and instead use the Kite9 Designer to knock up the business diagrams really quickly.
If you're interested in generating diagrams from your Java code, take a look at the Java Section of the Kite9.com site: The Java Hello World is a good place to start, and to really understand the benefits of using Kite9 in a project, have a look at this article about Class Diagrams.