Using XText to make Software Easy to Configure
The meaning of Configuration
Imagine a piece of software which can be run out of the box without needing any instance- specific information – without any configuration. Not that the software runs with overridable default values if you do not supply any; this software was developed perfectly, foreseeing all operating needs and circumstances and never needing to be configured. Of course, this rarely happens except for bespoke software created to be installed in just one static situation. Normally, there are many behavior-defining details which are unknowable as we develop our software because the required behavior varies somewhat from user to user or installation to installation. The variation arises from things like differing requirements according to line-of-business or integration between ours and other systems whose credentials are installation-specific. For this reason we anticipate the variation, declare those details configurable and build our software around them expecting them to be resolved to concrete values at runtime. Industrial scale systems typically require extensive configuration to tailor the system for any one customer’s needs and circumstances – a specialist’s task.
The configuration is often read at system startup, but this is not always the case. For example in the case of the Coucho Resin web server, the configuration file web.xml, in which website page redirections can be specified, may be modified and saved; this action immediately updates Resin’s web-serving behavior without it needing to be re-started.
Before we investigate the different ways of representing configuration information let us have a working definition of configuration:
The consolidation of information input to a system which is unknowable at build time, is read as the system is initialized or as changes are detected, accommodates installation-specific details and once interpreted, from that point onwards may direct the behavior of all other execution.
This definition covers what the configuration amounts to from the viewpoint of the system, but bear in mind we are also interested for this topic in the ease with which the user can edit the configuration. Yes, the user supplies values, but it helps if the process of supplying those values is guided by options and confirmed to form a meaningful set by validation.
Actually configuring software can often involve a mixture of input types: command line arguments, .ini files, XML files (for example: deployment descriptors or bean wiring), environment variables, admin screens and setup wizards not to mention plugins for advanced configuration and customization. Or the configuration settings could live in rows in a database table. So why propose yet another way to configure software? Well, if we look at some of the pros and cons of these conventional ways first then we will be in a position to judge how using a Domain Specific Language (DSL) measures up as an alternative.
Java frowns on environment variables due to their platform dependence, while .ini files and the command line offer only limited expression of a handful of configuration values. If our software needs any magnitude of structured, iterative, self-referencing, type-validated data to direct its execution, then the choice boils down to either grabbing values from an XML file, building user interface screens to guide the user through the process or declaring a plugin API.
Example: Logistics System
Suppose we are building a logistics monitoring system which polls a number of services, one of which is called the Cargo Booking Service, and notifies specified members of staff whenever there is a change to the services’ availabilities. In reality, such a change in availability may well have to trigger a range of different types of actions and make a variety of co-ordinating calls to other systems, but for the sake of simplicity let us assume the response is simply to send an email to an individual. This logistics system will be used as the example against which we will try different approaches to configuration.
The XML to configure our logistics system might look something like this:
<staff id="bsikes" name="Bill Sikes" email="email@example.com"/> <staff id="jdawkins" name="Jack Dawkins" email="firstname.lastname@example.org"/> <notification source="cargo-booking" event="unavailable"> <notify staff-ref="bsikes"/> </notification>
Two members of staff are declared, each with a unique id, real name for addressing them formally at the start of any message and their email address. These are followed by an instruction to notify the first staff member upon the Cargo Booking Service becoming unavailable.
Using XML for configuration in this case is fairly quick and convenient to develop and test. A third-party library does all the hard work of parsing and validating the XML for us and serves it up as a structure from which our software can just pluck the values it needs any place in the code it needs them. For example the Document Object Model (DOM) is one way of doing this. We can quickly lash together the first config file in XML letting its format firm up along-side the software that depends upon it, adding more tags and attributes as needed. But as well as making it easy for us programmers XML also offers help in various forms to the user doing the configuring (the stick man in the diagram above). XML is plain text; and plain text has advantages over a Graphical User Interface (GUI). Plain text nurtures collective knowledge because snippets can be easily dropped into an email or pasted to a bulletin board. Plain text is search engine friendly, so anyone looking for examples of how to configure our software can search on any XML tag and view theÂ examples which are thrown up. Furthermore, comparison between configuration file versions is simple with plain text: version control tools display who changed what – shedding light on why the system has started spamming some members of staff with cargo booking event notifications they do not care about. Finally, to give more guidance during configuration an XML schema can be declared which can direct an XML editor to enforce data types and apply structural constraints.
However, despite these advantages over a GUI, asking users to edit raw XML is only appropriate if they are already used to XML’s syntax of tags and attributes and if the configuration mainly contains small, unconnected structures.
The GUI to configure our logistics system could be a separate dedicated application just for editing a single configuration file or could be part of the same application with its configuration state persisted by an unknown persistence provider of some kind. For a GUI we would probably hide the configuration model and storage behind a facade. The point is that a GUI to guide the user through the process of setting up our software is more appropriate where the configuration requires constraints to be met as the configuration settings are entered.
Let us consider what the GUI for configuring our logistics system might look like.
Now we can assist editing of the configuration by offering a set of legal options such as theNotify list above being populated from previously created users. We can also validate the configuration by forcing the selection of one or more users before enabling the Createbutton. The GUI warns when deleting a member of staff would leave no-one notified when the Cargo Booking Service becomes unavailable.
But developing GUIs is slow and hinders agility; GUIs take time to test. Even with GUI testing tools effort is still required to cover all permutations of actions. Bear in mind also that whenever the underlying software which uses the configuration is updated or refactored then the GUI needs corresponding amendments to keep it and its tests in step; and all the screenshots in the documentation need to be revised too.
While a well crafted GUI can make light work of configuration for even the first-time user of our software, a poor GUI can make the same task irksome for all. Bleating error messages, poor explanations as to why an input is unacceptable and controls disabled forÂ no intuitive reason will quickly frustrate any user. The worst trait of a poor GUI is low visibility of the impact of a change. In the absence of any indication of how many outstanding issues need to be resolved before the configuration will reach an acceptable state, the GUI offers no guidance as to whether it is best to bash on with a particular configuration change or revert to a previous state and pursue a different strategy.
General Purpose Languages
We could expose an API to our logistics system in a General Purpose Language (GPL) such as Java or C#. The staff and notification objects would be expressed in the GPL in a plugin library which the user writes, compiles and places in a location discoverable by our system at run time.
ILogisticsConfigurationPlugin is an interface which we declare in our API, not shown here but can be assumed to declare the signatures of the two public methods. User and Notification are immutable classes also exposed by the API with constructors as called.
This is patently a heavyweight means of supplying configurability, effectively leaving a chunk of the coding to be completed at each installation. Yet it offers the ultimate in flexibility as the plugin is free to obtain the information programmatically from whomever, wherever and however it pleases; not necessarily hard-coded as in the example above.
The GPL is more than capable of expressing the configuration information our logistics system requires and the compiler checks that the contract of the interface is met and the statements are executable; but the compiler knows nothing of the notion that the Notification constructor’s User parameter must be an existing User and not a User constructed in-line.
If there really is no need for the degree of flexibility offered by a plugin then using a full- blown programming language just to express data values and structure invites problems. The configuration plugin writer could legally add a System.exit(), return null, or perform any of a hundred other undesirable actions in this plugin code. In addition, we have placed a condition on our software that those configuring it be proficient in the GPL.
The XText tool allows you to create your own programming language, a Domain Specific Language (DSL). You do this by specifying your language’s grammar then asking XText to generate a supporting model and editor.
The advantage of a DSL over a GPL is the higher degree of abstraction, or closeness of the language to the problem space. Information expressed in the DSL is more concise and readable than the same information expressed in a GPL. If the DSL’s grammar is designed thoughtfully, anyone familiar with the problem space will stand a fair chance of understanding a script written in the DSL on first sight. Furthermore, a DSL restricts the scope of what can be expressed to just constructs relevant to the domain space. This is a good thing. Narrowed down options make the language easier to learn and minimize the possibility of unexpected or harmful behavior.
So there it is. A readable script written in a language dedicated to the configuration of our logistics system. But there is more. The editor used to enter this script is not a plain text editor; the editor is dedicated to the DSL – it was generated from language grammar definition – and this is where the power of using a DSL really kicks in.
We define the syntax of our language (in this case using the XText language development framework) using the grammar definition language in four lines as follows:
A full description of how grammars are declared in XText is outside of the remit of this article, but to help make sense of this grammar its meaning can be paraphrased as follows:
(Line 1) The model consists of a list of config elements and (Line 2) a config element is either a Staff or a Notification declaration. (Line 3) Staff declarations start with the literalsStaff id followed by an identifier, then name, then a string, then email, then another string. (Line 4) Notification declarations start with Notify, followed by a staff id, then when, then either cargo or security, then becomes, then either unavailable or restricted.
This grammar is all that is needed to generate an editor which enforces the basic structural validation we need (in practice you would probably include validation of single fields such as the email address in the grammar). From the grammar definition the editor deduces how to validate the configuration, issuing warnings and errors.
The red underline, the error marker indication and the tooltip come for free. The generated editor also offers code assist:
Not bad for four lines of grammar code; and hence our software development stays fantastically agile. We can quickly drop in a new construct or a new validation rule as the need arises without burning up weeks of work re-jigging, re-testing and re-documenting a GUI. But not only does the user benefit from the text editor being aware of the structure of what is being entered, the software development benefits from the grammar definition forming the central definition of all configuration data and its constraints. Our software is then free to focus on its main line of business without concerning itself with validating configuration values.
Along with the editor, an Eclipse Modeling Framework (EMF) model is induced from the grammar. This EMF model loosely equates to the DOM in the XML case above. It is via this EMF model that our DSL-configured software accesses its configuration values.
Summary of Configuration Input Types and their Features
We want to make our software configurable in a way which is easy for the users to configure, encourages community knowledge about its configuration, guides the user through the configuration process as much as possible, allows easy comparison between versions, gives an indication of the impact of a change, and all without harming agility. Using a DSL for configuration yields all of these benefits and with a little care in grammar design can result in a configuration script which reads close to English.
Beck, K. (2000) Extreme Programming Explained, Addison Wesley
http://www.w3.org/DOM/ Steinberg, D. et al.
EMF: Eclipse Modeling Framework, Addison Wesley