Business Activity Monitoring (BAM) is a collective name for a class of software applications that aid in monitoring of business activities, as those activities are implemented in computer systems. The aim of this article is to introduce you to one such application, namely open source project Dashbuilder, present its main features and usage samples. The presentation will be accompanied by screenshots of the application's user interface.
The need for Business Activity Monitoring
Ability to collect, aggregate, and present data is a very common requirement for software systems, especially for those that support running business in a company. The raw data about business activities usually need to be presented in some user friendly format, be it in the form of tabular reports or in the form of visual charts and graphs. Target users of these features are often company managers, who can use these features to get immediate insight into the running processes and base their decisions on hard, ideally real-time data, instead of relying on intuition or hunch.
Data that contain answers to the questions that management might have are often available, spread across many database tables or buried in Excel files. Unfortunately, in general it is not easy to change the format of the reports or create new charts based on the available data because it requires non-trivial programming expertise and effort. Moreover by the time the required reporting capability is implemented, the answers provided by it might no longer be relevant.
This motivates the need for the kind of software systems, that would make it possible to create new data visualizations or define a new report format quickly and easily, ideally without the need for programmer intervention.
Dashbuilder is a web application, written in Java, which provides many BAM features. It was formerly developed by Spanish company Polymita and was a part of its Business Process Management solutions. In 2012 Polymita was acquired by Red Hat. Shortly after the acquisition, Red Hat open-sourced the Dashbuilder, which is now freely available on GitHub, licensed under the Apache License, Version 2.0.
As its name suggests, its primary feature is the ability to build dashboards - a set of interactive web pages that present data obtained from various data sources. Data can be presented in the form of various charts or tables. These data visualizations are interactive and can be for example filtered using various criteria. Creating and configuring dashboard does not require programming skills and is done in the application itself, using drag & drop and WYSIWYG editors. This means that business users can configure the dashboards themselves, without the need for programming skills.
Dashbuilder can be deployed in the form of java war file (Web Archive) into one of several widely used java application servers. Currently, it should be possible to deploy it to: JBoss AS 7.x, Widlfly 8.x, IBM Websphere 8.5, Weblogic 12c and Tomcat 7.
In order to try Dashbuilder we first need an application server, into which Dashbuilder war file can be deployed. I downloaded the latest community version of JBoss AS 7 (JBoss AS 7.1.1.Final), which is free and open source. The installation of the application server consists of just unzipping the distribution zip file. The only configuration that is needed is adding a user account, that can be used to login to Dashbuilder. That can be done by running a shell script
jboss-as-7.1.1.Final/bin/add-user.sh and filling in required user name, password and assigning (one of the) roles
admin. That is all you need to get started. Later, once you get acquainted with the main features, you can perform more advanced customization at the level of application server (e.g. specifying database connection parameters, creating additional user roles etc).
Afterwards, you will need a to get the
dashbuilder.war file that can be deployed to the application server. You can either download it here or you can build it from the sources yourself. To build the application from sources you need to follow the instructions on the project's GitHub page.
The installation of application is done by copying the
dashbuilder.war file to the application server's
deployments folder. Make sure you rename the war file using the command like
mv dashbuilder-6.2.0-SNAPSHOT-jboss-as7.war dashbuilder.war. This will make it simpler to access the application using your web browser. Now you just needed to start the application server using
jboss-as-7.1.1.Final/bin/standalone.sh. After the application server is up and running, open a browser window, navigate to
http://localhost:8080/dashbuilder/ and login to the application using the user account you created when setting up the application server.
Organization of dashboards - workspaces, pages and panels
There are three concepts the Dashbuilder uses to structure dashboards: Workspace, Page and Panel. Together they form a hierarchical structure: Workspace consist of Pages and Pages consist of Panels. I will try to explain each one in turn. Workspaces are primary means of organizing dashboards - they are basically a collection of web pages, which can share some common configuration. At the level of workspace you can for example set visual style of pages contained in it or restrict users that have access to it (e.g. one workspace for financial department, one for company managers etc.).
After you login to the Dashbuilder for the first time, you will see the so called Showcase workspace. As the name suggests, Showcase workspace is a demo created by application developers for the purpose of presenting most of the application's features. Users are expected to create their own workspaces either by starting with the Showcase and gradually reconfiguring it, or by creating an empty workspace from the scratch and incrementally adding content to it until it suites their needs.
I think it is not necessary to explain what a Page is - everyone who uses the Internet already knows that. From our point of view it will be more interesting how such a page can be composed in the Dashbuilder.
The main constituent part of the page is a Panel. Panel is a rectangular unit with some content, that can be added to the page using drag & drop. The application provides several kinds of panels out of the box: the most interesting is Key Performance Indicator (KPI) which has the capability to visualize data in the form of chart or tabular display - more on these later. The image on the right shows the variety of configuration options available for KPI panel. Closely related to the KPI panel is Filter & Drill down panel. Using this panel users can select various constraints on the data being displayed to filter the data or to focus on specific subset of the data. Next there is HTML Panel which basically enables embedding arbitrary HTML content in the page. The application contains embedded WYSIWYG editor for editing this type of panel. Then there are various navigation panels, that make it possible to quickly switch between pages in the given workspace or within workspaces themselves: Page menu (vertical/horizontal), Workspace menu, Tree menu and Bread crumb are just a few of the many navigation panel types. Last but not least there are several panel types, that serve for application configuration and administration. These panels can also be incorporated into dashboards and in this way the application's administration features can be customized: Data source manager - enables configuration of connections to external databases, Data provider manager - enables specifying what data will be extracted from given data sources. Export/Import workspace enables exporting of existing dashboard configuration (workspaces, pages, panels, data source configuration etc.) in the form of XML files that can backed up and later imported back into the application.
Retrieving data - databases and CSV files
In order to visually present data of interest to the users, Dashbuilder first needs to retrieve these data from some external data source. Our of the box it is possible to obtain data in two ways: from databases using JDBC and from CSV (Comma-Separated Values) files. In order to do that, user first need to do some configuration. This configuration is usually done once for given source of data and is then reused every time user needs to analyze the data from given data source.
The first way of obtaining data - from the JDBC databases - is more flexible of the two. It enables you to obtain data from any JDBC compliant database. For demonstration purposes I chose H2 database, which requires no setup, because the required JDBC driver is already provided as a part of JBoss Application server distribution. For other databases you need to make the appropriate JDBC driver available on the application's classpath. In JBoss Application Server this is done simply by putting the JDBC driver into the deployments folder or by creating new module (see this guide).
The rest of the configuration is done in the application's user interface as shown in the screenshot. The configuration consists of three steps: first you create a so called External connection, which specifies connection parameters for the database, like JDBC URL, user name and password. Alternatively you can also specify JNDI data source name and path in case you already defined these at the level of application server.
The second step is to create Data provider, which is basically an SQL query over the database. For given external connection many Data providers (i.e. Queries) can be created. The result of each SQL query is tabular data. Dashbuilder automatically extracts information about the columns and/or their data types. In the third and final configuration step you can modify column names or their data types. Now the data source is ready to be used - data contained therein can be extracted and presented by various charts.
Another way to obtain data is from CSV files. CSV files can be usually exported from spreadsheet applications, so they are easy to obtain, even for non technical users. Also the configuration in Dashbuilder is simple and is done in two steps. In the first step you specify where the file can be downloaded by providing the URL (it can also be a file from the local file system as the example shows) and what is the format of the data in the file : what characters are used for data separation, quoting and escaping and what date and number formats are used.
As with the databases, Dashbuilder automatically determines the structure of the data present in the CSV file (number of columns, their labels and data types) and lets you modify this information in the second step.
Presenting, filtering and analyzing data
Now that we have the JDBC or CSV data source configured, we can go on and create some visualizations based on those data. As mentioned before there is a special kind of panel designed for this purpose: KPI panel. Adding KPI panel to the page is done simply by selecting it from the list of available panels and then dragging and dropping it to the desired area of the page. Once the panel is dropped on the page you can configure it by first selecting the data source (which we configured in previous step) from which the panel will obtain data to display. After data source is selected, an interactive chart editor is displayed, in which you can specify various aspects of visualization (see image).
I will not describe all the possible settings you can make for KPI panel, but just to give you a flavor, you can choose:
- What type of chart you want: Pie, Bar or Line charts.
- What data are displayed on X and Y axis of the chart.
- Range of data to be displayed (e.g. for Date - based data you can choose to display data for last month only)
- Aggregation function to be applied to data (e.g. you can choose to display SUM of sales for given department, AVERAGE salary of employee in given department, or NUMBER OF DISTINCT tickets your employee was working on etc.)
- Whether data will be sorted: ascending/descending/unsorted based on name/value
- Various visual characteristics of the chart: its width, height, position inside the panel, alignment, whether title will be displayed, the angle at which the axis labels are displayed, etc.
Alternatively you can choose to display data in the form of table. The nice feature of table KPI panels is, that they enable you to export data in the form of Excel or CSV files, which you can download directly from the user interface. As with charts, tables too have a rich set of settings. You can choose:
- What columns the table will have and their order
- Whether it is sortable (in that case table data can be sorted by clicking column header)
- The style of header / table cells (using CSS)
- Whether filtering based on table data is enabled (more on this later)
- Number of rows per page
- Header position (top / bottom / not displayed)
- Whether data will be grouped by particular column value (similar to SQL feature GROUP BY statement)
Once you have some KPI panels configured you can start analyzing data presented in the dashboards. The primary feature to achieve this is filtering, also known as drill down. Filtering means setting various restrictions on data to be displayed in the charts, thus focusing on just a part of data set. It can be done in two ways: either by setting the restricting values in the Dashboard filter panel (see screenshot) or simply by clicking the charts themselves. For example you can click on the bar in a bar chart, which represents sales in France. This way only the data relevant for France will be displayed not only in the current chart, but also in the other charts.
A common concern users might have is the scalability of this approach. Let's say I have a database with hundreds of thousands of entries. Will it still be possible to present and analyze these data in Dashbuilder? The answer is Yes. The application offers two approaches to working with data: you can either let it load all data into memory (suitable for small to mid-sized data sets) or you can offload the task of data filtering to database (scales to large data sets). More on this can be found in this blog post.
Other interesting features
We've only scratched the surface of the configuration possibilities Dashbuilder has to offer. In fact the application provides a separate administration page, where a lot of additional settings can be tweaked. Let's just mention several possibilities accessible via the administration page.
Customizable security settings
You can grant or deny various permissions to the users based on the roles they have. For example you can give a permission to edit chart settings to the manager, or you can prevent accountant to view some of the dashboards that only managers should see. The roles are defined at the level of application server (for example you can have roles like accountant, analyst, manager, administrator) and each user is given one or more roles. The default security settings are mostly fine, but in case you need to tweak them, you can do so at four levels of granularity: global (will take effect for the whole application), workspace, page and panel level.
Internationalization (i18n) support
Out of the box it is possible to create dashboards in multiple language variants. All the textual content that you can create in the dashboards has a language associated with it. That means that you can for example create one welcome screen for English users and one for Spanish users. To switch the language the application provides Language panel, which can be seen in the screenshot in the top right corner. Using this panel users can switch the language of the content and based on that appropriate content is presented to them. It is possible to add / remove languages and locales based on your preferences.
Customizing page page layout and graphics
For those, that are not satisfied with the default look and feel of the application it is possible to change by using different background images, CSS style sheets and even organization and sizing of page regions, into which panels can be dropped. This too can be done in the user interface by uploading zip file with appropriately formatted content.
Export and import of dashboard configuration
Once you configure your dashboard it is sensible to backup your configuration. There is a feature that enables you to export all the information about the workspace configuration and all the dashboards contained in it. This exported configuration can then be downloaded in the form of XML files. You can use those either to quickly configure new installations of Dashbuilder or to restore the configuration to some previous state.
I hope I whetted your appetite to try Dashbuilder. It has the potential to satisfy many requirements usually required from BAM applications. It has rich customization possibilities and what I like most is, that all of this can be done without the need for programming. You can learn more at the official web site of the project - dashbuilder.org