BIRT - A Technical Interview With Evangelist Virgil Dodson
[img_assist|nid=3261|title=|desc=|link=none|align=right|width=120|height=120]BIRT is a top level project through Eclipse and one of the most popular tools for designing and developing reports for business. Web Builder Zone Leader, Schalk Neethling, recently had an interview with BIRT evangelist Virgil Dodson on the more technical aspects of BIRT as well as topics on best practices and getting help when you run into problems using BIRT. Here follows the interview with some really insightful answers provided by Mr. Dodson that we are sure developers will enjoy and find useful.
Schalk Neethling: Hi Virgil and thank you for granting us this interview about BIRT. Please introduce yourself and let us know a little about your background as well as the work you are involved in over at Actuate.
Virgil Dodson: Hi Schalk, thanks for putting this interview together. I am currently an Evangelist for BIRT and the BIRT related products from Actuate where I spend most of my time as forum moderator and blogger at the BIRT Exchange community site. I am a developer at heart and have been working in development related jobs for the past 14 years. I spent many years working with a Java spreadsheet component called e.Spreadsheet Engine (more widely known as Formula One). Since 2001, I have been helping Java developers get started with Actuate’s embedded reporting products.
Neethling: Now that we know who Virgil Dodson is, my next question is what exactly is BIRT?
Dodson: BIRT, which stands for Business Intelligence and Reporting Tools, is a top-level project through Eclipse. This project provides open source tools related to business intelligence and reporting. At a high level, BIRT provides a graphical report design environment, packaged either as a plug-in for the Eclipse IDE or as a standalone report designer, and a set of runtime APIs for integrating reports into your applications.
Neethling: I have had my fair share of writing reports using various tools from homegrown ones to commercial and open source tools. Why do you believe developers and report designers should choose BIRT over something like for example Jasper Reports?
Dodson: In 2004, using years of experience in the BI and Reporting industry, Actuate had the opportunity to build a reporting system from scratch. The result was BIRT which was built to cater to today’s web-based reporting scenarios.
Neethling: From inside the BIRT designer you are offered various options when it comes to datasources such as Flat file, JDBC, Web Service, Scripted etc. Although all of these options are provided as options, which would you consider to be the best choice or, is support and performance equal across all datasource options?
Dodson: Based on BIRT questions coming through, the JDBC data source is probably the most used however each are supported equally. As for performance, each data source will perform differently. For example, using a JDBC query where I can offload the sorting to the database will likely perform better than using a text file and sorting within a BIRT table.
Currently, you can connect to JDBC databases, XML, Web Services, flat files, and Java Objects through the Scripted Data Source… and you can mix and match these data sources with a Joint Data Set. The data sources within BIRT, as part of the Open Data Access (ODA) architecture, are also extend-able which means you can add you own custom data source.
Neethling: My next question is similar in nature to the previous but, more directed at how the data that the report design will be populated with is handled. Of the various methods I am aware of the options seems to be either using Rhino or Java event handlers. From a performance perspective, what would you suggest to a developer, handling the parsing of data from inside the design and/or the library via Rhino scripting or externally, using Java?
Neethling: I would like to present a short case study to you and would appreciate your comments:
Recently a developer using BIRT in their production environment complained about the peculiar performance he was getting from the BIRT engine. From what they could detect, it seemed that the larger the design file (rptdesign) becomes the slower the performance for reports both run asynchronously as well as when run in parallel. Further investigation made it clear that this particular design was uniquely different from their other reports in that the design itself was very large and also included a lot of conditional hiding and showing of certain sections. They have also decided to make the libraries that the design use smaller and more compact, but this led to a lot of small libraries that the design needed. They were also using Java event handlers as apposed to Rhino to handle the parsing of data and the population of the datasets.
From this quick overview of their architecture and design decisions, can you spot any particular area where they went wrong in their decision making or would you suggest a completely different approach?
Dodson: From reports we have seen in the broader community, BIRT performance is very strong -- with most performance tuning focused on external factors such as query execution time, as well as optimization needed for very complex reports. The size of the design file itself is generally not a good indicator of performance to expect, however, the more controls added to a report, the larger the design will be and the more processing will be needed.
For this scenario, I would likely focus on the number or controls and amount of data used within the report. For example, if this report gathers lots of information and then conditionally shows or hides controls for each user preference then we could be preparing more under the scenes than we actually need.
To check this, make sure you are not pulling more data than needed. Use the property binding tab within the dataset to modify the query at runtime. For individual elements or sections that are being hidden with the visibility property, consider adding a small script that drops an element from a report rather than just hiding it. As for embedded script vs external Java class performance, I haven’t seen much difference in my simple tests, but I would check to make sure only the minimum processing is happening in the Java event handler, especially if it is looping through all the data multiple times.
Neethling: With the previous question in mind, what are the key factors that influence the performance of BIRT reports?
Dodson: There are many factors that influence performance starting with the physical hardware like processor speed and amount of memory allocated. There are also several factors within the report design process that can affect performance. For example, as mentioned earlier, databases are very efficient at sorting data so including an ORDER BY clause in the query will likely be faster than sorting within a BIRT table. One of the recent webinars offered through BIRT Exchange was on designing high-performance BIRT reports. An archive is available at http://www.birt-exchange.com/modules/news-events/webinar.php?lid=347.
Neethling: Does Actuate or the Eclipse Foundation offer any specific technologies, either commercial or open source, that are focused on batch report generation? With this I mean that there is no use of the BIRT viewer in this scenario but, that multiple reports, between say 10,000 - 30,000, are generated in batches and emitted and saved to disk as either PDF or PostScript documents.
Dodson: BIRT includes a runtime API for integrating BIRT reports into an application and it's possible to write code around this to build basic batch scheduling capabilities. For out-of-the box scheduling, Actuate offers a product called the Actuate iServer Express. The iServer Express can securely deliver reports through a J2EE application, or they can be scheduled to run at a certain time and with certain permissions. Either an administrator, or the end-user, can schedule reports to run, have the reports emailed to them, and track which reports failed or succeeded.
Neethling: There are two technologies that I recently looked at from Actuate, these are the iServer and iPortal. Who is your target market for these two products and in which ways do these products make the lives of developers and designers alike easier?
Dodson: I'll start by talking about the iPortal, now called the BIRT Deployment Kit. This deployment kits saves the developers’ time in the code they would generally have to write to deploy BIRT (and e.Spreadsheet) reports. The Deployment Kit is delivered as a WAR file and can be easily integrated with a web application and supports branding to match the application style. Besides generating BIRT reports, the deployment kit comes with two additional options. One is called the BIRT Interactive Viewer, which lets web users customize their BIRT reports right in the browser. The other option is called BusinessReport Studio. This option allows end-users to design BIRT reports directly in the browser based on BIRT templates created by IT. The BIRT Deployment Kit can be downloaded for trial from www.birt-exchange.com.
The second product mentioned above, the iServer Express, is a report server for securely delivering BIRT reports. This product supports security integration, scheduling, report generation for BIRT (and e.Spreadsheet) reports. iServer Express also supports the Interactive Viewer option and the BusinessReport Studio option mentioned above. The iServer Express can be downloaded for trial from www.birt-exchange.com.
Neethling: I have voted for a couple of the accessibility related bugs on BIRT's bug tacking system. My viewpoint on these issues are the same as my views on accessibility as it relates to the web. So I must say I am very pleased that the developers are looking into these issues and have it as a high priority, an unfortunate rare occurrence in the development culture these days. With that said, what exactly are the plans for the next and future releases of BIRT from both the perspective of the viewer as well as the report designer.
Dodson: From the perspective of improved accessibility in BIRT 2.3, we have added 2 new properties ‘Caption’ and ‘Summary’ for report elements that generate HTML tables. (Please see http://www.w3.org/TR/html401/struct/tables.html#edef-CAPTION and http://www.w3.org/TR/html401/struct/tables.html#adef-summary for more information.) Please let us know what other enhancements you would like in this area.
Neethling: Will these improvements be available to both users of the Actuate BIRT designer pro as well as the open source alternative?
Dodson: The above mentioned enhancement is available in open source and Actuate BIRT Report Designer Pro.
Neethling: One thing I find seriously lacking with regards to BIRT, both for report designers and developers, are documentation and more specifically best practices. The BIRT-Exchange forum that is run by Actuate does seem to want to bridge this gap but, at the moment the information found there seems to be very sparse. Is their an active effort underway be either Actuate or the Eclipse Foundation to encourage contribution to make the BIRT-Exchange a more valuable resource?
Dodson: We also recognized a need for more information on using BIRT which is why we launched BIRT Exchange in the Fall of 2007. There is quite a bit of technical information about BIRT around the web and BIRT Exchange was designed to be the place that lumped it all together. On BIRT Exchange you will find the BIRT Documentation, forums to ask (or answer) questions, a mirror of the Eclipse BIRT newsgroup so that valuable information is searchable, a Wiki with an FAQ and troubleshooting tips, a monthly newsletter, bi-weekly technical webinars, and an area called the DevShare.
The DevShare contains many resources, like sample report designs and code, tutorials, quick tips, and links to technical blog posts or articles related to BIRT. While there are lots of BIRT resources available already we know the site could always use more. We are actively watching for BIRT resources to add to the site and as a community site, we are also encouraging contributions from outside Actuate.
You will also find several BIRT books available from Amazon.com. Two books that Actuate helps put together, BIRT: A Field Guide to Reporting and Integrating and Extending BIRT have recently been updated (Second Edition) and these include several hundred more pages than the previous edition. Watch for the new edition to hit Amazon soon. We have also seen two new BIRT books at Amazon. Practical Data Analysis and Reporting with BIRT from John Ward, and a German book called Eclipse BIRT from Ulrich Obst and Dennis Schladebeck.
Neethling: What are the steps and criteria for designs and developers that want to get involved with BIRT both from a documentation and active development perspective?
Dodson: To become an official BIRT committer, you should be nominated by another committer, or I think you can ask to be nominated. In either case, the rest of the committers will hold a vote to decide if you become an official committer. If you have input you want to provide to the BIRT team, you can submit bugs into bugzilla without being an official committer.
If you have tips other developers could benefit from, or best practices related to BIRT reporting, you can add this information to the BIRT Exchange community site by simple signing up for a free membership. Community members of BIRT Exchange can add items to DevShare, modify the FAQ and Wiki, ask and answer forum questions, and comment on other developers submissions.
Neethling: A common complaint I have read and heard from the community is that it is incredibly difficult to get help when one gets stuck with a problem using BIRT. Which resources would you suggest users make use of when trying to find answers to their questions?
Dodson: I would certainly start with BIRT Exchange to find information you need. The site’s global search will search across the DevShare, documentation, Wiki, and the Blog. If you don’t find what you need, you can ask a question in the forums. If you still need help, Actuate has support staff trained on BIRT and services staff to help get you started.
Neethling: Last but not least, what can the BIRT designers and developers out there expect from BIRT, the Eclipse Foundation and Actuate as you move towards a 3.0 release?
Dodson: We are currently in the release planning phase for BIRT. The project plan for the next release of BIRT (post 2.3) will be made available to the community when it is ready. Please stay tuned for more details. In the meanwhile if you have specific feedback, please file enhancement requests in Bugzilla.