Tomasz Dziurko

Think you shouldn't customize IntelliJ IDEA memory settings? You're wrong. Here's why you should?

from time to time there is a moment when we have to deal with xml data. and most of the time it is not the happiest day in our life. there is even a term “xml hell” describing situation when programmer has to deal with many xml configuration files that are hard to comprehend. but, like it or not, sometimes we have no choice, mostly because specification from client says something like “use configuration written in xml file” or something similar. and in such cases, xstream comes with its very cool features that make dealing with xml really less painful. overview xstream is a small library to serialize data between java objects and xml. it’s lightweight, small, has nice api and what is most important, it works with and without custom annotations that we might be not allowed to add when we are not the owner of java classes. first example suppose we have a requirement to load configuration from xml file: /users/tomek/work/mystuff/input.csv /users/tomek/work/mystuff/truststore.ts /users/tomek/work/mystuff/cn-user.jks password password user secret and we want to load it into configuration object: public class configuration { private string inputfile; private string user; private string password; private string truststorefile; private string keystorefile; private string keystorepassword; private string truststorepassword; // getters, setters, etc. } so basically what we have to do is: filereader filereader = new filereader("config.xml"); // load our xml file xstream xstream = new xstream(); // init xstream // define root alias so xstream knows which element and which class are equivalent xstream.alias("config", configuration.class); configuration loadedconfig = (configuration) xstream.fromxml(filereader); and that’s all, easy peasy something more serious ok, but previous example is very basic so now let’s do something more complicated: real xml returned by real webservice. 2013-03-09 john example 24 asd123123 2012-03-10 anna baker 26 axn567890 2010-12-05 tom meadow sgh08945 48 what we have here is simple list of bans written in xml. we want to load it into collection of ban objects. so let’s prepare some classes (getters/setters/tostring omitted): public class data { private list bans = new arraylist(); } public class ban { private string dateofupdate; private person person; } public class person { private string firstname; private string lastname; private int age; private string documentnumber; } as you can see there is some naming and type mismatch between xml and java classes (e.g. field name1->firstname, dateofupdate is string not a date), but it’s here for some example purposes. so the goal here is to parse xml and get data object with populated collection of ban instances containing correct data. let’s see how it can be achieved. parse with annotations first, easier way is to use annotations. and that’s the suggested approach in situation when we can modify java classes to which xml will be mapped. so we have: @xstreamalias("data") // maps data element in xml to this class public class data { // here is something more complicated. if we have list of elements that are // not wrapped in a element representing a list (like we have in our xml: // multiple elements not wrapped inside collection, // we have to declare that we want to treat these elements as an implicit list // so they can be converted to list of objects. @xstreamimplicit(itemfieldname = "ban") private list bans = new arraylist(); } @xstreamalias("ban") // another mapping public class ban { /* we want to have different field names in java classes so we define what element should be mapped to each field */ @xstreamalias("updated_at") // private string dateofupdate; @xstreamalias("troublemaker") private person person; } @xstreamalias("troublemaker") public class person { @xstreamalias("name1") private string firstname; @xstreamalias("name2") private string lastname; @xstreamalias("age") // string will be auto converted to int value private int age; @xstreamalias("number") private string documentnumber; and actual parsing logic is very short: filereader reader = new filereader("file.xml"); // load file xstream xstream = new xstream(); xstream.processannotations(data.class); // inform xstream to parse annotations in data class xstream.processannotations(ban.class); // and in two other classes... xstream.processannotations(person.class); // we use for mappings data data = (data) xstream.fromxml(reader); // parse // print some data to console to see if results are correct system.out.println("number of bans = " + data.getbans().size()); ban firstban = data.getbans().get(0); system.out.println("first ban = " + firstban.tostring()); as you can see annotations are very easy to use and as a result final code is very concise. but what to do in situation when we can’t modify mapping classes? we can use different approach that doesn’t require any modifications in java classes representing xml data. parse without annotations when we can’t enrich our model classes with annotations, there is another solution. we can define all mapping details using methods from xstream object: filereader reader = new filereader("file.xml"); // three first lines are easy, xstream xstream = new xstream(); // same initialisation as in the xstream.alias("data", data.class); // basic example above xstream.alias("ban", ban.class); // two more aliases to map... xstream.alias("troublemaker", person.class); // between node names and classes // we want to have different field names in java classes so // we have to use aliasfield(, , ) xstream.aliasfield("updated_at", ban.class, "dateofupdate"); xstream.aliasfield("troublemaker", ban.class, "person"); xstream.aliasfield("name1", person.class, "firstname"); xstream.aliasfield("name2", person.class, "lastname"); xstream.aliasfield("age", person.class, "age"); // notice here that xml will be auto-converted to int "age" xstream.aliasfield("number", person.class, "documentnumber"); /* another way to define implicit collection */ xstream.addimplicitcollection(bans.class, "bans"); data data = (data) xstream.fromxml(reader); // do the actual parsing // let's print results to check if data was parsed system.out.println("number of bans = " + data.getbans().size()); ban firstban = data.getbans().get(0); system.out.println("first ban = " + firstban.tostring()); as you can see xstream allows to easily convert more complicated xml structures into java objects, it also gives a possibility to tune results by using different names if this from xml doesn’t suit our needs. but there is one thing should catch your attention: we are converting xml representing a date into raw string which isn’t quite what we would like to get as a result. that’s why we will add converter to do some job for us. using existing custom type converter xstream library comes with set of built converters for most common use cases. we will use dateconverter. so now our class for ban looks like that: public class ban { private date dateofupdate; private person person; } and to use dateconverter we simply have to register it with date format that we expect to appear in xml data: xstream.registerconverter(new dateconverter("yyyy-mm-dd", new string[] {})); and that’s it. now instead of string our object is populated with date instance. cool and easy! but what about classes and situations that aren’t covered by existing converters? we could write our own. writing custom converter from scratch assume that instead of dateofupdate we want to know how many days ago update was done: public class ban { private int daysago; private person person; } of course we could calculate it manually for each ban object but using converter that will do this job for us looks more interesting. our daysagoconverter must implement converter interface so we have to implement three methods with signatures looking a little bit scary: public class daysagoconverter implements converter { @override public void marshal(object source, hierarchicalstreamwriter writer, marshallingcontext context) { } @override public object unmarshal(hierarchicalstreamreader reader, unmarshallingcontext context) { } @override public boolean canconvert(class type) { return false; } } last one is easy as we will convert only integer class. but there are still two methods left with these hierarchicalstreamwriter, marshallingcontext, hierarchicalstreamreader and unmarshallingcontext parameters. luckily, we could avoid dealing with them by using abstractsinglevalueconverter that shields us from so low level mechanisms. and now our class looks much better: public class daysagoconverter extends abstractsinglevalueconverter { @override public boolean canconvert(class type) { return type.equals(integer.class); } @override public object fromstring(string str) { return null; } public string tostring(object obj) { return null; } } additionally we must override method tostring(object obj) defined in abstractsinglevalueconverter as we want to store date in xml calculated from integer, not a simple object.tostring value which would be returned from default tostring defined in abstract parent. implementation code below is pretty straightforward, but most interesting lines are commented. i’ve skipped all validation stuff to make this example shorter. public class daysagoconverter extends abstractsinglevalueconverter { private final static string format = "yyyy-mm-dd"; // default date format that will be used in conversion private final datetime now = datetime.now().todatemidnight().todatetime(); // current day at midnight public boolean canconvert(class type) { return type.equals(integer.class); // converter works only with integers } @override public object fromstring(string str) { simpledateformat format = new simpledateformat(format); try { date date = format.parse(str); return days.daysbetween(new datetime(date), now).getdays(); // we simply calculate days between using jodatime } catch (parseexception e) { throw new runtimeexception("invalid date format in " + str); } } public string tostring(object obj) { if (obj == null) { return null; } integer daysago = ((integer) obj); return now.minusdays(daysago).tostring(format); // here we subtract days from now and return formatted date string } } usage to use our custom converter for a specific field we have to inform about it xstream object using registerlocalconverter: xstream.registerlocalconverter(ban.class, "daysago", new daysagoconverter()); we are using “local” method to apply this conversion only to specific field and not to every integer field in xml file. and after that we will get our ban objects populated with number of days instead of date. summary that’s all what i wanted to show you in this post. now you have basic knowledge about what xstream is capable of and how it can be used to easily map xml data to java objects. if you need something more advanced, please check project official page as it contains very good documentation and examples.

recently in our project we reported a strange bug. in one report where we display historical data provided by hibernate envers , users encountered duplicated records in the dropdown used for filtering. we tried to find the source of this bug, but after spending a few hours looking at the code responsible for this functionality we had to give up and ask for a dump from production database to check what actually is stored in one table. and when we got it and started investigating, it turned out that there is a bug in hibernate envers 3.6 that is a cause of our problems. but luckily after some investigation and invaluable help from adam warski (author of envers) we were able to fix this issue. bug itself let’s consider following scenario: a transaction is started. we insert some audited entities during it and then it is rolled back. the same entitymanager is reused to start another transaction second transaction is committed but when we check audit tables for entities that were created and then rolled back in step one, we will notice that they are still there and were not rolled back as we expected. we were able to reproduce it in a failing test in our project, so the next step was to prepare failing test in envers so we could verify if our fix is working. failing test the simplest test cases already present in envers are located in simple.java class and they look quite straightforward: public class simple extends abstractentitytest { private integer id1; public void configure(ejb3configuration cfg) { cfg.addannotatedclass(inttestentity.class); } @test public void initdata() { entitymanager em = getentitymanager(); em.gettransaction().begin(); inttestentity ite = new inttestentity(10); em.persist(ite); id1 = ite.getid(); em.gettransaction().commit(); em.gettransaction().begin(); ite = em.find(inttestentity.class, id1); ite.setnumber(20); em.gettransaction().commit(); } @test(dependsonmethods = "initdata") public void testrevisionscounts() { assert arrays.aslist(1, 2).equals(getauditreader().getrevisions(inttestentity.class, id1)); } @test(dependsonmethods = "initdata") public void testhistoryofid1() { inttestentity ver1 = new inttestentity(10, id1); inttestentity ver2 = new inttestentity(20, id1); assert getauditreader().find(inttestentity.class, id1, 1).equals(ver1); assert getauditreader().find(inttestentity.class, id1, 2).equals(ver2); } } so preparing my failing test executing scenario described above wasn’t a rocket science: /** * @author tomasz dziurko (tdziurko at gmail dot com) */ public class transactionrollbackbehaviour extends abstractentitytest { public void configure(ejb3configuration cfg) { cfg.addannotatedclass(inttestentity.class); } @test public void testauditrecordsrollback() { // given entitymanager em = getentitymanager(); em.gettransaction().begin(); inttestentity itetorollback = new inttestentity(30); em.persist(itetorollback); integer rollbackediteid = itetorollback.getid(); em.gettransaction().rollback(); // when em.gettransaction().begin(); inttestentity ite2 = new inttestentity(50); em.persist(ite2); integer ite2id = ite2.getid(); em.gettransaction().commit(); // then list revisionsforsavedclass = getauditreader().getrevisions(inttestentity.class, ite2id); assertequals(revisionsforsavedclass.size(), 1, "there should be one revision for inserted entity"); list revisionsforrolledbackclass = getauditreader().getrevisions(inttestentity.class, rollbackediteid); assertequals(revisionsforrolledbackclass.size(), 0, "there should be no revisions for insert that was rolled back"); } } now i could verify that tests are failing on the forked 3.6 branch and check if the fix that we had is making this test green. the fix after writing a failing test in our project, i placed several breakpoints in envers code to understand better what is wrong there. but imagine being thrown in a project developed for a few years by many programmers smarter than you. i felt overwhelmed and had no idea where the fix should be applied and what exactly is not working as expected. luckily in my company we have adam warski on board. he is the initial author of envers and actually he pointed us the solution. the fix itself contains only one check that registers audit processes that will be executed on transaction completion only when such processes iare still in the map for the given transaction. it sounds complicated, but if you look at the class auditprocessmanager in this commit it should be more clear what is happening there. official path besides locating a problem and fixing it, there are some more official steps that must be performed to have fix included in envers. step 1. create jira issue with bug - https://hibernate.onjira.com/browse/hhh-7682 step 2: create local branch envers-bugfix-hhh-7682 of forked hibernate 3.6 step 3: commit and push failing test and fix to your local and remote repository on github step 4: create pull request - https://github.com/hibernate/hibernate-orm/pull/393 step 5: wait for merge and that’s all. now fix is merged into main repository and we have one bug less in the world of open source