domain-driven design background there are a series of domain model patterns that describe objects and objects group built with domain-driven design. aggregates describe cohesive object graph with a single point of entry, called root: the internal objects of the aggregate cannot be persistently references from the outside. the domain classes whose instances are inside aggregates are subdivided into entities and value objects: the former have a lifecycle (like a post or a user), while the latter are just values with methods, equivalent to strings and other domain concepts. a prerequisite of these patterns is the immutability of value objects , which can then be shared between aggregates, just like string instances can be in many languages. value objects such as numbers and colors are modified by calling a method on them that return a new instance: every change to their state should produce a new value object. repositories are collection of aggregates: they model operations such as finding an aggregate or persisting a new one. a great departure of modern ddd from the entity/relationship modelling everyone knows is the duplication of data between aggregates to support new scenarios: it's possible some field or object is repeated in different aggregates. when there is an update to an aggregate, it's not necessarily atomically reflected to the other copies of its data. i'll refer to writing calls for generality, to indicate the command side of the command query sepration, which corresponds to everything that causes a change in state in the domain objects (in opposition to the reading side). events as mail messages thus it has become common to copy data between objects in different aggregates : for example, think of a document and invoice object that share the same start/finish date interval. traditionally this duplication is dealt with by extracting a common object, mapped to a common row in the database, with a name invented on the spot. domain events are an alternative that allows for duplicating these data: they reflect changes happened in a single aggregate, and are sent to other aggregates so that they can update themselves. technically speaking, domain events are plain old $yourlanguage objects, containing the modified data but not related to the orm like the main domain objects. domain events are handy for modelling "when" rules that should always be respected no matter who is writing to an aggregate; moreover, their handling can take place in the same transaction or even in a new one. my skeptic view of events was that it can be unclear which events are communicated between objects. after a while, i accepted that unit tests tell us that; moreover, communicating with events is a further level of abstraction which is unnecessary in simple domains but just a giant observer pattern in others. the underlying idea is that no matter who applies a command or modifies a domain object, we already configured the event handling mechanism so that consistency across aggregates is reached according to our policy defined in the event handlers (which may be immediate consistency, or eventual one. or it may result in sending a mail to a human asking him to review the changes: whatever you want.) the only alternative to propagate changes between aggregates would be to have many collaborators passed to the various repositories, but this solution couples the aggregates with each other in many way, while with events you're forced to define one-way messages. the event generator does not make any assumption about who will listen to the event and if it will be listened to at all: events are a point of decoupling like interface are for object collaboration. and it's not that we call static methods by passing a string. we have a clear contract, a domainevents static class, and we publish interesting events (like createdcar or updatedvoyageplan) as plain old domain object which contain all the information about the update, often even composing the relevant domain object. udi dahan discourages the reference to domain objects, and consider events just special value objects ; indeed as our solution matures we are moving towards simpler objects. this choice may force us to consider just what needs to be inserted in the message instead of a full reference (where and if serialization is used to transmit the event, it's simpler to use a value object in fact). moreover, it avoids possible further accidental writing calls to the domain object originating the event. in the application layer events are published by calling a static domain class: as a result event launchers cannot be decoupled from the event (as in udi dahan's approach). we launch events from the repository after an update has been performed, either by choosing an event class directly (in case of an update or creation) or by collecting the events from a queue on the relevant domain object, usually the root of the aggregate. this was a nice idea from a colleague of mine that let us decouple at least entities from the domainevents static class. for now we do not have the requirement to decouple the handling of events from the transaction , so the application layer (which is over the domain layer) open and commits/aborts a transaction, while reconstituting an aggregate, doing some "writing" work (updating it or executing a command) and saving it. the save triggers the event launch, which may trigger work on other aggregates through the configured handler: in case of an error the whole transaction is aborted, ensuring immediate consistency. so we aren't getting the scaling advantages of deferred handling (we're not interested in that for now), but the simplicity of communicating with events while writing code. dynamic language this a php-specific section: however, domain events are an approach typical of java or .net enterprise applications. we use php classes (or interfaces) for routing the events with instanceof; php is a shared nothing environment, so event configuration is done now on a per-action basis to avoid having to create all the objects handling events on each request. however, we want to move the configuration to the application level , with some lazy-loading: for example, configuring lazy event handlers as methods on factory objects that create the real handler and return it along with the name of the method to call. all communication between aggregates happen in a single process and a single address space (for now), so we don't use a bit of the decoupling properties of events. we map value objects into the relational database either as on the parent entity's table (decomposing their fields onto the entity) or as row of their own table. in any case, we have to ensure immutability via encapsulation and only assignign to $this->anyfield into the constructor. our standard pattern is to define setters as new self($this->field1, ..., $nwfieldvalue, ..., $this->fieldn); where n is a small number of fields. we map all domain object with the hibernate-equivalent doctrine 2. we are investigating how to deal with orphaned value objects, which are not reached anymore by any other entity.

In January, Google released LevelDB, "a fast and lightweight key/value database library." In a recent post on the "High Availability MySQL" blog has generated a discussion around the possibility of LevelDB being a storage engine for MySQL due to its performance benefits. The discussion generated some insight LevelDB's comparative performance to MySQL. The LevelDB site provides some insight into these performance benefits. When creating a brand new database, various methods shows a range of speeds from .4 MB/s to 62.7 MB/s in Write performance. In Read performance, LevelDB ranged from 152 MB/s to 232 MB/s. You can see a more detailed explanation of these benchmarks by checking out the LewisDB site here. The "High Availability MySQL" blog also suggests that LevelDB may be a "great fit" for MongoDB because it does not require multi-statement transactions. Commenters pointed out a few more details about LevelDB that may limit its performance: Unfortunately, there is a trade off between number of SST files and query latency variation: the larger single storage file is - the more time will require to compact it -- Vladmir Rodionov A recent GitHub post also compared MySQL and LevelDB. For sequential insert performance, LevelDB was found to get higher throughput/lower latency overall, although MySQL was more stable. For both average latency and update performance, MySQL and LevelDB performed essentially the same. Have you had a chance to use LevelDB? How does it compare to other libraries? Please post your comments below.

Since EF was released I have been a fan. However, every once in a while I’ll run into a table design situation that I am not sure how to handle with EF. This week, I needed to setup a self-referencing table in order to store some hierarchical data. A self referencing table is a table where the primary key on the table is also defined as a foreign key. Sounds a little confusing right? Let’s clarify the solution with an example. Let’s say I am building an application where I have a list of categories and subcategories. One of my top level categories is “Programming Languages” and under programming languages I have to subcategories which are “C#” and “Java”. In order to store this data I can use a single table with the following structure: The actual data would look like this: Just to clarify, a top level category will have a null value for the ParentId field. For all child categories the ParentId field is used as to represent its parent’s primary key value. As a programmer you may want to think about the ParentId field as a pointer. To complete the example lets take a look at the SQL used to create the table. CREATE TABLE [dbo].[Categories] ( [CategoryId] [int] IDENTITY(1,1) NOT NULL, [Name] [nvarchar](255) NOT NULL, [ParentId] [int] NULL, PRIMARY KEY CLUSTERED ( [CategoryId] ASC )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] ) ON [PRIMARY] GO ALTER TABLE [dbo].[Categories] WITH CHECK ADD CONSTRAINT [Category_Parent] FOREIGN KEY([ParentId]) REFERENCES [dbo].[Categories] ([CategoryId]) GO ALTER TABLE [dbo].[Categories] CHECK CONSTRAINT [Category_Parent] GO Upon examining the SQL, you should have noticed that the CategoryId is the primary key on the table and the ParentId field is a foreign key which points back to the CategoryId field. Since we have a key referencing a another key on the same table we can classify this this as a self-referencing table. Now that we fully understand what a self-referencing table is, we can move forward to the Entity Framework code. To get started we first need to create a simple C# object to represent the Category table. Of course, keep in mind that if you are using EF Code first you do not need to create the table or database ahead of time. I only showed the table first because I wanted to better illustrate what a self referencing table is. public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } } So far the Category class is very simple. However, we really want to add a few more properties in order to make this class useful. For example, if you are a child category you really want to be able to use dot notation to get the name of the parent category (e.g. subCategory.Parent.Name). Using EF, we will create a virtual property named Parent. By making the property virtual we are letting EF know that when this property is accessed we want to load some data. Based on your configuration settings and the code you use to retrieve your data (whether or not you used DbSet.Include), EF will lazy load or eager load this data. public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } public virtual Category Parent { get; set; } } Finally, we also want a property called Children so we can use dot notation to enumerate over the child categories. Once again, here is the modified class: public class Category { public int CategoryId { get; set; } public string Name { get; set; } public int? ParentId { get; set; } public virtual Category Parent { get; set; } public virtual ICollection Children { get; set; } } The final step is to let EF know how these properties are related to one another. This can be done using EF's fluent API. If you are new to EF and are unaware of the fluent API then you may want to read this article first. public class CommodityCategoryMap : EntityTypeConfiguration { public CommodityCategoryMap() { HasKey(x => x.CategoryId); Property(x => x.CategoryId) .HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity); Property(x => x.Name) .IsRequired() .HasMaxLength(255); HasOptional(x => x.Parent) .WithMany(x => x.Children) .HasForeignKey(x => x.ParentId) .WillCascadeOnDelete(false); } } Hopefully you paid careful attention to the last section of code where we state the a Category has an optional Parent property. In database speak, this simply means that the ParentID field is nullable. The code also states that if a Category object can have zero or many children. In order to specify that a record is a child, we leverage the ParentId field to hold the primary key value of the parent record. As I mentioned earlier, if you are a programmer its easier to think of the ParentId field as a pointer. Finally, I disabled the cascade on delete option. This step is optional and probably based on your own personal preferences. If you enable cascade on delete and you delete a category that has 100 children then you will effectively remove 101 records. For whatever reason this scares me a little bit. Perhaps, my short career as a DBA caused me to not trust people with large volume delete statements. However, you may decide differently depending on your circumstances. Hopefully, this short EF tutorial will help you if you are working through a scenario where you need to capture and manipulate hierarchical data. If you have any questions please leave a comment.

XML is for dinosaurs, right? Everybody uses JSON these days. So you do, don’t you? But what about things like XSD, XSLT, JAXB, XPath, etc – is it all evil? In this article, I’d like to introduce the StAXON project (APL2) which tries to give you the best from both worlds: JSON outside, but XML inside. One benefit from this is that you can integrate JSON with powerful XML-related technologies for free. StAXON lets you read and write JSON using the Java Streaming API for XML (javax.xml.stream), also known as StAX. More specifically, StAXON provides implementations of the StAX Cursor API (XMLStreamReader and XMLStreamWriter) StAX Event API (XMLEventReader and XMLEventWriter) StAX Factory API (XMLInputFactory and XMLOutputFactory) for JSON. You may know the Jettison project, which also has XMLStreamReader and XMLStreamWriter implementations. However, StAXON aims to provide a more comprehensive and consistent solution and tries to avoid some of the issues users are having with Jettison. Anyway, let’s get started and see what this “anti-aging substance” for XML can do. Setup Add the following dependency to your Maven POM file: de.odysseus.staxon staxon 1.0 or get the latest StAXON JAR from the Downloads page and add it to your classpath. Mapping Convention The purpose of StAXON’s mapping convention is to generate a more compact JSON. It borrows the "$" syntax for text elements from the Badgerfish convention but attempts to avoid needless text-only JSON objects: Element names become object properties: <–> {"alice":null} Attributes go in properties whose name begin with "@": <–> {"alice":{"@charlie":"david"} Text-only elements go to a simple key/value property: bob <–> {"alice":"bob"} Otherwise, text content is mapped to the "$" property: bob <–> {"alice":{"@charlie":"david","$":"bob"} Nested elements go to nested properties: charlie <–> {"alice":{"bob":"charlie"} A default namespace declaration goes in the element’s "@xmlns" property: <–> {"alice":{"@xmlns":"http://foo.com"} A prefixed namespace declaration goes in the element’s "@xmlns:" property: John Doe555-1111 However, with our JSON-based writer, the output is {"customer":{"name":"John Doe","phone":"555-1111"} Reading JSON Create a JSON-based reader: String json = "{\"customer\":{\"name\":\"John Doe\",\"phone\":\"555-1111\"}"; XMLInputFactory factory = new JsonXMLInputFactory(); XMLStreamReader reader = factory.createXMLStreamReader(new StringReader(json)); Read your document: assert reader.getEventType() == XMLStreamConstants.START_DOCUMENT; reader.nextTag(); assert reader.isStartElement() && "customer".equals(reader.getLocalName()); reader.next(); assert reader.isStartElement() && "name".equals(reader.getLocalName()); reader.next(); assert reader.hasText() && "John Doe".equals(reader.getText()); reader.nextTag(); assert reader.isEndElement(); reader.next(); assert reader.isStartElement() && "phone".equals(reader.getLocalName()); reader.next(); assert reader.hasText() && "555-111".equals(reader.getText()); reader.nextTag(); assert reader.isEndElement(); reader.next(); assert reader.isEndElement(); reader.next(); assert reader.getEventType() == XMLStreamConstants.END_DOCUMENT; reader.close(); Factory Configuration The JsonXMLInputFactory and JsonXMLOutputFactory classes can be configured via the standard setProperty(String, Object) API. The factory classes define several constants for properties they support. However, the JsonXMLConfig interface provides a convenient way to hold the configuration of both - input and output - factories: JsonXMLConfig config = new JsonXMLConfigBuilder(). virtualRoot("customer"). prettyPrint(true). build(); XMLInputFactory inputFactory = new JsonXMLInputFactory(config); ... XMLOutputFactory outputFactory = new JsonXMLOutputFactory(config); ... Virtual Roots Set the virtualRoot configuration property to strip the root element from the JSON representation, e.g. { "name" : "John Doe", "phone" : "555-1111" } As XML requires a single root element, but JSON documents often don’t have one, this is an important feature required to read and write existing JSON formats. Mastering Arrays What about JSON arrays? Unfortunately, there’s nothing like this in XML. And to be honest, this causes most of the trouble when writing JSON via an XML API like StAX. Simply omitting the array boundaries would lead to non-unique JSON properties, which is usually not desired. StAXON provides several ways to deal with JSON arrays. At the core is the idea to leverage XML processing instructions to tell the writer about to start an array: the processing instruction maps a sequence of XML elements with the same name to a JSON array. The processing instruction optionally takes the array element tag name (with prefix) as data. There’s no end array hint as StAXON detects the end of an array sequence and closes it automatically. Consider the following JSON document: { "alice" : { "bob" : [ "edgar", "charlie" ], "peter" : null } } In order to get a "bob" array instead of two separate "bob" properties, we need to provide XML events corresponding to edgar charlie I.e., with the cursor API, you would just insert writer.writeProcessingInstruction(JsonXMLStreamConstants.MULTIPLE_PI_TARGET); // to start an array. Initiating Arrays with Element Paths Sometimes it is not desired or even impossible to generate processing instruction to control arrays. This may be the case if the actual writing isn’t done by your code, but some other framework like JAXB or similar, and you only provide a stream writer. Addressing such a scenario, wouldn’t it be nice being able to tell the writer beforehand, which elements should trigger a JSON array? This is where the XMLMultipleStreamWriter and XMLMultipleEventWriter wrappers step in. E.g., to specify a sequence of bob elements below root element alice as a multiple path: writer = new XMLMultipleStreamWriter(writer, true, "/alice/bob"); The boolean parameter specifies whether our paths include the root node (alice) from the paths. That is, we could also use writer = new XMLMultipleStreamWriter(writer, false, "/bob"); To wrap all bob fields into arrays (not just alice children), we can use a relative path, without a leading slash: writer = new XMLMultipleStreamWriter(writer, false, "bob"); Now we (or some legacy code, framework, …) may write our document, and the writer will take care to trigger the bob array for us. Triggering Arrays automatically Finally, if nothing else works for you, you may also let StAXON fully automatically determine array boundaries. Use this only if you cannot provide processing instructions and cannot provide the paths of the elements that should be wrapped into JSON arrays. However, using this method has several drawbacks: The writer basically needs to cache the entire document in memory, eating both space and time. The writer will not be able to produce empty arrays or arrays with a single element. To enable this feature, set the JsonXMLOutputFactory.PROP_AUTO_ARRAY property to true. Triggering Document Arrays StAXON’s writer implementation allows you to wrap a sequence of documents into a JSON array. To do this, write the PI before writing anything else: writer.writeProcessingInstruction(JsonXMLStreamConstants.MULTIPLE_PI_TARGET); writer.writeStartDocument(); // first array component ... writer.writeEndDocument(); writer.writeStartDocument(); // second array component ... writer.writeEndDocument(); ... writer.close(); The writer.close() call is crucial here, as it will close the JSON array. Using JAXB Consider a JAXB-annotated Customer class: @JsonXML(virtualRoot = true, prettyPrint = true, multiplePaths = "phone") @XmlRootElement public class Customer { public String name; public List phone; } The @JsonXML annotation is used to configure the mapping details. In the above example, the customer root element is stripped from the JSON representation, phone elements are wrapped into an array and JSON output is nicely formatted, e.g. { "name" : "John Doe", "phone" : [ "555-1111" ] } Now, the JsonXMLMapper class enables for dead-simple mapping to and from JSON: /* * Create mapper instance. */ JsonXMLMapper mapper = new JsonXMLMapper(Customer.class); /* * Read customer. */ InputStream input = getClass().getResourceAsStream("input.json"); Customer customer = mapper.readObject(input); input.close(); /* * Write back to console */ mapper.writeObject(System.out, customer); Using JAX-RS StAXON provides the staxon-jaxrs module, which enables your RESTful services to serialize/deserialize JAXB-annotated classes to/from JSON. It includes the following JAX-RS @Provider classes: de.odysseus.staxon.json.jaxrs.jaxb.JsonXMLObjectProvider is used to read and write JSON objects de.odysseus.staxon.json.jaxrs.jaxb.JsonXMLArrayProvider is used to read and write JSON arrays In order to select the StAXON message body readers/writers for your resource, a @JsonXML annotation is required. When used with JAX-RS, the @JsonXML annotation can be placed on a model type (@XmlRootElement or @XmlType) to configure its serialization and deserialization a JAX-RS resource method to configure serialization of the result type a parameter of a JAX-RS resource method to configure deserialization of the parameter type If a @JsonXML annotation is present at a model type and a resource method or parameter, the latter will override the model type annotation. If neither is present, StAXON will not handle the resource. You can find a sample project using Jersey with StAXON here. Using XPath XPath is another standard that can be easily adopted for use with JSON. The Java XPath API (javax.xml.xpath) doesn’t let us provide an XMLStreamReader or similar as a source, but requires a Document Object Model (DOM). Therefore, we need to read our JSON into a DOM first to apply expressions against that DOM. This could be done by performing an XSLT identity transformation to a DOMResult. However, StAXON provides the DOMEventConsumer class to translate XML events to DOM nodes, which should be faster and simpler than leveraging XSLT. Once we have a DOM, there’s nothing special with applying XPath expressions. StringReader json = new StringReader("{\"edgar\":\"david\",\"bob\":\"charlie\"}"); /* * Our sample JSON has no root element, so specify "alice" as virtual root */ JsonXMLConfig config = new JsonXMLConfigBuilder().virtualRoot("alice").build(); /* * create event reader */ XMLEventReader reader = new JsonXMLInputFactory(config).createXMLEventReader(json); /* * parse JSON into Document Object Model (DOM) */ Document document = DOMEventConsumer.consume(reader); /* * evaluate an XPath expression */ XPath xpath = XPathFactory.newInstance().newXPath(); System.out.println(xpath.evaluate("//alice/bob", document)); Running the above sample will print charlie to the console. What else? In the end, using an XML API to read and write JSON may still look like a compromise, but it may turn out to be a good choice. The availability of a StAX implementation for JSON acts as a door opener to powerful XML related technologies and easily enables for dual-format (XML and JSON) services. There’s more we can do with StAXON: XSD, XSLT, XQuery, XML-JSON/JSON-XML conversions, to name a few. Please check the Wiki for some of those.

Even in languages where there are no constructs but classes, there is no constraint that can force a programmer into writing object-oriented code. In many cases, just wrapping a series of functions into classes do not result in the design. The Convert Procedural Design to Objects has great benefits, but it reaches a very large scale (potentially the whole application). What does object-oriented mean? In 2011, there is no reason to write procedural code anymore in a web application: all libraries and frameworks worth inclusion are object-oriented, even a part of the PHP code (SPL but most importantly PDO, and even DateTime). All other successful languages in the web space are either object-oriented, functional, or both. Software design literature is based on objects and their patterns. However, using class and extends keywords does not suffice to produce an object-oriented design; entire books are written on this topic. This refactoring tries to solve a common case of procedural design shoehorned into an object model: classes containing behavior, and depending on many other ones. dumb classes only being a container for data, or worse primitive types with no methods at all. It is common in procedural design to segregate responsibilities in this procedure/record pattern, but high level methods can be added on these dumb classes to encapsulate a bit of the data they are containing, and simplify the procedural classes using them. It is just a starting point towards "object-orientation", but often an overlooked one. The Tell Don't Ask principle summarizes what we would like to do in very few words: Procedural code gets information then makes decisions. Object-oriented code tells objects to do things. -- Alec Sharp Instead of an infinite series of calls from a procedure to getters and setters, we want to pass messages even to the lower level objects. Steps A preliminary step is to turn primitive data structures into a data object wrapping them and providing getters. If you see variables like arrays or strings passed around in the code to refactor, this step is necessary to provide a class to accomodate potential new methods. Inline the procedural code into a single class. This step makes us able to extract code along different lines than the original ones in the rest of the refactoring: for example, procedural code is often divided in temporal steps, while objects may segregate different parts of the available data instead. Extract methods on the procedural class. See the next steps for hints on what to extract. Methods that have one of the dumb objects as argument can be moved on the object itself, by eliminating it as a parameter but maintaining the remaining ones. Move Method should free the original giant class from any unrelated responsibilities. The goal is to remove logic from the procedural class as much as possibile, going into an opposite direction with regard to the original design; Fowler notes that in some cases the procedural class totally disappears. Example One of my popular examples is invoice calculation: the computation of fields like total price and due taxes from a series of information. In this procedural design, we have one invoice and a bunch of rows modelled with Primitive Obsession (as arrays). assertEquals(4640, $invoice->total()); } } class Invoice { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $rowTotal = $row[0] + $row[0] * $row[1] / 100; $total += $rowTotal; } return $total; } } We introduce the Row class, but the design is now worse: it adds a bunch of lines of code (the new class) without the new entity giving us something in return. The Row object has no responsibilities, and we just have to write getters and sometimes setters. At least we're writing down parts of our model for documentation (giving names to the net price and tax rate numbers), but we aren't sure this model is the most versatile one. assertEquals(4640, $invoice->total()); } } class Invoice { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $rowTotal = $row->getNetPrice() + $row->getTaxRate() * $row->getNetPrice() / 100; $total += $rowTotal; } return $total; } } class Row { public function __construct($netPrice, $taxRate) { $this->netPrice = $netPrice; $this->taxRate = $taxRate; } public function getNetPrice() { return $this->netPrice; } public function getTaxRate() { return $this->taxRate; } } For the scope of this small example all business logic is already in a single class, thus we don't have to inline anything. Let's extract a first method instead: class Invoice { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $total += $this->rowTotal($row); } return $total; } public function rowTotal($row) { return $row->getNetPrice() + $row->getTaxRate() * $row->getNetPrice() / 100; } } That was a small enough step. In a real situation, the extracted code may be 100-line long, so we would want to test the extraction has been successful before doing anything else. In fact, since the test still passes, we can notice this method has a Row object in its arguments, so it can be moved on Row now that its logic has been clearly isolated: $this->field references should become additional parameters of the method before moving it. Other parameters should just remain formal parameters. Calls to $this->anotherMethod() would be more difficult to treat, as you have the options of moving anothetMethod() in the Row class too, or to extract an interface containing anotherMethod() and pass $this. While moving the code, we change the references to $row to $this, and check that the method scope is public. We also rename the method to total() instead of rowTotal(). { private $rows; public function __construct($rows) { $this->rows = $rows; } public function total() { $total = 0; foreach ($this->rows as $row) { $total += $row->total(); } return $total; } } class Row { public function __construct($netPrice, $taxRate) { $this->netPrice = $netPrice; $this->taxRate = $taxRate; } public function getNetPrice() { return $this->netPrice; } public function getTaxRate() { return $this->taxRate; } public function total() { return $this->getNetPrice() + $this->getTaxRate() * $this->getNetPrice() / 100; } } Finally, we inline the getters, since they're not used from outside the Row class. They will be introduced again in the future in case there is a real need for them: as a rule of thumb we avoid exposing any state from Row that is not necessary. class Row { public function __construct($netPrice, $taxRate) { $this->netPrice = $netPrice; $this->taxRate = $taxRate; } public function total() { return $this->netPrice + $this->taxRate * $this->netPrice / 100; } }

Previously, we covered some tips and tricks for the Grid control. In this article, we will go over a few tips and tricks for the wx.ListCtrl widget when it’s in “report” mode. Take a look at the tips below: How to create a simple ListCtrl How to sort the rows of a ListCtrl How to make the ListCtrl cells editable in place Associating objects with ListCtrl rows Alternate the row colors of a ListCtrl How to create a simple ListCtrl The list control is a pretty common widget. In Windows, you will see the list control in Windows Explorer. It has four modes: icon, small icon, list, and report. They roughly match up with icons, tiles, list, and details views in Windows Explorer respectively. We’re going to focus on the ListCtrl in Report mode because that’s the mode that most developers use it in. Here’s a simple example of how to create a list control: import wx ######################################################################## class MyForm(wx.Frame): #---------------------------------------------------------------------- def __init__(self): wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") # Add a panel so it looks the correct on all platforms panel = wx.Panel(self, wx.ID_ANY) self.index = 0 self.list_ctrl = wx.ListCtrl(panel, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.InsertColumn(0, 'Subject') self.list_ctrl.InsertColumn(1, 'Due') self.list_ctrl.InsertColumn(2, 'Location', width=125) btn = wx.Button(panel, label="Add Line") btn.Bind(wx.EVT_BUTTON, self.add_line) sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) sizer.Add(btn, 0, wx.ALL|wx.CENTER, 5) panel.SetSizer(sizer) #---------------------------------------------------------------------- def add_line(self, event): line = "Line %s" % self.index self.list_ctrl.InsertStringItem(self.index, line) self.list_ctrl.SetStringItem(self.index, 1, "01/19/2010") self.list_ctrl.SetStringItem(self.index, 2, "USA") self.index += 1 #---------------------------------------------------------------------- # Run the program if __name__ == "__main__": app = wx.App(False) frame = MyForm() frame.Show() app.MainLoop() As you can probably tell from the code above, it’s really easy to create a ListCtrl instance. Notice that we set the style to report mode using the wx.LC_REPORT flag. To add column headers, we call the ListCtrl’s InsertColumn method and pass an integer to tell the ListCtrl which column is which and a string for the user’s convenience. Yes, the columns are zero-based, so the first column is number zero, the second column is number one, etc. The next important piece is contained in the button’s event handler, add_line, where we learn how to add rows of data to the ListCtrl. The typical method to use is the InsertStringItem method. If you wanted an image added to each row as well, then you’d use a more complicated method like InsertColumnInfo along with the InsertImageStringItem method. You can see how to use them in the wxPython demo. We’re sticking with the easy stuff in this article. Anyway, when you call InsertStringItem you give it the correct row index and a string. You use the SetStringItem method to set the data for the other columns of the row. Notice that the SetStringItem method requires three parameters: the row index, the column index and a string. Lastly, we increment the row index so we don’t overwrite anything. Now you can get out there and make your own! Let’s continue and find out how to sort rows! How to sort the rows of a ListCtrl The ListCtrl widget has had some extra scripts written for it that add functionality to the widget. These scripts are called mixins. You can read about them here. For this recipe, we’ll be using the ColumnSorterMixin mixin. The code below is a stripped down version of one of the wxPython demo examples. import wx import wx.lib.mixins.listctrl as listmix musicdata = { 0 : ("Bad English", "The Price Of Love", "Rock"), 1 : ("DNA featuring Suzanne Vega", "Tom's Diner", "Rock"), 2 : ("George Michael", "Praying For Time", "Rock"), 3 : ("Gloria Estefan", "Here We Are", "Rock"), 4 : ("Linda Ronstadt", "Don't Know Much", "Rock"), 5 : ("Michael Bolton", "How Am I Supposed To Live Without You", "Blues"), 6 : ("Paul Young", "Oh Girl", "Rock"), } ######################################################################## class TestListCtrl(wx.ListCtrl): #---------------------------------------------------------------------- def __init__(self, parent, ID=wx.ID_ANY, pos=wx.DefaultPosition, size=wx.DefaultSize, style=0): wx.ListCtrl.__init__(self, parent, ID, pos, size, style) ######################################################################## class TestListCtrlPanel(wx.Panel, listmix.ColumnSorterMixin): #---------------------------------------------------------------------- def __init__(self, parent): wx.Panel.__init__(self, parent, -1, style=wx.WANTS_CHARS) self.index = 0 self.list_ctrl = TestListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN |wx.LC_SORT_ASCENDING ) self.list_ctrl.InsertColumn(0, "Artist") self.list_ctrl.InsertColumn(1, "Title", wx.LIST_FORMAT_RIGHT) self.list_ctrl.InsertColumn(2, "Genre") items = musicdata.items() index = 0 for key, data in items: self.list_ctrl.InsertStringItem(index, data[0]) self.list_ctrl.SetStringItem(index, 1, data[1]) self.list_ctrl.SetStringItem(index, 2, data[2]) self.list_ctrl.SetItemData(index, key) index += 1 # Now that the list exists we can init the other base class, # see wx/lib/mixins/listctrl.py self.itemDataMap = musicdata listmix.ColumnSorterMixin.__init__(self, 3) self.Bind(wx.EVT_LIST_COL_CLICK, self.OnColClick, self.list_ctrl) sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- # Used by the ColumnSorterMixin, see wx/lib/mixins/listctrl.py def GetListCtrl(self): return self.list_ctrl #---------------------------------------------------------------------- def OnColClick(self, event): print "column clicked" event.Skip() ######################################################################## class MyForm(wx.Frame): #---------------------------------------------------------------------- def __init__(self): wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") # Add a panel so it looks the correct on all platforms panel = TestListCtrlPanel(self) #---------------------------------------------------------------------- # Run the program if __name__ == "__main__": app = wx.App(False) frame = MyForm() frame.Show() app.MainLoop() This code is a little on the odd side in that we have inherit the mixin in the wx.Panel based class rather than the wx.ListCtrl class. You can do it either way though as long as you rearrange the code correctly. Anyway, we are going to home in on the key differences between this example and the previous one. The first difference of major importance is in the looping construct where we insert the list control’s data. Here we include the list control’s SetItemData method to include the necessary inner-workings that allow the sorting to take place. As you might have guessed, this method associates the row index with the music data dict’s key. Next we instantiate the ColumnSorterMixin and tell it how many columns there are in the list control. We could have left the EVT_LIST_COL_CLICK binding off this example as it has nothing to do with the actual sorting of the rows, but in the interest of increasing your knowledge, it was left in. All it does is show you how to catch the user’s column click event. The rest of the code is self-explanatory. If you want to know about the requirements for this mixin, especially when you have images in your rows, please see the relevant section in the source (i.e. listctrl.py). Now, wasn’t that easy? Let’s continue our journey and find out how to make the cells editable! How to make the ListCtrl cells editable in place Sometimes, the programmer will want to allow the user to click on a cell and edit it in place. This is kind of a lightweight version of the wx.grid.Grid control. Here’s an example: import wx import wx.lib.mixins.listctrl as listmix ######################################################################## class EditableListCtrl(wx.ListCtrl, listmix.TextEditMixin): ''' TextEditMixin allows any column to be edited. ''' #---------------------------------------------------------------------- def __init__(self, parent, ID=wx.ID_ANY, pos=wx.DefaultPosition, size=wx.DefaultSize, style=0): """Constructor""" wx.ListCtrl.__init__(self, parent, ID, pos, size, style) listmix.TextEditMixin.__init__(self) ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = EditableListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "Editable List Control") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() In this script, we put the TextEditMixin in our wx.ListCtrl class instead of our wx.Panel, which is the opposite of the previous example. The mixin itself does all the heavy lifting. Again, you’ll have to check out the mixin’s source to really understand how it works. Associating objects with ListCtrl rows This subject comes up a lot: How do I associate data (i.e. objects) with my ListCtrl’s rows? Well, we’re going to find out exactly how to do that with the following code: import wx ######################################################################## class Car(object): """""" #---------------------------------------------------------------------- def __init__(self, make, model, year, color="Blue"): """Constructor""" self.make = make self.model = model self.year = year self.color = color ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [Car("Ford", "Taurus", "1996"), Car("Nissan", "370Z", "2010"), Car("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.Bind(wx.EVT_LIST_ITEM_SELECTED, self.onItemSelected) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 self.myRowDict = {} for row in rows: self.list_ctrl.InsertStringItem(index, row.make) self.list_ctrl.SetStringItem(index, 1, row.model) self.list_ctrl.SetStringItem(index, 2, row.year) self.list_ctrl.SetStringItem(index, 3, row.color) self.myRowDict[index] = row index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onItemSelected(self, event): """""" currentItem = event.m_itemIndex car = self.myRowDict[currentItem] print car.make print car.model print car.color print car.year ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() The list control widget actually doesn’t have a built-in way to accomplish this feat. If you want that, then you’ll want to check out the ObjectListView widget, which wraps the ListCtrl and gives it a lot more functionality. In the meantime, we’ll take a minute and go over the code above. The first piece is just a plain Car class with four attributes. Then in the MyPanel class, we create a list of Car objects that we’ll use for the ListCtrl’s data. To add the data to the ListCtrl, we use a for loop to iterate over the list. We also associate each row with a Car object using a Python dictionary. We use the row’s index for the key and the dict’s value ends up being the Car object. This allows us to access all the Car/row object’s data later on in the onItemSelected method. Let’s check that out! In onItemSelected, we grab the row’s index with the following little trick: event.m_itemIndex. Then we use that value as the key for our dictionary so that we can gain access to the Car object associated with that row. At this point, we just print out all the Car object’s attributes, but you could do whatever you want here. This basic idea could easily be extended to use a result set from a SqlAlchemy query for the ListCtrl’s data. Hopefully you get the general idea. Now if you were paying close attention, like Robin Dunn (creator of wxPython) was, then you might notice some really silly logic errors in this code. Did you find them? Well, you won’t see it unless you sort the rows, delete a row or insert a row. Do you see it now? Yes, I stupidly based the “unique” key in my dictionary on the row’s position, which will change if any of those events happen. So let’s look at a better example: import wx ######################################################################## class Car(object): """""" #---------------------------------------------------------------------- def __init__(self, make, model, year, color="Blue"): """Constructor""" self.id = id(self) self.make = make self.model = model self.year = year self.color = color ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [Car("Ford", "Taurus", "1996"), Car("Nissan", "370Z", "2010"), Car("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, size=(-1,100), style=wx.LC_REPORT |wx.BORDER_SUNKEN ) self.list_ctrl.Bind(wx.EVT_LIST_ITEM_SELECTED, self.onItemSelected) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 self.myRowDict = {} for row in rows: self.list_ctrl.InsertStringItem(index, row.make) self.list_ctrl.SetStringItem(index, 1, row.model) self.list_ctrl.SetStringItem(index, 2, row.year) self.list_ctrl.SetStringItem(index, 3, row.color) self.list_ctrl.SetItemData(index, row.id) self.myRowDict[row.id] = row index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) #---------------------------------------------------------------------- def onItemSelected(self, event): """""" currentItem = event.m_itemIndex car = self.myRowDict[self.list_ctrl.GetItemData(currentItem)] print car.make print car.model print car.color print car.year ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control Tutorial") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() In this example, we add a new attribute to our Car class that creates a unique id for each instance that is created using Python’s handy id builtin. Then in the loop where we add the data to the list control, we call the widget’s SetItemData method and give it the row index and the car instance’s unique id. Now it doesn’t matter where the row ends up because it’s had the unique id affixed to it. Finally, we have to modify the onItemSelected to get the right object. The magic happens in this code: # this code was helpfully provided by Robin Dunn car = self.myRowDict[self.list_ctrl.GetItemData(currentItem)] Cool, huh? Our last example will cover how to alternate the row colors, so let’s take a look! Alternate the row colors of a ListCtrl As this section’s title suggests, we will look at how to alternate colors of the rows of a ListCtrl. Here’s the code: import wx import wx.lib.mixins.listctrl as listmix ######################################################################## class MyPanel(wx.Panel): """""" #---------------------------------------------------------------------- def __init__(self, parent): """Constructor""" wx.Panel.__init__(self, parent) rows = [("Ford", "Taurus", "1996", "Blue"), ("Nissan", "370Z", "2010", "Green"), ("Porche", "911", "2009", "Red") ] self.list_ctrl = wx.ListCtrl(self, style=wx.LC_REPORT) self.list_ctrl.InsertColumn(0, "Make") self.list_ctrl.InsertColumn(1, "Model") self.list_ctrl.InsertColumn(2, "Year") self.list_ctrl.InsertColumn(3, "Color") index = 0 for row in rows: self.list_ctrl.InsertStringItem(index, row[0]) self.list_ctrl.SetStringItem(index, 1, row[1]) self.list_ctrl.SetStringItem(index, 2, row[2]) self.list_ctrl.SetStringItem(index, 3, row[3]) if index % 2: self.list_ctrl.SetItemBackgroundColour(index, "white") else: self.list_ctrl.SetItemBackgroundColour(index, "yellow") index += 1 sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(self.list_ctrl, 0, wx.ALL|wx.EXPAND, 5) self.SetSizer(sizer) ######################################################################## class MyFrame(wx.Frame): """""" #---------------------------------------------------------------------- def __init__(self): """Constructor""" wx.Frame.__init__(self, None, wx.ID_ANY, "List Control w/ Alternate Colors") panel = MyPanel(self) self.Show() #---------------------------------------------------------------------- if __name__ == "__main__": app = wx.App(False) frame = MyFrame() app.MainLoop() The code above will alternate each row’s background color. Thus you should see yellow and white rows. We do this by calling the ListCtrl instance’s SetItemBackgroundColour method. If you were using a virtual list control, then you’d want to override the OnGetItemAttr method. To see an example of the latter method, open up your copy of the wxPython demo; there’s one in there. Wrapping Up We’ve covered a lot of ground here. You should now be able to do a lot more with your wx.ListCtrl than when you started, assuming you’re new to using it, of course. Feel free to ask questions in the comments or suggest future recipes. I hope you found this helpful! Note: All examples were tested on Windows XP with Python 2.5 and wxPython 2.8.10.1. They were also tested on Windows 7 Professional with Python 2.6 Additional Reading The official wxPython wx.ListCtrl documentation The ListControls wiki page ListCtrl Tooltips wiki page The ObjectListView website The UltimateListCtrl, a pure Python implementation now included with wxPython Source Code listctrl.zip listctrl.tar Source: http://www.blog.pythonlibrary.org/2011/01/04/wxpython-wx-listctrl-tips-and-tricks/

I have been inserting log4j entries into a mongodb database and each entry has been given an ISODate timestamp: "timestamp" : ISODate("2012-01-17T22:30:19.839Z") To create a mapping for this, I had to manually add the timestamp as Spring Roo did not allow timestamp to be used as it was a reserved word. So I manually added: @DateTimeFormat(style="MM/dd/yyyy") private java.util.Date timestamp; But I started getting the following error: Invalid style specification: MM/dd/yyyy The stack trace for that error was: org.joda.time.format.DateTimeFormat.createFormatterForStyle(DateTimeFormat.java:702) org.joda.time.format.DateTimeFormat.patternForStyle(DateTimeFormat.java:212) com.comcast.uivr.web.LoggingController_Roo_Controller.ajc$interMethod$com_comcast_uivr_web_LoggingController_Roo_Controller$com_comcast_uivr_web_LoggingController$addDateTimeFormatPatterns(LoggingController_Roo_Controller.aj:98) com.comcast.uivr.web.LoggingController.ajc$interMethodDispatch2$com_comcast_uivr_web$addDateTimeFormatPatterns(LoggingController.java:1) com.comcast.uivr.web.LoggingController_Roo_Controller.ajc$interMethodDispatch1$com_comcast_uivr_web_LoggingController_Roo_Controller$com_comcast_uivr_web_LoggingController$addDateTimeFormatPatterns(LoggingController_Roo_Controller.aj) com.comcast.uivr.web.LoggingController_Roo_Controller.ajc$interMethod$com_comcast_uivr_web_LoggingController_Roo_Controller$com_comcast_uivr_web_LoggingController$list(LoggingController_Roo_Controller.aj:66) com.comcast.uivr.web.LoggingController.list(LoggingController.java:1) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) java.lang.reflect.Method.invoke(Method.java:597) org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:212) org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:900) org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:827) org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882) org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778) javax.servlet.http.HttpServlet.service(HttpServlet.java:617) javax.servlet.http.HttpServlet.service(HttpServlet.java:717) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) java.lang.Thread.run(Thread.java:662) To fix this I attempted to add the ISO date format for the @DateTimeFormat @DateTimeFormat(style="yyyyMMdd'T'HHmmss.SSSZ") private java.util.Date timestamp; Which still did not work and had the error. To resolve this I shitched to use ISO.DATE_TIME as the style: @DateTimeFormat(iso=ISO.DATE_TIME) private java.util.Date timestamp; From http://www.baselogic.com/blog/development/springframework/mapping-mongodb-isodate-spring-roo-entity/

Not too long ago I tweeted what I felt was a small triumph on my latest project, streaming files from MongoDB GridFS for downloads (rather than pulling the whole file into memory and then serving it up). I promised to blog about this but unfortunately my specific usage was a little coupled to the domain on my project so I couldn’t just show it off as is. So I’ve put together an example node.js+GridFS application and shared it on github and will use this post to explain how I accomplished it. GridFS Module First off, special props go to tjholowaychuk who responded in the #node.js irc channel when I asked if anyone has had luck with using GridFS from mongoose. A lot of my resulting code is derived from an gist he shared with me. Anyway, to the code. I’ll describe how I’m using gridfs and after setting the ground work illustrate how simple it is to stream files from GridFS. I created a gridfs module that basically accesses GridStore through mongoose (which I use throughout my application) that can also share the db connection created when connecting mongoose to the mongodb server. mongoose = require "mongoose" request = require "request" GridStore = mongoose.mongo.GridStore Grid = mongoose.mongo.Grid ObjectID = mongoose.mongo.BSONPure.ObjectID We can’t get files from mongodb if we cannot put anything into it, so let’s create a putFile operation. exports.putFile = (path, name, options..., fn) -> db = mongoose.connection.db options = parse(options) options.metadata.filename = name new GridStore(db, name, "w", options).open (err, file) -> return fn(err) if err file.writeFile path, fn parse = (options) -> opts = {} if options.length > 0 opts = options[0] if !opts.metadata opts.metadata = {} opts This really just delegates to the putFile operation that exists in GridStore as part of the mongodb module. I also have a little logic in place to parse options, providing defaults if none were provided. One interesting feature to note is that I store the filename in the metadata because at the time I ran into a funny issue where files retrieved from gridFS had the id as the filename (even though a look in mongo reveals that the filename is in fact in the database). Now the get operation. The original implementation of this simply passed the contents as a buffer to the provided callback by calling store.readBuffer(), but this is now changed to pass the resulting store object to the callback. The value in this is that the caller can use the store object to access metadata, contentType, and other details. The user can also determine how they want to read the file (either into memory or using a ReadableStream). exports.get = (id, fn) -> db = mongoose.connection.db id = new ObjectID(id) store = new GridStore(db, id, "r", root: "fs" ) store.open (err, store) -> return fn(err) if err # band-aid if "#{store.filename}" == "#{store.fileId}" and store.metadata and store.metadata.filename store.filename = store.metadata.filename fn null, store This code just has a small blight in that it checks to see if the filename and fileId are equal. If they are, it then checks to see if metadata.filename is set and sets store.filename to the value found there. I’ve tabled the issue to investigate further later. The Model In my specific instance, I wanted to attach files to a model. In this example, let’s pretend that we have an Application for something (job, a loan application, etc) that we can attach any number of files to. Think of tax receipts, a completed application, other scanned documents. ApplicationSchema = new mongoose.Schema( name: String files: [ mongoose.Schema.Mixed ] ) ApplicationSchema.methods.addFile = (file, options, fn) -> gridfs.putFile file.path, file.filename, options, (err, result) => @files.push result @save fn Here I define files as an array of Mixed object types (meaning they can be anything) and a method addFile which basically takes an object that at least contains a path and filename attribute. It uses this to save the file to gridfs and stores the resulting gridstore file object in the files array (this contains stuff like an id, uploadDate, contentType, name, size, etc). Handling Requests This all plugs in to the request handler to handle form submissions to /new. All this entails is creating an Application model instance, adding the uploaded file from the request (in this case we named the file field “file”, hence req.files.file) and saving it. app.post "/new", (req, res) -> application = new Application() application.name = req.body.name opts = content_type: req.files.file.type application.addFile req.files.file, opts, (err, result) -> res.redirect "/" Now the sum of all this work allows us to reap the rewards by making it super simple to download a requested file from gridFS. app.get "/file/:id", (req, res) -> gridfs.get req.params.id, (err, file) -> res.header "Content-Type", file.type res.header "Content-Disposition", "attachment; filename=#{file.filename}" file.stream(true).pipe(res) Here we simply look up a file by id and use the resulting file object to set Content-Type and Content-Disposition fields and finally make use of ReadableStream::pipe to write the file out to the response object (which is an instance of WritableStream). This is the piece of magic that streams data from MongoDB to the client side. Ideas This is just a humble beginning. Other ideas include completely encapsulating gridfs within the model. Taking things further we could even turn the gridfs model into a mongoose plugin to allow completely blackboxed usage of gridfs. Feel free to check the project out and let me know if you have ideas to take it even further. Fork away! Source: http://blog.james-carr.org/2012/01/09/streaming-files-from-mongodb-gridfs/

This is the forth of a series of articles about Persistence with Spring. This article will focus on the configuration and implementation of the persistence layer with Spring 3.1, JPA and Spring Data. For a step by step introduction about setting up the Spring context using Java based configuration and the basic Maven pom for the project, see this article. The Persistence with Spring series: Part 1 – The Persistence Layer with Spring 3.1 and Hibernate Part 3 – The Persistence Layer with Spring 3.1 and JPA Part 5 – Transaction configuration with JPA and Spring 3.1 No More DAO implementations As I discussed in a previous post, the DAO layer usually consists of a lot of boilerplate code that can and should be simplified. The advantages of such a simplification are many fold: a decrease in the number of artifacts that need to be defined and maintained, simplification and consistency of data access patterns and consistency of configuration. Spring Data takes this simplification one step forward and makes it possible to remove the DAO implementations entirely – the interface of the DAO is now the only artifact that need to be explicitly defined. The Spring Data managed DAO In order to start leveraging the Spring Data programming model with JPA, a DAO interface needs to extend the JPA specific Repository interface - JpaRepository – in Spring’s interface hierarchy. This will enable Spring Data to find this interface and automatically create an implementation for it. Also, by extending the interface we get most if not all relevant CRUD generic methods for standard data access available in the DAO. Defining custom access method and queries As discussed, by implementing one of the Repository interfaces, the DAO will already have some basic CRUD methods (and queries) defined and implemented. To define more specific access methods, Spring JPA supports quite a few options – you can either simply define a new method in the interface, or you can provide the actual JPQ query by using the @Query annotation. A third option to define custom queries is to make use of JPA Named Queries, but this has the disadvantage that it either involves XML or burdening the domain class with the queries. In addition to these, Spring Data introduces a more flexible and convenient API, similar to the JPA Criteria API, only more readable and reusable. The advantages of this API will become more pronounced when dealing with a large number of fixed queries that could potentially be more concisely expressed through a smaller number of reusable blocks that keep occurring in different combinations. Automatic Custom Queries When Spring Data creates a new Repository implementation, it analyzes all the methods defined by the interfaces and tries to automatically generate queries from the method name. While this has limitations, it is a very powerful and elegant way of defining new custom access methods with very little effort. For example, if the managed entity has a name field (and the Java Bean standard getter and setter for that field), defining the findByName method in the DAO interface will automatically generate the correct query: public interface IFooDAO extends JpaRepository< Foo, Long >{ Foo findByName( final String name ); } This is a relatively simple example; a much larger set of keywords is supported by query creation mechanism. In the case that the parser cannot match the property with the domain object field, the following exception is thrown: java.lang.IllegalArgumentException: No property nam found for type class org.rest.model.Foo Manual Custom Queries In addition to deriving the query from the method name, a custom query can be manually specified with the method level @Query annotation. For even more fine grained control over the creation of queries, such as using named parameters or modifying existing queries, the reference is a good place to start. Spring Data transaction configuration The actual implementation of the Spring Data managed DAO – SimpleJpaRepository – uses annotations to define and configure transactions. A read only @Transactional annotation is used at the class level, which is then overridden for the non read-only methods. The rest of the transaction semantics are default, but these can be easily overridden manually per method. Exception Translation without the template One of the responsibilities of Spring ORM templates (JpaTemplate, HibernateTemplate) is exception translation – translating JPA exceptions – which tie the API to JPA – to Spring’s DataAccessException hierarchy. Without the template to do that, exception translation can still be enabled by annotating the DAOs with the @Repository annotation. That, coupled with a Spring bean postprocessor will advice all @Repository beans with all the implementations of PersistenceExceptionTranslator found in the Container – to provide exception translation without using the template. The fact that exception translation is indeed active can easily be verified with an integration test: @Test( expected = DataAccessException.class ) public void whenAUniqueConstraintIsBroken_thenSpringSpecificExceptionIsThrown(){ String name = "randomName"; this.service.save( new Foo( name ) ); this.service.save( new Foo( name ) ); } Exception translation is done through proxies; in order for Spring to be able to create proxies around the DAO classes, these must not be declared final. Spring Data Configuration To activate the Spring JPA repository support, the jpa namespace is defined and used to specify the package where to DAO interfaces are located: At this point, there is no equivalent Java based configuration – support for it is however in the works. The Spring Java or XML configuration The JPA configuration with Spring 3.1 has already been carefully discussed in the previous article of this series. Spring Data also takes advantage of the Spring support for the JPA @PersistenceContext annotation which it uses to wire the EntityManager into the Spring factory bean responsible with creating the actual DAO implementations – JpaRepositoryFactoryBean. In addition to the already discussed configuration, there is one last missing piece – including the Spring Data XML configuration in the overall persistence configuration: @Configuration @EnableTransactionManagement @ImportResource( "classpath*:*springDataConfig.xml" ) public class PersistenceJPAConfig{ ... } The Maven configuration In addition to the Maven configuration for JPA defined in a previous article, the spring-data-jpa dependency is addeed: org.springframework.data spring-data-jpa 1.0.2.RELEASE Conclusion This article covered the configuration and implementation of the persistence layer with Spring 3.1, JPA 2 and Spring JPA (part of the Spring Data umbrella project), using both XML and Java based configuration. The various method of defining more advanced custom queries are discussed, as well as configuration with the new jpa namespace and transactional semantics. The final result is a new and elegant take on data access with Spring, with almost no actual implementation work. You can check out the full implementation in the github project. From the originalThe Persistence Layer with Spring Data JPA of the Persistence with Spring series

in the graph database space, there are two types of traversal engines: local and distributed. local traversal engines are typically for single-machine graph databases and are used for real-time production applications. distributed traversal engines are typically for multi-machine graph databases and are used for batch processing applications. this divide is quite sharp in the community, but there is nothing that prevents the unification of both models. a discussion of this divide and its unification is presented in this post. local traversal engines in a local traversal engine, there is typically a single processing agent that obeys a program. the agent is called a traverser and the program it follows is called a path description . in gremlin , a friend-of-a-friend path description is defined as such: g.v(1).oute('friend').inv.oute('friend').inv when this path description is interpreted by a traverser over a graph, an instance of the description is realized as actual paths in the graph that match that description. for example, the gremlin traverser starts at vertex 1 and then steps to its outgoing friend -edges. next, it will move to the head/target vertices of those edges (i.e. vertices 3 and 4). after that, it will go to the friend -edges of those vertices and finally, to the head of the previous edges (i.e. vertices 6 and 7). in this way, a single traverser is following all the paths that are exposed with each new atomic graph operation (i.e. each new step after a .). the abstract syntax being: step.step.step . what is returned by this friend-of-a-friend path description, on the example graph diagrammed, is vertices 6 and 7. in many situations, its not the end of the path that is desired, but some side-effect of the traversal. for example, as the traverser walks it can update some global data structure such as a ranking of the vertices. this idea is presented in the path description below, where the oute.inv path is looped over 1000 times. each time a vertex is traversed over, the map m is updated. this global map m maintains keys that are vertices and values that denote the number of times that each vertex has been touched ( groupcount ‘s behavior). m = [:] g.v(1).oute.inv.groupcount(m).loop(3){it.loops < 1000} the local traversal engine pattern is abstractly diagrammed on the right, where a single traverser is obeying some path description ( a.b.c ) and in doing so, moving around on a graph and updating a global data structure (the red boxed map). given the need for traversers to move from element to element, graph databases of this form tend to support strong data locality by means of a direct-reference graph data structure (i.e. vertices have pointers to edges and edges to vertices). a few examples of such graph databases include neo4j , orientdb , and dex . distributed traversal engines in a distributed traversal engine, a traversal is represented as a flow of messages between the elements of the graph. generally, each element (e.g. vertex) is operating independently of the other elements. each element is seen as its own processor with its own (usually homogenous) program to execute. elements communicate with each other via message passing . when no more messages have been passed, the traversal is complete and the results of the traversal are typically represented as a distributed data structure over the elements. graph databases of this nature tend to use the bulk synchronous parallel model of distributed computing. each step is synchronized in a manner analogous to a clock cycle in hardware. instances of this model include agrapa , pregel , trinity , and goldenorb . an example of distributed graph traversing is now presented using a ranking algorithm in java. [ note : in this example, edges are not first class citizens. this is typical of the state of the art in distributed traversal engines. they tend to be for single-relational, unlabeled-edge graphs.] public void evaluatestep(int step) { if(!this.inbox.isempty() && step < 1000) { this.rank = this.rank + this.inbox.size(); for(vertex vertex : this.adjacentvertices()) { for(int i=0; i

Learn the differences between GET and POST requests.

Lucene's 3.4.0 release adds a new feature called index-time join (also sometimes called sub-documents, nested documents or parent/child documents), enabling efficient indexing and searching of certain types of relational content. Most search engines can't directly index relational content, as documents in the index logically behave like a single flat database table. Yet, relational content is everywhere! A job listing site has each company joined to the specific listings for that company. Each resume might have separate list of skills, education and past work experience. A music search engine has an artist/band joined to albums and then joined to songs. A source code search engine would have projects joined to modules and then files. Perhaps the PDF documents you need to search are immense, so you break them up and index each section as a separate Lucene document; in this case you'll have common fields (title, abstract, author, date published, etc.) for the overall document, joined to the sub-document (section) with its own fields (text, page number, etc.). XML documents typically contain nested tags, representing joined sub-documents; emails have attachments; office documents can embed other documents. Nearly all search domains have some form of relational content, often requiring more than one join. If such content is so common then how do search applications handle it today? One obvious "solution" is to simply use a relational database instead of a search engine! If relevance scores are less important and you need to do substantial joining, grouping, sorting, etc., then using a database could be best overall. Most databases include some form a text search, some even using Lucene. If you still want to use a search engine, then one common approach is to denormalize the content up front, at index-time, by joining all tables and indexing the resulting rows, duplicating content in the process. For example, you'd index each song as a Lucene document, copying over all fields from the song's joined album and artist/band. This works correctly, but can be horribly wasteful as you are indexing identical fields, possibly including large text fields, over and over. Another approach is to do the join yourself, outside of Lucene, by indexing songs, albums and artist/band as separate Lucene documents, perhaps even in separate indices. At search-time, you first run a query against one collection, for example the songs. Then you iterate through all hits, gathering up (joining) the full set of corresponding albums and then run a second query against the albums, with a large OR'd list of the albums from the first query, repeating this process if you need to join to artist/band as well. This approach will also work, but doesn't scale well as you may have to create possibly immense follow-on queries. Yet another approach is to use a software package that has already implemented one of these approaches for you! elasticsearch, Apache Solr, Apache Jackrabbit, Hibernate Search and many others all handle relational content in some way. With BlockJoinQuery you can now directly search relational content yourself! Let's work through a simple example: imagine you sell shirts online. Each shirt has certain common fields such as name, description, fabric, price, etc. For each shirt you have a number of separate stock keeping units or SKUs, which have their own fields like size, color, inventory count, etc. The SKUs are what you actually sell, and what you must stock, because when someone buys a shirt they buy a specific SKU (size and color). Maybe you are lucky enough to sell the incredible Mountain Three-wolf Moon Short Sleeve Tee, with these SKUs (size, color): small, blue small, black medium, black large, gray Perhaps a user first searches for "wolf shirt", gets a bunch of hits, and then drills down on a particular size and color, resulting in this query: name:wolf AND size=small AND color=blue which should match this shirt. name is a shirt field while the size and color are SKU fields. But if the user drills down instead on a small gray shirt: name:wolf AND size=small AND color=gray then this shirt should not match because the small size only comes in blue and black. How can you run these queries using BlockJoinQuery? Start by indexing each shirt (parent) and all of its SKUs (children) as separate documents, using the new IndexWriter.addDocuments API to add one shirt and all of its SKUs as a single document block. This method atomically adds a block of documents into a single segment as adjacent document IDs, which BlockJoinQuery relies on. You should also add a marker field to each shirt document (e.g. type = shirt), as BlockJoinQuery requires a Filter identifying the parent documents. To run a BlockJoinQuery at search-time, you'll first need to create the parent filter, matching only shirts. Note that the filter must use FixedBitSet under the hood, like CachingWrapperFilter: Filter shirts = new CachingWrapperFilter( new QueryWrapperFilter( new TermQuery( new Term("type", "shirt")))); Create this filter once, up front and re-use it any time you need to perform this join. Then, for each query that requires a join, because it involves both SKU and shirt fields, start with the child query matching only SKU fields: BooleanQuery skuQuery = new BooleanQuery(); skuQuery.add(new TermQuery(new Term("size", "small")), Occur.MUST); skuQuery.add(new TermQuery(new Term("color", "blue")), Occur.MUST); Next, use BlockJoinQuery to translate hits from the SKU document space up to the shirt document space: BlockJoinQuery skuJoinQuery = new BlockJoinQuery( skuQuery, shirts, ScoreMode.None); The ScoreMode enum decides how scores for multiple SKU hits should be aggregated to the score for the corresponding shirt hit. In this query you don't need scores from the SKU matches, but if you did you can aggregate with Avg, Max or Total instead. Finally you are now free to build up an arbitrary shirt query using skuJoinQuery as a clause: BooleanQuery query = new BooleanQuery(); query.add(new TermQuery(new Term("name", "wolf")), Occur.MUST); query.add(skuJoinQuery, Occur.MUST); You could also just run skuJoinQuery as-is if the query doesn't have any shirt fields. Finally, just run this query like normal! The returned hits will be only shirt documents; if you'd also like to see which SKUs matched for each shirt, use BlockJoinCollector: BlockJoinCollector c = new BlockJoinCollector( Sort.RELEVANCE, // sort 10, // numHits true, // trackScores false // trackMaxScore ); searcher.search(query, c); The provided Sort must use only shirt fields (you cannot sort by any SKU fields). When each hit (a shirt) is competitive, this collector will also record all SKUs that matched for that shirt, which you can retrieve like this: TopGroups hits = c.getTopGroups( skuJoinQuery, skuSort, 0, // offset 10, // maxDocsPerGroup 0, // withinGroupOffset true // fillSortFields ); Set skuSort to the sort order for the SKUs within each shirt. The first offset hits are skipped (use this for paging through shirt hits). Under each shirt, at most maxDocsPerGroup SKUs will be returned. Use withinGroupOffset if you want to page within the SKUs. If fillSortFields is true then each SKU hit will have values for the fields from skuSort. The hits returned by BlockJoinCollector.getTopGroups are SKU hits, grouped by shirt. You'd get the exact same results if you had denormalized up-front and then used grouping to group results by shirt. You can also do more than one join in a single query; the joins can be nested (parent to child to grandchild) or parallel (parent to child1 and parent to child2). However, there are some important limitations of index-time joins: The join must be computed at index-time and "compiled" into the index, in that all joined child documents must be indexed along with the parent document, as a single document block. Different document types (for example, shirts and SKUs) must share a single index, which is wasteful as it means non-sparse data structures like FieldCache entries consume more memory than they would if you had separate indices. If you need to re-index a parent document or any of its child documents, or delete or add a child, then the entire block must be re-indexed. This is a big problem in some cases, for example if you index "user reviews" as child documents then whenever a user adds a review you'll have to re-index that shirt as well as all its SKUs and user reviews. There is no QueryParser support, so you need to programmatically create the parent and child queries, separating according to parent and child fields. The join can currently only go in one direction (mapping child docIDs to parent docIDs), but in some cases you need to map parent docIDs to child docIDs. For example, when searching songs, perhaps you want all matching songs sorted by their title. You can't easily do this today because the only way to get song hits is to group by album or band/artist. The join is a one (parent) to many (children), inner join. As usual, patches are welcome! There is work underway to create a more flexible, but likely less performant, query-time join capability, which should address a number of the above limitations. Source: http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html

1. Overview This is the third of a series of articles about Persistence with Spring. This article will focus on the configuration and implementation of Spring with JPA. For a step by step introduction about setting up the Spring context using Java based configuration and the basic Maven pom for the project, see this article. The Persistence with Spring series: Part 1 – The Persistence Layer with Spring 3.1 and Hibernate Part 2 – Simplifying the Data Access Layer with Spring and Java Generics Part 4 – The Persistence Layer with Spring Data JPA Part 5 – Transaction configuration with JPA and Spring 3.1 2. No More Spring Templates As of Spring 3.1, the JpaTemplate and the corresponding JpaDaoSupport have been deprecated in favor of using the native Java Persistence API. Also, both of these classes are only relevant for JPA 1 (from the JpaTemplate javadoc): Note that this class did not get upgraded to JPA 2.0 and never will. As a consequence, it is now best practice to use the Java Persistence API directly instead of the JpaTemplate, which will effectively decouple the DAO layer implementation from Spring entirely. Exception Translation without the template One of the responsibilities of JpaTemplate is exception translation – translating the low level exceptions – which tie the API to JPA – into higher level, generic Spring exceptions. Without the template to do that, exception translation can still be enabled by annotating the DAOs with the @Repository annotation. That, coupled with a Spring bean postprocessor will advice all @Repository beans with all the implementations of PersistenceExceptionTranslator found in the Container – to provide exception translation without using the template. Exception translation is done through proxies; in order for Spring to be able to create proxies around the DAO classes, these must not be declared final. Injecting the JPA EntityManager with Spring without the template The EntityManager is the API of the persistence context; this can be injected directly in the DAO. The Spring Container is capable of acting as a JPA container and of injecting the EntityManager by honoring the @PersistenceContext (both as field-level and a method-level annotation). For this to work, the PersistenceAnnotationBeanPostProcessor bean must exist in the Spring Container. The bean can be either created explicitly by defining it in the configuration, or automatically, by defining context:annotation-config or context:component-scan in the configuration. 3. The Spring Java configuration The EntityManager is set up in the configuration by creating a Spring factory bean to manage it; this will allow the PersistenceAnnotationBeanPostProcessor to retrieve it from the Container. There are two options to set this up – either the simpler LocalEntityManagerFactoryBean or the more flexible LocalContainerEntityManagerFactoryBean. The latter option is used here, so that additional properties can be configured on it: @Configuration @EnableTransactionManagement public class PersistenceJPAConfig{ @Bean public LocalContainerEntityManagerFactoryBean entityManagerFactoryBean(){ LocalContainerEntityManagerFactoryBean factoryBean = new LocalContainerEntityManagerFactoryBean(); factoryBean.setDataSource( this.restDataSource() ); factoryBean.setPackagesToScan( new String[ ] { "org.rest" } ); JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter(){ { // JPA properties ... } }; factoryBean.setJpaVendorAdapter( vendorAdapter ); factoryBean.setJpaProperties( this.additionlProperties() ); return factoryBean; } @Bean public DataSource restDataSource(){ DriverManagerDataSource dataSource = new DriverManagerDataSource(); dataSource.setDriverClassName( this.driverClassName ); dataSource.setUrl( this.url ); dataSource.setUsername( "restUser" ); dataSource.setPassword( "restmy5ql" ); return dataSource; } @Bean public PlatformTransactionManager transactionManager(){ JpaTransactionManager transactionManager = new JpaTransactionManager(); transactionManager.setEntityManagerFactory( this.entityManagerFactoryBean().getObject() ); return transactionManager; } @Bean public PersistenceExceptionTranslationPostProcessor exceptionTranslation(){ return new PersistenceExceptionTranslationPostProcessor(); } } Also, note that cglib must be on the classpath for Java @Configuration classes to work; to better understand the need for cglib as a dependency, see this article. 4. The Spring XML configuration The same Spring configuration with XML: There is a relatively small difference between the way Spring is configured in XML and the new Java based configuration – in XML, a reference to another bean can point to either the bean or a bean factory for that bean. In Java however, since the types are different, the compiler doesn’t allow it, and so the EntityManagerFactory is first retrieved from it’s bean factory and then passed to the transaction manager: txManager.setEntityManagerFactory( this.entityManagerFactoryBean().getObject() ); 5. Going full XML-less Usually, JPA defines a persistence unit through the META-INF/persistence.xml file. Starting with Spring 3.1, this XML file is no longer necessary – the LocalContainerEntityManagerFactoryBean now supports a ‘packagesToScan’ property where the packages to scan for @Entity classes can be specified. The persistence.xml file was the last piece of XML to be removed – now, JPA can be fully set up with no XML. 5.1. The JPA Properties JPA properties would usually be specified in the persistence.xml file; alternatively, the properties can be specified directly to the entity manager factory bean: factoryBean.setJpaProperties( this.additionlProperties() ); As a side-note, if Hibernate would be the persistence provider, then this would be the way to specify Hibernate specific properties. 5.2. The DAO Each DAO will be based on an parametrized, abstract DAO class class with support for the common generic operations: public abstract class AbstractJpaDAO< T extends Serializable > { private Class< T > clazz; @PersistenceContext EntityManager entityManager; public void setClazz( Class< T > clazzToSet ){ this.clazz = clazzToSet; } public T findOne( Long id ){ return this.entityManager.find( this.clazz, id ); } public List< T > findAll(){ return this.entityManager.createQuery( "from " + this.clazz.getName() ) .getResultList(); } public void save( T entity ){ this.entityManager.persist( entity ); } public void update( T entity ){ this.entityManager.merge( entity ); } public void delete( T entity ){ this.entityManager.remove( entity ); } public void deleteById( Long entityId ){ T entity = this.getById( entityId ); this.delete( entity ); } } A few aspects are interesting here – as discussed, the abstract DAO does not extend any Spring template (such as JpaTemplate). Instead, the JPA EntityManager is injected directly in the DAO, and will have the role of the main Persistence API Also, note that the entity Class is passed in the constructor to be used in the generic operations: @Repository public class FooDAO extends AbstractHibernateDAO< Foo > implements IFooDAO{ public FooDAO(){ setClazz(Foo.class ); } } 6. The Maven configuration In addition to the Maven configuration defined in a previous article, the following dependencies are addeed: spring-orm (which also has spring-tx as its dependency) and hibernate-entitymanager: org.springframework spring-orm 3.2.2.RELEASE org.hibernate hibernate-entitymanager 4.2.0.Final runtime mysql mysql-connector-java 5.1.24 runtime Note that the MySQL dependency is included as a reference – a driver is needed to configure the datasource, but any Hibernate supported database will do. 7. Conclusion This article covered the configuration and implementation of the persistence layer with JPA 2 and Spring 3.1, using both XML and Java based configuration. The reasons to stop relying on templates for the DAO layer was discussed, as well as getting rid of the last piece of XML usually associated with JPA – the persistence.xml. The final result is a lightweight, clean DAO implementation, with almost no compile-time reliance on Spring. You can check out the full implementation in the github project. OriginalThe Persistence Layer with Spring 3.1 and JPA from the Persistence with Spring series

1. Overview This is the second of a series of articles about Persistence with Spring. The previous article discussed setting up the persistence layer with Spring 3.1 and Hibernate, without using templates. This article will focus on simplifying the Data Access Layer by using a single, generified DAO, which will result in elegant data access, with no unnecessary clutter. Yes, in Java. The Persistence with Spring series: Part 1 – The Persistence Layer with Spring 3.1 and Hibernate Part 3 – The Persistence Layer with Spring 3.1 and JPA Part 4 – The Persistence Layer with Spring Data JPA Part 5 – Transaction configuration with JPA and Spring 3.1 2. The DAO mess Most production codebases have some kind of DAO layer. Usually the implementation ranges from a raw class with no inheritance to some kind of generified class, but one thing is consistent – there is always more then one. Most likely, there are as many DAOs as there are entities in the system. Also, depending on the level of generics involved, the actual implementations can vary from heavily duplicated code to almost empty, with the bulk of the logic grouped in an abstract class. 2.1. A Generic DAO Instead of having multiple implementations – one for each entity in the system – a single parametrized DAO can be used in such a way that it still takes full advantage of the type safety provided by generics. Two implementations of this concept are presented next, one for a Hibernate centric persistence layer and the other focusing on JPA. These implementation are by no means complete – only some data access methods are included, but they can be easily be made more thorough. 2.2. The Abstract Hibernate DAO public abstract class AbstractHibernateDAO< T extends Serializable > { private Class< T > clazz; @Autowired SessionFactory sessionFactory; public void setClazz( Class< T > clazzToSet ){ this.clazz = clazzToSet; } public T findOne( Long id ){ return (T) this.getCurrentSession().get( this.clazz, id ); } public List< T > findAll(){ return this.getCurrentSession() .createQuery( "from " + this.clazz.getName() ).list(); } public void save( T entity ){ this.getCurrentSession().persist( entity ); } public void update( T entity ){ this.getCurrentSession().merge( entity ); } public void delete( T entity ){ this.getCurrentSession().delete( entity ); } public void deleteById( Long entityId ){ T entity = this.getById( entityId ); this.delete( entity ); } protected Session getCurrentSession(){ return this.sessionFactory.getCurrentSession(); } } The DAO uses the Hibernate API directly, without relying on any Spring templates (such as HibernateTemplate). Using of templates, as well as management of the SessionFactory which is autowired in the DAO were covered in the previous post of the series. 2.3. The Abstract JPA DAO public abstract class AbstractJpaDAO< T extends Serializable > { private Class< T > clazz; @PersistenceContext EntityManager entityManager; public void setClazz( Class< T > clazzToSet ){ this.clazz = clazzToSet; } public T findOne( Long id ){ return this.entityManager.find( this.clazz, id ); } public List< T > findAll(){ return this.entityManager.createQuery( "from " + this.clazz.getName() ) .getResultList(); } public void save( T entity ){ this.entityManager.persist( entity ); } public void update( T entity ){ this.entityManager.merge( entity ); } public void delete( T entity ){ this.entityManager.remove( entity ); } public void deleteById( Long entityId ){ T entity = this.getById( entityId ); this.delete( entity ); } } Similar to the Hibernate DAO implementation, the Java Persistence API is used here directly, again not relying on the now deprecated Spring JpaTemplate. 2.4. The Generic DAO Now, the actual implementation of the generic DAO is as simple as it can be – it contains no logic. Its only purpose is to be injected by the Spring container in a service layer (or in whatever other type of client of the Data Access Layer): @Repository @Scope( BeanDefinition.SCOPE_PROTOTYPE ) public class GenericJpaDAO< T extends Serializable > extends AbstractJpaDAO< T > implements IGenericDAO< T >{ // } @Repository @Scope( BeanDefinition.SCOPE_PROTOTYPE ) public class GenericHibernateDAO< T extends Serializable > extends AbstractHibernateDAO< T > implements IGenericDAO< T >{ // } First, note that the generic implementation is itself parametrized – allowing the client to choose the correct parameter in a case by case basis. This will mean that the clients gets all the benefits of type safety without needing to create multiple artifacts for each entity. Second, notice the prototype scope of these generic DAO implementation. Using this scope means that the Spring container will create a new instance of the DAO each time it is requested (including on autowiring). That will allow a service to use multiple DAOs with different parameters for different entities, as needed. The reason this scope is so important is due to the way Spring initializes beans in the container. Leaving the generic DAO without a scope would mean using the default singleton scope, which would lead to a single instance of the DAO living in the container. That would obviously be majorly restrictive for any kind of more complex scenario. 3. The Service There is now a single DAO to be injected by Spring; also, the Class needs to be specified: @Service class FooService implements IFooService{ IGenericDAO< Foo > dao; @Autowired public void setDao( IGenericDAO< Foo > daoToSet ){ this.dao = daoToSet; this.dao.setClazz( Foo.class ); } // ... } Spring autowires the new DAO insteince using setter injection so that the implementation can be customized with the Class object. After this point, the DAO is fully parametrized and ready to be used by the service. 4. Conclusion This article discussed the simplification of the Data Access Layer by providing a single, reusable implementation of a generic DAO. This implementation was presented in both a Hibernate and a JPA based environment. The result is a streamlined persistence layer, with no unnecessary clutter. For a step by step introduction about setting up the Spring context using Java based configuration and the basic Maven pom for the project, see this article. The next article of the Persistence with Spring series will focus on setting up the DAL layer with Spring 3.1 and JPA. In the meantime, you can check out the full implementation in the github project. If you read this far, you should follow me on twitter here.

Neo4j is a high-performance, NOSQL graph database with all the features of a mature and robust database. In this post I will explain how to deploy a neo4j instance in Amazon EC2 web service. For this tutorial to take you no more than 10 minutes you should be able to execute properly some bash commands like mv, tar, ssh and scp (secure copy). I also assume that you have an account in Amazon Web Services and you are familiar to the process of launching instances. If not, I strongly recommend you to follow this starting guide and complete it till you manage to connect to your instance with ssh. Start downloading the latest stable version of neo4j. Which you can find here. The “Community Edition” fits well for development purposes. Do not forget to select the Unix version of the server. This will download a tar.gz file which you will copy to your EC2 instance later. While you download the neo4j server open the AWS Management Console and launch a Basic 32-bit Amazon Linux AMI. If you want to launch an Ubuntu AMI please notice that it doesn’t ship with Java, which is required for running neo4j. If you are not familiar with key pairs, pem files or security groups I insist you to follow the EC2 starting guide I mentioned above. You can either create a new security group or use the default, but you will need to configure a new security rule for the neo4j server port. After launching the instance, create a TCP rule on port 7474 with source 0.0.0.0/0. Here you are opening port 7474 for anyone. If you are planning to use the neo4j REST API and remotely call it from another server, for example a Rails application hosted in Heroku, for security reasons, you may want to change the source field to the address of your Heroku server. Do not forget to open port 22 (SSH), this is typically the first rule normal people create after launching an instance. You are almost done! You should now install neo4j in your instance. Open a terminal in your localhost and navigate to the path where you downloaded neo4j. Copy the file to your Amazon instance by using the scp command: scp -i your_pem_file.pem neo4j-community-1.6.M01-unix.tar.gz ec2-user@YOUR_PUBLIC_INSTANCE_DNS:/home/ec2-user Please notice that you will need to change the path to your pem file, typically placed in ~/.ssh, the filename of the neo4j server you just downloaded and the plublic DNS of your instance. Now connect to your instance with SSH: ssh -i your_pem_file.pem ec2-user@YOUR_PUBLIC_INSTANCE_DNS Untar the neo4j server: tar xvfz neo4j-community-1.6.M01-unix.tar.gz.tar.gz Move it to /usr/local and rename the folder to neo4j: sudo mv neo4j-community-1.6.M01 /usr/local/neo4j Almost done!!! You should now open neo4j-server.properties under the conf directory and add the following line: org.neo4j.server.webserver.address=0.0.0.0 This lines allows anyone to connect remotely to your neo4j database server. Now run the start script. From the neo4j server folder. sudo ./bin/neo4j start Finally, open a browser and access the webadmin interface of your neo4j database by typing http://YOUR_PUBLIC_INSTANCE_DNS:7474. You should see the Neo4j Monitoring and Management Tool, pretty cool! If not, ask me You can now try using the REST API and the curl bash command to insert nodes and relationships. I hope this post helped you, good luck! Follow me on Twitter @negarnil Source: http://www.cloudtmp.com/java/how-to-deploy-a-neo4j-instance-in-amazon-ec2-in-10-minutes/

Today, I was required to order a list of records based on a Name and then ID. A simple one, but I did spend some time on how to do it with Lambda Expression in C#. C# provides the OrderBy, OrderByDescending, ThenBy, ThenByDescending. You can use them in your lambda expression to order the records as per your requirement. Assuming your list is “Phones” and contains the following data public class Phone { public int ID { get; set; } public string Name { get; set; } } public class Phones : List { public Phones() { Add(new Phone { ID = 1, Name = "Windows Phone 7" }); Add(new Phone { ID = 5, Name = "iPhone" }); Add(new Phone { ID = 2, Name = "Windows Phone 7" }); Add(new Phone { ID = 3, Name = "Windows Mobile 6.1" }); Add(new Phone { ID = 6, Name = "Android" }); Add(new Phone { ID = 10, Name = "BlackBerry" }); } } If you were to use LINQ Query , the query will look like the one below dataGridView1.DataSource = (from m in new Phones() orderby m.Name, m.ID select m).ToList(); Simple isn’t it ? It very simple using Lamba expression too. Your Lambda’s expression for the above LINQ query will look like the one below dataGridView1.DataSource = new Phones().OrderByDescending(a => a.Name).ThenByDescending(a => a.ID).ToList();

In this post I am going to explain URL rewriting in greater details. This post will contain basic of URL rewriting and will explain how we can do URL rewriting in fewer lines of code. Why we need URL rewriting? Let’s consider a simpler scenario we want to display a customer details on a ASP.NET Page so how our page will know that for which customer we need to display details? The simplest way of doing is to use query string we will pass a customer id which uniquely identifies customer in query string. So our url will look like this. Customer.aspx?Id=1 This will work but the problem with above URL is that its not user friendly and search engine friendly. who is going to remember that what query string parameter I am going to pass and why we need that parameter. Also when search engine will crawl this site it will going to read this URL blindly as this url is not informative because it query string is not readable for search engine crawlers. So your search engine will be ranked lower as this URL is not readable to search engine crawlers.Now when do a URL rewriting our URL will be cleaner shorter and simpler like this. Customers/Id/1/ Here anybody in world can understand it talking about customer and this page will used to show customer details.Even search engine crawler will also know that you are talking about customers. That is why we need URL rewriting. URL rewriting and ASP.NET In earlier versions of ASP.NET we have to write lots of code for URL rewriting but Now with ASP.NET 4.0 you can easily rewrite in fewer lines of code. So let’s start a Demo where I will demonstrate you how we can easily rewrite URLs. So let’s first create a ASP. NET web form application via File->New->Project and a dialog box will open just like below and then created a empty project called URL rewriting. After creating a project I have added global.asax – where we are going to write url mapping logic and then I have added an asp.net page called which will display customer information just like below. So everything is now ready let’s start writing code. First thing we need to do is to define routes. Route will map a URL to physical page.First I will create static function called Register route which map route to particular file and then I am going to call this from application_start event of global.asax. Following is code for that. using System; using System.Web.Routing; namespace UrlRewriting { public class Global : System.Web.HttpApplication { protected void Application_Start(object sender, EventArgs e) { RegisterRoutes(RouteTable.Routes); } public static void RegisterRoutes(RouteCollection routeCollection) { routeCollection.MapPageRoute("RouteForCustomer", "Customer/{Id}", "~/Customer.aspx"); } } } Now as mapping code has been done let’s right code for customer.aspx page. I have have following code in page_load event of customer.aspx using System; namespace UrlRewriting { public partial class Customer : System.Web.UI.Page { protected void Page_Load(object sender, EventArgs e) { string id = Page.RouteData.Values["Id"].ToString(); Response.Write("Customer Details page"); Response.Write(string.Format("Displaying information for customer : {0}",id)); } } } Here in above code you can see that I am getting value from page route data and then just printing it. In real world it will fetch customer data from database and show customer details on page. Now let’s run that application. It will print a details of customer as I have passed Id in URL suppose you pass 1 as id in URL then it will look like following. Now if you put 2 in url it will print information about customer 2. That’s it. So we have enabled URL rewriting in asp.net in fewer lines of code. In next post I am going to explain redirection with URL rewriting. Hope you like this post. Stay tuned for more.. Till then Happy programming

this post presents an analysis of mysql (a relational database) and neo4j (a graph database) in a side-by-side comparison on a simple graph traversal. the data set that was used was an artificially generated graph with natural statistics. the graph has 1 million vertices and 4 million edges. the degree distribution of this graph on a log-log plot is provided below. a visualization of a 1,000 vertex subset of the graph is diagrammed above. loading the graph the graph data set was loaded both into mysql and neo4j. in mysql a single table was used with the following schema. create table graph ( outv int not null, inv int not null ); create index outv_index using btree on graph (outv); create index inv_index using btree on graph (inv); after loading the data, the table appears as below. the first line reads: “vertex 0 is connected to vertex 1.” mysql> select * from graph limit 10; +------+-----+ | outv | inv | +------+-----+ | 0 | 1 | | 0 | 2 | | 0 | 6 | | 0 | 7 | | 0 | 8 | | 0 | 9 | | 0 | 10 | | 0 | 12 | | 0 | 19 | | 0 | 25 | +------+-----+ 10 rows in set (0.04 sec) the 1 million vertex graph data set was also loaded into neo4j. in gremlin , the graph edges appear as below. the first line reads: “vertex 0 is connected to vertex 992915.” gremlin> g.e[1..10] ==>e[183][0-related->992915] ==>e[182][0-related->952836] ==>e[181][0-related->910150] ==>e[180][0-related->897901] ==>e[179][0-related->871349] ==>e[178][0-related->857804] ==>e[177][0-related->798969] ==>e[176][0-related->773168] ==>e[175][0-related->725516] ==>e[174][0-related->700292] warming up the caches before traversing the graph data structure in both mysql and neo4j, each database had a “ warm up ” procedure run on it. in mysql, a “select * from graph” was evaluated and all of the results were iterated through. in neo4j, every vertex in the graph was iterated through and the outgoing edges of each vertex were retrieved. finally, for both mysql and neo4j, the experiment discussed next was run twice in a row and the results of the second run were evaluated. traversing the graph the traversal that was evaluated on each database started from some root vertex and emanated n-steps out. there was no sorting, no distinct-ing, etc. the only two variables for the experiments are the length of the traversal and the root vertex to start the traversal from. in mysql, the following 5 queries denote traversals of length 1 through 5. note that the “?” is a variable parameter of the query that denotes the root vertex. select a.inv from graph as a where a.outv=? select b.inv from graph as a, graph as b where a.inv=b.outv and a.outv=? select c.inv from graph as a, graph as b, graph as c where a.inv=b.outv and b.inv=c.outv and a.outv=? select d.inv from graph as a, graph as b, graph as c, graph as d where a.inv=b.outv and b.inv=c.outv and c.inv=d.outv and a.outv=? select e.inv from graph as a, graph as b, graph as c, graph as d, graph as e where a.inv=b.outv and b.inv=c.outv and c.inv=d.outv and d.inv=e.outv and a.outv=? for neo4j, the blueprints pipes framework was used. a pipe of length n was constructed using the following static method. public static pipeline createpipeline(final integer steps) { final arraylist pipes = new arraylist(); for (int i = 0; i < steps; i++) { pipe pipe1 = new vertexedgepipe(vertexedgepipe.step.out_edges); pipe pipe2 = new edgevertexpipe(edgevertexpipe.step.in_vertex); pipes.add(pipe1); pipes.add(pipe2); } return new pipeline(pipes); } for both mysql and neo4j, the results of the query (sql and pipes) were iterated through. thus, all results were retrieved for each query. in mysql, this was done as follows. while (resultset.next()) { resultset.getint(finalcolumn); } in neo4j, this is done as follows. while (pipeline.hasnext()) { pipeline.next(); } experimental results the artificial graph dataset was constructed with a “ rich get richer “, preferential attachment model . thus, the vertices created earlier are the most dense (i.e. highest number of adjacent vertices). this property was used to limit the amount of time it would take to evaluate the tests for each traversal. only the first 250 vertices were used as roots of the traversals. before presenting timing results, note that all of these experiments were run on a macbook pro with a 2.66ghz intel core 2 duo and 4gigs of ram at 1067 mhz ddr3. the packages used were java 1.6, mysql jdbc 5.0.8, and blueprints pipes 0.1.2. java version "1.6.0_17" java(tm) se runtime environment (build 1.6.0_17-b04-248-10m3025) java hotspot(tm) 64-bit server vm (build 14.3-b01-101, mixed mode) the following java virtual machine parameters were used: -xmx1000m -xms500m below are the total running times for both mysql (red) and neo4j (blue) for traversals of length 1, 2, 3, and 4. the raw data is presented below along with the total number of vertices returned by each traversal—which, of course, is the same for both mysql and neo4j given that its the same graph data set being processed. also realize that traversals can loop and thus, many of the same vertices are returned multiple times. finally, note that only neo4j has the running time for a traversal of length 5. mysql did not finish after waiting 2 hours to complete. in comparison, neo4j took 14.37 minutes to complete a 5 step traversal. [mysql steps-1] time(ms):124 -- vertices_returned:11360 [mysql steps-2] time(ms):922 -- vertices_returned:162640 [mysql steps-3] time(ms):8851 -- vertices_returned:2206437 [mysql steps-4] time(ms):112930 -- vertices_returned:28125623 [mysql steps-5] n/a [neo4j steps-1] time(ms):27 -- vertices_returned:11360 [neo4j steps-2] time(ms):474 -- vertices_returned:162640 [neo4j steps-3] time(ms):3366 -- vertices_returned:2206437 [neo4j steps-4] time(ms):49312 -- vertices_returned:28125623 [neo4j steps-5] time(ms):862399 -- vertices_returned:358765631 next, the individual data points for both mysql and neo4j are presented in the plot below. each point denotes how long it took to return n number of vertices for the varying traversal lengths. finally, the data below provides the number of vertices returned per millisecond (on average) for each of the traversals. again, mysql did not finish in its 2 hour limit for a traversal of length 5. [mysql steps-1] vertices/ms:91.6128847554668 [mysql steps-2] vertices/ms:176.399127537985 [mysql steps-3] vertices/ms:249.286746556076 [mysql steps-4] vertices/ms:249.053599519823 [mysql steps-5] n/a [neo4j steps-1] vertices/ms:420.740351166341 [neo4j steps-2] vertices/ms:343.122344772028 [neo4j steps-3] vertices/ms:655.507125256186 [neo4j steps-4] vertices/ms:570.360621871775 [neo4j steps-5] vertices/ms:416.00886711325 conclusion in conclusion, given a traversal of an artificial graph with natural statistics, the graph database neo4j is more optimal than the relational database mysql. however, no attempts have been made to optimize the java vm, the sql queries, etc. these experiments were run with both neo4j and mysql “out of the box” and with a “natural syntax” for both types of queries. source: http://markorodriguez.com/2011/02/18/mysql-vs-neo4j-on-a-large-scale-graph-traversal/

Develop your own API service for your PHP projects.

For most large web applications, uptime is of foremost importants. Any outage can be seen by customers as a frustration, or opportunity to move to a competitor. What's more for a site that also includes e-commerce, it can mean real lost sales. Zero Downtime describes a site without service interruption. To achieve such lofty goals, redundancy becomes a critical requirement at every level of your infrastructure. If you're using cloud hosting, are you redundant to alternate availability zones and regions? Are you using geographically distributed load balancing? Do you have multiple clustered databases on the backend, and multiple webservers load balanced. All of these requirements will increase uptime, but may not bring you close to zero downtime. For that you'll need thorough testing. The solution is to pull the trigger on sections of your infrastructure, and prove that it fails over quickly without noticeable outage. The ultimate test is the outage itself. Sean Hull on Quora: What is zero downtime and why is it important? Source: http://www.iheavy.com/2011/06/23/zero-downtime-what-is-it-and-why-is-it-important/