a powerful multi-format xml schema (xsd) documentation generator and a tool for rapid development of custom xsd documentation generators according to user needs. about docflex/xml "xsddoc" template set template processor template designer integrations generation of xsd diagrams apache ant & maven links about docflex/xml docflex/xml is a java-based software system for development and execution of high performance template-driven documentation generators from any data stored in xml files. the actual doc/report generators are programmed in the form of special templates using a graphic template designer , which represents the templates visually in a form resembling the output they generate. further, the templates are interpreted by a template processor , which takes on input the xml files and produces by them the result documentation. this article describes an application of docflex/xml for the task of generation of high-quality xml schema documentation. that includes the following features of docflex/xml system: " xsddoc " template set that implements the ready-to-use xml schema documentation generator itself. template processor makes the templates works. currently, it provides three interchangeable output generators for html, rtf, txt (plain text) formats. template designer provides a high quality gui to design/modify templates. if you need a special xml schema doc generator, the simplest way to create it is to modify the standard xsddoc templates. the template designer enables you to do that. integrations with altova xmlspy and oxygen xml editor . if you are a user of one of those popular xml editors, you can turn it also into a dynamically linked diagramming engine for docflex, so that to include automatically the xsd diagrams generated by xmlspy/oxygenxml into the xml schema documentation generated by docflex (with the full support of hyperlinks). "xsddoc" template set it is the implementation of xml schema documentation itself, which provides the following functionality: generation of single documentation by any number of xml schema (xsd) files together, in particular: highly navigable framed (javadoc-like) html documentation single-file html documentation rtf documentation (further convertible to pdf) processing of any referenced xml schemas, in particular: correct processing of all , , elements found across all involved xsd files. automatic loading and processing (i.e. inclusion in the documentation scope) all directly/indirectly referenced xsd files. sophisticated documenting of xsd components , including: component diagrams (with hyperlinks to everything depicted on them; see also integrations ) xml representation summary (a textual alternative to diagrams) lists of related components. for elements this includes also the list of possible containing elements . (such a list is never present in the output generated by xslt-based doc generators). list of usage locations support of any xml schema design patterns . this comes down mainly to the following: special treatment of local elements (see below) support and documenting of substitution groups support of importing, inclusion and redefinition of schema files special documenting of local elements . local elements are those components that are declared locally within other xsd components. w3c xml schema spec allows you to declare any number of local elements that may share the same name but have different content. that's because their meaning is local and there will be no collisions with other declarations. that, however, creates a problem for documenting, because in a documentation both global and local elements may appear simultaneously in various lists according to their common properties. if each element component is identified only by its name, you will get the lists with multiple repeating names but little clue what they mean. moreover, some xml schemas may contain lots of identical local element declarations (that is, they have the same both name and content). so, you'll get in those lists a mess of repeating names, some of which referencing to effectively the same entities, whereas others to complete different ones. in xsddoc , those problems are solved in two ways: adding extensions to local element names. the extension provides more information about the element (e.g. where it can be inserted or its global type or where it is defined). that makes the whole string identifying the element unique. here is how it looks. the grey text is the name extension: unifying local elements by type. on the left you can see a documentation generated with such unification. on the right, all local elements are documented straight as they are. click on each screenshot to view the docs: we believe the first documentation (on the left) is easier to understand and use. processing of xhtml markup . you can format your xml schema annotations with xhtml tags, which will be recognized and rendered with the appropriate formatting in both html and rtf output, as shown on the following screenshots (click to see more details): here, on the left you can see the xml source of an xml schema, whose annotations are heavily laden with xhtml markup (including insertion of images). the next is the html documentation generated by that schema. on the right is a page of rtf documentation also generated by that schema. possibility of unlimited customization : xsddoc is controlled by more than 400 parameters, which allow you to adjust the generated documentation within huge range of included details. template parameters serve the same role as options in traditional doc generators. the difference is that docflex template architecture makes the support/implementation of template parameters very cheap (typically, the most of efforts takes writing their descriptions). so, there may be hundreds of parameters controlling a large template application. if parameters are not enough, you can modify the templates themselves using the template designer . in case of html output, you can also apply your own css styles to change how the generated documentation looks. template processor the template processor (also called simply "generator") makes everything work. it consists of two logical parts: 1. template interpreter 2. output generator the output generator actually has three different implementations for each currently supported output format: html, rtf, txt (plain text). the plain-text output can be used to generate documentation in formats not supported directly by docflex. the template processor is started directly from java command line with the following arguments: ● main template ● template parameters ● initial xsd files to be processed (documented) ● xml catalogs (to redirect physical location of input files) ● destination directory/file ● output format (this selects which output generator will be used) ● output format options (specify settings to control the selected output generator) actually, the number of settings may be so large that the template processor provides a special gui to specify everything interactively (click to enlarge): template designer although docflex templates are stored as plain-text files (with an xml-like format), they are not supposed for editing manually. rather, a special graphic template designer must be used, which visualizes the templates in the form of template components they are made of. those components are the actual constructs of the template language (not some textual statements, operators, blocks etc.) the following screenshots show templates open in the template designer (click to see a lot more): that approach has a number of advantages, among them: the processing structures represented by template components may be displayed in a way that visually expresses what a component does (for instance, it may resemble the output it generates). that representation may be both expressive and compact (after all, it is not just a text), which allows you easily to navigate a template, understand what it does and modify anything you need. as template components are visual and interactive, they may have very complex internal structure, for instance, contain lots of properties and nested components. at that, you don't need to scroll and navigate some kind of enormous text, which encodes all of this (as it would be in case of a script). rather, you just need to invoke some property dialogs and expand/collapse some component sections. a template component may be easily copied, pasted and deleted as a whole. at that, you don't need to bother that the template syntax is restored after that. the template designer will also ensure that each component is created, copied or moved only in the allowed place. the highly structured nature of templates eliminates the need for most of various named identifiers. many connections between different template components are also maintained by the template designer (i.e. modified automatically when necessary). as template files are stored and read only programmatically, there is no need to know and understand their syntax. there will be no syntax errors either. the actual syntax of template files may be optimized not for human programmers, but for faster loading and processing of templates by the template processor . there is no need in a compilation phase. the separation of template semantics from the particular structure of template files helps for faster and easier evolution of the template language. the obsolete constructs of older template versions can be automatically converted into new structures. both old and new templates will look and work up-to-date. integrations generation of xsd diagrams docflex/xml is able to work with any kind of diagrams (i.e. inserting them automatically in the generated output). that is supported on the level of templates, along with the generation of hypertext imagemaps, as shown on the following screenshot (click to see a lot more): docflex/xml provides no diagramming engine of its own. instead, it includes integrations with two most popular xml editors that do generate xsd diagrams: ● altova xmlspy ● oxygen xml editor effectively, the third-party software is used as dynamically linked diagramming engine. the advantage of such integrations is that when you are the user of one of those xml editors, you will get in the documentation generated by docflex the same diagrams as you see in your xml editor. here is how such a documentation with diagrams looks (click on a screenshot to view the real html): apache ant & maven as a pure java application, docflex/xml can be run in any environment that runs java itself. the template processor can be easily integrated with ant (that can be specified just in the ant build file). in case of maven, docflex/xml includes a simple maven plugin. it is possible also to use all diagraming integrations with both ant and maven. links docflex/xml (home page): http://www.filigris.com/docflex-xml/ docflex/xml xsddoc: http://www.filigris.com/docflex-xml/xsddoc/ xsddoc examples: http://www.filigris.com/docflex-xml/xsddoc/examples/ xmlspy integration: http://www.filigris.com/docflex-xml/xmlspy/ oxygenxml integration: http://www.filigris.com/docflex-xml/oxygenxml/ free downloads: http://www.filigris.com/downloads/ this original article: http://www.filigris.com/ann/docflex-xsd/

For some reason I needed extremely large, possibly even infinite InputStream that would simply return the same byte[]over and over. This way I could produce insanely big stream of data by repeating small sample. Sort of similar functionality can be found in Guava: Iterable Iterables.cycle(Iterable) and Iterator Iterators.cycle(Iterator). For example if you need an infinite source of 0 and 1, simply sayIterables.cycle(0, 1) and get 0, 1, 0, 1, 0, 1... infinitely. Unfortunately I haven't found such utility forInputStream, so I jumped into writing my own. This article documents many mistakes I made during that process, mostly due to overcomplicating and overengineering straightforward solution. We don't really need an infinite InputStream, being able to create very large one (say, 32 GiB) is enough. So we are after the following method: public static InputStream repeat(byte[] sample, int times) It basically takes sample array of bytes and returns an InputStream returning these bytes. However when sample runs out, it rolls over, returning the same bytes again - this process is repeated given number of times, until InputStreamsignals end. One solution that I haven't really tried but which seems most obvious: public static InputStream repeat(byte[] sample, int times) { final byte[] allBytes = new byte[sample.length * times]; for (int i = 0; i < times; i++) { System.arraycopy(sample, 0, allBytes, i * sample.length, sample.length); } return new ByteArrayInputStream(allBytes); } I see you laughing there! If sample is 100 bytes and we need 32 GiB of input repeating these 100 bytes, generatedInputStream shouldn't really allocate 32 GiB of memory, we must be more clever here. As a matter of fact repeat()above has another subtle bug. Arrays in Java are limited to 231-1 entries (int), 32 GiB is way above that. The reason this program compiles is a silent integer overflow here: sample.length * times. This multiplication doesn't fit in int. OK, let's try something that at least theoretically can work. My first idea was as follows: what if I create manyByteArrayInputStreams sharing the same byte[] sample (they don't do an eager copy) and somehow join them together? Thus I needed some InputStream adapter that could take arbitrary number of underlying InputStreams and chain them together - when first stream is exhausted, switch to next one. This awkward moment when you look for something in Apache Commons or Guava and apparently it was in the JDK forever... java.io.SequenceInputStream is almost ideal. However it can only chain precisely two underlying InputStreams. Of course since SequenceInputStreamis an InputStream itself, we can use it recursively as an argument to outer SequenceInputStream. Repeating this process we can chain arbitrary number of ByteArrayInputStreams together: public static InputStream repeat(byte[] sample, int times) { if (times <= 1) { return new ByteArrayInputStream(sample); } else { return new SequenceInputStream( new ByteArrayInputStream(sample), repeat(sample, times - 1) ); } } If times is 1, just wrap sample in ByteArrayInputStream. Otherwise use SequenceInputStream recursively. I think you can immediately spot what's wrong with this code: too deep recursion. Nesting level is the same as times argument, which will reach millions or even billions. There must be a better way. Luckily minor improvement changes recursion depth from O(n) to O(logn): public static InputStream repeat(byte[] sample, int times) { if (times <= 1) { return new ByteArrayInputStream(sample); } else { return new SequenceInputStream( repeat(sample, times / 2), repeat(sample, times - times / 2) ); } } Honestly this was the first implementation I tried. It's a simple application of divide and conquer principle, where we produce result by evenly splitting it into two smaller sub-problems. Looks clever, but there is one issue: it's easy to prove we create t (t = times) ByteArrayInputStreams and O(t) SequenceInputStreams. While sample byte array is shared, millions of various InputStream instances are wasting memory. This leads us to alternative implementation, creating just one InputStream, regardless value of times: import com.google.common.collect.Iterators; import org.apache.commons.lang3.ArrayUtils; public static InputStream repeat(byte[] sample, int times) { final Byte[] objArray = ArrayUtils.toObject(sample); final Iterator infinite = Iterators.cycle(objArray); final Iterator limited = Iterators.limit(infinite, sample.length * times); return new InputStream() { @Override public int read() throws IOException { return limited.hasNext() ? limited.next() & 0xFF : -1; } }; } We will use Iterators.cycle() after all. But before we have to translate byte[] into Byte[] since iterators can only work with objets, not primitives. There is no idiomatic way to turn array of primitives to array of boxed types, so I useArrayUtils.toObject(byte[]) from Apache Commons Lang. Having an array of objects we can create an infiniteiterator that cycles through values of sample. Since we don't want an infinite stream, we cut off infinite iterator usingIterators.limit(Iterator, int), again from Guava. Now we just have to bridge from Iterator toInputStream - after all semantically they represent the same thing. This solution suffers two problems. First of all it produces tons of garbage due to unboxing. Garbage collection is not that much concerned about dead, short-living objects, but still seems wasteful. Second issue we already faced previously:sample.length * times multiplication can cause integer overflow. It can't be fixed because Iterators.limit() takesint, not long - for no good reason. BTW we avoided third problem by doing bitwise and with 0xFF - otherwise byte with value -1 would signal end of stream, which is not the case. x & 0xFF is correctly translated to unsigned 255 (int). So even though implementation above is short and sweet, declarative rather than imperative, it's too slow and limited. If you have a C background, I can imagine how uncomfortable you were seeing me struggle. After all the most straightforward, painfully simple and low-level implementation was the one I came up with last: public static InputStream repeat(byte[] sample, int times) { return new InputStream() { private long pos = 0; private final long total = (long)sample.length * times; public int read() throws IOException { return pos < total ? sample[(int)(pos++ % sample.length)] : -1; } }; } GC free, pure JDK, fast and simple to understand. Let this be a lesson for you: start with the simplest solution that jumps to your mind, don't overengineer and don't be too smart. My previous solutions, declarative, functional, immutable, etc. - maybe they looked clever, but they were neither fast nor easy to understand. The utility we just developed was not just a toy project, it will be used later in subsequent article.

This is a first post in a series of short code snippets that will present the configuration of Spring beans from XML to Java. XML: resource.loader=class class.resource.loader.class=org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader Java @Bean public VelocityEngine velocityEngine() throws VelocityException, IOException{ VelocityEngineFactoryBean factory = new VelocityEngineFactoryBean(); Properties props = new Properties(); props.put("resource.loader", "class"); props.put("class.resource.loader.class", "org.apache.velocity.runtime.resource.loader." + "ClasspathResourceLoader"); factory.setVelocityProperties(props); return factory.createVelocityEngine(); }

this article illustrates the general time - memory tradeoff with the example of different hash table implementations in java. the more memory a hash table takes, the faster each operation (e. g. getting a value by key or putting an entry) is performed. benchmarking method hash maps with int keys and int values were tested. memory measure is relative usage over theoretical minimum. for example, 1000 entries of int key and value take at least (4 (size of int) + 4) * 1000 = 8000 bytes. if the hash map implementation takes 20 000 bytes, it's memory overuse is (20 000 - 8000) / 8000 = 1.5. each implementation was benchmarked on 9 different load levels (load factors). on each load level, each map was filled with 10 numbers of entries, logariphmically evenly distributed bewteen 1000 and 10 000 000 (to study caching effects). then, for the same implementation and load level, memory metrics and average operation throughputs are averaged independently, over 3 smallest sizes (small sizes), 3 largest sizes (large sizes) and all 10 sizes from 1000 to 10 000 000 (all sizes). implementations: higher frequency trading collections (hftc) high performance primitive collections (hppc) fastutil collections goldman sachs collections (gs) trove collections mahout collections java.util.hashmap as a reference get value by key (successful) only looking at these charts, you can suppose that hftc, trove and mahout on the one size, fastutil, hppc and gs on the another use the same hash table algorithm. (in fact, it is not quite true.) sparser hash table on average performs less lookups during key search, therefore less memory reads, therefore the operation finishes earlier. notice, that on small sizes the largest maps are the fastest for all implementations, but on large and all sizes there isn't visible progress starting from memory overuse ~4. that's because when the total memory taken by the map goes beyond cpu cache capacity, cache misses become more often when the map is getting larger. this effect compensates algorithic trend. update (increment) value by key update operation behaves pretty similar to get(). fastutil wasn't benchmarked, because there aren't fairly performant method for this task in it's api. put an entry (key was absent) in this case, maps were gradually filled in with the entries from the size 0 to the target size (1000 - 10 000 000). rehash shouldn't occur, because maps were constructed with the target size provided. for small sizes, plots still looks like hyperbolas, but i can't explain so dramatic change on large sizes and differences between hftc and other primitive implementations. internal iteration (foreach) iteration is getting slower with memory usage growth. interesting thing about external iteration: for all open hash table implementations throughtput depends only memory usage, not even on load factor (which differs for implementations for the same memory usage). also, foreach throughput don't depend on open hash table size. external iteration (via iterator or cursor) external iteration performance is more varying than internal, because there is more freedom for optimization. hftc and trove employ own iteration interfaces, other libraries use standard java.util.iterator . footnote raw benchmark results from which the pictures were built with a link to the benchmarking code and information about the runsite in description.

JAX-RS 2.0 is the latest version of the JSR 311 specification and it was released along with Java EE 7.

In this post we explore 5 reasons to use a Java Data Grid for caching Java objects in-memory in your applications. In a later post we will explore some of the other data grid capabilities, beyond data storage, that can revolutionize your Java architectures, like on-grid computation and events. Memory is Fast Java Data Grids store Java objects in memory. Memory access is fast with low latency. So if access to data storage either disk or database is the primary bottleneck in your application then using a data grid as an in-memory cache in front of your storage tier will give you a performance boost. Scale out your Application Shared State If you need to share state across JVMs to scale out your application then using a Java Data Grid rather than a database will increase your scalability. A typical shared state architecture is shown below, the application server tier stores shared Java objects in the data grid and these objects are available to all application server nodes in your architecture. Separating the data grid tier from the application server tier has a number of advantages; Applications can be redeployed and restarted without losing the shared state Data Grid JVMs and Application JVMs can be tuned separately State can be shared across multiple different applications. Each tier can be scaled horizontally separately depending on work load Typical use cases for shared state include; PCI compliant storage of card security codes; In-game state in online games; web session data; prices and catalogues in ecommerce. Anything that needs low latency access can be stored in the shared data grid. High Availability for In-Memory Data As well as low latency access and scaling out shared state. Java Data Grids also provide high availability for your in-memory data. When storing Java objects in a data grid a primary object is stored in one of the Data Grid JVMs and secondary back up copies of the object are stored in different Data Grid JVM node, ensuring that if you lose a node then you don't lose any data. Clients of the data grid do not need to know where data is to access it so high availability is transparent to your application. Scale Out In-Memory Data Volumes Java objects, in data grids, aren't fully replicated across all Data Grid JVMs but are stored as a primary object and a secondary object. This means the more Data Grid JVM nodes we add the more JVM heap we have for storing Java objects in-memory (and remember memory is fast). For example if we build a Data Grid with 20 JVMs each with 4Gb free heap (after per JVM overhead) we could theoretically store 80Gb (4 times 20) of shared Java objects. If we assume we have 1 duplicate for high availability this cuts our storage in half so we can store 40Gb (.5 time 4 times 20 ) of Java Objects in memory. Native Integration with JPA Java Data Grids have native integration with JPA frameworks like TopLink and Hibernate whereby the Data Grid can act as a second level cache between JPA and the database. This can give a large performance boost to your database driven application if latency associated with database access is a key performance bottleneck.

While reading ‘Why The R Programming Language Is Good For Business‘ I came across Udacity’s ‘Data Analysis with R‘ courses – part of which focuses exploring data sets using visualisations, something I haven’t done much of yet. I thought it’d be interesting to create some visualisations around the times that people RSVP ‘yes’ to the various Neo4j events that we run in London. I started off with the following query which returns the date time that people replied ‘Yes’ to an event and the date time of the event: library(Rneo4j) query = "MATCH (e:Event)<-[:TO]-(response {response: 'yes'}) RETURN response.time AS time, e.time + e.utc_offset AS eventTime" allYesRSVPs = cypher(graph, query) allYesRSVPs$time = timestampToDate(allYesRSVPs$time) allYesRSVPs$eventTime = timestampToDate(allYesRSVPs$eventTime) > allYesRSVPs[1:10,] time eventTime 1 2011-06-05 12:12:27 2011-06-29 18:30:00 2 2011-06-05 14:49:04 2011-06-29 18:30:00 3 2011-06-10 11:22:47 2011-06-29 18:30:00 4 2011-06-07 15:27:07 2011-06-29 18:30:00 5 2011-06-06 20:21:45 2011-06-29 18:30:00 6 2011-07-04 19:49:04 2011-07-27 19:00:00 7 2011-07-05 16:40:10 2011-07-27 19:00:00 8 2011-08-19 07:41:10 2011-08-31 18:30:00 9 2011-08-24 12:47:40 2011-08-31 18:30:00 10 2011-08-18 09:56:53 2011-08-31 18:30:00 I wanted to create a bar chart showing the amount of time in advance of a meetup that people RSVP’d ‘yes’ so I added the following column to my data frame: allYesRSVPs$difference = allYesRSVPs$eventTime - allYesRSVPs$time > allYesRSVPs[1:10,] time eventTime difference 1 2011-06-05 12:12:27 2011-06-29 18:30:00 34937.55 mins 2 2011-06-05 14:49:04 2011-06-29 18:30:00 34780.93 mins 3 2011-06-10 11:22:47 2011-06-29 18:30:00 27787.22 mins 4 2011-06-07 15:27:07 2011-06-29 18:30:00 31862.88 mins 5 2011-06-06 20:21:45 2011-06-29 18:30:00 33008.25 mins 6 2011-07-04 19:49:04 2011-07-27 19:00:00 33070.93 mins 7 2011-07-05 16:40:10 2011-07-27 19:00:00 31819.83 mins 8 2011-08-19 07:41:10 2011-08-31 18:30:00 17928.83 mins 9 2011-08-24 12:47:40 2011-08-31 18:30:00 10422.33 mins 10 2011-08-18 09:56:53 2011-08-31 18:30:00 19233.12 mins I then tried to use ggplot to create a bar chart of that data: > ggplot(allYesRSVPs, aes(x=difference)) + geom_histogram(binwidth=1, fill="green") Unfortunately that resulted in this error: Don't know how to automatically pick scale for object of type difftime. Defaulting to continuous Error: Discrete value supplied to continuous scale I couldn’t find anyone who had come across this problem before in my search but I did find the as.numeric function which seemed like it would put the difference into an appropriate format: allYesRSVPs$difference = as.numeric(allYesRSVPs$eventTime - allYesRSVPs$time, units="days") > ggplot(allYesRSVPs, aes(x=difference)) + geom_histogram(binwidth=1, fill="green") that resulted in the following chart: We can see there is quite a heavy concentration of people RSVPing yes in the few days before the event and then the rest are scattered across the first 30 days. We usually announce events 3/4 weeks in advance so I don’t know that it tells us anything interesting other than that it seems like people sign up for events when an email is sent out about them. The date the meetup was announced (by email) isn’t currently exposed by the API but hopefully one day it will be. The code is on github if you want to have a play – any suggestions welcome.

I blogged about this a few days ago, but I think I'm ready to really release my Cordova Brackets extension. The code is pretty much crap and it's really lacking in providing good feedback while it works, but the initial feature list is complete. Assuming you've got the Cordova CLI installed already, you can, via Brackets: Add and remove platforms. Warning - if Cordova needs to grab the bits for the platform (it does this once), it will be slow, and again, my extension isn't providing feedback about the operation until it is done, so, like, don't do anything. Go get coffee. Emulate for a platform. Not use run. It doesn't work, which means I kinda lied when I said the initial feature list was complete, but I'm OK with that. List plugins, along with the version. Add plugins, and I've got a nice autocomplete so you can quickly find the right plugin. Remove plugins too. There are bugs around this, so, be careful. You can see this in action in this thrilling video below. Oddly it is a bit out of focus for the first few seconds, but then it clears up. I blame gremlins. It is now available via the Extension Manager so you can install it directly from within Brackets. If you want to help work on the source, pull a fork over at https://github.com/cfjedimaster/Cordova-Extension. If you really like it, visit the Amazon wishlist and pick up that Marvel Lego game. ;)

recently, i had to put on my system administrator cap for couple of hours. i was to set up two dell poweredge 1720 server computers for use as test servers. the two servers were to run microsoft windows server 2012 r2 and that .net framework 3.5 must be enabled on the servers as it is one of the requirements needed by the enterprise application to be installed on the servers. enabling .net framework 3.5 on windows server 2012 r2 can be done through the server manager by clicking on add roles and features and then under features selecting .net framework 3.5 or through command prompt or windows power shell. as it is microsoft’s custom to put the required files for required role or features in the side-by-side folder. one would have thought that .net framework 3.5 files should be in the folder also as it is in windows server 2012’s side-by-side folder. so an attempt to enable .net framework 3.5 on windows server 2012 r2 would give an error “that installation of one or more roles, role services or features failed. the source file could not be found” like the one shown in the picture below. this means that the .net framework 3.5 (which includes 3.0 and 2.0) files are not included in windows server 2012 r2 setup. if the server is connected to the internet and it has windows update enabled, the framework can be pulled from the internet and added to the windows installation folder by the os when enabling the feature. but the servers i was configuring had no internet connections which means that there was no way the framework could be pulled from the internet. what i did was to copy the side-by-side folder from windows server 2012 setup, (not windows server 2012 r2 setup) since it contains the files needed to install the .net 3.5 framework, i then ran command prompt as an administrator with the following commands. dism.exe /online /enable-feature /featurename:netfx3serverfeatures /source:d:\sxs to enable the feature dism.exe /online /enable-feature /featurename:netfx3 /source:d:\sxs /limitaccess to install the feature where d:\sxs is the path of the folder containing the required files that i copied from windows server 2012 setup side-by-side folder. i hope i have been able to save someone precious time.

This is a sample that I have wanted to try for sometime - A Websocket application to tail the contents of a file. The following is the final view of the web-application: There are a few parts to this application: Generating a File to tail: I chose to use a set of 100 random quotes as a source of the file content, every few seconds the application generates a quote and writes this quote to the temporary file. Spring Integration is used for wiring this flow for writing the contents to the file: Just a quick note, Spring Integration flows can now also be written using a Java Based DSL, and this flow using Java is available here Tailing the file and sending the content to a broker The actual tailing of the file itself can be accomplished by OS specific tail command or by using a library like Apache Commons IO. Again in my case I decided to use Spring Integration which provides Inbound channel adapters to tail a file purely using configuration, this flow looks like this: and its working Java equivalent There is a reference to a "fileContentRecordingService" above, this is the component which will direct the lines of the file to a place where the Websocket client will subscribe to. Websocket server configuration Spring Websocket support makes it super simple to write a Websocket based application, in this instance the entire working configuration is the following: @Configuration @EnableWebSocketMessageBroker public class WebSocketDefaultConfig extends AbstractWebSocketMessageBrokerConfigurer { @Override public void configureMessageBroker(MessageBrokerRegistry config) { //config.enableStompBrokerRelay("/topic/", "/queue/"); config.enableSimpleBroker("/topic/", "/queue/"); config.setApplicationDestinationPrefixes("/app"); } @Override public void registerStompEndpoints(StompEndpointRegistry registry) { registry.addEndpoint("/tailfilesep").withSockJS(); } } This may seem a little over the top, but what these few lines of configuration does is very powerful and the configuration can be better understood by going through the reference here. In brief, it sets up a websocket endpoint at '/tailfileep' uri, this endpoint is enhanced with SockJS support, Stomp is used as a sub-protocol, endpoints `/topic` and `/queue` is configured to a real broker like RabbitMQ or ActiveMQ but in this specific to an in-memory one. Going back to the "fileContentRecordingService" once more, this component essentially takes the line of the file and sends it this in-memory broker, SimpMessagingTemplate facilitates this wiring: public class FileContentRecordingService { @Autowired private SimpMessagingTemplate simpMessagingTemplate; public void sendLinesToTopic(String line) { this.simpMessagingTemplate.convertAndSend("/topic/tailfiles", line); } } Websocket UI configuration The UI is angularjs based, the client controller is set up this way and internally uses the javascript libraries for sockjs and stomp support: var tailFilesApp = angular.module("tailFilesApp",[]); tailFilesApp.controller("TailFilesCtrl", function ($scope) { function init() { $scope.buffer = new CircularBuffer(20); } $scope.initSockets = function() { $scope.socket={}; $scope.socket.client = new SockJS("/tailfilesep); $scope.socket.stomp = Stomp.over($scope.socket.client); $scope.socket.stomp.connect({}, function() { $scope.socket.stomp.subscribe("/topic/tailfiles", $scope.notify); }); $scope.socket.client.onclose = $scope.reconnect; }; $scope.notify = function(message) { $scope.$apply(function() { $scope.buffer.add(angular.fromJson(message.body)); }); }; $scope.reconnect = function() { setTimeout($scope.initSockets, 10000); }; init(); $scope.initSockets(); }); The meat of this code is the "notify" function which the callback acting on the messages from the server, in this instance the new lines coming into the file and showing it in a textarea. This wraps up the entire application to tail a file. A complete working sample without any external dependencies is available at this github location, instructions to start it up is also available at that location. Conclusion Spring Websockets provides a concise way to create Websocket based applications, this sample provides a good demonstration of this support. I had presented on this topic recently at my local JUG (IndyJUG) and a deck with the presentation is available here

Continuing from the first article, this time we will write some more useful custom collectors: for grouping by given criteria, sampling input, batching and sliding over with fixed size window. Grouping (counting occurrences, histogram) Imagine you have a collection of some items and you want to calculate how many times each item (with respect to equals()) appears in this collection. This can be achieved using CollectionUtils.getCardinalityMap() from Apache Commons Collections. This method takes an Iterable and returns Map, counting how many times each item appeared in the collection. However sometimes instead of usingequals() we would like to group by an arbitrary attribute of input T. For example say we have a list of Person objects and we would like to compute the number of males vs. females (i.e. Map) or maybe an age distribution. There is a built-in collector Collectors.groupingBy(Function classifier) - however it returns a map from key to all items mapped to that key. See: import static java.util.stream.Collectors.groupingBy; //... final List people = //... final Map> bySex = people .stream() .collect(groupingBy(Person::getSex)); It's valuable, but in our case unnecessarily builds two List. I only want to know the number of people. There is no such collector built-in, but we can compose it in a fairly simple manner: import static java.util.stream.Collectors.counting; import static java.util.stream.Collectors.groupingBy; //... final Map bySex = people .stream() .collect( groupingBy(Person::getSex, HashMap::new, counting())); This overloaded version of groupingBy() takes three parameters. First one is the key (classifier) function, as previously. Second argument creates a new map, we'll see shortly why it's useful. counting() is a nested collector that takes all people with same sex and combines them together - in our case simply counting them as they arrive. Being able to choose map implementation is useful e.g. when building age histogram. We would like to know how many people we have at given age - but age values should be sorted: final TreeMap byAge = people .stream() .collect( groupingBy(Person::getAge, TreeMap::new, counting())); byAge .forEach((age, count) -> System.out.println(age + ":\t" + count)); We ended up with a TreeMap from age (sorted) to count of people having that age. Sampling, batching and sliding window IterableLike.sliding() method in Scala allows to view a collection through a sliding fixed-size window. This window starts at the beginning and in each iteration moves by given number of items. Such functionality, missing in Java 8, allows several useful operators like computing moving average, splitting big collection into batches (compare with Lists.partition() in Guava) or sampling every n-th element. We will implement collector for Java 8 providing similar behaviour. Let's start from unit tests, which should describe briefly what we want to achieve: import static com.nurkiewicz.CustomCollectors.sliding @Unroll class CustomCollectorsSpec extends Specification { def "Sliding window of #input with size #size and step of 1 is #output"() { expect: input.stream().collect(sliding(size)) == output where: input | size | output [] | 5 | [] [1] | 1 | [[1]] [1, 2] | 1 | [[1], [2]] [1, 2] | 2 | [[1, 2]] [1, 2] | 3 | [[1, 2]] 1..3 | 3 | [[1, 2, 3]] 1..4 | 2 | [[1, 2], [2, 3], [3, 4]] 1..4 | 3 | [[1, 2, 3], [2, 3, 4]] 1..7 | 3 | [[1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 7]] 1..7 | 6 | [1..6, 2..7] } def "Sliding window of #input with size #size and no overlapping is #output"() { expect: input.stream().collect(sliding(size, size)) == output where: input | size | output [] | 5 | [] 1..3 | 2 | [[1, 2], [3]] 1..4 | 4 | [1..4] 1..4 | 5 | [1..4] 1..7 | 3 | [1..3, 4..6, [7]] 1..6 | 2 | [[1, 2], [3, 4], [5, 6]] } def "Sliding window of #input with size #size and some overlapping is #output"() { expect: input.stream().collect(sliding(size, 2)) == output where: input | size | output [] | 5 | [] 1..4 | 5 | [[1, 2, 3, 4]] 1..7 | 3 | [1..3, 3..5, 5..7] 1..6 | 4 | [1..4, 3..6] 1..9 | 4 | [1..4, 3..6, 5..8, 7..9] 1..10 | 4 | [1..4, 3..6, 5..8, 7..10] 1..11 | 4 | [1..4, 3..6, 5..8, 7..10, 9..11] } def "Sliding window of #input with size #size and gap of #gap is #output"() { expect: input.stream().collect(sliding(size, size + gap)) == output where: input | size | gap | output [] | 5 | 1 | [] 1..9 | 4 | 2 | [1..4, 7..9] 1..10 | 4 | 2 | [1..4, 7..10] 1..11 | 4 | 2 | [1..4, 7..10] 1..12 | 4 | 2 | [1..4, 7..10] 1..13 | 4 | 2 | [1..4, 7..10, [13]] 1..13 | 5 | 1 | [1..5, 7..11, [13]] 1..12 | 5 | 3 | [1..5, 9..12] 1..13 | 5 | 3 | [1..5, 9..13] } def "Sampling #input taking every #nth th element is #output"() { expect: input.stream().collect(sliding(1, nth)) == output where: input | nth | output [] | 1 | [] [] | 5 | [] 1..3 | 5 | [[1]] 1..6 | 2 | [[1], [3], [5]] 1..10 | 5 | [[1], [6]] 1..100 | 30 | [[1], [31], [61], [91]] } } Using data driven tests in Spock I managed to write almost 40 test cases in no-time, succinctly describing all requirements. I hope these are clear for you, even if you haven't seen this syntax before. I already assumed existence of handy factory methods: public class CustomCollectors { public static Collector>> sliding(int size) { return new SlidingCollector<>(size, 1); } public static Collector>> sliding(int size, int step) { return new SlidingCollector<>(size, step); } } The fact that collectors receive items one after another makes are job harder. Of course first collecting the whole list and sliding over it would have been easier, but sort of wasteful. Let's build result iteratively. I am not even pretending this task can be parallelized in general, so I'll leave combiner() unimplemented: public class SlidingCollector implements Collector>, List>> { private final int size; private final int step; private final int window; private final Queue buffer = new ArrayDeque<>(); private int totalIn = 0; public SlidingCollector(int size, int step) { this.size = size; this.step = step; this.window = max(size, step); } @Override public Supplier>> supplier() { return ArrayList::new; } @Override public BiConsumer>, T> accumulator() { return (lists, t) -> { buffer.offer(t); ++totalIn; if (buffer.size() == window) { dumpCurrent(lists); shiftBy(step); } }; } @Override public Function>, List>> finisher() { return lists -> { if (!buffer.isEmpty()) { final int totalOut = estimateTotalOut(); if (totalOut > lists.size()) { dumpCurrent(lists); } } return lists; }; } private int estimateTotalOut() { return max(0, (totalIn + step - size - 1) / step) + 1; } private void dumpCurrent(List> lists) { final List batch = buffer.stream().limit(size).collect(toList()); lists.add(batch); } private void shiftBy(int by) { for (int i = 0; i < by; i++) { buffer.remove(); } } @Override public BinaryOperator>> combiner() { return (l1, l2) -> { throw new UnsupportedOperationException("Combining not possible"); }; } @Override public Set characteristics() { return EnumSet.noneOf(Characteristics.class); } } I spent quite some time writing this implementation, especially correct finisher() so don't be frightened. The crucial part is a buffer that collects items until it can form one sliding window. Then "oldest" items are discarded and window slides forward by step. I am not particularly happy with this implementation, but tests are passing. sliding(N)(synonym to sliding(N, 1)) will allow calculating moving average of N items.sliding(N, N) splits input into batches of size N. sliding(1, N) takes every N-th element (samples). I hope you'll find this collector useful, enjoy!

In this post, I will show how to integrate Apache Tiles 3 with Spring MVC.

Read about enhanced identifier generators, like JPA and Hibernate.

Retrieving JMX information for a Java process is very easy when using a tool such as JConsole or JVisualVM. These provide an interface that allows viewing of information such as CPU usage, memory usage, threads active and more. This blog post gives an example of how to retrieve such information programmatically. In order to retrieve JMX information from a Java application, the target application must be configured to expose JMX information. This link shows how to do this. As an example, we shall be retrieving CPU and memory usage from a standalone Mule instance. In Mule, the JMX agent may be configured from a Mule configuration file. Through this, one may set the address that a JMX client can use to retrieve information; this is how. The following Java code allows for polling the JMX agent and retrieving memory, CPU usage and also shows how to remotely invoke the garbage collector: // create jmx connection with mules jmx agent JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:1098/server"); JMXConnector jmxc = JMXConnectorFactory.connect(url, null); jmxc.connect(); //create object instances that will be used to get memory and operating system Mbean objects exposed by JMX; create variables for cpu time and system time before Object memoryMbean = null; Object osMbean = null; long cpuBefore = 0; long tempMemory = 0; CompositeData cd = null; cpuBefore = Long.parseLong(a.toString()); // call the garbage collector before the test using the Memory Mbean jmxc.getMBeanServerConnection().invoke(new ObjectName("java.lang:type=Memory"), "gc", null, null); //create a loop to get values every second (optional) for (int i = 0; i < samplesCount; i++) { //get an instance of the HeapMemoryUsage Mbean memoryMbean = jmxc.getMBeanServerConnection().getAttribute(new ObjectName("java.lang:type=Memory"), "HeapMemoryUsage"); cd = (CompositeData) memoryMbean; //get an instance of the OperatingSystem Mbean osMbean = jmxc.getMBeanServerConnection().getAttribute(new ObjectName("java.lang:type=OperatingSystem"),"ProcessCpuTime"); System.out.println("Used memory: " + " " + cd.get("used") + " Used cpu: " + osMbean); //print memory usage tempMemory = tempMemory + Long.parseLong(cd.get("used").toString()); Thread.sleep(1000); //delay for one second } //get system time and cpu time from last poll long cpuAfter = Long.parseLong(osMbean.toString()); long cpuDiff = cpuAfter - cpuBefore; //find cpu time between our first and last jmx poll System.out.println("Cpu diff in milli seconds: " + cpuDiff / 1000000); //print cpu time in miliseconds System.out.println("average memory usage is: " + tempMemory / samplesCount);//print average memory usage The above example prints: ... Used memory: 23376624 Used cpu: 38060000000 Used memory: 24020624 Used cpu: 38080000000 Used memory: 24621920 Used cpu: 38090000000 Cpu diff in milli seconds: 4230 average memory usage is: 28028204 When the JMX agent may not be enabled for a Java process, it is also possible to retrieve information by fetching the process by id, for example. The following code shows how to do this using Sigar API: //create a sigar object Sigar sigar = new Sigar(); for (int i = 0; i < 100; i++) { ProcessFinder find = new ProcessFinder(sigar); //get the list of current java processes, and optionally query the list to choose which process to monitor long[] pidList = sigar.getProcList(); //assuming we know the process id, we may query the process finder long pid = find.findSingleProcess("Pid.Pid.eq=54730"); //get memory info for the process id ProcMem memory = new ProcMem(); memory.gather(sigar, pid); //get cou info for the oricess id ProcCpu cpu = new ProcCpu(); cpu.gather(sigar, pid); //print the memory used by the process id System.out.println("Current memory used: " + Long.toString(memory.getSize())); //print all memory info System.out.println(memory.toMap()); //print all cpu info System.out.println(cpu.toMap()); Thread.sleep(1000); } This is displayed when running the above example: Current memory used: 3257659392 {Resident=258789376, PageFaults=109613, Size=3257659392} {User=34973, LastTime=1404467787774, Percent=0.0, StartTime=1404467383826, Total=38121, Sys=3148}

Get an overview of the Java observer pattern using an inventory example.

Get familiar with the Angular JS origin story.

in my last post on extjs 4 mvc, i have demonstrated the use of extjs 4 mvc to create a simple create-read-update-delete application using extjs only. today we will go to see how to use that extjs part for ui and use spring mvc to manage the books records on server side using spring. let first start by creating spring’s webmvc configurer class. i am using spring’s new xml-free approach and if you are new to it, i recommend you to please go through my previous post. package org.techzoo.springmvc.bootstrap; import org.springframework.context.annotation.bean; import org.springframework.context.annotation.componentscan; import org.springframework.context.annotation.configuration; import org.springframework.web.servlet.config.annotation.enablewebmvc; import org.springframework.web.servlet.config.annotation.resourcehandlerregistry; import org.springframework.web.servlet.config.annotation.viewcontrollerregistry; import org.springframework.web.servlet.config.annotation.webmvcconfigureradapter; import org.springframework.web.servlet.view.internalresourceviewresolver; @configuration @enablewebmvc @componentscan(basepackages = {"org.techzoo.springmvc"}) public class mywebmvcconfigurer extends webmvcconfigureradapter { @override public void addresourcehandlers(resourcehandlerregistry registry) { registry.addresourcehandler("/resources/**") .addresourcelocations("/resources/"); } @override public void addviewcontrollers(viewcontrollerregistry registry) { registry.addviewcontroller("/") .setviewname("index"); } @bean public internalresourceviewresolver viewresolver() { internalresourceviewresolver resolver = new internalresourceviewresolver(); resolver.setprefix("/web-inf/views/"); resolver.setsuffix(".jsp"); return resolver; } } create a book bean now, this bean is work as a value object and represent a single record in grid. package org.techzoo.springmvc.vo; /** * author: tousif khan */ public class book { private int id; private string title; private string author; private int price; private int qty; public book() {} public book(int id, string title, string author, int price, int qty) { super(); this.id = id; this.title = title; this.author = author; this.price = price; this.setqty(qty); } //getter-setters goes here... @override public string tostring() { return string.format("book [title = %s, author = %s]", title, author); } } now create a bookdao interface and implement it. package org.techzoo.springmvc.dao; import java.util.list; import org.springframework.stereotype.component; import org.techzoo.springmvc.vo.book; @component public interface bookdao { public boolean addbook(book book); public void updatebook(book book); public list listbooks(); public book getbookbyid(integer bookid); public boolean removebook(book book); } to make it more simple, i am using book list but you can extend it to save it relational table. see my springmvc-hibernate grud tutorial for more details. package org.techzoo.springmvc.dao; import java.util.arraylist; import java.util.list; import org.springframework.stereotype.repository; import org.techzoo.springmvc.vo.book; @repository public class bookdaoimpl implements bookdao { private static list books = new arraylist(); static { books.add(new book(1001, "jdbc, servlet and jsp", "santosh kumar", 300, 12000)); books.add(new book(1002, "head first java", "kathy sierra", 550, 2500)); books.add(new book(1003, "java scjp certification", "khalid mughal", 650, 5500)); books.add(new book(1004, "spring and hinernate", "santosh kumar", 350, 2500)); books.add(new book(1005, "mastering c++", "k. r. venugopal", 400, 1200)); } @override public boolean addbook(book book) { return books.add(book); } @override public boolean removebook(book book) { return books.remove(book); } @override public list listbooks() { return books; } @override public void updatebook(book book) { int index = books.indexof(book); if(index != -1) { books.add(index, book); } } @override public book getbookbyid(integer bookid) { book b = null; for(book b1 : books) { if(b1.getid() == bookid) { return b1; } } return b; } } now it’s time to create a book controller. package org.techzoo.springmvc.controller; import java.util.hashmap; import java.util.list; import java.util.map; import org.springframework.beans.factory.annotation.autowired; import org.springframework.stereotype.controller; import org.springframework.validation.objecterror; import org.springframework.web.bind.methodargumentnotvalidexception; import org.springframework.web.bind.annotation.exceptionhandler; import org.springframework.web.bind.annotation.requestbody; import org.springframework.web.bind.annotation.requestmapping; import org.springframework.web.bind.annotation.requestmethod; import org.springframework.web.bind.annotation.responsebody; import org.techzoo.springmvc.dao.bookdao; import org.techzoo.springmvc.vo.book; @controller @requestmapping("api") public class bookcontroller { bookdao bookbao; @autowired public bookcontroller(bookdao bookbao) { this.bookbao = bookbao; } @requestmapping(value = "/index") public string index() { return "index"; } @requestmapping (value = "book/save", method = requestmethod.post) @responsebody public boolean savebook(@requestbody book book) { return bookbao.addbook(book); } @responsebody @requestmapping (value = "book/loadbooks") public map> loadallbooks() { map> books = new hashmap>(); books.put("books", bookbao.listbooks()); return books; } @requestmapping (value = "book/delete", method = requestmethod.post) @responsebody public boolean deletebooks(@requestbody book book) { return bookbao.removebook(book); } @requestmapping (value = "book/updatebook", method = requestmethod.post) @responsebody public boolean updatebooks(@requestbody book book) { bookbao.updatebook(book); return true; } } extjs code looks similar to previous example with a little change. i have changes the local store to server ajax proxy and after each delete/add/update operation, we will load the all record from server to make grid update. ext.onready(function () { ext.define('techzoo.model.book', { extend: 'ext.data.model', fields: [ {name: 'id', type: 'int'}, {name: 'title', type: 'string'}, {name: 'author', type: 'string'}, {name: 'price', type: 'int'}, {name: 'qty', type: 'int'} ] }); ext.define('techzoo.store.books', { extend : 'ext.data.store', storeid: 'bookstore', model : 'techzoo.model.book', fields : ['id', 'title', 'author','price', 'qty'], proxy: { type: 'ajax', url: '/springmvc-extjs-crud/api/book/loadbooks', reader: { type: 'json', root: 'books' } }, autoload: true }); ext.define('techzoo.view.bookslist', { extend: 'ext.grid.panel', alias: 'widget.bookslist', title: 'books list - (extjs springmvc example - @ tousif khan)', store: 'books', initcomponent: function () { this.tbar = [{ text : 'add book', action : 'add', iconcls : 'book-add' }]; this.columns = [ { header: 'title', dataindex: 'title', flex: 1 }, { header: 'author', dataindex: 'author' }, { header: 'price', dataindex: 'price' , width: 60 }, { header: 'quantity', dataindex: 'qty', width: 80 }, { header: 'action', width: 50, renderer: function (v, m, r) { var id = ext.id(); ext.defer(function () { ext.widget('image', { renderto: id, name: 'delete', src : 'resources/images/book_delete.png', listeners : { afterrender: function (me) { me.getel().on('click', function() { var grid = ext.componentquery.query('bookslist')[0]; if (grid) { var sm = grid.getselectionmodel(); var rs = sm.getselection(); if (!rs.length) { ext.msg.alert('info', 'no book selected'); return; } ext.msg.confirm('remove book', 'are you sure you want to delete?', function (button) { if (button == 'yes') { //grid.store.remove(rs[0]); var book = rs[0].getdata(); ext.ajax.request({ url: '/springmvc-extjs-crud/api/book/delete', method : 'post', jsondata: book, success: function(response){ var grid = ext.componentquery.query('bookslist')[0]; grid.getstore().load(); } }); } }); } }); } } }); }, 50); return ext.string.format('', id); } } ]; this.callparent(arguments); } }); ext.define('techzoo.view.booksform', { extend : 'ext.window.window', alias : 'widget.booksform', title : 'add book', width : 350, layout : 'fit', resizable: false, closeaction: 'hide', modal : true, config : { recordindex : 0, action : '' }, items : [{ xtype : 'form', layout: 'anchor', bodystyle: { background: 'none', padding: '10px', border: '0' }, defaults: { xtype : 'textfield', anchor: '100%' }, items : [{ name : 'title', fieldlabel: 'book title' },{ name: 'author', fieldlabel: 'author name' },{ name: 'price', fieldlabel: 'price' },{ name: 'qty', fieldlabel: 'quantity' }] }], buttons: [{ text: 'ok', action: 'add' },{ text : 'reset', handler : function () { this.up('window').down('form').getform().reset(); } },{ text : 'cancel', handler: function () { this.up('window').close(); } }] }); ext.define('techzoo.controller.books', { extend : 'ext.app.controller', stores : ['books'], views : ['bookslist', 'booksform'], refs : [{ ref : 'formwindow', xtype : 'booksform', selector: 'booksform', autocreate: true }], init: function () { this.control({ 'bookslist > toolbar > button[action=add]': { click: this.showaddform }, 'bookslist': { itemdblclick: this.onrowdblclick }, 'booksform button[action=add]': { click: this.doaddbook } }); }, onrowdblclick: function(me, record, item, index) { var win = this.getformwindow(); win.settitle('edit book'); win.setaction('edit'); win.setrecordindex(index); win.down('form').getform().setvalues(record.getdata()); win.show(); }, showaddform: function () { var win = this.getformwindow(); win.settitle('add book'); win.setaction('add'); win.down('form').getform().reset(); win.show(); }, doaddbook: function () { var win = this.getformwindow(); var store = this.getbooksstore(); var values = win.down('form').getvalues(); var action = win.getaction(); // var book = ext.create('techzoo.model.book', values); var url = ''; if(action == 'edit') { url = '/springmvc-extjs-crud/api/book/updatebook'; } else { url = '/springmvc-extjs-crud/api/book/save'; } ext.ajax.request({ url: url, method : 'post', jsondata: values, success: function(response){ store.load(); } }); win.close(); } }); ext.application({ name : 'techzoo', controllers: ['books'], launch: function () { ext.widget('bookslist', { width : 500, height: 300, renderto: 'output' }); } } ); }); output: open the html file in any browser, the output will look similar to below. you can click to add new book record in grid.

I've begun work on trying to answer the questions I gathered concerning Cordova's FileSystem support. As I work through the questions I'm trying to build "real" samples to go along with the text. My first sample is a simple one, but I think it is pretty relevant for the types of things folks may do with Cordova and the file system - checking to see if a file exists locally and if not - fetching it. I'll begin by sharing the code and then explaining the parts. Here is the entire JavaScript file for the application. (Earlier today, Andrew Grieve shared a way my code could be simplified by a good 1/3rd. The code below reflects his update and has been changed since my original writing of the blog post.) document.addEventListener("deviceready", init, false); //The directory to store data var store; //Used for status updates var $status; //URL of our asset var assetURL = "https://raw.githubusercontent.com/cfjedimaster/Cordova-Examples/master/readme.md"; //File name of our important data file we didn't ship with the app var fileName = "mydatafile.txt"; function init() { $status = document.querySelector("#status"); $status.innerHTML = "Checking for data file."; store = cordova.file.dataDirectory; //Check for the file. window.resolveLocalFileSystemURL(store + fileName, appStart, downloadAsset); } function downloadAsset() { var fileTransfer = new FileTransfer(); console.log("About to start transfer"); fileTransfer.download(assetURL, store + fileName, function(entry) { console.log("Success!"); appStart(); }, function(err) { console.log("Error"); console.dir(err); }); } //I'm only called when the file exists or has been downloaded. function appStart() { $status.innerHTML = "App ready!"; } Ok, let's break it down. The first step is to check to see if our file exists already. The question is - where should we store the file? If you look at the docs for the FileSystem, you will see that the latest version of the plugin adds some useful aliases for common folders. Unfortunately, the docs are not exactly clear about how some of these aliases work. I asked for help (both on the PhoneGap Google group and the Cordova development list) and got some good responses from Kerri Shotts and Julio Sanchez. The directory that I thought made sense, cordova.file.applicationStorageDirectory, is incorrectly documented as being writeable in iOS. A pull request has already been filed to fix this mistake. For my application, the most appropriate directory is the next one, cordova.file.dataDirectory. Once I have my directory alias, I can make use of resolveLocalFileSystem on the directory plus desired file name to see if it exists. The third argument, downloadAsset, will only be run on an error, in this case a file not existing. If the file does not exist, we then have to download it. For this we use a second plugin, FileTransfer. This is where one more point of confusion comes in. We need to convert that earlier DirectoryEntry object, the one we used to get an API for files and directories, back to a URL so we can give a path to the Download API. So to recap - we've got a few moving parts here. We've got a directory alias, built into the plugin for easily finding common folders for our application. Again, the docs here are currently a bit wrong but they should be corrected soon. From that we can quickly see if our desired file exists, and if not, use the FileTransfer plugin to download it. Simple... but even a simple application caused me a bit of trouble, so hopefully this helps others. You can get the full source code here: https://github.com/cfjedimaster/Cordova-Examples/tree/master/checkanddownload

Learn about Identity, Sequence, and Table in Hibernate.

When I was learning programming, back in the days of Turbo Pascal, I managed to list files in directory using FindFirst,FindNext and FindClose functions. First I came up with a procedure printing contents of a given directory. You can imagine how proud I was to discover I can actually call that procedure from itself to traverse file system recursively. Well, I didn't know the term recursion back then, but it worked. Similar code in Java would look something like this: public void printFilesRecursively(final File folder) { for (final File entry : listFilesIn(folder)) { if (entry.isDirectory()) { printFilesRecursively(entry); } else { System.out.println(entry.getAbsolutePath()); } } } private File[] listFilesIn(File folder) { final File[] files = folder.listFiles(); return files != null ? files : new File[]{}; } Didn't know File.listFiles() can return null, did ya? That's how it signals I/O errors, like if IOException never existed. But that's not the point. System.out.println() is rarely what we need, thus this method is neither reusable nor composable. It is probably the best counterexample of Open/Closed principle. I can imagine several use cases for recursive traversal of file system: Getting a complete list of all files for display purposes Looking for all files matching given pattern/property (also check out File.list(FilenameFilter)) Searching for one particular file Processing every single file, e.g. sending it over network Every use case above has a unique set of challenges. For example we don't want to build a list of all files because it will take a significant amount of time and memory before we can start processing it. We would like to process files as they are discovered and lazily - by pipe-lining computation (but without clumsy visitor pattern). Also we want to short-circuit searching to avoid unnecessary I/O. Luckily in Java 8 some of these issues can be addressed with streams: final File home = new File(FileUtils.getUserDirectoryPath()); final Stream files = Files.list(home.toPath()); files.forEach(System.out::println); Remember that Files.list(Path) (new in Java 8) does not look into subdirectories - we'll fix that later. The most important lesson here is: Files.list() returns a Stream - a value that we can pass around, compose, map, filter, etc. It's extremely flexible, e.g. it's fairly simple to count how many files I have in a directory per extension: import org.apache.commons.io.FilenameUtils; //... final File home = new File(FileUtils.getUserDirectoryPath()); final Stream files = Files.list(home.toPath()); final Map> byExtension = files .filter(path -> !path.toFile().isDirectory()) .collect(groupingBy(path -> getExt(path))); byExtension. forEach((extension, matchingFiles) -> System.out.println( extension + "\t" + matchingFiles.size())); //... private String getExt(Path path) { return FilenameUtils.getExtension(path.toString()).toLowerCase(); } OK, just another API, you might say. But it becomes really interesting once we need to go deeper, recursively traversing subdirectories. One amazing feature of streams is that you can combine them with each other in various ways. Old Scala saying "flatMap that shit" is applicable here as well, check out this recursive Java 8 code: //WARNING: doesn't compile, yet: private static Stream filesInDir(Path dir) { return Files.list(dir) .flatMap(path -> path.toFile().isDirectory() ? filesInDir(path) : singletonList(path).stream()); } Stream lazily produced by filesInDir() contains all files within directory including subdirectories. You can use it as any other stream by calling map(), filter(), anyMatch(), findFirst(), etc. But how does it really work?flatMap() is similar to map() but while map() is a straightforward 1:1 transformation, flatMap() allows replacing single entry in input Stream with multiple entries. If we had used map(), we would have end up with Stream>(or maybe Stream>). But flatMap() flattens this structure, in a way exploding inner entries. Let's see a simple example. Imagine Files.list() returned two files and one directory. For files flatMap() receives a one-element stream with that file. We can't simply return that file, we have to wrap it, but essentially this is no-operation. It gets way more interesting for a directory. In that case we call filesInDir() recursively. As a result we get a stream of contents of that directory, which we inject into our outer stream. Code above is short, sweet and... doesn't compile. These pesky checked exceptions again. Here is a fixed code, wrapping checked exceptions for sanity: public static Stream filesInDir(Path dir) { return listFiles(dir) .flatMap(path -> path.toFile().isDirectory() ? filesInDir(path) : singletonList(path).stream()); } private static Stream listFiles(Path dir) { try { return Files.list(dir); } catch (IOException e) { throw Throwables.propagate(e); } } Unfortunately this quite elegant code is not lazy enough. flatMap() evaluates eagerly, thus it always traverses all subdirectories, even if we barely ask for first file. You can try with my tiny LazySeq library that tries to provide even lazier abstraction, similar to streams in Scala or lazy-seq in Clojure. But even standard JDK 8 solution might be really helpful and simplify your code significantly.