Recently a respected member of the Apache community tried Log4j 2 and wrote on Twitter: (Quote from Mark Struberg: @TheASF #log4j2 rocks big times! Performance is close to insane ^^ http://logging.apache.org/log4j/2.x/ ) It happened shortly after Remko Popma contributed something which is now called the “AsyncLoggers”. Some of you might know Log4j 2 has AsyncAppenders already. They are similar like the ones you can find in Log4j 1 and other logging frameworks. I am honest: I wasn’t so excited about the new feature until I read the tweet on its performance and became curious. Clearly Java logging has many goals. Among them: logging must be as fast as hell. Nobody wants his logging framework to become a bottleneck. Of course you’ll always have a cost when logging. There is some operation the CPU must perform. Something is happening, even when you decide NOT to write a log statement. Logging is expected to be invisible. Until now, the well-known logging frameworks were similar in speed. Benchmarks are unreliable after all. We have made some benchmarks over at Apache Logging. Sometimes one logging frameworks wins, sometimes the other. But at the end of the day you can say they are all very good and you can choose whatever your liking is. Until we got Remko’s contribution and Log4j 2 became “insanely fast”. Small software projects running one thread might not care about performance so much. When running a SaaS you simply don’t know when your app gets so much attraction that you need to scale. Then you suddenly need some extra power. With Log4j 2, running 64 threads might bring you twelve times more logging throughput than with comparable frameworks. We speak of more than 18,000,000 messages per second, while others do around 1,500,000 or less in the same environment. I saw the chart, but simply couldn’t believe it. There must be something wrong. I rechecked. I ran the tests myself. It’s like that: Log4j 2 is insanely fast. Async Performance, last read on July 19, 2013 As of now, we have a logging framework which performs lots better than every other logging framework out there. As of now we need to justify our decision when we do not want to use Log4j 2, if speed matters.Everything else than Log4j 2 can become a bottleneck and a risk. With such a fast logging framework you might even consider to log a bit more in production than you did before. Eventually I wrote Remko an e-mail and asked him what exactly the difference between the old AsyncAppenders and the new Asynchronous Loggers is. The difference between old AsynAppenders and new AsyncLoggers “The Asynchronous Loggers do two things differently than the AsyncAppender”, he told me, “they try to do the minimum amount of work before handing off the log message to another thread, and they use a different mechanism to pass information between the producer and consumer threads. AsyncAppender uses an ArrayBlockingQueue to hand off the messages to the thread that writes to disk, and Asynchronous Loggers use the LMAX Disruptor library. Especially the Disruptor has made a large performance difference.” In other terms, the AsyncAppender use a first-in-first-out Queue to work through messages. But the Async Logger uses something new – the Disruptor. To be honest, I had never heard of it. And furthermore, I never thought much about scaling my logging framework. When somebody said “scale the system”, I thought about the database, the app server and much more, but usually not logging. In production, logging was off. End of story. But Remko thinks about scaling when it comes to logging. “Looking at the performance test results for the Asynchronous Loggers, the first thing you notice is that some ways of logging scale much better than others. By scaling better I mean that you get more throughput when you add more threads. If your throughput increases a constant amount with every thread you add, you have linear scalability. This is very desirable but can be difficult to achieve.”, he wrote me. “Comparing synchronous to asynchronous, you would expect any asynchronous mechanism to scale much better than synchronous logging because you don’t do the I/O in the producing thread any more, and we all know that ‘I/O is slow’ (and I’ll get back to this in a bit)”. Yes, exactly my understanding. I thought it would be enough to send something to a queue, and something else would pick it up and write the message. The app would go on. This is exactly what the old AsyncAppender does, wrote Remko: “With AsyncAppender, all your application thread needs to do is create a LogEvent object and put it on the ArrayBlockingQueue; the consuming thread will then take these events off the queue and do all the time-consuming work. That is, the work of turning the event into bytes and writing these bytes to the I/O device. Since the application threads do not need to do the I/O, you would expect this to scale better, meaning adding threads will allow you to log more events.” If you believed that like me, take a seat and a deep breath. We were wrong. “What may surprise you is that this is not the case.”, he wrote. “If you look at the performance numbers for the AsyncAppenders of all logging frameworks, you’ll see that every time you double the number of threads, your throughput per thread roughly halves.” “So your total throughput remains more or less flat! AsyncAppenders are faster than synchronous logging, but they are similar in the sense that neither of them gives you more total throughput when you add more threads.”, he told me. It hit me like a hammer. Basically instead of making your logging faster with adding more threads you made basically: nothing. After all Appenders didn’t scale until now. I asked Remko why this was the case. “It turns out that queues are not the most optimal data structure to pass information between threads. The concurrent queues that are part of the standard Java libraries use locks to make sure that values don’t get corrupted and to ensure data visibility between threads.”. LMAX Disruptor? “The LMAX team did a lot of research on this and found that these queues have a lot of lock contention. An interesting thing they found is that queues are always either full or empty: If your producer is faster, your queue will be full most of the time (and that may be a problem in itself ). If your consumer is fast enough, your queue will be empty most of the time. Either way, you will have contention on the head or on the tail of the queue, where both the producer and the consumer thread want to update the same field. To resolve this, the LMAX team came up with the Disruptor library, which is a lock-free data structure for passing messages between threads. Here is a performance comparison between the Disruptor and ArrayBlockingQueue:Performance Comparison.” Wow. After all these years of Java programming I actually felt a bit like a Junior programmer again. I missed the LMAX disruptor and even never considered it a performance problem to use the Queue. I wonder what other performance problems I did not discover so far. I realized, I had to re-learn Java. I asked Remko how he could find a library like the LMAX disruptor. I mean nobody writes software, creates an instance of a Queue-class, doubts its performance and finally searches the internet for “something better”. Or are there really people of that kind? “How I found about the Disruptor? The short answer is, it was all a mistake.”, he started. “Okay, perhaps that was a bit too short, so here is the longer answer: a colleague of mine wrote a small logger, essentially adding a time-stamped log message to a queue, with a background thread that took these strings off the queue and wrote them to disk. He did this because he needed better performance than what he could get with log4j-1.x. I did some testing and found it was faster, I don’t remember exactly by how much. I was quite surprised because I had been using log4j for years and had never thought it would be easily outperformed. Until then I had assumed that the well-known libraries would be fast, because, well… To be honest, I had just assumed. So this was a bit of an eye-opener for me. However, the custom logger was a bit bare-bones in terms of functionality so I started to look around for alternatives.” “Before I start talking about the Disruptor, I have to confess something. I recently went back to see how much faster the custom logger was than log4j-1.x, but when I measured it it was actually slower! It turned out that I had been comparing the custom logger to an old beta of log4j-2.0, I think beta3 or beta4. AsyncAppender in those betas still had a performance issue (LOG4J2-153 if you’re curious). If I had compared the custom logger to the AsyncAppender in log4j-1.x, I would have found that log4j-1.x was faster and I would not have thought about it further. But because I made this mistake I started to look for other high-performance logging libraries that were richer in functionality. I did not find such a logging library, but I ran into a whole bunch of other interesting stuff, including the Disruptor. Eventually I decided to try to combine Log4j-2, which has a very nice code base, with the Disruptor. The result of this was eventually accepted into Log4j-2 itself, and the rest, as they say, was history.” “One thing I came across that I should mention here is Peter Lawrey’sChronicle library. Chronicle uses memory-mapped files to write tens of millions of messages per second to disk with very low latency. Remember that above I said that “we all know that I/O is slow”? Chronicle shows that synchronous I/O can be very, very fast.“. “It was via Peter’s work that I came across the Disruptor. There is a lot of good material out there about the Disruptor. Just to give you a few pointers: Martin Fowler: LMAX Trisha Lee on LMAX under the hood (slightly outdated now but the most detailed material I know of) …and video presentations like this The Disruptor google group is also highly recommended. Recommended readings on Java performance in general are: Martin Thompson’s “Mechanical Sympathy” Martin Thompson Presentations. Martin Thompson has done a number of articles and presentations on various aspects of high performance computing in java. He does a great job of making the complex stuff that is going on under the hood accessible.” My bookmarks folder went full after reading this e-mail, and I appreciate the lots of starting points for improving my knowledge on Java performance. Should I use AsyncLoggers by default? I was sure I want to use the new Async Loggers. This all sounds just fantastic. But on the other hand, I am a bit scared and even a little paranoid to include new dependencies or new technologies like the new Log4j 2 Async Loggers. I asked Remko if he would use the new feature by default or if he would enable them just for a few, limited use cases. “I use Async Loggers by default, yes.”, he wrote me. “One use case when you would _not_ want to use asynchronous logging is when you use logging for audit purposes. In that case a logging error is a problem that your application needs to know about and deal with. I believe that most applications are different, in that they don’t care too much about logging errors. Most applications don’t want to stop if a logging exception occurs, in fact, they don’t even want to know about it. By default, appenders in Log4j-2.0 will suppress exceptions so the application doesn’t need to try/catch every log statement. If that is your usage, then you will not lose anything by using asynchronous loggers, so you get only the benefits, which is improved performance.” “One nice little detail I should mention is that both Async Loggers and Async Appenders fix something that has always bothered me in Log4j-1.x, which is that they will flush the buffer after logging the last event in the queue. With Log4j-1.x, if you used buffered I/O, you often could not see the last few log events, as they were still stuck in the memory buffer. Your only option was setting immediateFlush to true, which forces disk I/O on every single log event and has a performance impact. With Async Loggers and Appenders in Log4j-2.0 your log statements are all flushed to disk, so they are always visible, but this happens in a very efficient manner.” Isn’t it risky to log to use Log4js AsyncLoggers? But considering that Log4j-1 had serious threading issues and the modern world uses cloud computing and clustering all the time to scale their apps,isn’t asynchronous logging some kind of additional risk? Or is it safe? I knew my questions would sound like the questions of a decision maker, not of an developer. But the whole LMAX thing was so new to me and since I maintain the old and really ugly Log4j 1 code, I simply had to ask. Remko: “There are a number of questions in there. First, is Log4j-2 safer from a concurrency perspective than Log4j-1.x? I believe so. The Log4j-2 team has put in considerable effort to support multi-threaded applications, and the asynchronous loggers are just a very recent and relatively small addition to the project. Log4j-2 uses more granular locking than log4j-1.x, and is architecturally simpler, which should result in fewer issues, and any issues that do come up will be easier to fix.” “On the other hand, Log4j-2 is still in beta and is under active development, although recently I think most effort is being spent on fixing things and tying up loose ends rather than adding new features. I believe it is stable enough for production use. If you are considering using Log4j-2, for performance or other reasons, I’d suggest you do your due diligence and test, just like you would before adopting any other 3rd party library in your project.” (Sidenote: A stable version of Log4j2 can be expected soon, most likely autumn 2013). Sounded good to me. And yes, I can perfectly agree with that from my own observations on the project, though I personally did not write code in the Log4j 2 repository. “The other question I see is: Is asynchronous logging riskier than synchronous logging? I don’t think so, in fact, if your application is multi threaded the opposite may be the case: once the log event has been handed off to the consumer thread that does the I/O, there is only that one thread dealing with the layouts, appenders and all the other logging-related components. So after the hand-off you’re single-threaded and you don’t need to worry about any threading issues like deadlock and liveliness etc any more.” “You can take this one step further and make your business logic completely single-threaded, using the disruptor for all I/O or communication with external systems. Single-threaded business logic without lock contention can be blazingly fast. The results at LMAX (6 million transactions/sec, with less than 10 ms latency) speak for themselves.” Reading Remko’s message I learned three things. First, I had to learn more about Java performance. Second, I definitely want to make my applications use Log4j 2. As first step, I will enable it in my Struts 2 apps, which I use often. Third, a web application framework using the LMAX Disruptor might blow us all away. I would like to give a big thank you and a hug to Remko Popma for answering my questions and working on this blog post with me. All errors are my own.

In my previous post quite sometime back I had written about Internal DSLs using Java. In the book Domain Specific Languages by Martin Fowler, he discusses about another type of DSL called external DSLs in which the DSL is written in another language which is then parsed by the host language to populate the semantic model. In the previous example I was discussing about creating a DSL for defining a graph. The advantage of using an external dsl is that any change in the graph data would not require recompilation of the program instead the program can just load the external dsl, create a parse tree and then populate the semantic model. The semantic model will remain the same and the advantage of using the semantic model is that one can make modification to the DSL without making much changes to the semantic model. In the example between Internal DSLs and external DSLs I have not modified the semantic model. To create an external DSL I am making use of ANTLR. What is ANTLR? The definition as given on the official site is: ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It’s widely used to build languages, tools, and frameworks. From a grammar, ANTLR generates a parser that can build and walk parse trees. The notable features of ANTLR from the above definition are: parser generator for structured text or binary files can build and walk parse trees Semantic Model In this example I will exploit the above features of ANTLR to parse a DSL and then walk through the parse tree to populate the Semantic model. To recap, the semantic model consists of Graph, Edge and Vertex classes which represent a Graph and an Edge and a Vertex of the Graph respectively. The below code shows the class definitions: public class Graph { private List edges; private Set vertices; public Graph() { edges = new ArrayList<>(); vertices = new TreeSet<>(); } public void addEdge(Edge edge){ getEdges().add(edge); getVertices().add(edge.getFromVertex()); getVertices().add(edge.getToVertex()); } public void addVertice(Vertex v){ getVertices().add(v); } public List getEdges() { return edges; } public Set getVertices() { return vertices; } public static void printGraph(Graph g){ System.out.println("Vertices..."); for (Vertex v : g.getVertices()) { System.out.print(v.getLabel() + " "); } System.out.println(""); System.out.println("Edges..."); for (Edge e : g.getEdges()) { System.out.println(e); } } } public class Edge { private Vertex fromVertex; private Vertex toVertex; private Double weight; public Edge() { } public Edge(Vertex fromVertex, Vertex toVertex, Double weight) { this.fromVertex = fromVertex; this.toVertex = toVertex; this.weight = weight; } @Override public String toString() { return fromVertex.getLabel() + " to " + toVertex.getLabel() + " with weight " + getWeight(); } public Vertex getFromVertex() { return fromVertex; } public void setFromVertex(Vertex fromVertex) { this.fromVertex = fromVertex; } public Vertex getToVertex() { return toVertex; } public void setToVertex(Vertex toVertex) { this.toVertex = toVertex; } public Double getWeight() { return weight; } public void setWeight(Double weight) { this.weight = weight; } } public class Vertex implements Comparable { private String label; public Vertex(String label) { this.label = label.toUpperCase(); } @Override public int compareTo(Vertex o) { return (this.getLabel().compareTo(o.getLabel())); } public String getLabel() { return label; } public void setLabel(String label) { this.label = label; } } Creating the DSL Lets come up with the structure of the language before going into creating grammar rules. The structure which I am planning to come up is something like: Graph { A -> B (10) B -> C (20) D -> E (30) } Each line in the Graph block represents an edge and the vertices involved in the edge and the value in the braces represent the weight of the edge. One limitation which I am enforcing is that the Graph cannot have dangling vertices i.e vertices which are not part of any edge. This limitation can be removed by slightly changing the grammar, but I would leave that as an exercise for the readers of this post. The first task in creating the DSL is to define the grammar rules. These are the rules which your lexer and parser will use to convert the DSL into a Abstract Syntax tree/parse tree. ANTLR then makes use of this grammar to generate the Parser, Lexer and a Listener which are nothing but java classes extending/implementing some classes from the ANTLR library. The creators of the DSL must make use of these java classes to load the external DSL, parse it and then using the listener populate the semantic model as and when the parser encounters certain nodes (think of this as a variant of SAX parser for XML) Now that we know in very brief what ANTLR can do and the steps in using ANTLR, we would have to setup ANTLR i.e download ANTLR API jar and setup up some scripts for generating the parser and lexer and then trying out the language via the command line tool. For that please visit this official tutorial from ANTLR which shows how to setup ANTLR and a simple Hello World example. Grammar for the DSL Now that you have ANTLR setup let me dive into the grammar for my DSL: grammar Graph; graph: 'Graph {' edge+ '}'; vertex: ID; edge: vertex '->' vertex '(' NUM ')' ; ID: [a-zA-Z]+; NUM: [0-9]+; WS: [ \t\r\n]+ -> skip; Lets go rule: graph: 'Graph {' edge+ '}'; The above grammar rule which is the start rule says that the language should start with ‘Graph {‘ and end with ‘}’ and has to contain at lease one edge or more than one edge. vertex: ID; edge: vertex '->' vertex '(' NUM ')' ; ID: [a-zA-Z]+; NUM: [0-9]+; The above four rules say that a vertex should have atleast one character or more than one character. And an edge is defined as collection of two vertices separated by a ‘->’ and with the some digits in the ‘()’. I have named the grammar language as “Graph” and hence once we use ANTLR to generate the java classes i.e parser and lexer we will end up seeing the following classes: GraphParser, GraphLexer, GraphListener and GraphBaseListener. The first two classes deal with the generation of parse tree and the last two classes deal with the parse tree walk through. GraphListener is an interface which contains all the methods for dealing with the parse tree i.e dealing with events such as entering a rule, exiting a rule, visiting a terminal node and in addition to these it contains methods for dealing with events related to entering the graph rule, entering the edge rule and entering the vertex rule. We will be making use of these methods to intercept the data present in the dsl and then populate the semantic model. Populating the semantic model I have created a file graph.gr in the resource package which contains the DSL for populating the graph. As the files in the resource package are available to the ClassLoader at runtime, we can use the ClassLoader to read the DSL script and then pass it on to the Lexer and parser classes. The DSL script used is: Graph { A -> B (10) B -> C (20) D -> E (30) A -> E (12) B -> D (8) } And the code which loads the DSL and populates the semantic model: //Please resolve the imports for the classes used. public class GraphDslAntlrSample { public static void main(String[] args) throws IOException { //Reading the DSL script InputStream is = ClassLoader.getSystemResourceAsStream("resources/graph.gr"); //Loading the DSL script into the ANTLR stream. CharStream cs = new ANTLRInputStream(is); //Passing the input to the lexer to create tokens GraphLexer lexer = new GraphLexer(cs); CommonTokenStream tokens = new CommonTokenStream(lexer); //Passing the tokens to the parser to create the parse trea. GraphParser parser = new GraphParser(tokens); //Semantic model to be populated Graph g = new Graph(); //Adding the listener to facilitate walking through parse tree. parser.addParseListener(new MyGraphBaseListener(g)); //invoking the parser. parser.graph(); Graph.printGraph(g); } } /** * Listener used for walking through the parse tree. */ class MyGraphBaseListener extends GraphBaseListener { Graph g; public MyGraphBaseListener(Graph g) { this.g = g; } @Override public void exitEdge(GraphParser.EdgeContext ctx) { //Once the edge rule is exited the data required for the edge i.e //vertices and the weight would be available in the EdgeContext //and the same can be used to populate the semantic model Vertex fromVertex = new Vertex(ctx.vertex(0).ID().getText()); Vertex toVertex = new Vertex(ctx.vertex(1).ID().getText()); double weight = Double.parseDouble(ctx.NUM().getText()); Edge e = new Edge(fromVertex, toVertex, weight); g.addEdge(e); } } And the output when the above would be executed would be: Vertices... A B C D E Edges... A to B with weight 10.0 B to C with weight 20.0 D to E with weight 30.0 A to E with weight 12.0 B to D with weight 8.0 To summarize, this post creates a external DSL for populating the data for graphs by making use of ANTLR. I will enhance this simple DSL and expose it as an utility which can be used by programmers working on graphs. The post is very heavy on concepts and code, feel free to drop in any queries you have so that I can try to address them for benefit of others as well.

An overview starting with some basic classes of the Java 8 package: Instant, LocalDate, LocalTime, and LocalDateTime.

Fake system clock is a design pattern addressing testability issues of programs heavily relying on system time. If business logic flow depends on current system time, testing various flows becomes cumbersome or even impossible. Examples of such problematic scenarios include: certain business flow runs only (or is ignored) during weekends some logic is triggered only after an hour since some other event when two events occur at the exact same time (typically 1 ms precision), something should happen … Each scenario above poses unique set of challenges. Taken literally our unit tests would have to run only on specific day (1) or sleep for an hour to observe some behaviour. Scenario (3) might even be impossible to test under some circumstances since system clock can tick 1 millisecond at any time, thus making test unreliable. Fake system clock addresses these issues by abstracting system time over simple interface. Essentially you never call new Date(), new GregorianCalendar() or System.currentTimeMillis() but always rely on this: import org.joda.time.{DateTime, Instant} trait Clock { def now(): Instant def dateNow(): DateTime } As you can see I am depending on Joda Time library. Since we are already in the Scala land, one might consider scala-timeor nscala-time wrappers. Moreover the abstract name Clock is not a coincidence. It’s short and descriptive, but more importantly it mimics java.time.Clock class from Java 8 - that happens to address the same problem discussed here at the JDK level! But since Java 8 is still not here, let’s stay with our sweet and small abstraction. The standard implementation that you would normally use simply delegates to system time: import org.joda.time.{Instant, DateTime} object SystemClock extends Clock { def now() = Instant.now() def dateNow() = DateTime.now() } For the purposes of unit testing we will develop other implementations, but first let’s focus on usage scenarios. In a typical Spring/JavaEE applications fake system clock can be turned into a dependency that the container can easily inject. This makes dependence on system time explicit and manageable, especially in tests: @Controller class FooController @Autowired() (fooService: FooService, clock: Clock) { def postFoo(name: String) = fooService store new Foo(name, clock) } Here I am using Spring constructor injection asking the container to provide some Clock implementation. Of course in this case SystemClock is marked as @Service. In unit tests I can pass fake implementation and in integration tests I can place another, @Primary bean in the context, shadowing the SystemClock. This works great, but becomes painful for certain types of objects, namely entity/DTO beans and utility (static) classes. These are typically not managed by Spring so it can’t inject Clock bean to them. This forces us to pass Clock manually from the last “managed” layer: class Foo(fooName: String, clock: Clock) { val name = fooName val time = clock.dateNow() } similarly: object TimeUtil { def firstFridayOfNextMonth(clock: Clock) = //... } It’s not bad from design perspective. Both Foo constructor and firstFridayOfNextMonth() method do rely on system time so let’s make it explicit. On the other hand Clock dependency must be dragged, sometimes through many layers, just so that it can be used in one single method somewhere. Again, this is not bad per se. If your high level method has Clockparameter you know from the beginning that it relies on current time. But still is seems like a lot of boilerplate and overhead for little gain. Luckily Scala can help us here with: implicit parameters Let us refactor our solution a little bit so that Clock is an implicit parameter: @Controller class FooController(fooService: FooService) { def postFoo(name: String)(implicit clock: Clock) = fooService store new Foo(name) } @Service class FooService(fooRepository: FooRepository) { def store(foo: Foo)(implicit clock: Clock) = fooRepository storeInFuture foo } @Repository class FooRepository { def storeInFuture(foo: Foo)(implicit clock: Clock) = { val friday = TimeUtil.firstFridayOfNextMonth() //... } } object TimeUtil { def firstFridayOfNextMonth()(implicit clock: Clock) = //... } Notice how we call fooRepository storeInFuture foo ignoring second clock parameter. However this alone is not enough. We still have to provide some Clock instance as second parameter, otherwise compilation error strikes: could not find implicit value for parameter clock: com.blogspot.nurkiewicz.foo.Clock controller.postFoo("Abc") ^ not enough arguments for method postFoo: (implicit clock: com.blogspot.nurkiewicz.foo.Clock)Unit. Unspecified value parameter clock. controller.postFoo("Abc") ^ The compiler tried to find implicit value for Clock parameter but failed. However we are really close, the simplest solution is to use package object: package com.blogspot.nurkiewicz.foo package object foo { implicit val clock = SystemClock } Where SystemClock was defined earlier. Here is what happens: every time I call a function with implicit clock: Clock parameter inside com.blogspot.nurkiewicz.foo package, the compiler will discover foo.clock implicit variable and pass it transparently. In other words the following code snippets are equivalent but the second one provides explicit Clock, thus ignoring implicits: TimeUtil.firstFridayOfNextMonth() TimeUtil.firstFridayOfNextMonth()(SystemClock) also equivalent (first form is turned into the second by the compiler): fooService.store(foo) fooService.store(foo)(SystemClock) Interestingly in the bytecode level, implicit parameters aren’t any different from normal parameters so if you want to call such method from Java, passing Clock instance is mandatory and explicit. implicit clock parameter seems to work quite well. It hides ubiquitous dependency while still giving possibility to override it. For example in: fooService.store(foo) fooService.store(foo)(SystemClock) Tests The whole point of abstracting system time was to enable unit testing by gaining full control over time flow. Let us begin with a simple fake system clock implementation that always returns the same, specified time: class FakeClock(fixed: DateTime) extends Clock { def now() = fixed.toInstant def dateNow() = fixed } Of course you are free to put any logic here: advancing time by arbitrary value, speeding it up, etc. You get the idea. Now remember, the reason for implicit parameter was to hide Clock from normal production code while still being able to supply alternative implementation. There are two approaches: either pass FakeClock explicitly in tests: val fakeClock = new FakeClock( new DateTime(2013, 7, 15, 0, 0, DateTimeZone.UTC)) controller.postFoo("Abc")(fakeClock) or make it implicit but more specific to the compiler resolution mechanism: implicit val fakeClock = new FakeClock( new DateTime(2013, 7, 15, 0, 0, DateTimeZone.UTC)) controller.postFoo("Abc") The latter approach is easier to maintain as you don’t have to remember about passing fakeClock to method under test all the time. Of course fakeClock can be defined more globally as a field or even inside test package object. No matter which technique of providing fakeClock we choose, it will be used throughout all calls to service, repository and utilities. The moment we given explicit value to this parameter, implicit parameter resolution is ignored. Problems and summary Solution above to testing systems heavily dependant on time is not free from issues on its own. First of all the implicitClock parameter must be propagated throughout all the layers up to the client code. Notice that Clock is only needed in repository/utility layer while we had to drag it up to the controller layer. It’s not a big deal since the compiler will fill it in for us, but sooner or later most of our methods will include this extra parameter. Also Java and frameworks working on top of our code are not aware of Scala implicit resolution happening at compile time. Therefore e.g. our Spring MVC controller will not work as Spring is not aware of SystemClock implicit variable. It can be worked around though with WebArgumentResolver. Fake system clock pattern in general works only when used consistently. If you have even one place when real time is used directly as opposed to Clock abstraction, good luck in finding test failure reason. This applies equally to libraries and SQL queries. Thus if you are designing a library relying on current time, consider providing pluggable Clock abstraction so that client code can supply custom implementation like FakeClock. In SQL, on the other hand, do not rely on functions likeNOW() but always explicitly provide dates from your code (and thus from custom Clock).

I previously wrote a blog post describing how I’ve been trying to learn more about network sockets in which I created some server sockets and connected to them using netcat. The next step was to do the same thing in Java and I started out by writing a server socket which echoed any messages sent by the client: public class EchoServer { public static void main(String[] args) throws IOException { int port = 4444; ServerSocket serverSocket = new ServerSocket(port, 50, InetAddress.getByAddress(new byte[] {0x7f,0x00,0x00,0x01})); System.err.println("Started server on port " + port); while (true) { Socket clientSocket = serverSocket.accept(); System.err.println("Accepted connection from client: " + clientSocket.getRemoteSocketAddress() ); In in = new In (clientSocket); Out out = new Out(clientSocket); String s; while ((s = in.readLine()) != null) { out.println(s); } System.err.println("Closing connection with client: " + clientSocket.getInetAddress()); out.close(); in.close(); clientSocket.close(); } } } public final class In { private Scanner scanner; public In(java.net.Socket socket) { try { InputStream is = socket.getInputStream(); scanner = new Scanner(new BufferedInputStream(is), "UTF-8"); } catch (IOException ioe) { System.err.println("Could not open " + socket); } } public String readLine() { String line; try { line = scanner.nextLine(); } catch (Exception e) { line = null; } return line; } public void close() { scanner.close(); } } public class Out { private PrintWriter out; public Out(Socket socket) { try { out = new PrintWriter(socket.getOutputStream(), true); } catch (IOException ioe) { ioe.printStackTrace(); } } public void close() { out.close(); } public void println(Object x) { out.println(x); out.flush(); } } I ran the main method of the class and this creates a server socket on port 4444 listening on the 127.0.0.1 interface and we can connect to it using netcat like so: $ nc -v 127.0.0.1 4444 Connection to 127.0.0.1 4444 port [tcp/krb524] succeeded! hello hello The output in my IntelliJ console looked like this: Started server on port 4444 Accepted connection from client: /127.0.0.1:63222 Closing connection with client: /127.0.0.1 Using netcat is fine but what I actually wanted to do was write some test code which would check that I’d made sure the server socket on port 4444 was accessible via all interfaces i.e. bound to 0.0.0.0. There are actually some quite nice classes in Java which make this very easy to do and wiring those together I ended up with the following client code: public static void main(String[] args) throws IOException { Enumeration nets = NetworkInterface.getNetworkInterfaces(); for (NetworkInterface networkInterface : Collections.list(nets)) { for (InetAddress inetAddress : Collections.list(networkInterface.getInetAddresses())) { Socket socket = null; try { socket = new Socket(inetAddress, 4444); System.out.println(String.format("Connected using %s [%s]", networkInterface.getDisplayName(), inetAddress)); } catch (ConnectException ex) { System.out.println(String.format("Failed to connect using %s [%s]", networkInterface.getDisplayName(), inetAddress)); } finally { if (socket != null) { socket.close(); } } } } } } If we run the main method of that class we’ll see the following output (on my machine at least!): Failed to connect using en0 [/fe80:0:0:0:9afe:94ff:fe4f:ee50%4] Failed to connect using en0 [/192.168.1.89] Failed to connect using lo0 [/0:0:0:0:0:0:0:1] Failed to connect using lo0 [/fe80:0:0:0:0:0:0:1%1] Connected using lo0 [/127.0.0.1] Interestingly we can’t even connect via the loopback interface using IPv6 which is perhaps not that surprising in retrospect given we bound using an IPv4 address. If we tweak the second line of EchoServer from: ServerSocket serverSocket = new ServerSocket(port, 50, InetAddress.getByAddress(new byte[] {0x7f,0x00,0x00,0x01})); to ServerSocket serverSocket = new ServerSocket(port, 50, InetAddress.getByAddress(new byte[] {0x00,0x00,0x00,0x00})); And restart the server before re-running the client we can now connect through all interfaces: Connected using en0 [/fe80:0:0:0:9afe:94ff:fe4f:ee50%4] Connected using en0 [/192.168.1.89] Connected using lo0 [/0:0:0:0:0:0:0:1] Connected using lo0 [/fe80:0:0:0:0:0:0:1%1] Connected using lo0 [/127.0.0.1] We can then wrap the EchoClient code into our testing framework to assert that we can connect via all the interfaces.

There are quite a few configuration libraries available in Java, such as this one available from Apache Commons, and they usually follow a very similar pattern: they parse a variety of configuration files and in the end, give you a Property or Map like structure where you can query your values: Double double = config.getDouble("number"); Integer integer = config.getInteger("number"); I have always been unsatisfied with this approach for a couple of reasons: A lot of boiler plate to retrieve these parameters. Having to share the whole configuration object even if I only need one parameter from it. It’s very easy to misspell a property and received incorrect values. A while ago, I was reading the Guice documentation and I came across a paragraph that made me realize that maybe, we could do better. Here is the relevant excerpt: Guice supports binding annotations that have attribute values. In the rare case that you need such an annotation: Create the annotation @interface. Create a class that implements the annotation interface. Follow the guidelines for equals() and hashCode() specified in the Annotation Javadoc. Pass an instance of this to the annotatedWith() binding clause. I thought that using this technique might be exactly what I needed to create a smarter configuration framework, even though I had different plans than using this trick with the annotatedWith method, as suggested by this paragraph. The relevance of this snippet will become clear later, so let’s start with the goals. Objectives I want to: Be able to inject individual configuration values anyhere in my code base and I want this to be type safe. No @Named or other string-based lookup. Have a canonical list of all the properties available to the application, with their full type, default value, documentation and leaving the door open for improvements (e.g. is this option mandatory or optional, detecting when some properties are not used anywhere, deprecation, aliasing, etc…). I don’t care much about the front end: how these properties get gathered is not relevant to this framework, they can come from XML, JSON, the network, a database, and they can have arbitrarily complex resolution and overriding rules, let’s save this for a future post. The input of this framework is a Map of properties and I take it from there. By the time we’re done, we will be able to do something like this: # Some property file host=foo.com port=1234 Using these configuration values in your code: public class A { @Inject @Prop(Property.HOST) private String host; @Inject @Prop(Property.PORT) private Integer port; // ... } Implementation The definition of the Prop annotation is trivial: @Retention(RUNTIME) @Target({ ElementType.FIELD, ElementType.PARAMETER }) @BindingAnnotation public @interface Prop { Property value(); } Property is an enum that captures all the information necessary for all your properties. In our case: public enum Property { HOST("host", "The host name", new TypeLiteral() {}, "foo.com"), PORT("port", "The port", new TypeLiteral() {}, 1234); } This enum contains the string name of the property, a description, its default value and its type. Note that this type is a TypeLiteral, so we can even offer properties that have generic types that would otherwise be erased, a trick that comes in handy to inject caches or other generic collections. Obviously, you can have additional parameters as you see fit (e.g. “boolean deprecated“). The next step is to tie all the properties that we parsed as input — we’ll use a Map called "allProps"— into our module so that Guice knows how to inject them. In order to do this, we iterate all these properties and bind them to their own provider. Because we are using typed names, note the use of Key.get from the Guice API, which lets us specifically target each property with a specific annotation: for (Property prop : Property.values()) { Object value = PropertyConverters.getValue(prop.getType(), prop, allProps.asMap()); binder.bind(Key.get(prop.getType(), new PropImpl(prop))) .toProvider(new PropertyProvider(prop, value)); } There are three classes in this piece of code that I haven’t explained yet. The first one isPropertyConverters, which simply reads the string version of the property and converts it to a Java type. The second one is PropertyProvider, a trivial Guice provider: public class PropertyProvider implements Provider { private final T value; private final Property property; public PropertyProvider(Property property, T value) { this.property = property; this.value = value; } @Override public T get() { return value; } } PropImpl is more tricky and also the one thing that has always prevented me from implementing such a framework, until I came across this obscure tidbit of the Guice documentation quoted above. In order to understand the necessity of its existence, we need to understand how Guice’s Key.get() works. Guice uses this class to translate a type into a unique key that it can use to inject the correct value. The important part here is to notice that not only does this method work with both Class andTypeLiteral (which we are using), but it can also be given a specific annotation. This annotation can be @Named, which I’m not a big fan of because it’s a string, so susceptible to typos, or a real annotation, which is what we want. However, annotations are special beasts in Java and you can’t get an instance of them just like that. This is where the trick mentioned at the top of this article comes into play: Java actually allows you to implement an annotation with a regular class. The implementation turns out to be fairly trivial, the difficulty was realizing that this was possible at all. Now that we have all this in place, let’s back track and dissect how the magic happens: @Inject @Prop(Property.HOST) private String host; When Guice encounters this injection point, it looks into its binders and it finds multiple bindings forStrings. However, because they have all been bound with a Key, the key is actually a pair: (String, a Prop). In this case, it will look up the pair String, Property.HOST and it will find a provider there. This provider was instantiated with the value found in the property file, so it knows what value to return. Generalizing Once I had the basic logic in place, I wondered if I could turn this mini framework into a library so that others could use it. The only missing piece would be to allow the specification of a more general Propannotation. In the example above, this annotation has a value of type Property, which is specific to my application: @Retention(RUNTIME) @Target({ ElementType.FIELD, ElementType.PARAMETER }) @BindingAnnotation public @interface Prop { Property value(); } In order to make this more general, I need to make this attribute return an enum instead of my own: @Retention(RUNTIME) @Target({ ElementType.FIELD, ElementType.PARAMETER }) @BindingAnnotation public @interface Prop { Enum value(); } Unfortunately, this is not legal Java, because according to the JLS section 8.9, Enum and its generic variants are not enum types, something that Josh Bloch confirmed, to my consternation. Therefore, this cannot be turned into a library, so if you are interested in using it in your project, you will have to copy the source and make a few modifications to adjust it to your needs, starting by havingProp#value have the type of the enum that captures your configuration. You can find a small proof of concept here, which I hope you’ll find useful. Note: this is a copy of the article I posted on our work blog.

As you know, there are many frameworks for mobile app development and a growing number of them are based on HTML5. These next-generation tools help developers create mobile apps for phones and tablets without the steep learning curve associated with native SDKs and other programming languages like Objective-C or Java. For countless developers throughout the world, HTML5 represents the future for cross-platform mobile app development. But the question is why? Why has HTML5 become so popular? The widespread adoption of HTML5 involves the emergence of the bring your own device (BYOD) movement. BYOD means that developers can no longer limit application usage to a single platform because consumers want their apps to run on the devices they use every day. HTML5 allows developers to target multiple devices with a single codebase and deliver experiences that closely emulate native solutions, without writing the same application multiple times, using multiple languages or SDKs. The evolution of modern web browsers means that HTML5 can deliver cross-platform, multi-device solutions that mirror behaviors and experiences of “native” apps to the point that it’s often difficult to distinguish between an app written using native development tools and those using HTML. Multiple platform support, time to market and lower maintenance costs are just a few of the advantages inherent in HTML/JavaScript. It’s advantages don’t stop there. HTML’s ability to mitigate long-term risks associated with emerging technologies such as WinRT, ChromeOS, FirefoxOS, and Tizen is unmatched. Simply said, the only code that will work on all these platforms is HTML/JavaScript. Is there a price? Yes, certainly native app consumes less memory and will have faster or more responsive user experience. But in all cases where a web app would work, you can make a step further and create a mobile web app or even packaged application for the store, for multiple platforms, from single codebase. PhoneJS lets you get started fast. PhoneJS for Your Next Mobile Project PhoneJS is a cross-platform HTML5 mobile app development framework that was built to be versatile, flexible and efficient. PhoneJS is a single page application (SPA) framework, with view management and URL routing. Its layout engine allows you to abstract navigation away from views, so the same app can be rendered differently across different platforms or form factors. PhoneJS includes a rich collection of touch-optimized UI widgets with built-in styles for today’s most popular mobile platforms including iOS, Android, and Windows Phone 8. To better understand the principles of PhoneJS development and how you can create and publish applications in platform stores, let’s take a look at a simple demo app called TipCalculator. This application calculates tip amounts due based on a restaurant bill. The complete source code for this app is available here. The app can be found in the AppStore, Google Play, and Windows Store. PhoneJS Application Layout and Routing TipCalculator is a Single-Page Application (SPA) built with HTML5. The start page is index.html, with standard meta tags and links to CSS and JavaScript resources. It includes a script reference to the JavaScript file index.js, where you’ll find the code that configures PhoneJS app framework logic: TipCalculator.app = new DevExpress.framework.html.HtmlApplication({ namespace: TipCalculator, defaultLayout: "empty" }); Within this section, we must specify the default layout for the app. In this example, we’ll use the simplest option, an empty layout. More advanced layouts are available with full support for interactive navigation styles described in the following images: PhoneJS uses well-established layout methodologies supported by many server-side frameworks, including Ruby on Rails and ASP.NET MVC. Detailed information about Views and Layouts can be found in our online documentation. To configure view routing in our SPA, we must add an additional line of code in index.js: TipCalculator.app.router.register(":view", { view: "home" }); This registers a simple route that retrieves the view name from the URL (from the hash segment of the URL). The home view is used by default. Each view is defined in its own HTML file and is linked into the main application page index.html like this: PhoneJS ViewModel A viewmodel is a representation of data and operations used by the view. Each view has a function with the same base name as the view itself and returns the viewmodel for the view. For the home view, the views/home.js script defines the function home which creates the corresponding viewmodel. TipCalculator.home = function(params) { ... }; Three input parameters are used for the tip calculation algorithm: bill total, the number of people sharing the bill, and a tip percentage. These variables are defined as observables, which will be bound to corresponding UI widgets. Note: Observables functionality is supplied by Knockout.js, an important foundation for viewmodels used in PhoneJS. You can learn more about Knockout.js here. This is the code used in the home function to initialize the variables: var billTotal = ko.observable(), tipPercent = ko.observable(DEFAULT_TIP_PERCENT), splitNum = ko.observable(1); The result of the tip calculation is represented by four values: totalToPay, totalPerPerson, totalTip, tipPerPerson. Each value is a dependent observable (a computed value), which is automatically recalculated when any of the observables used in its definition change. Again, this is standard Knockout.js functionality. var totalTip = ko.computed(...); var tipPerPerson = ko.computed(...); var totalPerPerson = ko.computed(...); var totalToPay = ko.computed(...); For an example of business logic implementation in a viewmodel, let’s take a closer look at the observable totalToPay. The total sum to pay is usually rounded. For this purpose, we have two functions roundUp and roundDown that change the value of roundMode (another observable). These changes cause recalculation of totalToPay, because roundMode is used in the code associated with the totalToPay observable. var totalToPay = ko.computed(function() { var value = totalTip() + billTotalAsNumber(); switch(roundMode()) { case ROUND_DOWN: if(Math.floor(value) >= billTotalAsNumber()) return Math.floor(value); return value; case ROUND_UP: return Math.ceil(value); default: return value; } }); When any input parameter in the view changes, rounding should be disabled to allow the user to view precise values. We subscribe to the changes of the UI-bound observables to achieve this: billTotal.subscribe(function() { roundMode(ROUND_NONE); }); tipPercent.subscribe(function() { roundMode(ROUND_NONE); }); splitNum.subscribe(function() { roundMode(ROUND_NONE); }); The complete viewmodel can be found in home.js. It represents a simple example of a typical viewmodel. Note: In a more complex app, it may be useful to implement a structure that modularizes your viewmodels separate from view implementation files. In other words, a file like home.js need not contain the code to implement the viewmodel and instead call a helper function elsewhere for this purpose. In this walkthrough we’re trying to keep things structurally simple. PhoneJS Views Let’s now turn to the markup of the view located in the view/home.html file. The root div element represents a view with the name ‘home’. Within it is a div containing markup for a placeholder called ‘content’. ... A toolbar is located at the top of the view: dxToolbar is a PhoneJS UI widget. It’s defined in the markup using Knockout.js binding. A fieldset appears below the toolbar. To display a fieldset, we use two special CSS classes understood by PhoneJS: dx-fieldset and dx-field. The fieldset contains a text field for the bill total and two sliders for the tip percentage and the number of diners. Two buttons (dxButton) are displayed below the editors, allowing the user to round the total sum to pay. The remaining view displays fieldsets used for calculated results. Total to pay Total per person Total tip Tip per person This completes the description of the files required to create a simple app using PhoneJS. As you’ve seen, the process is simple, straightforward and intuitive. Start, Debug and Build for Stores Starting and debugging a PhoneJS app is just like any other HTML5 based app. You must deploy the folder containing HTML and JavaScript sources, along with any other required file to your web server. Because there is no server-side component to the architectural model, it doesn’t matter which web server you use as long as it can provide file access through HTTP. Once deployed, you can open the app on a device, in an emulator or a desktop browser by simply navigating app’s start page URL. If you want to view the app as it will appear in a phone or tablet within a desktop browser, you will have to override the UserAgent in the browser. Fortunately, this is easy to do with the developer tools that ship as part of today’s modern browsers: If you prefer not to modify UserAgent settings, you can use the Ripple Emulator to emulate multiple device types. At this point you have a web application that will work in the browser on the mobile device and look like native app. Modern mobile browsers provide access to local storage, location api, camera, so good chances are that your app already has anything it needs. Creating Store Ready Applications Using PhoneJS and PhoneGap But what if you need access to device features that browser does not provide? What if you want an app in the app store, not just a webpage. Then you’ll have to create a hybrid application and de-facto standard for such an app is Apache Cordova aka PhoneGap. PhoneGap project for each platform is a native app project that contains WebView (browser control) and a “bridge” that lets your JavaScript code inside WebView access native functions provided by PhoneGap libraries and plugins. To use it, you need to have SDK for each platform you are targeting, but you don’t need to know details of native development, you just need to put your HTML, CSS, JS files into right places and specify your app’s properties like name, version, icons, splashcreens and so on. To be able to publish your app, you will need to register as developer in the respective development portal. This process is well documented for each store and beyond the scope of this article. After that you’ll be able to receive certificates to sign your app package. The need to have SDK for each platform installed sounds challenging - especially after “write one, run everywhere” promise of HTML5/JS approach. This is a small price to pay for building hybrid application and have everything under control. But still there are several services and products that solves this problem for you. One is Adobe’s online service - PhoneGap Build which allows you to build one app for free (to build more, you’ll need a paid account). If you have all the required platform certificate files, the service can build your app for all supported platforms with a few mouse clicks. You only need to prepare app descriptions, promotional and informational content and icons in order to submit your app to an individual store. For Visual Studio developers, DevExpress offers a product called DevExtreme (it includes PhoneJS), which can build applications for iOS, Android and Windows Phone 8 directly within the Microsoft Visual Studio IDE. To summarize, if you need a web application that looks and feels like native on a mobile device, you need PhoneJS - it contains everything required to build touch-enabled, native-looking web application. If you want to go further and access device features, like the contact list or camera, from JavaScript code, you will need Cordova aka PhoneGap. PhoneGap also lets you compile your web app into a native app package. If you don’t want to install an SDK for each platform you are targeting, you can use the PhoneGapBuild service to build your package. Finally, if you have DevExtreme, you can build packages right inside Visual Studio.

Overview OpenHFT/Java Lang started as an Apache 2.0 library to provide the low level functionality used by Java Chronicle without the need to persist to a file. This allows serializable and deserialization of data and random access to memory in native space (off heap) It supports writing and reading enumerable types with object pooling. e.g. writing and reading String without creating an object (if it has been pooled). It also supports writing and read primitive types in binary and text without creating any garbage. Small messages can be serialized and deserialized in under a micro-second. Recent additions Java Lang supports a DirectStore which is like a ByteBuffer but can be any size (up to 40 to 48-bit on most systems) It support 64-bit sizes and offset. It support compacted types, and object serialization. It also supports thread safety features such as volatile reads, ordered (lazy) writes, CAS operations and using an int (4 bytes) as a lock in native memory. Testing a native memory lock in Java This test has one lock and a value which is toggled. One thread changes the value from 0 to 1 and the other switches it from 1 to 0. This goes around 20 million times, but has been run for longer final DirectStore store1 = DirectStore.allocate(1L << 12); final int lockCount = 20 * 1000 * 1000; new Thread(new Runnable() { @Override public void run() { manyToggles(store1, lockCount, 1, 0); } }).start(); manyToggles(store1, lockCount, 0, 1); store1.free(); The manyToggles method is more interesting. Note is using the 4 bytes at offset 0 as a lock. You can arrange any number of locks in native space this way. E.g. you might have fixed length records and want to be able to lock them before updating or access them. You can place a lock at the "head" of the record. private void manyToggles(DirectStore store1, int lockCount, int from, int to) { long id = Thread.currentThread().getId(); assertEquals(0, id >>> 24); System.out.println("Thread " + id); DirectBytes slice1 = store1.createSlice(); for (int i = 0; i < lockCount; i++) { assertTrue( slice1.tryLockNanosInt(0L, 10 * 1000 * 1000)); int toggle1 = slice1.readInt(4); if (toggle1 == from) { slice1.writeInt(4L, to); } else { i--; } slice1.unlockInt(0L); } } The size of the DataStore and the offsets within it are long, allowing you to allocate a continuous block of native memory into the many GB, and access it as you required. On my 2.6 GHz i5 laptop I get the following output for this test Contended lock rate was 9,096,824 per second This looks great but under heavy contention, one thread can be staved out. This is more useful for lots of locks and lower contention. Note: if I drop the timeout from 10 ms to 1 ms, it eventually fails meaning sometimes it takes more then 1 ms to get a lock! Conclusion The Java Lang library is taking the step of making it easier to use native memory with the same functionality available on the heap. The language support is not as good, but if you need to store say 128 GB of data you will get a much better GC behavior using off heap memory.

A couple of weeks ago Jim and I were building out a neo4j unmanaged extension from which we wanted to return the results of a traversal which had a lot of paths. Our code initially looked a bit like this: package com.markandjim @Path("/subgraph") public class ExtractSubGraphResource { private final GraphDatabaseService database; public ExtractSubGraphResource(@Context GraphDatabaseService database) { this.database = database; } @GET @Produces(MediaType.TEXT_PLAIN) @Path("/{nodeId}/{depth}") public Response hello(@PathParam("nodeId") long nodeId, @PathParam("depth") int depth) { Node node = database.getNodeById(nodeId); final Traverser paths = Traversal.description() .depthFirst() .relationships(DynamicRelationshipType.withName("whatever")) .evaluator( Evaluators.toDepth(depth) ) .traverse(node); StringBuilder allThePaths = new StringBuilder(); for (org.neo4j.graphdb.Path path : paths) { allThePaths.append(path.toString() + "\n"); } return Response.ok(allThePaths.toString()).build(); } } We then compiled that into a JAR, placed it in ‘plugins’ and added the following line to ‘conf/neo4j-server.properties’: org.neo4j.server.thirdparty_jaxrs_classes=com.markandjim=/unmanaged After we’d restarted the neo4j server we were able to call this end point using cURL like so: $ curl -v http://localhost:7474/unmanaged/subgraph/1000/10 This approach works quite well but Jim pointed out that it was quite inefficient to load all those paths up into memory so we thought it would be quite cool if we could stream it as we got to each path. Traverser wraps an iterator so we are lazily evaluating the result set in any case. After a bit of searching we came StreamingOutput which is exactly what we need. We adapted our code to use that instead: package com.markandjim @Path("/subgraph") public class ExtractSubGraphResource { private final GraphDatabaseService database; public ExtractSubGraphResource(@Context GraphDatabaseService database) { this.database = database; } @GET @Produces(MediaType.TEXT_PLAIN) @Path("/{nodeId}/{depth}") public Response hello(@PathParam("nodeId") long nodeId, @PathParam("depth") int depth) { Node node = database.getNodeById(nodeId); final Traverser paths = Traversal.description() .depthFirst() .relationships(DynamicRelationshipType.withName("whatever")) .evaluator( Evaluators.toDepth(depth) ) .traverse(node); StreamingOutput stream = new StreamingOutput() { @Override public void write(OutputStream os) throws IOException, WebApplicationException { Writer writer = new BufferedWriter(new OutputStreamWriter(os)); for (org.neo4j.graphdb.Path path : paths) { writer.write(path.toString() + "\n"); } writer.flush(); } }; return Response.ok(stream).build(); } As far as I can tell the only discernible difference between the two approaches is that you get an almost immediate response from the streamed approached whereas the first approach has to put everything in the StringBuilder first. Both approaches make use of chunked transfer encoding which according to tcpdump seems to have a maximum packet size of 16332 bytes: 00:10:27.361521 IP localhost.7474 > localhost.55473: Flags [.], seq 6098196:6114528, ack 179, win 9175, options [nop,nop,TS val 784819663 ecr 784819662], length 16332 00:10:27.362278 IP localhost.7474 > localhost.55473: Flags [.], seq 6147374:6163706, ack 179, win 9175, options [nop,nop,TS val 784819663 ecr 784819663], length 16332

A recent Java production performance problem forced me to revisit and truly appreciate the Java VM Just-In-Time (JIT) compiler. Most Java developers and support individuals have heard of this JVM run time performance optimization but how many truly understand and appreciate its benefits? This article will share with you a troubleshooting exercise I was involved with following the addition of a new virtual server (capacity improvement and horizontal scaling project). For a more in-depth coverage of JIT, I recommend the following articles: ## Just-in-time compilation http://en.wikipedia.org/wiki/Just-in-time_compilation ## The Java HotSpot Performance Engine Architecture http://www.oracle.com/technetwork/java/whitepaper-135217.html ## Understanding Just-In-Time Compilation and Optimization http://docs.oracle.com/cd/E15289_01/doc.40/e15058/underst_jit.htm ## How the JIT compiler optimizes code http://pic.dhe.ibm.com/infocenter/java7sdk/v7r0/index.jsp?topic=%2Fcom.ibm.java.zos.70.doc%2Fdiag%2Funderstanding%2Fjit_overview.html JIT compilation overview The JIT compilation is essentially a process that improves the performance of your Java applications at run time. The diagram below illustrates the different JVM layers and interaction. It describes the following high level process: Java source files are compiled by the Java compiler into platform independent bytecode or Java class files. After your fire your Java application, the JVM loads the compiled classes at run time and execute the proper computation semantic via the Java interpreter. When JIT is enabled, the JVM will analyze the Java application method calls and compile the bytecode (after some internal thresholds are reached) into native, more efficient, machine code. The JIT process is normally prioritized by the busiest method calls first. Once such method call is compiled into machine code, the JVM executes it directlyinstead of “interpreting” it. The above process leads to improved run time performance over time. Case study Now here is the background on the project I was referring to earlier. The primary goal was to add a new IBM P7 AIX virtual server (LPAR) to the production environment in order to improve the platform’s capacity. Find below the specifications of the platform itself: Java EE server: IBM WAS 6.1.0.37 & IBM WCC 7.0.1 OS: AIX 6.1 JDK: IBM J2RE 1.5.0 (SR12 FP3 +IZ94331) @64-bit RDBMS: Oracle 10g Platform type: Middle tier and batch processing In order to achieve the existing application performance levels, the exact same hardware specifications were purchased. The AIX OS version and other IBM software’s were also installed using the same version as per existing production. The following items (check list) were all verified in order to guarantee the same performance level of the application: Hardware specifications (# CPU cores, physical RAM, SAN…). OS version and patch level; including AIX kernel parameters. IBM WAS & IBM WCC version, patch level; including tuning parameters. IBM JRE version, patch level and tuning parameters (start-up arguments, Java heap size…). The network connectivity and performance were also assessed properly. After the new production server build was completed, functional testing was performed which did also confirm a proper behaviour of the online and batch applications. However, a major performance problem was detected on the very first day of its production operation. You will find below a summary matrix of the performance problems observed. Production server Operation elapsed time Volume processed (# orders) CPU % (average) Middleware health Existing server 10 hours 250 000 (baseline) 20% healthy *New* server 10 hours 50 000 -500% 80% +400% High thread utilization As you can see from the above view, the performance results were quite disastrous on the first production day. Not only much less orders were processed by the new production server but the physical resource utilization such as CPU % was much higher compared with the existing production servers. The situation was quite puzzling given the amount of time spent ensuring that the new server was built exactly like the existing ones. At that point, another core team was engaged in order to perform extra troubleshooting and identify the source of the performance problem. Troubleshooting: searching for the culprit... The troubleshooting team was split in 2 in order to focus on the items below: Identify the source of CPU % from the IBM WAS container and compare the CPU footprint with the existing production server. Perform more data and file compares between the existing and new production server. In order to understand the source of the CPU %, we did perform an AIX CPU per Thread analysis from the IBM JVM running IBM WAS and IBM WCC. As you can see from the screenshot below, many threads were found using between 5-20% each. The same analysis performed on the existing production server did reveal fewer # of threads with CPU footprint always around 5%. Conclusion: the same type of business process was using 3-4 times more CPU vs. the existing production server. In order to understand the type of processing performed, JVM thread dumps were captured at the same time of the CPU per Thread data. Now the first thing that we realized after reviewing the JVM thread dump (Java core) is that JIT was indeed disabled! The problem was also confirmed by running the java –version command from the running JVM processes. This finding was quite major, especially given that JIT was enabled on the existing production servers. Around the same time, the other team responsible of comparing the servers did finally find differences between the environment variables of the AIX user used to start the application. Such compare exercise was missed from the earlier gap analysis. What they found is that the new AIX production server had the following extra entry: JAVA_COMPILER=NONE As per the IBM documentation, adding such environment variable is one of the ways to disableJIT. Complex root cause analysis, simple solution In order to understand the impact of disabling JIT in our environment, you have to understand its implication. Disabling JIT essentially means that the entire JVM is now running ininterpretation mode. For our application, running in full interpretation mode not only reduces the application throughput significantly but also increases the pressure point on the server CPU utilization since each request/thread takes 3-4 more times CPU than a request executed with JIT (remember, when JIT is enabled, the JVM will perform many calls to the machine/native code directly). As expected, the removal of this environment variable along with the restart of the affected JVM processes did resolve the problem and restore the performnance level. Assessing the JIT benefits for your application I hope you appreciated this case study and short revisit of the JVM JIT compilation process. In order to understand the impact of not using JIT for your Java application, I recommend that you preform the following experiment: Generate load to your application with JIT enabled and capture some baseline data such as CPU %, response time, # requests etc. Disable JIT. Redo the same testing and compare the results. I’m looking forward for your comments and please share any experience you may have with JIT.

Strategy Pattern is one of the patterns from the Design Patterns : Elements of Reusable Object book. The intent of the strategy pattern as stated in the book is: Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it. In this post I would like to give an example or two on strategy pattern and then rewrite the same example using lambda expressions to be introduced in Java 8. Strategy Pattern: An example Consider an interface declaring the strategy: interface Strategy{ public void performTask(); } Consider two implementations of this strategy: class LazyStratgey implements Strategy{ @Override public void performTask() { System.out.println("Perform task a day before deadline!"); } } class ActiveStratgey implements Strategy{ @Override public void performTask() { System.out.println("Perform task now!"); } } The above strategies are naive and I have kept it simple to help readers grasp it quickly. And lets see these strategies in action: public class StartegyPatternOldWay { public static void main(String[] args) { List strategies = Arrays.asList( new LazyStratgey(), new ActiveStratgey() ); for(Strategy stg : strategies){ stg.performTask(); } } } The output for the above is: Perform task a day before deadline! Perform task now! Strategy Pattern: An example with Lambda expressions Lets look at the same example using Lambda expressions. For this we will retain our Strategy interface, but we need not create different implementation of the interface, instead we make use of lambda expressions to create different implementations of the strategy. The below code shows it in action: import java.util.Arrays; import java.util.List; public class StrategyPatternOnSteroids { public static void main(String[] args) { System.out.println("Strategy pattern on Steroids"); List strategies = Arrays.asList( () -> {System.out.println("Perform task a day before deadline!");}, () -> {System.out.println("Perform task now!");} ); strategies.forEach((elem) -> elem.performTask()); } } The output for the above is: Strategy pattern on Steroids Perform task a day before deadline! Perform task now! In the example using lambda expression, we avoided the use of class declaration for different strategies implementation and instead made use of the lambda expressions. Strategy Pattern: Another Example This example is inspired from Neal Ford’s article on IBM Developer works: Functional Design Pattern-1. The idea of the example is exactly similar, but Neal Ford uses Scala and I am using Java for the same with a few changes in the naming conventions. Lets look at an interface Computation which also declares a generic type T apart from a method compute which takes in two parameters. interface Computation { public T compute(T n, T m); } We can have different implementations of the computation like: IntSum – which returns the sum of two integers, IntDifference – which returns the difference of two integers and IntProduct – which returns the product of two integers. class IntSum implements Computation { @Override public Integer compute(Integer n, Integer m) { return n + m; } } class IntProduct implements Computation { @Override public Integer compute(Integer n, Integer m) { return n * m; } } class IntDifference implements Computation { @Override public Integer compute(Integer n, Integer m) { return n - m; } } Now lets look at these strategies in action in the below code: public class AnotherStrategyPattern { public static void main(String[] args) { List computations = Arrays.asList( new IntSum(), new IntDifference(), new IntProduct() ); for (Computation comp : computations) { System.out.println(comp.compute(10, 4)); } } }public class AnotherStrategyPattern { public static void main(String[] args) { List computations = Arrays.asList( new IntSum(), new IntDifference(), new IntProduct() ); for (Computation comp : computations) { System.out.println(comp.compute(10, 4)); } } } The output for the above is: 14 6 40 Strategy Pattern: Another Example with lambda expressions Now lets look at the same example using Lambda expressions. As in the previous example as well we need not declare classes for different implementation of the strategy i.e the Computation interface, instead we make use of lambda expressions to achieve the same. Lets look at an example: public class AnotherStrategyPatternWithLambdas { public static void main(String[] args) { List> computations = Arrays.asList( (n, m)-> { return n+m; }, (n, m)-> { return n*m; }, (n, m)-> { return n-m; } ); computations.forEach((comp) -> System.out.println(comp.compute(10, 4))); } } The output for above is 14 6 40 From the above examples we can see that using Lambda expressions will help in reducing lot of boilerplate code to achieve more concise code. And with practice one can get used to reading lambda expressions.

in-line predicates can create a maintenance nightmare. writing in-line lambda expressions and using the stream interfaces to perform common operations on collections can be awesome. assume the following example: list getadultmales (list persons) { return persons.stream().filter(p -> p.getage() > adult && p.getsex() == sexenum.male ).collect(collectors.tolist()); } that’s fun! but things like this also lead to software that is costly to maintain. at least in an enterprise application, where most of your code handles business logic, your development team will grow the tenancy to write the same similar set of predicate rules again and again. that is not what you want on your project. it breaks three important principles for growing maintainable and stable enterprise applications: dry (don’t repeat yourself): writing code more than once is not a good fit for a lazy developer it also makes your software more difficult to maintain because it becomes harder to make your business logic consistent readability : following clean-code best practices, 80% of writing code is reading the code that already exists. having complicated lambda expressions is still a bit hard to read compared to a simple one-line statement. testability : your business logic needs to be well-tested. it is adviced to unit-test your complex predicates. and that is just much easier to do when you separate your business predicate from your operational code. and from a personal point of view… that method still contains too much boilerplate code… imports to the rescue! fortunately, we have a very good suggestion in the world of unit testing on how we could improve on this. imagine the following example: import static somepackage.personpredicate; ... list getadultmales (list persons) { return persons.stream().filter( isadultmale() ).collect(collectors.tolist()); } what we did here was: create a personpredicate class define a “factory” method that creates the lambda predicate for us statically import the factory method into our old class this is how such a predicate class could look like, located next to your person domain entity: public personpredicate { public static predicate isadultmale() { return p -> p.getage() > adult && p.getsex() == sexenum.male; } } wait… why don’t we just create a “ismaleadult” boolean function on the person class itself like we would do in domain driven development? i agreed, that is also an option… but as time goes on and your software project becomes bigger and loaded with functionality and data… you will again break your clean code principles: the class becomes bloated with all kind of function and conditions your class and tests become huge, more difficult to handle and change (*) (*) and yes… even if you do your best to separate your concerns and use composition patterns adding some defaults… working with domain objects, we can imagine that some operations (such as filter) are often executed on domain entities. taking that into account, it would make sense to let our entities implement some interface that offers us some default methods. for example: public interface domainoperations { default list filter(predicate predicate) { return persons.stream().filter( predicate ) .collect(collectors.tolist()); } } when our person entity implements this interface, we can clean-up our code even more: list getadultmales (list persons) { return persons.filter( isadultmale() ); } and there we go… conclusion moving your predicates to a predicate helper class offers some good advantages in the long run: predicate classes are easy to test and change your domain objects remain clean and focussed on representing your domain, not your business logic you optimize the re-usability of your code and, in the end, reduce your maintenance you seperate your business from operational concerns references clean code: a handbook of agile software craftsmanship [robert c. martin] practical unit testing with junit and mockito [tomek kaczanowski] state of the collections [http://cr.openjdk.java.net/~briangoetz/lambda/collections-overview.html] notes the code above is served as an example to illustrate the principles i wanted to discuss. however, i did not proof-run this code yet (it’s still on my todo list). some modifications may be needed for your project.

This is a package of C#, VB.NET Example Project for Spire.BarCode for .NET. Spire.BarCode for .NET is a professional and reliable barcode generation and recognition component. It enables developers to quickly and easily add barcode generation and recognition functionality to their Microsoft .NET applications (ASP.NET, WinForms and .NET) and it supports in C#, VB.NET. Spire.Barcode for.NET is 100% FREE barcode component. First Glance of Spire.BarCode for .NET http://www.e-iceblue.com/images/url/BarCode.png Spire.BarCode for .NET Main Features: Supports rich Barcode types, more than 37 different barcodes. • Code bar Barcode • Code 1 of 1 Barcode • Standard 2 of 5 barcode • Code 3 of 9 barcode • Extended Code 3 of 9 barcode • Code 9 of 3 Barcode • Extended Code 9 of 3 Barcode • Code 128 barcode • EAN-8 barcode • EAN-13 barcode • EAN-128 barcode • EAN-14 barcode • SCC14 barcode • SSCC18 barcode • ITF14 Barcode • ITF-6 Barcode • UPCA barcode • UPCE barcode • Postnet barcode • Planet barcode • MSI barcode • 2D Barcode DataMatrix • QR Code barcode • Pdf417 barcode • Pdf417 Macro barcode • RSS14 barcode • RSS-14 Truncated barcode • RSS Limited Barcode • RSS Expanded Barcode • USPS OneCode barcode • Swiss Post Parcel Barcode • PZN Barcode • OPC(Optical Product Code) Barcode • Deutschen Post Barcode • Deutsche Post Leitcode Barcode • Royal Mail 4-state Customer Code Barcode • Singapore Post Barcode 1.Robust Barcode Recognize and Generation 1D & 2D Barcode. Developers can read most often used Linear, 2D and Postal barcodes, detecting them anywhere, with any orientation. 2.High performance for generating and reading barcode image Developers can create barcode images in any desired output image format like Bitmap, JPG, PNG, EMF, TIFF, GIF and WMF. 3.Superior performance support for reading and writing barcode Developers can easily set barcode image borders, border colors, style, margins and width. You can also rotate barcode images to any angle and produce high quality barcode images. 4.Easy Integration Spire.Barcode for .NET can be easily integrated into any .net applications. There are two main ways to integrate Spire.Barcode in .NET applications, API Mode and Component Mode. • API Mode is just one line of code to create, recognizes barcode. • Component Mode use Visual way to create barcode, then drag Spire.Barcode component to your .NET, Windows or ASP.NET Form. No more code needs. Download Spire.BarCode: Spire.BarCode for .NET is a free barcode library used in .NET applications (in ASP.NET, WinForms and .NET). And you can download Spire.BarCode for .NET and install it on your system. With Spire.BarCode, you can add Enterprise-level barcode formats to your NET applications easily and quickly. Feedback and Support E-iceblue welcomes any kind of questions, bug reports and feedback about this product from our customers. As long as you leave them on Spire.BarCode for .NET Forum or contact us by e-mail, we will offer full support within one business day. Related Links Website:www.e-iceblue.com Product Introduction: http://www.e-iceblue.com/Introduce/barcode-for-net-introduce.html#.UdEuk9hp4Zu Download: http://www.e-iceblue.com/Download/download-barcode-for-net-now.html . Spire.BarCode for .NET is a FREE and professional barcode component specially designed for .NET developers (C#, VB.NET, ASP.NET) to generate, read 1D & 2D barcodes. Developers and programmers can use Spire.BarCode to add Enterprise-Level barcode formats to their .net applications (ASP.NET, WinForms and Web Service) quickly and easily. Spire.BarCode for .NET provides a very easy way to integrate barcode processing. With just one line of code to create, read 1D & 2D barcode, Spire.BarCode supports variable common image formats, such as Bitmap, JPG, PNG, EMF, TIFF, GIF and WMF. Spire.BarCode for .NET is 100% FREE BarCode component, no risk to integrate in your .NET application.

What is the Default? Usually in most installations, binary log files are located in the MySQL default directory (/var/lib/mysql) just next to the data files. Why Should I Move the Binary Logs to Another Directory? Each data modification (INSERT, UPDATE, DELETE...) and data definition (ALTER, ADD, DROP...) statement that you perform in your server are recorded in the Log files. Therefore, each time you make any of these statements, you actually update both your data files and your log files. The result is high IO utilization that is focused on a specific disk area. A common recommendation in the database field is to separate these files to two different disks in order to get a better performance. How to Perform it? Change the log-bin variable in the my.cnf to log-bin=/path/to/new/directory/mysql-bin Purge as many files as you can (PURGE BINLOG...) in order to minimize the number of moved files (see stop 4). Stop the master (service mysql stop). Move the files to the new directory: mv /var/lib/mysql/mysql-bin.* /path/to/new/directory Start the master again (service mysql start). Bottom Line Few steps and your server is ready for more traffic and data. Keep Performing, Moshe Kaplan

longpolling ; also known as reverse ajax or comet, is a push method that operates seamlessly in javascript aided web browsers. although advanced push techniques such as sse and websocket with html 5 are already available, either these technologies are still in the process of development or they are not supported entirely by web browsers, longpolling and similar techniques be in demand. in longpolling technique, the web browser makes a request to the server, and this request is suspended on the server until a ready response ( resource ) is founded. the pending http request is sent back to the available source web browser when a response is obtained. the web browser transmits a new polling message as soon as it gets the http response, and this process repeats itself until a new http response is ready.. from time to time, the failure of the pending polling request originating from server, web browser or current network environment can also be a matter. in this case, web browser takes the error information from the server, sent a request once more and then the polling starts again. in longpolling technique, xmlhttprequest object which is a javascript object can be used on the web browser side. forthcoming asyncronousresponse objects with jax-rs 2 can be used on the server side in terms of flexibility and convenience. now, i would like to share with you a chat application instance that contains the association of longpolling technique and jax-rs 2 within java ee 7 . dependent library chat application can easily operate on a servlet container, but available container environment must comply with the standard servlet 3 because jax-rs 2 uses forthcoming asynchronous handlers together with servlet 3 on the container. org.glassfish.jersey.containers jersey-container-servlet 2.0 note: jersey is the reference implementer library of the standard jax-rs. http://jersey.java.net/ jetty 9 or tomcat 7 can be used for the servlet 3 support . entry point of the chat application (rest endpoint) @applicationpath("app") public class app extends application { @override public set> getclasses() { set> sets=new hashset<>(); sets.add(chat.class); return sets; } } the above application class type object that is wrapped with @applicationpath notation contains the entry point of the chat application. the defined chat class into the hashset is described as a rest resource belonging to the chat application. chat resource @path("/chat") public class chat { final static map waiters = new concurrenthashmap<>(); final static executorservice ex = executors.newsinglethreadexecutor(); @path("/{nick}") @get @produces("text/plain") public void hangup(@suspended asyncresponse asyncresp, @pathparam("nick") string nick) { waiters.put(nick, asyncresp); } @path("/{nick}") @post @produces("text/plain") @consumes("text/plain") public string sendmessage(final @pathparam("nick") string nick, final string message) { ex.submit(new runnable() { @override public void run() { set nicks = waiters.keyset(); for (string n : nicks) { // sends message to all, except sender if (!n.equalsignorecase(nick)) waiters.get(n).resume(nick + " said that: " + message); } } }); return "message is sent.."; } } asyncresponse type parameter that is found as a parameter on hangup(..) method defines the longpolling response object which will be suspended on the server. while asyncresponse type objects can be provided from jax-rs environment automatically, these suspended objects can be submitted to the user (web browser) as a response at any time. the second parameter [ @pathparam("nick") string nick ] found on the hangup(..) method obtains the user nick name which is sent to the method on the url. the information about suspended asyncresponse objects belong to which user is needed subsequently. concurrenthashmap that is a structure of a concurrent map can be used for this purpose. in this way,each nick -> asyncresponse object that is sent to thehangup(..) method will be kept under record. the second method of the chat class sendmessage(..) can be accessed with http /post method and operates as restful method which derives chat messages from the web browser. when users submit a chat message with the knowledge of nick name to the server, sendmessage(..) restful method obtains this information from the method parameters, and this information is being transacted as asynchronous through the executorservice object. of course a simple thread can be used instead of executorservice object that is used in here. just as new thread( new runnable(){ … } ).start(); . the key issue here is the running of the sent message at the background and to prevent the interruption of default /post request, more precisely, to prevent the waiting of message information made push to all users. the nick -> asyncresponse pair that is filled in the previous hangup(..) method in the concurrenthashmap is consumed into the runnable task object which is found in the sendmessage(..) method, and this chat message is transmitted to all suspended asyncresponse objects except the user itself. the resume() method included in the asyncresponse objects allows the @suspended response object to respond. that is to say, ( suspend ) before, and then ( resume ) when chat message resource become ready. html components nick:message: post in the above html page, there are a nick input object, a text area object where the message will be entered and an html button where the message will be sent and these are lined in an html table object. the textarea is made as disabled initially because it is expected to enter the nick name firstly from the user when the application is started. when the user sends the first request (nick name), unchangeable state of this field is modified and is made message writable. also, when nick name is sent to the server (i.e. when the first polling request is made), html table row element which is a class type ( ) is hidden. chat application is required to provide xmlhttprequest objects and http requests as asynchronous. in this application we will benefit from jquery ajax library which facilitates these operations and uses xmlhttprequest objects in the background. in addition to jquery dependence, there is a chat.js script that was written by us in the project. chat.js // the polling function must be called once, //it will call itself recursively in case of error or performance. var poolit = function () { $.ajax({ type: "get", url: "/app/chat/" + $("#nick").val(), // access to the hangup(..) method datatype: "text", // incoming data type text success: function (message) { // the message is added to the element when it is received. $("ul").append("" + message + ""); poolit(); // link to the re-polling when a message is consumed. }, error: function () { poolit(); // start re-polling if an error occurs. } }); } // when the submit button is clicked; $("button").click(function () { if ($(".nick").css("visibility") === "visible") { // if line is visible; $("textarea").prop("disabled", false); // able to enter data $(".nick").css("visibility", "hidden"); // make line invisible; $("span").html("chat started.."); // information message // polling operation must be initiated at a time poolit(); } else // if it is not the first time ; $.ajax({ type: "post", // http post request url: "/app/chat/" + $("#nick").val(),// access to the sendmessage(..) method. datatype: "text", // incoming data type -> text data: $("textarea").val(), // chat message to send contenttype: "text/plain", // the type of the sent message success: function (data) { $("span").html(data); // it writes [message is sent..] if successful. // blink effect $("span").fadeout('fast', function () { $(this).fadein('fast'); }); } }); }); testing the application because the jetty 9.0.0.rc2 plugin is included in the pom.xml configuration file of the application, the application can be easily run with > mvn jetty:run-war command. the application can be accessed at http://localhost:8080/ after above goal run. you can access source code and live example follow the next url => http://en.kodcu.com/2013/06/jax-rs-2-and-longpolling-based-chat-application/

This technical tip shows how to Add, Delete & Get Attachment in a PDF Document using Aspose.Pdf for Java. In order to add attachment in a PDF document, you need to create a FileSpecification object with the file, which needs to be added, and the file description. After that the FileSpecification object can be added to EmbeddedFiles collection of Document object using add(..) method of EmbeddedFiles collection. The attachments of the PDF document can found in the EmbeddedFiles collection of the Document object. In order to delete all the attachments, you only need to call the delete(..) method of the EmbeddedFiles collection and then save the updated file using save method of the Document object. //Add attachment in a PDF document. //open document com.aspose.pdf.Document pdfDocument = new com.aspose.pdf.Document("input.pdf"); //setup new file to be added as attachment com.aspose.pdf.FileSpecification fileSpecification = new com.aspose.pdf.FileSpecification("sample.txt", "Sample text file"); //add attachment to document's attachment collection pdfDocument.getEmbeddedFiles().add(fileSpecification); // Save updated document containing table object pdfDocument.save("output.pdf"); //Delete all the attachments from the PDF document. //open document com.aspose.pdf.Document pdfDocument = new com.aspose.pdf.Document("input.pdf"); //delete all attachments pdfDocument.getEmbeddedFiles().delete(); //save updated file pdfDocument.save("output.pdf"); //Get an individual attachment from the PDF document. //open document com.aspose.pdf.Document pdfDocument = new com.aspose.pdf.Document("input.pdf"); //get particular embedded file com.aspose.pdf.FileSpecification fileSpecification = pdfDocument.getEmbeddedFiles().get_Item(1); //get the file properties System.out.printf("Name: - " + fileSpecification.getName()); System.out.printf("\nDescription: - " + fileSpecification.getDescription()); System.out.printf("\nMime Type: - " + fileSpecification.getMIMEType()); // get attachment form PDF file try { InputStream input = fileSpecification.getContents(); File file = new File(fileSpecification.getName()); // create path for file from pdf file.getParentFile().mkdirs(); // create and extract file from pdf java.io.FileOutputStream output = new java.io.FileOutputStream(fileSpecification.getName(), true); byte[] buffer = new byte[4096]; int n = 0; while (-1 != (n = input.read(buffer))) output.write(buffer, 0, n); // close InputStream object input.close(); output.close(); } catch (IOException e) { e.printStackTrace(); } // close Document object pdfDocument.dispose();

Hi, I'm excited to announce the release of our QuartzDesk product. QuartzDesk is an advanced Java Quartz scheduler management and monitoring GUI / tool with many powerful and unique features. To name just a few: Support for Quartz 1.x and 2.x schedulers. Persistent job execution history. Job execution log message capturing. Notifications (email, all popular IM protocols, web-service). Interactive execution statistics and charts. REST API for job / trigger / scheduler monitoring. QuartzAnywhere web-service to manage / monitor Quartz schedulers from applications. and more To keep this announcement short, I kindly refer you to the QuartzDesk Features page for details and screenshots. The product is aimed at Java developers and system administrators. Jan Moravec (Founder) & The QuartzDesk Team

in the beginning, before you start making a new game, you need to prepare the concept (logic) of the game. you need to have a clear idea about it. you need to develop a high level of understanding, without going into the fine details. in the first phase, we find the answer "what" we want to create, and we reserve the answers to "how to do" for the upcoming stages. let me illustrate this method with an example. let’s develop the popular game "snake". this game was popular long ago, even on consoles and old cell phones with text user interfaces. concept of the game initially a small sized snake appears on the screen which keeps running for the rest of the game like endless runner. player can change the direction only. speed of the snake increases with time. length of the snake also increases after eating randomly appearing food. increasing length and speed of the snake adds difficulty to the game over time. we can use storyboard technique to graphically represent our idea. according to wikipedia: "storyboards are graphic organizers in the form of illustrations or images displayed in sequence for the purpose of pre-visualizing a motion picture, animation, motion graphic or interactive media sequence." here is our storyboard: for your convenience i am adding the description of each storyboard screen: screen 1: snake (square) waiting for a key-press to start moving. circle is shown as food item. screen 2: after eating (hitting) food item the snake’s length increases and food item appears at another random location. screen 3: snake can re-enter the playing area from the opposite edge after leaving it from an edge. screen 4: snake dies after biting itself here, uml statechart diagram may also help to understand different "states" of the snake during a game. according to wikipedia: "a state diagram is a type of diagram used in computer science and related fields to describe the behavior of systems. state diagrams require that the system described is composed of a finite number of states; sometimes, this is indeed the case, while at other times this is a reasonable abstraction. many forms of state diagrams exist, which differ slightly and have different semantics." here is our statechart diagram: in the diagram, edges represent the actions and the ovals represent the states a snake is in after the specific action. game starts with an idle snake. you can move snake in all four directions using arrow keys. while moving in any direction when a key is pressed, the snake goes to "deciding" state to determine which key was pressed and then again goes to the respective direction. snake eats food when encountered while moving. there is also a "dead" state if snake hits itself while moving. you may also want to add more state diagrams for prominent objects to clarify. there are other uml diagrams which may help you describe your game project. these diagrams are not only helpful for yourself but also helps if you are working in a team making team communication easy and unambiguous. structure of the game the play area in virtually divided into a 200?200-pixel grid having 10?10-pixel cells. the initgrid() function prepares the grid. as you can guess from the code that the snake’s width and height is also 10?10 pixels. so as a programming trick, i used the height and width of the snake to represent the dimensions of a single cell. function initgrid() { //*****initialize grid ... cell = {"col":col*snakeheight, "row":row*snakewidth}; ... } you are right if you are thinking about the usage of this virtual grid. in fact, during the initialization of the game structure, this grid helps us to identify the places (cells…to be precise) where we can put the snake and food randomly. a random number generator function randomcell(cells) gives us a random number which we use as an index to the grid array and get the coordinates stored against that specific index. here is the random number generator function… function randomcell(cells) { return math.floor((math.random()*cells)); } math.random and math.floor are both javascript functions. the following code shows the usage of grid and random number generator function… var initsnakeposition = randomcell(grid.length - 1); //pass number of cells var snakepart = new kinetic.rect({ name: 'snake', x: grid[initsnakeposition].col, y: grid[initsnakeposition].row, width: snakewidth, height: snakeheight, fill: 'black', }); kinetic.rect constructor constructs the initial single rectangle to represent the snake. later, when the snake would grow after eating the food, we would be adding more rectangles. each rectangle is assigned a number to represent its actual position in the snake array. as there is only one rectangle at the moment, we assign it position 1. snakepart.position = snakeparts; snakeparts is a counter which keeps counting the number of parts in the snake array. you might be wondering that we have not created any array of snakepart objects but we are talking about array? in fact if you keep the value of name: property same for all the snakepart objects, kineticjs would return all those objects as an array if you ask like this… var snakepartsarray = stage.get('.snake'); you will see the usage of this feature in action later in the code. snakepart.position shows how you can add custom properties to kinetic.rect object dynamically, or to any other object. why we need the position when kineticjs can return indexed array? please don’t bother yourself with this question at the moment, you will find the answer if you keep reading. two more identifications are required to make the job more easy to manage the snake actions and movements, snake head and tail. there is only one snake-part (rectangle) to begin with therefore both head and tail pointers point to the same rectangle. var snaketail; var snakehead; ... snakehead = snakepart; snaketail = snakepart; we are done with setting up the snake. to construct the food which is a simple circle of radius 5 see the following code… var randomfoodcell = randomcell(grid.length - 1); var food = new kinetic.circle({ id: 'food', x:grid[randomfoodcell].col+5, y:grid[randomfoodcell].row+5, radius: 5, fill: 'black' }); kinetic.circle constructs the food for our snake game. here adding +5 to x and y coordinates to place the circle exactly in the centre of a 10?10 cell provided that the radius of the circle is 5. after we are done with the creation of basic shapes for our game and their positions on the game area/grid we need to add those shapes to a kinetic.layer and then add that layer to the kinetic.stage. // add the shapes (sanke and food) to the layer var layer = new kinetic.layer(); layer.add(snakepart); layer.add(food); // add the layer to the stage stage.add(layer); the ‘stage’ object used in the cod above has already been created in the beginning using the following code snippet… //stage var stagewidth = 200; var stageheight = 200; var stage = new kinetic.stage({ container: 'container', width: stagewidth, height: stageheight }); container property of stage needs to know the id of the div where we want to show our html5 canvas. initial screen of the game looks like this once our structure is complete… after setting up the environment/structure let’s deal with the user stories / use cases one by one. the main game loop executing after a set interval var gameinterval = self.setinterval(function(){gameloop()},70); setinterval is a javascript function which makes a function called asynchronously that is passed as an argument, after the set intervals. in our case gameloop() is the function which drives the whole game. have a look at it… function gameloop() { if(checkgamestatus()) move(where); else { clearinterval(gameinterval);//stop calling gameloop() alert('game over!'); } } well, the behaviour of gameloop() is pretty obvious. it moves the snake according to the arrow key pressed. and if snake hits himself then display a game over message to the player and also stop the asynchronous calls by calling a javascript clearinterval() method. to capture the arrow keys i have used the jquery’s keydown event handler to respond to the keys pressed. it sets a variable ‘where’ that is eventually used by gameloop() to pass the code to actual move() function. $( document ).ready(function() { $(document).keydown(function(e) { switch(e.keycode) { // user pressed "up" arrow case up_arrow: where = up_arrow; break; // user pressed "down" arrow case down_arrow: where = down_arrow; break; // user pressed "right" arrow case right_arrow: where = right_arrow; break; // user pressed "left" arrow case left_arrow: where = left_arrow; break; } }); }); as you might have guessed already that move() function is actual brain of this game and kinetic.animation handles the actual movement of the objects. all you have to do is to set new locations for your desired objects and then call start() method. to prevent the animation from running infinitely call stop() method immediately after start(). try removing the stop() method and see what happens yourself. function move(direction) { //super hint: only move the tail var foodhit = false; switch(direction) { case down_arrow: foodhit = snakeeatsfood(direction); var anim2 = new kinetic.animation(function(frame) { if(foodhit) { snakehead.sety(snakehead.gety()+10); growsnake(direction); if(snakehead.gety() == stageheight) snakehead.sety(0); relocatefood(); } else { snaketail.sety(snakehead.gety()+10); snaketail.setx(snakehead.getx()); if(snaketail.gety() == stageheight) snaketail.sety(0); reposition(); } }, layer); anim2.start(); anim2.stop(); break; case up_arrow: ... ... move the snake snake is divided into small 10?10-pixels squares. the square on the front is head and the back most square is tail. the technique to move the snake is simple. we pick the tail and put it before the head except when there is only one snake part. the position number assigned to each part of the snake is out of sequence now. head and tail pointers are pointing towards wrong parts. we need to reposition the pointers and position numbers. it is done by calling reposition(). it works as shown in the diagram below… grow the snake when it eats food snake eats food when snake’s head is on the food. once this condition is met, food is relocated to some other cell of the grid. the decision is made inside snakeeatsfood() function. the algorithm used is commonly known as bounding box collision detection for 2d objects. to grow the snake, head moves one step ahead by leaving an empty cell behind. a new rectangle is created at that empty cell to give the impression of the growth of the snake. //grow snake length after eating food function growsnake(direction) { switch(direction) { case down_arrow: var x, y; x = snakehead.getx(); y = snakehead.gety()-10; resetpositions(createsnakepart(x,y)); break; ... ... resetpositions() is almost identical to reposition(), see the detils under "move the snake" heading. assign the food a new location once the snake eats the food, the food is assigned a new location on the grid. relocatefood() performs this function. it prepares a new grid skipping all the positions occupied by the snake. after creating a new grid array, random number generator function generates a number which is used as an index to the grid and eventually we get the coordinates where we can place the food without overlapping the snake. re-enter the snake from the opposite edge when it meets and passes an edge this is really simple. we let snake finish its move, then we check if the head is out of boundary. if it is, we assign it new coordinates to make it appear from the opposite end. the code given below works when snake is moving down, for example… if(snakehead.gety() == stageheight) snakehead.sety(0); end the game when snake hits himself or it occupies the whole ground after each move, checkgamestatus() is called by gameloop() to check if snake has hit himself or not. logic is fairly simple. the same bounding box collision detection method for 2d objects is used here. if coordinates of head matches the coordinates of any other part of the snake, the snake is dead – game end! live demo download in package today we prepared another one good tutorial using kineticjs and html5. i hope you enjoyed our lesson. good luck and welcome back.

Lots and lots of web applications need to page through information. From customer records, to the albums in your itunes collection. So as web developers and architects, it’s important that we do all this efficiently. Start by looking at how you’re fetching information from your MySQL database. We’ve outlined three ways to do just that. 1. Paging without discarding records Ultimately we’re trying to avoid discarding records. After all if the server doesn’t fetch them, we save big. How else can we avoid this extra work. How about remember the last name. For example: select id, name, address, phone FROM customers WHERE id > 990 ORDER BY id LIMIT 10; Of course such a solution would only work if you were paging by ID. If you page by name, it might get messier as there may be more than one person with the same name. If ID doesn’t work for your application, perhaps returning paged users by USERNAME might work. Those would be unique: SELECT id, username FROM customers WHERE username > '[email protected]' ORDER BY username LIMIT 10; Paging queries can be slow with SQL as they often involve the OFFSET keyword which instructs the server you only want a subset. However it typically scans collects and then discards those rows first. With deferred join or by maintaining a place or position column you can avoid this, and speedup your database dramatically. 2. Try using a Deferred Join This is an interesting trick. Suppose you have pages of customers. Each page displays ten customers. The query will use LIMIT to get ten records, and OFFSET to skip all the previous page results. When you get to the 100th page, it’s doing LIMIT 10 OFFSET 990. So the server has to go and read all those records, then discard them. SELECT id, name, address, phone FROM customers ORDER BY name LIMIT 10 OFFSET 990; MySQL is first scanning an index then retrieving rows in the table by primary key id. So it’s doing double lookups and so forth. Turns out you can make this faster with a tricky thing called a deferred join. The inside piece just uses the primary key. An explain plan shows us “using index” which we love! SELECT id FROM customers ORDER BY name LIMIT 10 OFFSET 990; Now combine this using an INNER JOIN to get the ten rows and data you want: SELECT id, name, address, phone FROM customers INNER JOIN ( SELECT id FROM customers ORDER BY name LIMIT 10 OFFSET 990) AS my_results USING(id); That’s pretty cool! 3. Maintain a Page or Place column Another way to trick the optimizer from retrieving rows it doesn’t need is to maintain a column for the page, place or position. Yes you need to update that column whenever you (a) INSERT a row (b) DELETE a row ( c) move a row with UPDATE. This could get messy with page, but a straight place or position might work easier. SELECT id, name, address, phone FROM customers WHERE page = 100 ORDER BY name; Or with place column something like this: SELECT id, name, address, phone FROM customers WHERE place BETWEEN 990 AND 999 ORDER BY name;

During any sort of programming you will always get situations where you need to compare values with each other, if the values are boolean or integers then the comparison is simple. But if you want to compare strings or parts of strings then there can be more to the comparison such as case of the string you are comparing. In this tutorial we are going to look at all the different ways you can compare strings in PHP using a number of built in PHP functions. == operator The most common way you will see of comparing two strings is simply by using the == operator if the two strings are equal to each other then it returns true. if('string1' == 'string1') { echo ' Strings match. '; } else { echo ' Strings do not match. '; } This code will return that the strings match, but what if the strings were not in the same case it will not match. If all the letters in one string were in uppercase then this will return false and that the strings do not match. if('string1' == 'STRING1') { echo ' Strings match. '; } else { echo ' Strings do not match. '; } This means that we can't use the == operator when comparing strings from user inputs, even if the first letter is in uppercase it will still return false. So we need to use some other function to help compare the strings. strcmp Function Another way to compare strings is to use the PHP function strcmp, this is a binary safe string comparison function that will return a 0 if the strings match. if(strcmp('string1', 'string1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } This if statement will return true and echo that the strings match. But this function is case sensitive so if one of the strings has an uppercase letter then the function will not return 0. strcasecmp Function The previous examples will not allow you to compare different case strings, the following function will allow you to compare case insensitive strings. if(strcasecmp('string1', 'string1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } if(strcasecmp('string1', 'String1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } if(strcasecmp('string1', 'STRING1') == 0) { echo ' Strings match. '; } else { echo ' Strings do not match. '; } All of these if statements will return that the strings match, which means that we can use this function when comparing strings that are input by the user.