What is JIT Compiler? The Just In Time Compiler (JIT) concept and more generally adaptive optimization is well known concept in many languages besides Java (.Net, Lua, JRuby). In order to explain what is JIT Compiler I want to start with a definition of compiler concept. According to wikipedia compiler is "a computer program that transforms the source language into another computer language (the target language)". We are all familiar with static java compiler (javac) that compiles human readable .java files to a byte code that can be interpreted by JVM - .class files. Then what does JIT compile? The answer will given a moment later after explanation of what is "Just in Time". According to most researches, 80% of execution time is spent in executing 20% of code. That would be great if there was a way to determine those 20% of code and to optimize them. That's exactly what JIT does - during runtime it gathers statistics, finds the "hot" code compiles it from JVM interpreted bytecode (that is stored in .class files) to a native code that is executed directly by Operating System and heavily optimizes it. Smallest compilation unit is single method. Compilation and statistics gathering is done in parallel to program execution by special threads. During statistics gathering the compiler makes hypotheses about code function and as the time passes tries to prove or to disprove them. If the hypothesis is dis-proven the code is deoptimized and recompiled again. The name "Hotspot" of Sun (Oracle) JVM is chosen because of the ability of this Virtual Machine to find "hot" spots in code. What optimizations does JIT? Let's look closely at more optimizations done by JIT. Inline methods - instead of calling method on an instance of the object it copies the method to caller code. The hot methods should be located as close to the caller as possible to prevent any overhead. Eliminate locks if monitor is not reachable from other threads Replace interface with direct method calls for method implemented only once to eliminate calling of virtual functions overhead Join adjacent synchronized blocks on the same object Eliminate dead code Drop memory write for non-volatile variables Remove prechecking NullPointerException and IndexOutOfBoundsException Et cetera When the Java VM invokes a Java method, it uses an invoker method as specified in the method block of the loaded class object. The Java VM has several invoker methods, for example, a different invoker is used if the method is synchronized or if it is a native method. The JIT compiler uses its own invoker. Sun production releases check the method access bit for value ACC_MACHINE_COMPILED to notify the interpreter that the code for this method has already been compiled and stored in the loaded class. JIT compiler compiles the method block into native code for this method and stores that in the code block for that method. Once the code has been compiled the ACC_MACHINE_COMPILED bit, which is used on the Sun platform, is set. How do we know what JIT is doing in our program and how can it be controlled? First of all to disable JIT Djava.compiler=NONE parameter can be used. There are 2 types of JIT compilers in Hotspot - one is used for client program and one for server (-server option in VM parameters). Program, running on server enjoys usually from more resources than program running on client and to server program top throughput is usually more important. Hence JIT in server is more resource consuming and gathering statistics takes more time to make the statistics more accurate. For client program gathering statics for a method lasts 1500 method calls, for server 15000. These default values can be changed by -XX:CompileThreshold=XXX VM parameter. In order to find out whether default value is good for you try enabling "XX:+PrintCompilation" and "-XX:-CITime" parameters that print JIT statistics and time CPU spent by JIT. Benchmarks Most of the benchmarks show that JITed code runs 10 to 20 times faster than interpreted code. There are many benchmarks done. Below given result graphs of two of them: Its worth to mention that programs that run in JIT mode, but are still in "learning mode" run much slower than non JITed programs. Drawbacks of JIT JIT Increases level of unpredictability and complexity in Java program. It adds another layer that developers don't really understand. Example of possible bugs - 'happens before relations" in concurrency. JIT can easily reorder code if the change is safe for a program running in single thread. To solve this problem developers make hints to JIT using "synchronized" word or explicit locking. Increases non heap memory footprint - JITed code is stored in "Code Cache" generation. Advanced JIT JIT and garbage collection. For GC to occur program must reach safe points. For this purpose JIT injects yieldpoints at regular intervals in native code. In addition to scanning of stack to find root references, registers must be scanned as they may hold objects created by JIT Comments are appresiated. The article can be found also at: artiomg.blogspot.com/2011/10/just-in-time-compiler-jit-in-hotspot.html

How often have we all heard that “batching” will increase latency? As someone with a passion for low-latency systems this surprises me. In my experience when batching is done correctly, not only does it increase throughput, it can also reduce average latency and keep it consistent. Well then, how can batching magically reduce latency? It comes down to what algorithm and data structures are employed. In a distributed environment we are often having to batch up messages/events into network packets to achieve greater throughput. We also employ similar techniques in buffering writes to storage to reduce the number of IOPS. That storage could be a block device backed file-system or a relational database. Most IO devices can only handle a modest number of IO operations per second, so it is best to fill those operations efficiently. Many approaches to batching involve waiting for a timeout to occur and this will by its very nature increase latency. The batch can also get filled before the timeout occurs making the latency even more unpredictable. Figure 1. Figure 1. above depicts decoupling the access to an IO device, and therefore the contention for access to it, by introducing a queue like structure to stage the messages/events to be sent and a thread doing the batching for writing to the device. The Algorithm An approach to batching uses the following algorithm in Java pseudo code: public final class NetworkBatcher implements Runnable { private final NetworkFacade network; private final Queue queue; private final ByteBuffer buffer; public NetworkBatcher(final NetworkFacade network, final int maxPacketSize, final Queue queue) { this.network = network; buffer = ByteBuffer.allocate(maxPacketSize); this.queue = queue; } @Override public void run() { while (!Thread.currentThread().isInterrupted()) { while (null == queue.peek()) { employWaitStrategy(); // block, spin, yield, etc. } Message msg; while (null != (msg = queue.poll())) { if (msg.size() > buffer.remaining()) { sendBuffer(); } buffer.put(msg.getBytes()); } sendBuffer(); } } private void sendBuffer() { buffer.flip(); network.send(buffer); buffer.clear(); } } Basically, wait for data to become available and as soon as it is, send it right away. While sending a previous message or waiting on new messages, a burst of traffic may arrive which can all be sent in a batch, up to the size of the buffer, to the underlying resource. This approach can use ConcurrentLinkedQueue which provides low-latency and avoid locks. However it has an issue in not creating back pressure to stall producing/publishing threads if they are outpacing the batcher whereby the queue could grow out of control because it is unbounded. I’ve often had to wrap ConcurrentLinkedQueue to track its size and thus create back pressure. This size tracking can add 50% to the processing cost of using this queue in my experience. This algorithm respects the single writer principle and can often be employed when writing to a network or storage device, and thus avoid lock contention in third party API libraries. By avoiding the contention we avoid the J-Curve latency profile normally associated with contention on resources, due to the queuing effect on locks. With this algorithm, as load increases, latency stays constant until the underlying device is saturated with traffic resulting in a more "bathtub" profile than the J-Curve. Let’s take a worked example of handling 10 messages that arrive as a burst of traffic. In most systems traffic comes in bursts and is seldom uniformly spaced out in time. One approach will assume no batching and the threads write to device API directly as in Figure 1. above. The other will use a lock free data structure to collect the messages plus a single thread consuming messages in a loop as per the algorithm above. For the example let’s assume it takes 100µs to write a single buffer to the network device as a synchronous operation and have it acknowledged. The buffer will ideally be less than the MTU of the network in size when latency is critical. Many network sub-systems are asynchronous and support pipelining but we will make the above assumption to clarify the example. If the network operation is using a protocol like HTTP under REST or Web Services then this assumption matches the underlying implementation. Best (µs) Average (µs) Worst (µs) Packets Sent Serial 100 500 1,000 10 Smart Batching 100 150 200 1-2 The absolute lowest latency will be achieved if a message is sent from the thread originating the data directly to the resource, if the resource is un-contended. The table above shows what happens when contention occurs and a queuing effect kicks in. With the serial approach 10 individual packets will have to be sent and these typically need to queue on a lock managing access to the resource, therefore they get processed sequentially. The above figures assume the locking strategy works perfectly with no perceivable overhead which is unlikely in a real application. For the batching solution it is likely all 10 packets will be picked up in first batch if the concurrent queue is efficient, thus giving the best case latency scenario. In the worst case only one message is sent in the first batch with the other nine following in the next. Therefore in the worst case scenario one message has a latency of 100µs and the following 9 have a latency of 200µs thus giving a worst case average of 190µs which is significantly better than the serial approach. This is one good example when the simplest solution is just a bit too simple because of the contention. The batching solution helps achieve consistent low-latency under burst conditions and is best for throughput. It also has a nice effect across the network on the receiving end in that the receiver has to process fewer packets and therefore makes the communication more efficient both ends. Most hardware handles data in buffers up to a fixed size for efficiency. For a storage device this will typically be a 4KB block. For networks this will be the MTU and is typically 1500 bytes for Ethernet. When batching, it is best to understand the underlying hardware and write batches down in ideal buffer size to be optimally efficient. However keep in mind that some devices need to envelope the data, e.g. the Ethernet and IP headers for network packets so the buffer needs to allow for this. There will always be an increased latency from a thread switch and the cost of exchange via the data structure. However there are a number of very good non-blocking structures available using lock-free techniques. For the Disruptor this type of exchange can be achieved in as little as 50-100ns thus making the choice of taking the smart batching approach a no brainer for low-latency or high-throughput distributed systems. This technique can be employed for many problems and not just IO. The core of the Disruptor uses this technique to help rebalance the system when the publishers burst and outpace the EventProcessors. The algorithm can be seen inside the BatchEventProcessor. Note: For this algorithm to work the queueing structure must handle the contention better than the underlying resource. Many queue implementations are extremely poor at managing contention. Use science and measure before coming to a conclusion. Batching with the Disruptor The code below shows the same algorithm in action using the Disruptor's EventHandler mechanism. In my experience, this is a very effective technique for handling any IO device efficiently and keeping latency low when dealing with load or burst traffic. public final class NetworkBatchHandler implements EventHander { private final NetworkFacade network; private final ByteBuffer buffer; public NetworkBatchHandler(final NetworkFacade network, final int maxPacketSize) { this.network = network; buffer = ByteBuffer.allocate(maxPacketSize); } public void onEvent(Message msg, long sequence, boolean endOfBatch) throws Exception { if (msg.size() > buffer.remaining()) { sendBuffer(); } buffer.put(msg.getBytes()); if (endOfBatch) { sendBuffer(); } } private void sendBuffer() { buffer.flip(); network.send(buffer); buffer.clear(); } } The endOfBatch parameter greatly simplifies the handling of the batch compared to the double loop in the algorithm above. I have simplified the examples to illustrate the algorithm. Clearly error handling and other edge conditions need to be considered. Separation of IO from Work Processing There is another very good reason to separate the IO from the threads doing the work processing. Handing off the IO to another thread means the worker thread, or threads, can continue processing without blocking in a nice cache friendly manner. I've found this to be critical in achieving high-performance throughput. If the underlying IO device or resource becomes briefly saturated then the messages can be queued for the batcher thread allowing the work processing threads to continue. The batching thread then feeds the messages to the IO device in the most efficient way possible allowing the data structure to handle the burst and if full apply the necessary back pressure, thus providing a good separation of concerns in the workflow. Conclusion So there you have it. Smart Batching can be employed in concert with the appropriate data structures to achieve consistent low-latency and maximum throughput. From http://mechanical-sympathy.blogspot.com/2011/10/smart-batching.html

as a user interface fan, i could not miss the development with html 5. so the goal of this post is to walk through a graphic application that uses javascript and html 5. we will see through examples one way (among others) to develop this kind of project. application overview tools the html 5 page data gathering cards loading & cache handling cards display mouse management state storage animations handling multi-devices conclusion to go further application overview we will produce an application that will let us display a magic the gathering ©(courtesy of www.wizards.com/magic ) cards collection. users will be able to scroll and zoom using the mouse (like bing maps, for example). you can see the final result here: http://bolaslenses.catuhe.com the project source files can be downloaded here: http://www.catuhe.com/msdn/bolaslenses.zip cards are stored on windows azure storage and use the azure content distribution network ( cdn : a service that deploys data near the final users) in order to achieve maximum performances. an asp.net service is used to return cards list (using json format). tools to write our application, we will use visual studio 2010 sp1 with web standards update . this extension adds intellisense support in html 5 page (which is a really important thing ). so, our solution will contain an html 5 page side by side with .js files (these files will contain javascript scripts). about debug, it is possible to set a breakpoint directly in the .js files under visual studio. it is also possible to use the developer bar of internet explorer 9 (use f12 key to display it). debug with visual studio 2010 debug with internet explorer 9 (f12/developer bar) so, we have a modern developer environment with intellisense and debug support. therefore, we are ready to start and first of all, we will write the html 5 page. the html 5 page our page will be built around an html 5 canvas which will be used to draw the cards: cards scanned by mwshq team magic the gathering official site : http://www.wizards.com/magic bolas lenses your browser does not support html5 canvas. loading data... if we dissect this page, we can note that it is divided into two parts: the header part with the title, the logo and the special mentions the main part (section) holds the canvas and the tooltips that will display the status of the application. there is also a hidden image ( backimage ) used as source for not yet loaded cards. to build the layout of the page, a style sheet ( full.css ) is applied. style sheets are a mechanism used to change the tags styles (in html, a style defines the entire display options for a tag): html, body { height: 100%; } body { background-color: #888888; font-size: .85em; font-family: "segoe ui, trebuchet ms" , verdana, helvetica, sans-serif; margin: 0; padding: 0; color: #696969; } a:link { color: #034af3; text-decoration: underline; } a:visited { color: #505abc; } a:hover { color: #1d60ff; text-decoration: none; } a:active { color: #12eb87; } header, footer, nav, section { display: block; } table { width: 100%; } header, #header { position: relative; margin-bottom: 0px; color: #000; padding: 0; } #title { font-weight: bold; color: #fff; border: none; font-size: 60px !important; vertical-align: middle; margin-left: 70px } #legal { text-align: right; color: white; font-size: 14px; width: 50%; position: absolute; top: 15px; right: 10px } #leftheader { width: 50%; vertical-align: middle; } section { margin: 20px 20px 20px 20px; } #maincanvas{ border: 4px solid #000000; } #cardscount { font-weight: bolder; font-size: 1.1em; } .tooltip { position: absolute; bottom: 5px; color: black; background-color: white; margin-right: auto; margin-left: auto; left: 35%; right: 35%; padding: 5px; width: 30%; text-align: center; border-radius: 10px; -webkit-border-radius: 10px; -moz-border-radius: 10px; box-shadow: 2px 2px 2px #333333; } #bolaslogo { width: 64px; height: 64px; } #picturecell { float: left; width: 64px; margin: 5px 5px 5px 5px; vertical-align: middle; } thus, this sheet is responsible for setting up the following display: style sheets are powerful tools that allow an infinite number of displays. however, they are sometimes complicated to setup (for example if a tag is affected by a class, an identifier and its container). to simplify this setup, the development bar of internet explorer 9 is particularly useful because we can use it to see styles hierarchy that is applied to a tag. for example let’s take a look at the waittext tooltip with the development bar. to do this, you must press f12 in internet explorer and use the selector to choose the tooltip: once the selection is done, we can see the styles hierarchy: thus, we can see that our div received its styles from the body tag and the . tooltip entry of the style sheet. with this tool, it becomes possible to see the effect of each style (which can be disabled). it is also possible to add new style on the fly. another important point of this window is the ability to change the rendering mode of internet explorer 9. indeed, we can test how, for example, internet explorer 8 will handle the same page. to do this, go to the [ browser mode ] menu and select the engine of internet explorer 8. this change will especially impact our tooltip as it uses border-radius (rounded edge) and box-shadow that are features of css 3: internet explorer 9 internet explorer 8 our page provides a graceful degradation as it still works (with no annoying visual difference) when the browser does not support all the required technologies. now that our interface is ready, we will take a look at the data source to retrieve the cards to display. the server provides the cards list using json format on this url: http://bolaslenses.catuhe.com/ home/listofcards/?colorstring=0 it takes one parameter ( colorstring ) to select a specific color (0 = all). when developing with javascript, there is a good reflex to have (reflex also good in other languages too, but really important in javascript): one must ask whether what we want to develop has not been already done in an existing framework. indeed, there is a multitude of open source projects around javascript. one of them is jquery which provides a plethora of convenient services. thus, in our case to connect to the url of our server and get the cards list, we could go through a xmlhttprequest and have fun to parse the returned json. or we can use jquery . so we will use the getjson function which will take care of everything for us: function getlistofcards() { var url = "http://bolaslenses.catuhe.com/home/listofcards/?jsoncallback=?"; $.getjson(url, { colorstring: "0" }, function (data) { listofcards = data; $("#cardscount").text(listofcards.length + " cards displayed"); $("#waittext").slidetoggle("fast"); }); } as we can see, our function stores the cards list in the listofcards variable and calls two jquery functions: text that change the text of a tag slidetoggle that hides (or shows) a tag by animating its height the listofcards list contains objects whose format is: id : unique identifier of the card path : relative path of the card (without the extension) it should be noted that the url of the server is called with the “ ?jsoncallback=? ” suffix. indeed, ajax calls are constrained in terms of security to connect only to the same address as the calling script. however, there is a solution called jsonp that will allow us to make a concerted call to the server (which of course must be aware of the operation). and fortunately, jquery can handle it all alone by just adding the right suffix. once we have our cards list, we can set up the pictures loading and caching. cards loading & cache handling the main trick of our application is to draw only the cards effectively visible on the screen. the display window is defined by a zoom level and an offset (x, y) in the overall system. var visucontrol = { zoom : 0.25, offsetx : 0, offsety : 0 }; the overall system is defined by 14819 cards that are spread over 200 columns and 75 rows. also, we must be aware that each card is available in three versions: high definition: 480x680 without compression (.jpg suffix) medium definition: 240x340 with standard compression (.50.jpg suffix) low definition: 120x170 with strong compression (.25.jpg suffix) thus, depending on the zoom level, we will load the correct version to optimize networks transfer. to do this we will develop a function that will give an image for a given card. this function will be configured to download a certain level of quality. in addition it will be linked with lower quality level to return it if the card for the current level is not yet uploaded: function imagecache(substr, replacementcache) { var extension = substr; var backimage = document.getelementbyid("backimage"); this.load = function (card) { var localcache = this; if (this[card.id] != undefined) return; var img = new image(); localcache[card.id] = { image: img, isloaded: false }; currentdownloads++; img.onload = function () { localcache[card.id].isloaded = true; currentdownloads--; }; img.onerror = function() { currentdownloads--; }; img.src = "http://az30809.vo.msecnd.net/" + card.path + extension; }; this.getreplacementfromlowercache = function (card) { if (replacementcache == undefined) return backimage; return replacementcache.getimageforcard(card); }; this.getimageforcard = function(card) { var img; if (this[card.id] == undefined) { this.load(card); img = this.getreplacementfromlowercache(card); } else { if (this[card.id].isloaded) img = this[card.id].image; else img = this.getreplacementfromlowercache(card); } return img; }; } an imagecache is built by giving the associated suffix and the underlying cache. here you can see two important functions: load : this function will load the right picture and will store it in a cache (the msecnd.net url is the azure cdn address of the cards) getimageforcard : this function returns the card picture from the cache if already loaded. otherwise it requests the underlying cache to return its version (and so on) so to handle our 3 levels of caches, we have to declare three variables: var imagescache25 = new imagecache(".25.jpg"); var imagescache50 = new imagecache(".50.jpg", imagescache25); var imagescachefull = new imagecache(".jpg", imagescache50); selecting the right cover is only depending on zoom: function getcorrectimagecache() { if (visucontrol.zoom <= 0.25) return imagescache25; if (visucontrol.zoom <= 0.8) return imagescache50; return imagescachefull; } to give a feedback to the user, we will add a timer that will manage a tooltip that indicates the number of images currently loaded: function updatestats() { var stats = $("#stats"); stats.html(currentdownloads + " card(s) currently downloaded."); if (currentdownloads == 0 && statsvisible) { statsvisible = false; stats.slidetoggle("fast"); } else if (currentdownloads > 1 && !statsvisible) { statsvisible = true; stats.slidetoggle("fast"); } } setinterval(updatestats, 200); again we note the use of jquery to simplify animations. we will now discuss the display of cards. cards display to draw our cards, we need to actually fill the canvas using its 2d context (which exists only if the browser supports html 5 canvas): var maincanvas = document.getelementbyid("maincanvas"); var drawingcontext = maincanvas.getcontext('2d'); the drawing will be made by processlistofcards function (called 60 times per second): function processlistofcards() { if (listofcards == undefined) { drawwaitmessage(); return; } maincanvas.width = document.getelementbyid("center").clientwidth; maincanvas.height = document.getelementbyid("center").clientheight; totalcards = listofcards.length; var localcardwidth = cardwidth * visucontrol.zoom; var localcardheight = cardheight * visucontrol.zoom; var effectivetotalcardsinwidth = colscount * localcardwidth; var rowscount = math.ceil(totalcards / colscount); var effectivetotalcardsinheight = rowscount * localcardheight; initialx = (maincanvas.width - effectivetotalcardsinwidth) / 2.0 - localcardwidth / 2.0; initialy = (maincanvas.height - effectivetotalcardsinheight) / 2.0 - localcardheight / 2.0; // clear clearcanvas(); // computing of the viewing area var initialoffsetx = initialx + visucontrol.offsetx * visucontrol.zoom; var initialoffsety = initialy + visucontrol.offsety * visucontrol.zoom; var startx = math.max(math.floor(-initialoffsetx / localcardwidth) - 1, 0); var starty = math.max(math.floor(-initialoffsety / localcardheight) - 1, 0); var endx = math.min(startx + math.floor((maincanvas.width - initialoffsetx - startx * localcardwidth) / localcardwidth) + 1, colscount); var endy = math.min(starty + math.floor((maincanvas.height - initialoffsety - starty * localcardheight) / localcardheight) + 1, rowscount); // getting current cache var imagecache = getcorrectimagecache(); // render for (var y = starty; y < endy; y++) { for (var x = startx; x < endx; x++) { var localx = x * localcardwidth + initialoffsetx; var localy = y * localcardheight + initialoffsety; // clip if (localx > maincanvas.width) continue; if (localy > maincanvas.height) continue; if (localx + localcardwidth < 0) continue; if (localy + localcardheight < 0) continue; var card = listofcards[x + y * colscount]; if (card == undefined) continue; // get from cache var img = imagecache.getimageforcard(card); // render try { if (img != undefined) drawingcontext.drawimage(img, localx, localy, localcardwidth, localcardheight); } catch (e) { $.grep(listofcards, function (item) { return item.image != img; }); } } }; // scroll bars drawscrollbars(effectivetotalcardsinwidth, effectivetotalcardsinheight, initialoffsetx, initialoffsety); // fps computefps(); } this function is built around many key points: if the cards list is not yet loaded, we display a tooltip indicating that download is in progress: var pointcount = 0; function drawwaitmessage() { pointcount++; if (pointcount > 200) pointcount = 0; var points = ""; for (var index = 0; index < pointcount / 10; index++) points += "."; $("#waittext").html("loading...please wait" + points); subsequently, we define the position of the display window (in terms of cards and coordinates), then we proceed to clean the canvas: function clearcanvas() { maincanvas.width = document.body.clientwidth - 50; maincanvas.height = document.body.clientheight - 140; drawingcontext.fillstyle = "rgb(0, 0, 0)"; drawingcontext.fillrect(0, 0, maincanvas.width, maincanvas.height); } then we browse the cards list and call the drawimage function of the canvas context. the current image is provided by the active cache (depending on the zoom): // get from cache var img = imagecache.getimageforcard(card); // render try { if (img != undefined) drawingcontext.drawimage(img, localx, localy, localcardwidth, localcardheight); } catch (e) { $.grep(listofcards, function (item) { return item.image != img; }); we also have to draw the scroll bar with the roundedrectangle function that uses quadratic curves: function roundedrectangle(x, y, width, height, radius) { drawingcontext.beginpath(); drawingcontext.moveto(x + radius, y); drawingcontext.lineto(x + width - radius, y); drawingcontext.quadraticcurveto(x + width, y, x + width, y + radius); drawingcontext.lineto(x + width, y + height - radius); drawingcontext.quadraticcurveto(x + width, y + height, x + width - radius, y + height); drawingcontext.lineto(x + radius, y + height); drawingcontext.quadraticcurveto(x, y + height, x, y + height - radius); drawingcontext.lineto(x, y + radius); drawingcontext.quadraticcurveto(x, y, x + radius, y); drawingcontext.closepath(); drawingcontext.stroke(); drawingcontext.fill(); } function drawscrollbars(effectivetotalcardsinwidth, effectivetotalcardsinheight, initialoffsetx, initialoffsety) { drawingcontext.fillstyle = "rgba(255, 255, 255, 0.6)"; drawingcontext.linewidth = 2; // vertical var totalscrollheight = effectivetotalcardsinheight + maincanvas.height; var scaleheight = maincanvas.height - 20; var scrollheight = maincanvas.height / totalscrollheight; var scrollstarty = (-initialoffsety + maincanvas.height * 0.5) / totalscrollheight; roundedrectangle(maincanvas.width - 8, scrollstarty * scaleheight + 10, 5, scrollheight * scaleheight, 4); // horizontal var totalscrollwidth = effectivetotalcardsinwidth + maincanvas.width; var scalewidth = maincanvas.width - 20; var scrollwidth = maincanvas.width / totalscrollwidth; var scrollstartx = (-initialoffsetx + maincanvas.width * 0.5) / totalscrollwidth; roundedrectangle(scrollstartx * scalewidth + 10, maincanvas.height - 8, scrollwidth * scalewidth, 5, 4); } and finally, we need to compute the number of frames per second: function computefps() { if (previous.length > 60) { previous.splice(0, 1); } var start = (new date).gettime(); previous.push(start); var sum = 0; for (var id = 0; id < previous.length - 1; id++) { sum += previous[id + 1] - previous[id]; } var diff = 1000.0 / (sum / previous.length); $("#cardscount").text(diff.tofixed() + " fps. " + listofcards.length + " cards displayed"); } drawing cards relies heavily on the browser's ability to speed up canvas rendering. for the record, here are the performances on my machine with the minimum zoom level (0.05): browser fps internet explorer 9 30 firefox 5 30 chrome 12 17 ipad (with a zoom level of 0.8) 7 windows phone mango (with a zoom level of 0.8) 20 (!!) the site even works on mobile phones and tablets as long as they support html 5. here we can see the inner power of html 5 browsers that can handle a full screen of cards more than 30 times per second! mouse management to browse our cards collection, we have to manage the mouse (including its wheel). for the scrolling, we'll just handle the onmouvemove , onmouseup and onmousedown events. onmouseup and onmousedown events will be used to detect if the mouse is clicked or not: var mousedown = 0; document.body.onmousedown = function (e) { mousedown = 1; getmouseposition(e); previousx = posx; previousy = posy; }; document.body.onmouseup = function () { mousedown = 0; }; the onmousemove event is connected to the canvas and used to move the view: var previousx = 0; var previousy = 0; var posx = 0; var posy = 0; function getmouseposition(eventargs) { var e; if (!eventargs) e = window.event; else { e = eventargs; } if (e.offsetx || e.offsety) { posx = e.offsetx; posy = e.offsety; } else if (e.clientx || e.clienty) { posx = e.clientx; posy = e.clienty; } } function onmousemove(e) { if (!mousedown) return; getmouseposition(e); mousemovefunc(posx, posy, previousx, previousy); previousx = posx; previousy = posy; } this function (onmousemove) calculates the current position and provides also the previous value in order to move the offset of the display window: function move(posx, posy, previousx, previousy) { currentaddx = (posx - previousx) / visucontrol.zoom; currentaddy = (posy - previousy) / visucontrol.zoom; } mousehelper.registermousemove(maincanvas, move); note that jquery also provides tools to manage mouse events. for the management of the wheel, we will have to adapt to different browsers that do not behave the same way on this point: function wheel(event) { var delta = 0; if (event.wheeldelta) { delta = event.wheeldelta / 120; if (window.opera) delta = -delta; } else if (event.detail) { /** mozilla case. */ delta = -event.detail / 3; } if (delta) { wheelfunc(delta); } if (event.preventdefault) event.preventdefault(); event.returnvalue = false; } we can see that everyone does what he wants :). the function to register with this event is: mousehelper.registerwheel = function (func) { wheelfunc = func; if (window.addeventlistener) window.addeventlistener('dommousescroll', wheel, false); window.onmousewheel = document.onmousewheel = wheel; }; and we will use this function to change the zoom with the wheel: // mouse mousehelper.registerwheel(function (delta) { currentaddzoom += delta / 500.0; }); finally we will add a bit of inertia when moving the mouse (and the zoom) to give some kind of smoothness: // inertia var inertia = 0.92; var currentaddx = 0; var currentaddy = 0; var currentaddzoom = 0; function doinertia() { visucontrol.offsetx += currentaddx; visucontrol.offsety += currentaddy; visucontrol.zoom += currentaddzoom; var effectivetotalcardsinwidth = colscount * cardwidth; var rowscount = math.ceil(totalcards / colscount); var effectivetotalcardsinheight = rowscount * cardheight var maxoffsetx = effectivetotalcardsinwidth / 2.0; var maxoffsety = effectivetotalcardsinheight / 2.0; if (visucontrol.offsetx < -maxoffsetx + cardwidth) visucontrol.offsetx = -maxoffsetx + cardwidth; else if (visucontrol.offsetx > maxoffsetx) visucontrol.offsetx = maxoffsetx; if (visucontrol.offsety < -maxoffsety + cardheight) visucontrol.offsety = -maxoffsety + cardheight; else if (visucontrol.offsety > maxoffsety) visucontrol.offsety = maxoffsety; if (visucontrol.zoom < 0.05) visucontrol.zoom = 0.05; else if (visucontrol.zoom > 1) visucontrol.zoom = 1; processlistofcards(); currentaddx *= inertia; currentaddy *= inertia; currentaddzoom *= inertia; // epsilon if (math.abs(currentaddx) < 0.001) currentaddx = 0; if (math.abs(currentaddy) < 0.001) currentaddy = 0; } this kind of small function does not cost a lot to implement, but adds a lot to the quality of user experience. state storage also to provide a better user experience, we will save the display window’s position and zoom. to do this, we will use the service of localstorage (which saves pairs of keys / values for the long term (the data is retained after the browser is closed) and only accessible by the current window object): function saveconfig() { if (window.localstorage == undefined) return; // zoom window.localstorage["zoom"] = visucontrol.zoom; // offsets window.localstorage["offsetx"] = visucontrol.offsetx; window.localstorage["offsety"] = visucontrol.offsety; } // restore data if (window.localstorage != undefined) { var storedzoom = window.localstorage["zoom"]; if (storedzoom != undefined) visucontrol.zoom = parsefloat(storedzoom); var storedoffsetx = window.localstorage["offsetx"]; if (storedoffsetx != undefined) visucontrol.offsetx = parsefloat(storedoffsetx); var storedoffsety = window.localstorage["offsety"]; if (storedoffsety != undefined) visucontrol.offsety = parsefloat(storedoffsety); } animations to add even more dynamism to our application we will allow our users to double-click on a card to zoom and focus on it. our system should animate three values: the two offsets (x, y) and the zoom. to do this, we will use a function that will be responsible of animating a variable from a source value to a destination value with a given duration: var animationhelper = function (root, name) { var paramname = name; this.animate = function (current, to, duration) { var offset = (to - current); var ticks = math.floor(duration / 16); var offsetpart = offset / ticks; var tickscount = 0; var intervalid = setinterval(function () { current += offsetpart; root[paramname] = current; tickscount++; if (tickscount == ticks) { clearinterval(intervalid); root[paramname] = to; } }, 16); }; }; the use of this function is: // prepare animations parameters var zoomanimationhelper = new animationhelper(visucontrol, "zoom"); var offsetxanimationhelper = new animationhelper(visucontrol, "offsetx"); var offsetyanimationhelper = new animationhelper(visucontrol, "offsety"); var speed = 1.1 - visucontrol.zoom; zoomanimationhelper.animate(visucontrol.zoom, 1.0, 1000 * speed); offsetxanimationhelper.animate(visucontrol.offsetx, targetoffsetx, 1000 * speed); offsetyanimationhelper.animate(visucontrol.offsety, targetoffsety, 1000 * speed); the advantage of the animationhelper function is that it is able to animate as many parameters as you wish (and that only with the settimer function!) handling multi-devices finally we will ensure that our page can also be seen on tablets pc and even on phones. to do this, we will use a feature of css 3: the media-queries . with this technology, we can apply style sheets according to some queries such as a specific display size: here we see that if the screen width is less than 480 pixels, the following style sheet will be added: #legal { font-size: 8px; } #title { font-size: 30px !important; } #waittext { font-size: 12px; } #bolaslogo { width: 48px; height: 48px; } #picturecell { width: 48px; } finally we will ensure that our page can also be seen on tablets pc and even on phones. to do this, we will use a feature of css 3: #legal { font-size: 8px; } #title { font-size: 30px !important; } #waittext { font-size: 12px; } #bolaslogo { width: 48px; height: 48px; } #picturecell { width: 48px; } conclusion html 5 / css 3 / javascript and visual studio 2010 allow to develop portable and efficient solutions (within the limits of browsers that support html 5 of course) with some great features such as hardware accelerated rendering. this kind of development is also simplified by the use of frameworks like jquery. also, i am especially fan of javascript that turns out to be a very powerful dynamic language. of course, c# or vb.net developers have to change theirs reflexes but for the development of web pages it's worth. in conclusion, i think that the best to be convinced is to try! to go further internet explorer test drive: http://ie.microsoft.com/testdrive/ internet explorer 9 guide for developer : http://msdn.microsoft.com/en-us/ie/ff468705 w3c site for html 5 : http://dev.w3.org/html5/spec/overview.html internet explorer site : http://msdn.microsoft.com/en-us/ie/aa740469 about the author david catuhe is a developer evangelist for microsoft france in charge of user experience development tools (from xaml to directx/xna and html5). he defines himself as a geek and likes coding all that refer to graphics. before working for microsoft, he founded a company that developed a realtime 3d engine written with directx ( www.vertice.fr ). source: http://blogs.msdn.com/b/eternalcoding/archive/2011/07/25/feedback-of-a-graphic-development-using-html5-amp-javascript.aspx

This tutorial will walk you throughout how to save and load an image from database (MySQL) using Hibernate. Requirements For this sampel project, we are going to use: Eclipse IDE (you can use your favorite IDE); MySQL (you can use any other database, make sure to change the column type if required); Hibernate jars and dependencies (you can download the sample project with all required jars); JUnit - for testing (jar also included in the sample project). PrintScreen When we finish implementing this sample projeto, it should look like this: Database Model Before we get started with the sample projet, we have to run this sql script into MySQL: DROP SCHEMA IF EXISTS `blog` ; CREATE SCHEMA IF NOT EXISTS `blog` DEFAULT CHARACTER SET latin1 COLLATE latin1_swedish_ci ; USE `blog` ; -- ----------------------------------------------------- -- Table `blog`.`BOOK` -- ----------------------------------------------------- DROP TABLE IF EXISTS `blog`.`BOOK` ; CREATE TABLE IF NOT EXISTS `blog`.`BOOK` ( `BOOK_ID` INT NOT NULL AUTO_INCREMENT , `BOOK_NAME` VARCHAR(45) NOT NULL , `BOOK_IMAGE` MEDIUMBLOB NOT NULL , PRIMARY KEY (`BOOK_ID`) ) ENGINE = InnoDB; This script will create a table BOOK, which we are going to use in this tutorial. Book POJO We are going to use a simple POJO in this project. A Book has an ID, a name and an image, which is represented by an array of bytes. As we are going to persist an image into the database, we have to use the BLOB type. MySQLhas some variations of BLOBs, you can check the difference between them here. In this example, we are going to use the Medium Blob, which can store L + 3 bytes, where L < 2^24. Make sure you do not forget to add the column definition on the Column annotation. package com.loiane.model; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.Id; import javax.persistence.Lob; import javax.persistence.Table; @Entity @Table(name="BOOK") public class Book { @Id @GeneratedValue @Column(name="BOOK_ID") private long id; @Column(name="BOOK_NAME", nullable=false) private String name; @Lob @Column(name="BOOK_IMAGE", nullable=false, columnDefinition="mediumblob") private byte[] image; public long getId() { return id; } public void setId(long id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public byte[] getImage() { return image; } public void setImage(byte[] image) { this.image = image; } } Hibernate Config This configuration file contains the required info used to connect to the database. com.mysql.jdbc.Driver jdbc:mysql://localhost/blog root root org.hibernate.dialect.MySQLDialect 1 true Hibernate Util The HibernateUtil class helps in creating the SessionFactory from the Hibernate configuration file. package com.loiane.hibernate; import org.hibernate.SessionFactory; import org.hibernate.cfg.AnnotationConfiguration; import com.loiane.model.Book; public class HibernateUtil { private static final SessionFactory sessionFactory; static { try { sessionFactory = new AnnotationConfiguration() .configure() .addPackage("com.loiane.model") //the fully qualified package name .addAnnotatedClass(Book.class) .buildSessionFactory(); } catch (Throwable ex) { System.err.println("Initial SessionFactory creation failed." + ex); throw new ExceptionInInitializerError(ex); } } public static SessionFactory getSessionFactory() { return sessionFactory; } } DAO In this class, we created two methods: one to save a Book instance into the database and another one to load a Book instance from the database. package com.loiane.dao; import org.hibernate.HibernateException; import org.hibernate.Session; import org.hibernate.Transaction; import com.loiane.hibernate.HibernateUtil; import com.loiane.model.Book; public class BookDAOImpl { /** * Inserts a row in the BOOK table. * Do not need to pass the id, it will be generated. * @param book * @return an instance of the object Book */ public Book saveBook(Book book) { Session session = HibernateUtil.getSessionFactory().openSession(); Transaction transaction = null; try { transaction = session.beginTransaction(); session.save(book); transaction.commit(); } catch (HibernateException e) { transaction.rollback(); e.printStackTrace(); } finally { session.close(); } return book; } /** * Delete a book from database * @param bookId id of the book to be retrieved */ public Book getBook(Long bookId) { Session session = HibernateUtil.getSessionFactory().openSession(); try { Book book = (Book) session.get(Book.class, bookId); return book; } catch (HibernateException e) { e.printStackTrace(); } finally { session.close(); } return null; } } Test To test it, first we need to create a Book instance and set an image to the image attribute. To do so, we need to load an image from the hard drive, and we are going to use the one located in the images folder. Then we can call the DAO class and save into the database. Then we can try to load the image. Just to make sure it is the same image we loaded, we are going to save it in the hard drive. package com.loiane.test; import static org.junit.Assert.assertNotNull; import java.io.File; import java.io.FileInputStream; import java.io.FileOutputStream; import org.junit.AfterClass; import org.junit.BeforeClass; import org.junit.Test; import com.loiane.dao.BookDAOImpl; import com.loiane.model.Book; public class TestBookDAO { private static BookDAOImpl bookDAO; @BeforeClass public static void runBeforeClass() { bookDAO = new BookDAOImpl(); } @AfterClass public static void runAfterClass() { bookDAO = null; } /** * Test method for {@link com.loiane.dao.BookDAOImpl#saveBook()}. */ @Test public void testSaveBook() { //File file = new File("images\\extjsfirstlook.jpg"); //windows File file = new File("images/extjsfirstlook.jpg"); byte[] bFile = new byte[(int) file.length()]; try { FileInputStream fileInputStream = new FileInputStream(file); fileInputStream.read(bFile); fileInputStream.close(); } catch (Exception e) { e.printStackTrace(); } Book book = new Book(); book.setName("Ext JS 4 First Look"); book.setImage(bFile); bookDAO.saveBook(book); assertNotNull(book.getId()); } /** * Test method for {@link com.loiane.dao.BookDAOImpl#getBook()}. */ @Test public void testGetBook() { Book book = bookDAO.getBook((long) 1); assertNotNull(book); try{ //FileOutputStream fos = new FileOutputStream("images\\output.jpg"); //windows FileOutputStream fos = new FileOutputStream("images/output.jpg"); fos.write(book.getImage()); fos.close(); }catch(Exception e){ e.printStackTrace(); } } } To verify if it was really saved, let’s check the table Book: and if we right click… and choose to see the image we just saved, we will see it: Source Code Download You can download the complete source code (or fork/clone the project – git) from: Github: https://github.com/loiane/hibernate-image-example BitBucket: https://bitbucket.org/loiane/hibernate-image-example/downloads Happy Coding! From http://loianegroner.com/2011/10/how-to-load-or-save-image-using-hibernate-mysql/

Version 2.3 of the Java servlet spec introduced the concept of filters. According to the documentation from Oracle’s site: “A filter dynamically intercepts requests and responses to transform or use the information contained in the requests or responses”. Today I’ll show you how to build a simple filter to intercept the response HTTP response code using annotations introduced in the Servlet 3.0 specification. With NetBeans IDE 7 create a new Maven Java Web Application called: Intercept Delete the index.jsp file under the Web Pages folder. Right-click on the project and add a new servlet called: MainServlet Since we are using the new Servlet 3 annotations we don’t need to set a whole lot of properties. Maven generates a decent MainServlet.java file for us, I just removed the comments for the output. My file looks like this: package com.giantflyingsaucer.intercept; import java.io.IOException; import java.io.PrintWriter; import javax.servlet.ServletException; import javax.servlet.annotation.WebServlet; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; @WebServlet(name = "MainServlet", urlPatterns = {"/"}) public class MainServlet extends HttpServlet { protected void processRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { response.setContentType("text/html;charset=UTF-8"); PrintWriter out = response.getWriter(); try { out.println(""); out.println(""); out.println(""); out.println(""); out.println(""); out.println("Servlet MainServlet"); out.println(""); out.println(""); } finally { out.close(); } } // /** * Handles the HTTP GET method. * @param request servlet request * @param response servlet response * @throws ServletException if a servlet-specific error occurs * @throws IOException if an I/O error occurs */ @Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { processRequest(request, response); } /** * Handles the HTTP POST method. * @param request servlet request * @param response servlet response * @throws ServletException if a servlet-specific error occurs * @throws IOException if an I/O error occurs */ @Override protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { processRequest(request, response); } /** * Returns a short description of the servlet. * @return a String containing servlet description */ @Override public String getServletInfo() { return "Short description"; }// } Right-click on the project and add a Filter called: InterceptFilter We will add the following two lines to the doFilter method. HttpServletResponse hsr = (HttpServletResponse) response; System.out.println("HTTP Status: " + hsr.getStatus()); My doFilter method looks like this: @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { if (debug) { log("InterceptFilter:doFilter()"); } doBeforeProcessing(request, response); HttpServletResponse hsr = (HttpServletResponse) response; System.out.println("HTTP Status: " + hsr.getStatus()); Throwable problem = null; try { chain.doFilter(request, response); } catch (Throwable t) { problem = t; t.printStackTrace(); } doAfterProcessing(request, response); if (problem != null) { if (problem instanceof ServletException) { throw (ServletException) problem; } if (problem instanceof IOException) { throw (IOException) problem; } sendProcessingError(problem, response); } } Clean and Build the project and deploy it to Apache Tomcat. Access the URL with a browser and take a look at your catalina.out file and you should see the HTTP response code. Note: You shouldn’t need to do any changes to the web.xml file for this project to work. From http://www.giantflyingsaucer.com/blog/?p=3279

this article is the fourth in a series about my adventures developing a fitness tracking application with html5, play scala, coffeescript and jade. previous articles can be found at: integrating scalate and jade with play 1.2.3 trying to make coffeescript work with scalate and play integrating html5 boilerplate with scalate and play developing features after getting my desired infrastructure setup, i started coding like a madman. the first feature i needed was a stopwatch to track the duration of a workout, so i started writing one with coffeescript. after spending 20 minutes playing with dates and settimeout, i searched and found a stopwatch jquery plug-in . i added this to my app, deployed it to heroku , brought up the app on my iphone 3g, clicked start and started riding my bike to work. when i arrived, i unlocked my phone and discovered that the time had stopped. at first, i thought this was a major setback. my disappointed disappeared when i found a super neat javascript stopwatch and kåre byberg's version that worked just fine. this stopwatch used settimeout, so by keeping the start time, the app on the phone would catch up as soon as you unlocked it. i ported kåre's script to coffeescript and rejoiced in my working stopwatch. # created by kåre byberg © 21.01.2005. please acknowledge if used # on other domains than http://www.timpelen.com. # ported to coffeescript by matt raible. also added hours support. flagclock = 0 flagstop = 0 stoptime = 0 refresh = null clock = null start = (button, display) -> clock = display startdate = new date() starttime = startdate.gettime() if flagclock == 0 $(button).html("stop") flagclock = 1 counter starttime, display else $(button).html("start") flagclock = 0 flagstop = 1 counter = (starttime) -> currenttime = new date() timediff = currenttime.gettime() - starttime timediff = timediff + stoptime if flagstop == 1 if flagclock == 1 $(clock).val formattime timediff, "" callback = -> counter starttime refresh = settimeout callback, 10 else window.cleartimeout refresh stoptime = timediff formattime = (rawtime, roundtype) -> if roundtype == "round" ds = math.round(rawtime / 100) + "" else ds = math.floor(rawtime / 100) + "" sec = math.floor(rawtime / 1000) min = math.floor(rawtime / 60000) hour = math.floor(rawtime / 3600000) ds = ds.charat(ds.length - 1) start() if hour >= 24 sec = sec - 60 * min + "" sec = prependzerocheck sec min = min - 60 * hour + "" min = prependzerocheck min hour = prependzerocheck hour hour + ":" + min + ":" + sec + "." + ds prependzerocheck = (time) -> time = time + "" # convert from int to string unless time.charat(time.length - 2) == "" time = time.charat(time.length - 2) + time.charat(time.length - 1) else time = 0 + time.charat(time.length - 1) reset = -> flagstop = 0 stoptime = 0 window.cleartimeout refresh if flagclock == 1 resetdate = new date() resettime = resetdate.gettime() counter resettime else $(clock).val "00:00:00.0" @stopwatch = { start: start reset: reset } the scalate/jade template to render this stopwatch looks as follows: script(type="text/javascript" src={uri("/public/javascripts/stopwatch.coffee")}) #display input(id="clock" class="xlarge" type="text" value="00:00:00.0" readonly="readonly") #controls button(id="start" type="button" class="btn primary") start button(id="reset" type="button" class="btn :disabled") reset :plain next, i wanted to create a map that would show your location. for this, i used merge design's html 5 geolocation demo as a guide. the html5 geo api is pretty simple, containing only three methods: // gets the users current position navigator.geolocation.getcurrentposition(successcallback, errorcallback, options); // request repeated updates of position watchid = navigator.geolocation.watchposition(successcallback, errorcallback); // cancel the updates navigator.geolocation.clearwatch(watchid); after rewriting the geolocation example in coffeescript, i ended up with the following code in my map.coffee script. you'll notice it uses google maps javascript api to show an actual map with a marker. # geolocation with html 5 and google maps api based on example from maxheapsize: # http://maxheapsize.com/2009/04/11/getting-the-browsers-geolocation-with-html-5/ # this script is by merge database and design, http://merged.ca/ -- if you use some, # all, or any of this code, please offer a return link. map = null mapcenter = null geocoder = null latlng = null timeoutid = null initialize = -> if modernizr.geolocation navigator.geolocation.getcurrentposition showmap showmap = (position) -> latitude = position.coords.latitude longitude = position.coords.longitude mapoptions = { zoom: 15, maptypeid: google.maps.maptypeid.roadmap } map = new google.maps.map(document.getelementbyid("map"), mapoptions) latlng = new google.maps.latlng(latitude, longitude) map.setcenter(latlng) geocoder = new google.maps.geocoder() geocoder.geocode({'latlng': latlng}, addaddresstomap) addaddresstomap = (results, status) -> if (status == google.maps.geocoderstatus.ok) if (results[1]) marker = new google.maps.marker({ position: latlng, map: map }) $('#location').html('your location: ' + results[0].formatted_address) else alert "sorry, we were unable to geocode that address." start = -> timeoutid = settimeout initialize, 500 reset = -> if (timeoutid) cleartimeout timeoutid @map = { start: start reset: reset } the template to show the map is a mere 20 lines of jade: script(type="text/javascript" src="http://www.google.com/jsapi") script(type="text/javascript" src="http://maps.googleapis.com/maps/api/js?sensor=false") :css .demo-map { border: 1px solid silver; height: 200px; margin: 10px auto; width: 280px; } #map(class="demo-map") p(id="location") span(class="label success") new | fetching your location with html 5 geolocation... script(type="text/javascript" src={uri("/public/javascripts/map.coffee")}) :javascript map.start(); the last two features i wanted were 1) distance traveled and 2) drawing the route taken on the map. for this i learned from a simple trip meter using the geolocation api . as i was beginning to port the js to coffeescript, i thought, "there's got to be a better way." i searched and found js2coffee to do most of the conversion for me. if you know javascript and you're learning coffeescript, this is an invaluable tool. i tried out the trip meter that night evening on a bike ride and noticed it said i'd traveled 3 miles when i'd really gone 6. i quickly figured out it was only calculating start point to end point and not taking into account all the turns in between. to view what was happening, i integrated my odometer.coffee with my map using google maps polylines . upon finishing the integration, i discovered two things, 1) html5 geolocation was highly inaccurate and 2) geolocation doesn't run in the background . i was able to solve the first problem by passing in {enablehighaccuracy: true} to navigator.geolocation.watchposition(). below are two screenshots showing before high accuracy and after. both screenshots are from the same two-block walk. the second issue is a slight show-stopper. phonegap might be able to solve the problem, but i'm currently using a workaround → turning off auto-lock and keeping safari in the foreground. making it look good after i got all my desired features developed, i moved onto making the app look good. i started by using sass for my css and installed play's sass module . i then switched to less when i discovered and added twitter's bootstrap to my project. at first i used play's less module (version 0.3), but ran into compilation issues . i then tried play's greenscript module , but gave up on it when i found it was incompatible with the coffeescript module . switching back to the less module and using the "0.3.compatibility" version solved all remaining issues. you might remember that i integrated html5 boilerplate and wondering why i have both bootstrap and boilerplate in my project. at this point, i don't think boilerplate is needed, but i've kept it just in case it's doing something for html5 cross-browser compatibility. i've renamed its style.css to style.less and added the following so it has access to bootstrap's variables. /* variables from bootstrap */ @import "libs/variables.less"; then i made my app look a lot better with layouts, stylish forms , a fixed topbar and alerts . for example, here's the coffeescript i wrote to display geolocation errors: geolocationerror = (error) -> msg = 'unable to locate position. ' switch error.code when error.timeout then msg += 'timeout.' when error.position_unavailable then msg += 'position unavailable.' when error.permission_denied then msg += 'please turn on location services.' when error.unknown_error then msg += error.code $('.alert-message').remove() alert = $('') alert.html('×' + msg); alert.insertbefore($('.span10')) then i set about styling up the app so it looked good on a smartphone with css3 media queries . below is the less code i used to hide elements and squish the widths for smaller devices. @media all and (max-device-width: 480px) { /* hide scrollbar on mobile */ html { overflow-y:hidden } /* hide sidebar on mobile */ .home .span4, .home .page-header, .topbar form { display: none } .home .container { width: 320px; } .about { .container, .span10 { width: 280px; } .span10 { padding-top: 0px; } } tools in the process of developing a stopwatch, odometer, displaying routes and making everything look good, i used a number of tools. i started out primarily with textmate and its bundles for less , coffeescript and jade . when i started writing more scala, i installed the scala textmate bundle . when i needed some debugging, i switched to intellij and installed its scala plugin. coffeescript, less and haml plugins (for jade) were already installed by default. i also used james ward's setup play framework with scala in intellij . issues i think it's obvious that my biggest issue so far is the fact that a webapp can't multitask in the background like a native app can. beyond that, there's accuracy issues with html5's geolocation that i haven't seen in native apps. i also ran into a caching issue when calling getcurrentposition(). it only worked the first time and i had to refresh my browser to get it to work again. strangely enough, this only happened on my desktop (in safari and firefox) and worked fine on my iphone. unfortunately, it looks like phonegap has issues similar to this. my workaround for no webapp multitasking is turning off auto-lock and leaving the browser in the foreground while i exercise. the downside to this is it really drains the battery quickly (~ 3 hours). i constantly have to charge my phone if i'm testing it throughout the day. the testing is a real pain too. i have to deploy to heroku (which is easy enough), then go on a walk or bike ride. if something's broke, i have to return home, tweak some things, redeploy and go again. also, there's been a few times where safari crashes halfway through and i lose all the tracking data. this happens with native apps too, but seemingly not as often. if you'd like to try the app on your mobile phone and see if you experience these issues, checkout play-more.com . summary going forward, there's still more html5 features i'd like to use. in particular, i'd like to play music while the fitness tracker is running. i'd love it if cloud music services (e.g. pandora or spotify) had an api i could use to play music in a webapp. soundcloud might be an option, but i've also thought of just uploading some mp3s and playing them with the tag. i've really enjoyed developing with all these technologies and haven't experienced much frustration so far. the majority has come from integrating scalate into play, but i've resolved most problems. next, i'll talk about how i've improved play's scalate support and my experience working with anorm . source: http://raibledesigns.com/rd/entry/developing_with_html5_coffeescript_and

Subtyping Issues in Java Behavior customization is a common requirement for large software systems. A software product cannot fully satisfy requirements of all customers; there are always differences that must be implemented, which are specific to each customer. Thus, software products must be flexible enough to allow these customizations in behavior even without changes in product's source code. In Java-based software products, a solution for such problems is usually based on a principle called subtyping. Subtyping can be defined on an example of two types A and B: if type B is a subtype of type A, then all code that operates correctly on objects of type A will operate correctly on objects of type B. Naturally, a software product behavior can be customized by replacing some default class A with custom class B where B is a subtype of A. If subtyping is based on an interface, then implementations of the interface can be independent of each other, which decreases complexity of the system and benefits modularity. At the same time such independence limits code reusability. A custom implementation can use Delegation pattern to call methods, but it is impossible to override a method in the base class. Another approach for subtyping is direct subclassing of the implementation class. Subclassing allows method overriding in the base class and also introduces design-time dependencies on that class. These dependencies limit subclass reusability. As you can see standard Java subtyping techniques limit reusability of either a base class or a subclass. Dynamic subtyping The approach described below combines the advantages of standard Java subtyping techniques. It allows creation of a new class, which can override methods in an original class, but at the same time, is dependent only on the interface of the original. Let's start with the following example: Suppose we have an interface Action as described below and its implementation in form of class ActionImpl. //This is an interface of Action public interface Action{ void doAction1(); void doAction2(); void doAll(); } //This is a default implementation of the interface Action class ActionImpl implements Action{ public void doAction1(){ ... } public void doAction2(){ ... } public void doAll(){ doAction1(); doAction2(); } } Class ActionImpl implements all methods of the interface and is used as a default implementation. Now let's suppose we want to override method doAction2 in the ActionImpl in such a way that the new class won't depend on ActionImpl. Let's create a new class ActionExtension as the following: //This is a custom implementation that overrides one method. abstract class ActionExtension extends ImplementationOf implements Action{ public void doAction2(){ … Super.doAction2(); } } This class extends a simple base class which looks like this: //This is base class for dynamic subclasses public abstract class ImplementationOf{ /** * A field for calling super class methods. * Should be used only in expressions where super keyword can be used. * */ protected final T Super = null; } Note that ActionExtension class is an abstract class, so we don't have to define all methods, just those that we want to override in ActionImpl. That's all we need to do at design time. Now let's look how it works at run-time. At the point of code where instance of Action is expected we need to create a new class: //Fragment of code that creates dynamic subclass of ActionImpl Class dynamicSubclass = DynamicClassExtender.extend(ActionImpl.class, Action.class, ActionExtension.class); The created class is a subclass of ActionImpl and contains all methods copied from ActionExtension class and modified in such a way that all calls to methods of Super field are directed to ActionImpl class. This class can be used to create an instance which can be used instead of instance of ActionImpl class: Action action = (Action) dynamicSubclass.newInstance(); In the real product the code above should be integrated into a dependency injection framework of your choice, and actual mapping of extension classes to implementations should be configurable. Dynamic class extender From the previous chapter we saw that all the complexities of dynamic subclassing are hidden in DynamicClassExtender class, which uses a byte code manipulation library to create a direct subclass of ActionImpl from ActionExtension class. In order to see what kind of byte code manipulation is involved let's start with ActionExtender class decompiled using standard javap utility: public abstract class ActionExtension extends ImplementationOf implements Action{ public ActionExtension(); Code: 0: aload_0 1: invokespecial #10; //Method ImplementationOf."":()V 4: return public void doAction2(); Code: 0: aload_0 1: getfield #17; //Field Super:Ljava/lang/Object; 4: checkcast #5; //class Action 7: invokeinterface #21, 1; //InterfaceMethodAction.doAction2:()V 12: return } Now if we manually create a class that extends ActionImpl and overrides doAction2 method and then decompile it with javap we'll see the following byte code: public class ActionExtension$ActionImpl extends ActionImpl{ public ActionExtension$ActionImpl(); Code: 0: aload_0 1: invokespecial #8; //Method ActionImpl."":()V 4: return public void doAction2(); Code: 0: aload_0 1: invokespecial #15; //Method ActionImpl.doAction2:()V 4: return } After comparing byte codes from both classes we can figure out a list of manipulations that DynamicClassExtender must apply on ActionExtension class in order to create a dynamic subclass of ActionImpl: change class name to ActionExtension$ActionImpl change access attribute to public, remove abstract attribute change super class name to ActionImpl replace references to ImplementationOf with references to ActionImpl replace method invocations on Super field (instructions 1,4,7 in ActionExtension.doAction2) with method invocations in super class (instruction 1 in ActionExtension$ActionImpl.doAction2) You can find a proof-of-concept implementation of DynamicClassExtender in the zip file attached to this article. The source code is distributed under the terms of BSD License. Conclusion The dynamic approach for subtyping, presented above, may help to create customizable Java applications without sharing implementation source code and without sacrificing code reusability.

i guess, i’m writing this post after a long time. this time, i’m writing about apache tika api that a friend of mine and i tried out to extract/retrieve metadata information from audio files supported by it – .mp3, .aiff, .au, .midi, .wav. to make it clear, here’s a screenshot of the information shown by windows vista about an audio file: we wanted to extract this using java and with googling, found that apache tika would help. we needed this metadata to index audio files for it to be searchable in a search application that we’re building using apache lucene . here’s a sample java program that extracts metadata from an mp3 file: package singz.samples.search.audio.metadata; import java.io.file; import java.io.fileinputstream; import java.io.filenotfoundexception; import java.io.ioexception; import java.io.inputstream; import org.apache.tika.exception.tikaexception; import org.apache.tika.metadata.metadata; import org.apache.tika.parser.parsecontext; import org.apache.tika.parser.parser; import org.apache.tika.parser.mp3.mp3parser; import org.xml.sax.contenthandler; import org.xml.sax.saxexception; import org.xml.sax.helpers.defaulthandler; /** * @author singaram subramanian * extract metadata of an audio file using apache tika api * */ public class audiometadataextractordemo { public static void main(string[] args) { // this audio file has metadata embedded in xmp (extensible metadata platform) standard // created by adobe systems inc. xmp standardizes the definition, creation, and // processing of extensible metadata. string audiofileloc = "c:\\pop\\backstreetboys_showmethemeaningofbeinglonely.mp3"; try { inputstream input = new fileinputstream(new file(audiofileloc)); contenthandler handler = new defaulthandler(); metadata metadata = new metadata(); parser parser = new mp3parser(); parsecontext parsectx = new parsecontext(); parser.parse(input, handler, metadata, parsectx); input.close(); // list all metadata string[] metadatanames = metadata.names(); for(string name : metadatanames){ system.out.println(name + ": " + metadata.get(name)); } // retrieve the necessary info from metadata // names - title, xmpdm:artist etc. - mentioned below may differ based // on the standard used for processing and storing standardized and/or // proprietary information relating to the contents of a file. system.out.println("title: " + metadata.get("title")); system.out.println("artists: " + metadata.get("xmpdm:artist")); system.out.println("genre: " + metadata.get("xmpdm:genre")); } catch (filenotfoundexception e) { e.printstacktrace(); } catch (ioexception e) { e.printstacktrace(); } catch (saxexception e) { e.printstacktrace(); } catch (tikaexception e) { e.printstacktrace(); } } } maven pom xml 4.0.0 singz.samples.search.audio audiometadataextractor 0.0.1 jar audiometadataextractor http://maven.apache.org utf-8 org.apache.tika tika-core 0.10 org.apache.tika tika-parsers 0.10 output xmpdm:releasedate: 2001 xmpdm:audiochanneltype: stereo xmpdm:album: top 100 pop author: backstreet boys xmpdm:artist: backstreet boys channels: 2 xmpdm:audiosamplerate: 44100 xmpdm:logcomment: eng xmpdm:tracknumber: 04 version: mpeg 3 layer iii version 1 xmpdm:composer: null xmpdm:audiocompressor: mp3 title: show me the meaning of being lonely samplerate: 44100 xmpdm:genre: pop content-type: audio/mpeg title: show me the meaning of being lonely artists: backstreet boys genre: pop about apache tika http://tika.apache.org/index.html “the apache tika™ toolkit detects and extracts metadata and structured text content from various documents using existing parser libraries.” http://www.lucidimagination.com/devzone/technical-articles/content-extraction-tika#article.tika “apache tika is a content type detection and content extraction framework. tika provides a general application programming interface that can be used to detect the content type of a document and also parse textual content and metadata from several document formats. tika does not try to understand the full variety of different document formats by itself but instead delegates the real work to various existing parser libraries such as apache poi for microsoft formats, pdfbox for adobe pdf, neko html for html etc. the grand idea behind tika is that it offers a generic interface for parsing multiple formats. the tika api hides the technical differences of the various parser implementations. this means that you don’t have to learn and consume one api for every format you use but can instead use a single api – the tika api. internally tika usually delegates the parsing work to existing parsing libraries and adapts the parse result so that client applications can easily manage variety of formats. tika aims to be efficient in using available resources (mainly ram) while parsing. the tika api is stream oriented so that the parsed source document does not need to be loaded into memory all at once but only as it is needed. ultimately, however, the amount of resources consumed is mandated by the parser libraries that tika uses. at the time of writing this, tika supports directly around 30 document formats. see list of supported document formats . the list of supported document formats is not limited by tika in any way. in the simplest case you can add support for new document formats by implementing a thin adapter that that implements the parser interface for the new document format.” about xmp standard http://en.wikipedia.org/wiki/extensible_metadata_platform “the adobe extensible metadata platform ( xmp ) is a standard, created by adobe systems inc. , for processing and storing standardized and proprietary information relating to the contents of a file. xmp standardizes the definition, creation, and processing of extensible metadata . serialized xmp can be embedded into a significant number of popular file formats, without breaking their readability by non-xmp-aware applications. embedding metadata avoids many problems that occur when metadata is stored separately. xmp is used in pdf , photography and photo editing applications. xmp can be used in several file formats such as pdf , jpeg , jpeg 2000 , jpeg xr , gif , png , html , tiff , adobe illustrator , psd , mp3 , mp4 , audio video interleave , wav , rf64 , audio interchange file format , postscript , encapsulated postscript , and proposed for djvu . in a typical edited jpeg file, xmp information is typically included alongside exif and iptc information interchange model data.” from http://singztechmusings.wordpress.com/2011/10/17/how-to-retrieveextract-metadata-information-from-audio-files-using-java-and-apache-tika-api/

our development team wanted to get notified as soon as something goes wrong in our production system, a critical java web application serving thousands of customers daily. the idea was to let it send us an email when there are too many errors, indicating usually a problem with a database, an external web service, or something really bad with the application itself. in this post i want to present a simple solution we have implemented using a custom log4j appender based on stats4j and an smtpappender (which is more difficult to configure and troubleshoot than you might expect) and in the following post i explore how to achieve the same effect with the open-source hyperic hq monitoring sw. the challenge we faced the following challenges with the logs: it’s unfortunately normal to have certain number of exceptions (customers select search criteria yielding no results, temporary, unimportant outages of external services etc.) and we certainly don’t want to be spammed because of that. so the solution must have a configurable threshold and only send an alert when it is exceeded. the failure rate should be computed for a configurable period (long enough not to trigger an alert because of few-minutes outages yet short enough for the team to be informed asap when something serious happens). once an alert is send, no further alerts should be send again for some time (ideally until the original problem is fixed), we don’t want to be spammed because of a problem we already know about. the solution we’ve based our solution on lara d’abreo’s stat4j , which provides a custom log4j appender that uses the logs to compute configurable measures and triggers alerts when they exceed their warning or critical thresholds. it is couple of years old, alpha-quality (regarding its generality and flexibility) open-source library, which is fortunately simple enough to be modified easily for one’s needs. so we have tweaked stat4j to produce alerts when the number of alerts exceeds thresholds and keep quiet thereafter and combined that with a log4j smtpappender that listens for the alerts and sends them via e-mail to the team. stat4j tweaking the key components of stat4j are the stat4jappender for log4j itself, calculators (measures) that aggregate the individual logs (e.g. by counting them or extracting some number form them), statistics that define which logs to consider via regular expressions and how to process them by referencing a calculator, and finally alerts that log a warning when the value of a statistics exceeds its limits. you can learn more in an article that introduces stat4j . we have implemented a custom measure calculator, runningrate (to count the number of failures in the last n minutes) and modified stat4j as follows: we’ve enhanced alert to support a new attribute, quietperiod , so that once triggered, subsequent alerts will be ignored for that duration (unless the previous alert was just a warning while the new one is a critical one) we’ve modified the appender to include the log’s throwable together with the log message, which is then passed to the individual statistics calcualtors, so that we could filter more precisely what we want to count finally we’ve modified alert to log alerts as errors instead of warnings so that the smtpappender wouldn’t ignore them get our modified stat4j from github (sources or a compiled jar ). disclaimer: it is one day’s hack and i’m not proud of the code. stat4j configuration take the example stat4j.properties and put it on the classpath. it is already configured with the correct calculator, statistics, and alert. see this part: ... ### jakub holy - my config calculator.minuterate.classname=net.sourceforge.stat4j.calculators.runningrate # period is in [ms] 1000 * 60 * 10 = 10 min: calculator.minuterate.period=600000 statistic.runningerrorrate.description=errors per 10 minutes statistic.runningerrorrate.calculator=minuterate # regular expression to match " <- " statistic.runningerrorrate.first.match=.*exception.* # error rate alert.toomanyerrorsrecently.description=too many errors in the log alert.toomanyerrorsrecently.statistic=runningerrorrate alert.toomanyerrorsrecently.warn= >=3 alert.toomanyerrorsrecently.critical= >=10 alert.toomanyerrorsrecently.category=alerts # ignore following warnings (or criticals, after the first critical) for the given amount of time: # 1000 * 60 * 100 = 100 min alert.toomanyerrorsrecently.quietperiod=6000000 the important config params are calculator.minuterate.period (in ms) – count errors over this period, reset the count at its end; a reasonable value may be 10 minutes alert.toomanyerrorsrecently.warn and .critical – trigger the alert when so many errors in the period has been encountered; reasonable values depend on your application’s normal error rate alert.toomanyerrorsrecently.quietperiod (in ms) – don’t send further alerts for this period not to spam in a persistent failure situation; the reasonable value depends on how quickly you usually fix problems, 1 hour would seem ok to me notice that statistic.runningerrorrate.first.match is a regular expression defining which logs to count; “.*” would include any log, “your\.package\..*exception” any exception in the package and so on, you can even specify logs to exclude using a negative lookahead ( (?! x )) log4j configuration now we need to tell log4j to use the stat4j appender to count error occurences and to send alerts via email: log4j.rootcategory=debug, console, fileappender, stat4jappender ... ### stat4jappender & emailalertsappender ### # collects statistics about logs and sends alerts when there # were too many failures in cooperation with the emailalertsappender ## stat4jappender log4j.appender.stat4jappender=net.sourceforge.stat4j.log4j.stat4jappender log4j.appender.stat4jappender.threshold=error # for configuration see stat4j.properties ## emailalertsappender # beware: smtpappender ignores its thresholds and only evers sends error or higher messages log4j.category.alerts=error, emailalertsappender log4j.appender.emailalertsappender=org.apache.log4j.net.smtpappender [email protected] # beware: the address below must have a valid domain or some receivers will reject it (e.g. gmail) [email protected] log4j.appender.emailalertsappender.smtphost=172.20.20.70 log4j.appender.emailalertsappender.buffersize=1 log4j.appender.emailalertsappender.subject=[stat4j] too many exceptions in log log4j.appender.emailalertsappender.layout=org.apache.log4j.patternlayout log4j.appender.emailalertsappender.layout.conversionpattern=%d{iso8601} %-5p %x{clientidentifier} %c %x - %m%n comments #8 specify the stat4j appender #9 only send errors to stat4j, we are not interested in less serious exceptions #14 “alerts” is the log category used by stat4jappender to log alerts (the same you would create via logger.getlogger(“alerts”)); as mentioned, smtpappender will without respect to the configuration only process errors and higher issues with the smtpappender it is quite tricky to get the smtpappender working. some pitfall: smtpappender ignores all logs that are not error or higher without respect to how you set its threshold if you specify a non-existing from domain then some recipient’s mail servers can just delete the email as spam (e.g. gmail) to send emails, you of course need mail.jar (and for older jvms also activation.jar), here are instructions for tomcat and one $100 tip: to debug it, run your application in the debug mode and set a method breakpoint on javax.mail.transport#send (you don’t need the source code) and when there, set this.session.debug to true to get a very detailed log of the following smtp communication in the server log. sidenote the fact that this article is based on log4j doesn’t mean i’d personally choose it, it just came with the project. i’d at least consider using the newer and shiny logback instead . conclusion stat4j + smtpappender are a very good base for a rather flexible do-it-yourself alerting system based on logs and e-mail. you can achieve the same thing out-out-the-box with hyperic hq plus much much more (provided that you get your admins to open two ports for it), which i will describe in the next blog post. links an alternative for preventing the smtpappender from spamming in persisten failure situations (aside of its built-in buffer size): log4j-email-throttle eventconsolidatingappender – announced via mailing list in 2/2011 – “the purpose of this appender is to consolidate multiple events that are received by a single logger within a specified number of seconds into a single event; this single consolidated event is then forwarded to a ‘downstream’ appender” from http://theholyjava.wordpress.com/2011/10/15/aggregating-error-logs-to-send-a-warning-email-when-too-many-of-them-log4j-stat4j-smtpappender/

If you've looked at my recent posts, you know I'm working on a plugin for VisualVM, a very useful tool supplied with the JDK. In one example, I showed how to attach to a waiting Java application using a socket-based AttachingConnector. At that time I said that there were two primary ways of attaching to a process with JDI -- via shared memory, and with a socket. It turns out there is a "third way". Following is an example of why this way is useful, and why it was provided. When I last wrote JDI programs (in Java 5), I would notice that my target application would start up and print (to stdout) the port on which it was listening, as in the following: Listening for transport dt_socket at address: 55779 In Java 5, if you detached your debugger from this process, you would get another line to stdout in the target's console, like this: Listening for transport dt_socket at address: 55779 and this would go on for as long as you chose to attach and detach, etc. At some point (and I don't know when this started happening), the port on which the target is listening started changing on each detach of an external debugger. If in Java 6 (I'm using u20), you repeatedly attach and detach from the target process, you'll see the following out in the target's console: Listening for transport dt_socket at address: 55837 ERROR: transport error 202: recv error: Connection reset by peer Listening for transport dt_socket at address: 55844 ERROR: transport error 202: recv error: Connection reset by peer Listening for transport dt_socket at address: 55846 ERROR: transport error 202: recv error: Connection reset by peer Listening for transport dt_socket at address: 55911 If you're writing an application that attaches using the debug port, each time you attach you need to find out what port the target is using. This information is not available from the process itself; in other words, you have to play the usual unpleasant game of capturing console output to know what the port is. Even if you specify a port at target start, you still need to get your hands on the value. You can still find the original request for a feature to attach to a process by its process ID if you search around the old Java bug reports. The long and short of it: a new AttachingConnector was created, one which attaches by PID. As you know, sometimes it isn't much fun finding a process's PID either. In my case, however, I am writing a plugin for VisualVM, and one thing you get for free when you do that is Visual VM's API, which as you might expect includes calls to get the PID. My goal, then, is to use this new connector in my VisualVM plugin, and I thought it might be appreciated if I shared the details. I've adapted my test program from an earlier post so that it now outputs the details of each AttachingConnector; the changed code fragment is shown here: List attachingConnectors = vmMgr.attachingConnectors(); for (AttachingConnector ac: attachingConnectors) { Map paramsMap = ac.defaultArguments(); Iterator keyIter = paramsMap.keySet().iterator(); System.out.println("AttachingConnector: '" + ac.getClass().getName() + "'"); System.out.println(" name: '" + ac.name() + "'"); System.out.println(" description: '" + ac.description() + "'"); System.out.println(" transport name: '" + ac.transport().name() + "'"); System.out.println(" default arguments:"); while (keyIter.hasNext()) { String nextKey = keyIter.next(); System.out.println(" key: '" + nextKey + "'; value: '" + paramsMap.get(nextKey) + "'"); } } The output from this code is shown below: AttachingConnector: 'com.sun.tools.jdi.SocketAttachingConnector' name: 'com.sun.jdi.SocketAttach' description: 'Attaches by socket to other VMs' transport name: 'dt_socket' default arguments: key: 'timeout'; value: 'timeout=' key: 'hostname'; value: 'hostname=AdamsResearch' key: 'port'; value: 'port=' AttachingConnector: 'com.sun.tools.jdi.SharedMemoryAttachingConnector' name: 'com.sun.jdi.SharedMemoryAttach' description: 'Attaches by shared memory to other VMs' transport name: 'dt_shmem' default arguments: key: 'timeout'; value: 'timeout=' key: 'name'; value: 'name=' AttachingConnector: 'com.sun.tools.jdi.ProcessAttachingConnector' name: 'com.sun.jdi.ProcessAttach' description: 'Attaches to debuggee by process-id (pid)' transport name: 'local' default arguments: key: 'pid'; value: 'pid=' key: 'timeout'; value: 'timeout=' A couple of things I hadn't noticed before is that the socket-based connector comes with the hostname argument pre-set to my machine's hostname, and that all three connectors have a timeout default argument. The first observation brings up an interesting point: if you use the local, PID-based connector, remember that you'll only be attaching to processes on your debugger's host. I changed my test program to use the local connector and it works as before! Well, no, actually, it does not. Here's what I now get: java.lang.UnsatisfiedLinkError: no attach in java.library.path Exception in thread "main" java.io.IOException: no providers installed at com.sun.tools.jdi.ProcessAttachingConnector.attach(ProcessAttachingConnector.java:86) at com.adamsresearch.jdiDemo.JDIDemo.main(JDIDemo.java:70) Does this mean the local connector isn't exactly ready for use? No, but I have been burned by the same issue that has plagued a number of others (scroll down in that page -- the issue was found by a reader of that post and was solved, partially, by another reader of that post). I'm working on a Windows platform, and when you do that you have to be a little careful ;-> . In this case, the problem is caused by 1) using the java interpreter as found on the system path, and 2) not making sure that path points directly to your JDK or JRE directory. The executable will look in a path relative to itself for the needed libraries, and when Windows copies the java executable to C:\Windows\system32 (or similar) -- and if you use that executable -- that relative path is broken. I believe this is the true issue, unlike described in the comments on the above post, where the distinction is made between using the JRE java and the JDK java. I don't think that's the issue. For example, below are the results of my attach test in 3 different scenarios: Using java from my path, the first hit of which comes from C:\Windows\system32: java -cp c:\jdk1.6.0_20\lib\tools.jar;. com.adamsresearch.jdiDemo.JDIDemo 10816 863 fileName ... java.lang.UnsatisfiedLinkError: no attach in java.library.path Exception in thread "main" java.io.IOException: no providers installed at com.sun.tools.jdi.ProcessAttachingConnector.attach(ProcessAttachingConnector.java:86) at com.adamsresearch.jdiDemo.JDIDemo.main(JDIDemo.java:70) Using the full path to the JRE bin java: c:\jdk1.6.0_20\jre\bin\java -cp c:\jdk1.6.0_20\lib\tools.jar;. com.adamsresearch.jdiDemo.JDIDemo 10816 863 fileName ... Attached to process 'Java HotSpot(TM) 64-Bit Server VM' Using the full path to the JDK bin java: c:\jdk1.6.0_20\bin\java -cp c:\jdk1.6.0_20\lib\tools.jar;. com.adamsresearch.jdiDemo.JDIDemo 10816 863 fileName ... Attached to process 'Java HotSpot(TM) 64-Bit Server VM' As you can see, the above seems to support my theory that it's not the JRE vs the JDK, but rather the context-poor placement of the java executable in the "usual" Windows binaries directory, that caused the problem. That posting is several years old, so it is possible that at that time, the needed JDI libraries actually were not included in the JRE, but it is clear that today, you will see the same exception if you use the java executable found in Windows' default binaries directory. Now, if I run my JDI application against my JarView utility, searching for AttachingConnector in the JDK installation directory, I get the following output: Breakpoint at line 863: fileName = 'AttachingConnector.class' Breakpoint at line 863: fileName = 'GenericAttachingConnector$1.class' Breakpoint at line 863: fileName = 'GenericAttachingConnector.class' Breakpoint at line 863: fileName = 'ProcessAttachingConnector$1.class' Breakpoint at line 863: fileName = 'ProcessAttachingConnector$2.class' Breakpoint at line 863: fileName = 'ProcessAttachingConnector.class' Breakpoint at line 863: fileName = 'SharedMemoryAttachingConnector$1.class' Breakpoint at line 863: fileName = 'SharedMemoryAttachingConnector.class' Breakpoint at line 863: fileName = 'SocketAttachingConnector$1.class' Breakpoint at line 863: fileName = 'SocketAttachingConnector.class' and so have done what I set out to do, which is 1) debug-attach by process ID, and 2) thrash through the inevitable hiccups and share the solutions. Hopefully this will be useful to you, too. Note: actually, there are even more ways to attach to a Java process. JPDA Connection and Invocation is the definitive guide, from Oracle. If you're going to be writing debuggers, you can't go wrong reading this page first. From http://wayne-adams.blogspot.com/2011/10/jdi-three-ways-to-attach-to-java.html

Let’s say you are using some external components in your web page that renders a button as below: The above code will call the delete() method when user clicks on the button. But, what if you want to show a confirmation box before calling the delete()? Since the code is rendered by a external component, most probably you won’t be having control over how its code generated. So, you have to intercept the onclick event and block it until your method is called and then invoke the original method. Lets do it.. var btn = $("#btn"); btn.data("funcToCall", btn.attr("onclick")); $("#btn").removeAttr("onclick"); $("#btn").bind("click", function(e){ if(confirm("Are you sure?")){ var func = $(this).data("funcToCall"); eval(func); } }); As you can see, the onclick function is first stored into that element and then the onclick attribute is removed. Finally when the button in clicked, our custom method is called first and based on the user decision, the delete() method is either called or not called. From http://veerasundar.com/blog/2011/10/intercepting-onclick-event-in-javascript-using-jquery/

I had to answer the following question: Shall we use a Portal and if yes, should it be Liferay Portal or Oracle Portal? Or shall we use just one or more Java web frameworks? This article shows my result. I had to look especially at Liferay and Oracle products, nevertheless the result can be used for other products, too. The short answer: A Portal makes sense only in a few use cases, in the majority of cases you should not use one. In my case, we will not use one. What is a Portal? It is important to know that we are talking about an Enterprise Portal. Wikipedia has a good definition: „An enterprise portal [...] is a framework for integrating information, people and processes across organizational boundaries. It provides a secure unified access point, often in the form of a web-based user interface, and is designed to aggregate and personalize information through application-specific portlets.“ Several Portal server are available in the Java / JVM environment. Liferay Portal and GateIn Portal (former JBoss Portal) are examples for open source products while Oracle Portal or IBM WebSphere Portal are proprietary products. You develop Portlets („simple“ web applications) and deploy them in your portal. If you need to know more about a Portal or the Portlet JSR standards, ask Wikipedia: http://en.wikipedia.org/wiki/Portlet. Should we use a Portal or not? I found several pros and cons for using a Portal instead of just web applications. Disadvantages of using a Portal: Higher complexity Additional configuration (e.g. portlet.xml, Portal server) Communication between Portlets using Events is not trivial (it is also not trivial if two applications communicate without portlets, of course) Several restrictions when developing a web application within a Portlet Additional testing efforts (test your web applications and test it within a Portal and all its Portal features) Additional costs Open source usually offers enterprise editions which include support (e.g. Liferay) Proprietary products have very high initial costs. Besides, you need support, too (e.g. Oracle) You still have to customize the portal and integrate applications. A portal product does not give you corporate identity or systems integration for free. Software licensing often is only ten percent of the total price. Developers need additional skills besides using a web framework Several restrictions must be considered choosing a web-framework and implement the web application Rethinking about web application design is necessary Portlets use other concepts such as events or an action and render phase instead of only one phase Frameworks (also called bridges) help to solve this problem (but these are standardized for JSF only, a few other plugins are available, e.g. for GWT or Grails) Actually, IMO you have to use JSF if you want to realize Portlets in a stable, relatively „easy“ and future-proof way. There is no standard bridge for other frameworks. There are no books, best practices, articles or conference sessions about Portlets without JSF, right? Advantages of using a Portal Important: Many of the pros can be realized by oneself with relatively low efforts (see the "BUT" notes after each bullet point). Single Sign On (SSO) BUT: Several Java frameworks are available, e.g OpenSSO (powerful, but complicated) or JOSSO (not so powerful, but easy to use). Good products are available, e.g. Atlassian Crowd (I love Atlassian products such as Crowd, Jira or Confluence, because they are very intuitive and easy to use). Integration of several applications within one GUI A portal gives you layout and sequence of the applications for free (including stuff such as drag & drop, minimizing windows, and so on) Communication between Portlets (i.e. between different applications) BUT: This is required without a portal, too. Several solutions can be used, such as a database, messaging, web services, events, and so on. Even „push“ is possible for some time now (using specific web framework features or HTML 5 websockets). Uniform appearence BUT: CSS can solve this problem (the keyword „corporate identity“ exists in almost every company). Create a HTML template and include your applications within this template. Done. Personalization Regarding content, structure or graphical presentation Based on individual preferences or metadata BUT: Some of these features can be realized very easily by oneself (e.g. a simple role concept). Nevertheless, GUI features such as drag & drop are more effort (although component libraries can help you a lot). Many addons are included Search Content management Document management Web 2.0 tools (e.g. blogs or wikis) Collaboration suites (e.g. team pages) Analytics and reporting Development platforms BUT: A) Do you really need these Features? B) Is the offered functionality sufficent? Portals only offer „basic“ versions of stand-alone products. For instance, the content management system or search engine of a Portal is less powerful than other „real“ products offering this functionality. Thus, you have to think about the following central question: Do we really need all those features offered by a portal? Conclusion: The total cost of ownership (TCO) is much higher when using a portal. You have to be sure, that you really need the offered features. In some situations, you can defer your decision. Create your web applications as before. You can still integrate them in a Portal later, if you really need one. For instance, the following Oracle blog describes how you can use iFrames to do this: http://blogs.oracle.com/jheadstart/entry/integrating_a_jsf_application If you decide to use a Portal, you have to choose a Portal product. Should we use an Open Source or Proprietary Portal Product? Both, open source and proprietary Portal products have pros and cons. I especially looked at Oracle Portal and Liferay Portal, but probably most aspects can be considered when evaluating other products, too. Advantages of Oracle Portal Oracle offers a full-stack suite for development (including JSF and Portlets): Oracle Application Development Framework (ADF) Oracle JDeveloper offers good support for ADF. Everything from one product line increases efficiency (database, application server, ESB, IDE, Portal, …) – at least in theory :-) Disadvantages of Oracle Portal: High initial costs (I heard something about 200K in our company) Complex, heavyweight product (compared to Liferay Portal) Proprietary Communication between Portlets is not implemented using the standard JSR-286, but a custom proprietary solution (Source: http://www.contribute.be/web/contribute/news/-/journal_content/56_INSTANCE_pdF5/10234/21893) Advantages of Liferay Portal: Open source Drastically lower initial costs Lightwight product (1-Click-Install, etc.) Disadvantages of Liferay Portal: Not everything is from one product line (this cannot be considered as disadvantage always, but in our case the customer preferred very few different vendors (keyword “IT consolidation”) Portlets are still Portlets. Although Liferay is lightweight, realizing Portlets still sucks as it does with a proprietary product When to use a Portal? Well, the conclusion is difficult. In my opinion, it does make sense only in a few use cases. If you really need many or all of those Portal features, and they are also sufficient, then use a Portal product. Though, usually it is much easier to create a simple web application which integrates your applications. Use a SSO framework, create a template, and you are done. Your developers will appreciate not to work with Portlets and its increased complexity and restrictions. Did I miss any pros or cons? Do you have another opinion (probably, many people do???), then please write a comment and let’s discuss… Best regards, Kai Wähner (Twitter: @KaiWaehner) [Content from my Blog: Kai Wähner's Blog: Pros and Cons - When to use a Portal and Portlets instead of just Java Web-Frameworks]

If you need to rename the Java package of a 3rd party library, e.g. to include it directly in your project while avoiding possible conflicts, you can use one of the following Maven plugins (and they may be more) in the package lifecycle phase: Uberize plugin (latest – org.fusesource.mvnplugins:maven-uberize-plugin:1.20) – originally inspired by the Shade plugin, intended to overcome some of its limitations. Intended primarily to merge your code and dependencies into one jar. Shade plugin package-rename-task, Ant-based Maven plugin – I’m not sure whether this is further maintained From http://theholyjava.wordpress.com/2011/10/06/tools-for-renaming-the-package-of-a-dependency-with-maven/

This may not be strictly true, but it seems to me that the Guy’s at Spring are always banging on about AspectJ and Aspect Oriented Programming (AOP), to the point where I suspect that it’s used widely under the hood and is an integral part of Spring and I say “widely used under the hood”, because I haven’t come across too many projects that do use AspectJ or AOP in general. I suspect that part of the reason for this is possibly down to the fact that AOP different is a concept to Object Oriented Programming (OOP). AOP, they say, is all to do with an application’s cross-cutting concerns, which translates to mean stuff that’s common to all classes within your application. The usual example given here is logging and example code usually demonstrates logging all method entry and exit details, which is something that I’ve never found that useful. The other reason that I’ve not seen it used is that the AspectJ documentation is a bit ropey. Today’s blog is a demonstration of how to implement AspectJ’s @AfterThrowing advice in a Spring application. The idea of the after throwing advice is that you intercept an exception after it’s thrown, but before it’s caught - as shown in this rather simplistic diagram: I said above that the AOP sample code usually demonstrates logging, and in that respect this blog is no different. The idea in this contrived scenario is to log to a simple Apache Commons log any exceptions thrown. The class that actually does all this is IncidentThrowsAdvice: @Aspect public class IncidentThrowsAdvice { // Obtain a suitable logger. private static Log logger = LogFactory.getLog(IncidentThrowsAdvice.class); /** * Called between the throw and the catch */ @AfterThrowing(pointcut = "execution(* *(String, ..))", throwing = "e") public void myAfterThrowing(JoinPoint joinPoint, Throwable e) { System.out.println("Okay - we're in the handler..."); Signature signature = joinPoint.getSignature(); String methodName = signature.getName(); String stuff = signature.toString(); String arguments = Arrays.toString(joinPoint.getArgs()); logger.info("Write something in the log... We have caught exception in method: " + methodName + " with arguments " + arguments + "\nand the full toString: " + stuff + "\nthe exception is: " + e.getMessage(), e); } } The class itself is fairly straight forward. It has one method myAfterThrowing(...), which takes two arguments of type: JoinPoint and Throwable. Taking each of these in turn, JoinPoint just an class that holds information that describes a point within your code. Applying it to this particular after throwing advice means that it’s the point in your code where the exception occurred. The second argument is more straight forward as it’s the actual exception that’s currently being thrown There are two annotations applied to this class: @Aspect and @AfterThrowing. @Aspect marks the AnyOldExampleBean class as an AspectJ class, whilst the second annotation: @AfterThrowing is more interesting. This annotation tells AspectJ to call the myAfterThrowing() method when an exception occurs. It has two attributes, pointcut and throwing. pointcut defines the circumstances in which the myAfterThrowing() method is called as defined by the expression * *.*(..). This, like the rest of AspectJ seems remarkably badly documented, however you can determine that this signifies a method signature. In this case we’re checking every method in every class. This expression breaks down as follows: * - The first star is method visibility and or the method return type. The following will work in this example: * public void void whereas public on its own throws a BeanCreationException exception when loading the Spring config. *.* - this represents the package and method, again, using the same wild card formatting; hence, the following are all valid: example_10_annotations.afterthrowing_annotation.Sneeze.sneeze *.* *.sneeze example_10_annotations.*.Sneeze.sneeze ...however, example_10_annotations.*.sneeze won’t match for some reason... (..) - Defines the method arguments. In this example, the following will match: (..) (String, String, int) (String, ..) Having defined an AspectJ after throwing advice, the next step is to integrate it into the Spring application and this is a matter of adding one line to your Spring config file together with the appropriate schema reference in your : XML element: The important line in this file is: ...as it switches AOP on. The bean definitions that follow it demonstrate how the after throws advice works. The Sneeze class simply throws an exception when its called and the AnyOldExampleBean is a simple test class that calls Sneeze.sneeze(...) as shown below: public class Sneeze { /** * Throw an exception */ public void sneeze(String arg0, String arg1, int i) throws Exception { throw new Exception("Simulate an error"); } } public class AnyOldExampleBean { private Sneeze sneeze; /** * @return */ public Sneeze getSneeze() { return sneeze; } /** * @param sneeze */ public void setSneeze(Sneeze sneeze) { this.sneeze = sneeze; } /** * Do something * * @return */ public void run() { try { sneeze.sneeze("arg0", "arg1", 42); } catch (Exception e) { System.out.println("Caught e"); } finally { System.out.println("the end..."); } } } The code to run the above is: ApplicationContext ctx = new ClassPathXmlApplicationContext("example10_throwsadvice.xml"); AnyOldExampleBean myExampleBean = ctx.getBean(AnyOldExampleBean.class); myExampleBean.run(); ...and when running this code, you should get the following output: This is the after throwing exception handler 13:41:46,399 INFO ClassPathXmlApplicationContext:456 - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@4ce2cb55: startup date [Sun Aug 14 13:41:46 BST 2011]; root of context hierarchy 13:41:46,462 INFO XmlBeanDefinitionReader:315 - Loading XML bean definitions from class path resource [example10_throwsadvice.xml] 13:41:46,863 INFO DefaultListableBeanFactory:555 - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@663257b8: defining beans [org.springframework.aop.config.internalAutoProxyCreator,exceptionHandler,example_10_annotations.afterthrowing_annotation.AfterThrowingBean#0,sneeze]; root of factory hierarchy Okay - we're in the handler... 13:41:47,171 INFO IncidentThrowsAdvice:42 - Write something in the log... We have caught exception in method: sneeze with arguments [arg0, arg1, 42] and the full toString: void example_10_annotations.afterthrowing_annotation.Sneeze.sneeze(String,String,int) the exception is: Simulate an error java.lang.Exception: Simulate an error at example_10_annotations.afterthrowing_annotation.Sneeze.sneeze(Sneeze.java:20) at example_10_annotations.afterthrowing_annotation.Sneeze$$FastClassByCGLIB$$5c47789.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149) at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:55) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ...THE REST HAS BEEN REMOVED FOR CLARITY... Caught e the end... As I said above, AspectJ seems very badly documented as can be borne out by the JavaDocs, which if you look through means that the code contains very little documentation, which in turn means that I’m glad I’m not working on it... Finally, in covering the after throwing advice, this blog only really touches on AspectJ and there are other useful AspectJ advice annotations that I will probably be covering in the near future. From http://www.captaindebug.com/2011/09/using-aspectjs-afterthrowing-advice-in.html

Everyone in the Java world seems to use various opensource libraries and frameworks... and why not, there are hundreds available covering virtually every type of programming problem you’re likely to come across in today’s programming landscape. This blog takes a quick look at the reasons for using opensource artifacts and examines what could go wrong... The first reason for using them is reduced cost as it’s cheaper for your project to grab hold of an opensource library than it is for your to write the same thing yourself. The second reason for using opensource artifacts is reduced cost: you get free support from a bunch of capable and enthusiastic developers, usually in the form of copious amounts of documentation and forums. The third reason is reduced cost: you get free updates and enhancements from the opensource community and free bug fixes, although you don’t get to choose which enhancements are added to the project. Some projects, such as Tomcat, have a mechanism for voting on what enhancements are made, but at the end of the day it’s down to what really interests the developers. There are also a couple of unspoken reasons for using popular opensource libraries and frameworks: firstly, they make your CV look good. If opensource X is popular and you put that on your CV then your chances of getting a pay rise or a better job will improve. Secondly, if you work on one of the opensource projects, then you’ll earn some kudos, which, again, makes your CV look good improves the chances of you increasing the size of your pay-packet. There is an obvious downside to using opensource artifacts and that is all projects have an natural life-cycle. New versions of libraries are released, old libraries are deprecated, falling out of use because the technology’s too old, the developers have lost interest or moved on, or the rest of the community found something else that’s better and jumped on that bandwagon deserting yours. So, the problems of finding yourself saddled with retired and deprecated opensource libraries are firstly extra cost: there’s no support, no forum and no bug fixes. You’re on your own. You can often manage to download the source code to retired projects and support it yourself, but that’s not guaranteed and that costs money. The second problem of using deprecated code is extra cost: old code usually encompasses obsolete architecture and patterns, which contain known flaws and problems - after all, that’s why they’re obsolete. Using obsolete patterns and architecture encourages and in some cases forces developers to write bad code, not because your developers are bad, but that’s just the way it is... For example, there are some very obsolete JSP tags that blatantly mix database calls with business and presentation logic, which is a well know way of producing crumby, unmaintainable, spaghetti code. The third problem is, believe it or not, extra cost: I’ve recently come across a project where the code is so old that there are JAR file clashes, with different JARs containing different versions of the same API being dragged into the classpath. Certain bits of the code use one version of the API whilst other bits use the other version. eclipse didn’t know what to make of it all. There are also hidden costs: no one in there right mind wants to work on obsolete spaghetti code - it damages moral and saps the will to live, whilst damaging your ability to find that next, more highly paid, job. Plus, when people do leave, you’ve got the extra cost of finding and training their replacements. Never forget that the best people will be the first to leave, leaving you with the less experienced developers, again driving up your cost So, what can you do when faced with obsolete opensource libraries and frameworks? 1) Do nothing, continue using the obsolete library and hope everything will be alright. 2) Scrap the whole project and start again from scratch - the Big Bang Theory. 3) Refactor vigorously to remove the obsolete opensource code. This could also be seen as a way of changing the architecture of an application, updating the programming practices of the team and improving the code and whole build process. From the above I guess that you can figure out that in my opinion I prefer option 3. Option 1 is very risky, but then again, so is option 2: starting from scratch wastes time simply re-inventing the wheel, and whilst you do that, you don’t have a product, plus you may also end up with a big a mess as you started with. Option 3 is evolution and not revolution, quite the most sensible way to go. Having said all this, I definitely won’t stop using opensource code... From http://www.captaindebug.com/2011/09/benefits-and-dangers-of-using.html

Learn how to add SLF4J, or Simple Logging Facade for Java, to your Maven project in this tutorial.

Even a basic planning problem, such as bin packing, can be notoriously hard to solve and scale. One might consider the brute force algorithm. Let's take a look at how that algorithm works out on the cloud balance example of Drools Planner: Given a set of servers with different hardware (CPU, memory and network bandwidth) and given a set of processes with different hardware requirements, assign each process to 1 server and minimize the total cost of the active servers. The brute force algorithm is simple: try every combination between processes where each process is assigned to each server. For example, if we have 6 processes (P0, P1, P2, P3, P4, P5) and 2 servers (S0, S1), we'd try these solutions: P0->S0, P1->S0, P2->S0, P3->S0, P4->S0, P5->S0 P0->S0, P1->S0, P2->S0, P3->S0, P4->S0, P5->S1 P0->S0, P1->S0, P2->S0, P3->S0, P4->S1, P5->S0 P0->S0, P1->S0, P2->S0, P3->S0, P4->S1, P5->S1 ... P0->S1, P1->S1, P2->S1, P3->S1, P4->S1, P5->S1 On my machine, it takes 15ms to calculate the score of these 2^6 combinations. When I scale out to 9 processes and 3 servers, which are 3^9 combinations, it becomes 1582ms. So it scales like this: Notice that despite that the number of processes has not even doubled, the running time multiplied by 100! For comparison, I 've added the running time of the First Fit algorithm. And it gets worse: for 12 processes and 4 servers, which are 4^12 combinations, it take more than 17 minutes: What if we want to distribute 3000 processes over 1000 servers? With this kind of scalability, it will take too long. In fact, the brute force algorithm is useless. Luckily, Drools Planner implements several other optimization algorithms, which can handle such loads. If you want to know more about them, take a look at the Drools Planner manual or come to my talk at JUDCon London (31 Oct - 1 Nov). This article was originally posted on the Drools & jBPM blog.

So i thought to do a series of hibernate examples showing various features of hibernate. In the first part i wanted to show about the Delete Orphan feature and how it may be used with the use of a story line. So let us begin :) Prerequisites: In order for you to try out the following example you will need the below mentioned JAR files: org.springframework.aop-3.0.6.RELEASE.jar org.springframework.asm-3.0.6.RELEASE.jar org.springframework.aspects-3.0.6.RELEASE.jar org.springframework.beans-3.0.6.RELEASE.jar org.springframework.context.support-3.0.6.RELEASE.jar org.springframework.context-3.0.6.RELEASE.jar org.springframework.core-3.0.6.RELEASE.jar org.springframework.jdbc-3.0.6.RELEASE.jar org.springframework.orm-3.0.6.RELEASE.jar org.springframework.transaction-3.0.6.RELEASE.jar. org.springframework.expression-3.0.6.RELEASE.jar commons-logging-1.0.4.jar log4j.jar aopalliance-1.0.jar dom4j-1.1.jar hibernate-commons-annotations-3.2.0.Final.jar hibernate-core-3.6.4.Final.jar hibernate-jpa-2.0-api-1.0.0.Final.jar javax.persistence-2.0.0.jar jta-1.1.jar javassist-3.1.jar slf4j-api-1.6.2.jar mysql-connector-java-5.1.13-bin.jar commons-collections-3.0.jar For anyone who want the eclipse project to try this out, you can download it with the above mentioned JAR dependencies here. Introduction: Its year 2011. And The Justice League has grown out of proportion and are searching for a developer to help with creating a super hero registering system. A developer competent in Hibernate and ORM is ready to do the system and handle the persistence layer using Hibernate. For simplicity, He will be using a simple stand alone application to persist super heroes. This is how this example will layout: Table design Domain classes and Hibernate mappings DAO & Service classes Spring configuration for the application A simple main class to show how it all works Let the Journey Begin...................... Table Design: The design consists of three simple tables as illustrated by the diagram below; As you can see its a simple one-to-many relationship linked by a Join Table. The Join Table will be used by Hibernate to fill the Super hero list which is in the domain class which we will go on to see next. Domain classes and Hibernate mappings: There are mainly only two domain classes as the Join table in linked with the primary owning entity which is the Justice League entity. So let us go on to see how the domain classes are constructed with annotations; package com.justice.league.domain; import java.io.Serializable; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.Table; import org.hibernate.annotations.Type; @Entity @Table(name = "SuperHero") public class SuperHero implements Serializable { /** * */ private static final long serialVersionUID = -6712720661371583351L; @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "super_hero_id") private Long superHeroId; @Column(name = "super_hero_name") private String name; @Column(name = "power_description") private String powerDescription; @Type(type = "yes_no") @Column(name = "isAwesome") private boolean isAwesome; public Long getSuperHeroId() { return superHeroId; } public void setSuperHeroId(Long superHeroId) { this.superHeroId = superHeroId; } public String getName() { return name; } public void setName(String name) { this.name = name; } public String getPowerDescription() { return powerDescription; } public void setPowerDescription(String powerDescription) { this.powerDescription = powerDescription; } public boolean isAwesome() { return isAwesome; } public void setAwesome(boolean isAwesome) { this.isAwesome = isAwesome; } } As i am using MySQL as the primary database, i have used the GeneratedValue strategy as GenerationType.AUTO which will do the auto incrementing whenever a new super hero is created. All other mappings are familiar to everyone with the exception of the last variable where we map a boolean to a Char field in the database. We use Hibernate's @Type annotation to represent true & false as Y & N within the database field. Hibernate has many @Type implementations which you can read about here. In this instance we have used this type. Ok now that we have our class to represent the Super Heroes, lets go on to see how our Justice League domain class looks like which keeps tab of all super heroes who have pledged allegiance to the League. package com.justice.league.domain; import java.io.Serializable; import java.util.ArrayList; import java.util.List; import javax.persistence.CascadeType; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.FetchType; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.JoinColumn; import javax.persistence.JoinTable; import javax.persistence.OneToMany; import javax.persistence.Table; @Entity @Table(name = "JusticeLeague") public class JusticeLeague implements Serializable { /** * */ private static final long serialVersionUID = 763500275393020111L; @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "justice_league_id") private Long justiceLeagueId; @Column(name = "justice_league_moto") private String justiceLeagueMoto; @Column(name = "number_of_members") private Integer numberOfMembers; @OneToMany(cascade = { CascadeType.ALL }, fetch = FetchType.EAGER, orphanRemoval = true) @JoinTable(name = "JUSTICE_LEAGUE_SUPER_HERO", joinColumns = { @JoinColumn(name = "justice_league_id") }, inverseJoinColumns = { @JoinColumn(name = "super_hero_id") }) private List superHeroList = new ArrayList(0); public Long getJusticeLeagueId() { return justiceLeagueId; } public void setJusticeLeagueId(Long justiceLeagueId) { this.justiceLeagueId = justiceLeagueId; } public String getJusticeLeagueMoto() { return justiceLeagueMoto; } public void setJusticeLeagueMoto(String justiceLeagueMoto) { this.justiceLeagueMoto = justiceLeagueMoto; } public Integer getNumberOfMembers() { return numberOfMembers; } public void setNumberOfMembers(Integer numberOfMembers) { this.numberOfMembers = numberOfMembers; } public List getSuperHeroList() { return superHeroList; } public void setSuperHeroList(List superHeroList) { this.superHeroList = superHeroList; } } The important fact to note here is the annotation @OneToMany(cascade = { CascadeType.ALL }, fetch = FetchType.EAGER, orphanRemoval = true). Here we have set orphanRemoval = true. So what does that do exactly? Ok so say that you have a group of Super Heroes in your League. And say one Super Hero goes haywire. So we need to remove Him/Her from the League. With JPA cascade this is not possible as it does not detect Orphan records and you will wind up with the database having the deleted Super Hero(s) whereas your collection still has a reference to it. Prior to JPA 2.0 you did not have the orphanRemoval support and the only way to delete orphan records was to use the following Hibernate specific(or ORM specific) annotation which is now deprecated; @org.hibernate.annotations.Cascade(org.hibernate.annotations.CascadeType.DELETE_ORPHAN) But with the introduction of the attribute orphanRemoval, we are now able to handle the deletion of orphan records through JPA. Now that we have our Domain classes DAO & Service classes: To keep with good design standards i have separated the DAO(Data access object) layer and the service layer. So let us see the DAO interface and implementation. Note that i have used HibernateTemplatethrough HibernateDAOSupportso as to keep away any Hibernate specific detail out and access everything in a unified manner using Spring. package com.justice.league.dao; import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.annotation.Transactional; import com.justice.league.domain.JusticeLeague; @Transactional(propagation = Propagation.REQUIRED, readOnly = false) public interface JusticeLeagueDAO { public void createOrUpdateJuticeLeagure(JusticeLeague league); public JusticeLeague retrieveJusticeLeagueById(Long id); } In the interface layer i have defined the Transaction handling as Required. This is done so that whenever you do not need a transaction you can define that at the method level of that specific method and in more situations you will need a transaction with the exception of data retrieval methods. According to the JPA spec you need a valid transaction for insert/delete/update functions. So lets take a look at the DAO implementation; package com.justice.league.dao.hibernate; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.orm.hibernate3.support.HibernateDaoSupport; import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.annotation.Transactional; import com.justice.league.dao.JusticeLeagueDAO; import com.justice.league.domain.JusticeLeague; @Qualifier(value="justiceLeagueHibernateDAO") public class JusticeLeagueHibernateDAOImpl extends HibernateDaoSupport implements JusticeLeagueDAO { @Override public void createOrUpdateJuticeLeagure(JusticeLeague league) { if (league.getJusticeLeagueId() == null) { getHibernateTemplate().persist(league); } else { getHibernateTemplate().update(league); } } @Transactional(propagation = Propagation.NOT_SUPPORTED, readOnly = false) public JusticeLeague retrieveJusticeLeagueById(Long id){ return getHibernateTemplate().get(JusticeLeague.class, id); } } Here i have defined an @Qualifier to let Spring know that this is the Hibernate implementation of the DAO class. Note the package name which ends with hibernate. This as i see is a good design concept to follow where you separate your implementation(s) into separate packages to keep the design clean. Ok lets move on to the service layer implementation. The service layer in this instance is just acting as a mediation layer to call the DAO methods. But in a real world application you will probably have other validations, security related procedures etc handled within the service layer. package com.justice.league.service; import com.justice.league.domain.JusticeLeague; public interface JusticeLeagureService { public void handleJusticeLeagureCreateUpdate(JusticeLeague justiceLeague); public JusticeLeague retrieveJusticeLeagueById(Long id); } package com.justice.league.service.impl; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Component; import com.justice.league.dao.JusticeLeagueDAO; import com.justice.league.domain.JusticeLeague; import com.justice.league.service.JusticeLeagureService; @Component("justiceLeagueService") public class JusticeLeagureServiceImpl implements JusticeLeagureService { @Autowired @Qualifier(value = "justiceLeagueHibernateDAO") private JusticeLeagueDAO justiceLeagueDAO; @Override public void handleJusticeLeagureCreateUpdate(JusticeLeague justiceLeague) { justiceLeagueDAO.createOrUpdateJuticeLeagure(justiceLeague); } public JusticeLeague retrieveJusticeLeagueById(Long id){ return justiceLeagueDAO.retrieveJusticeLeagueById(id); } } Few things to note here. First of all the @Component binds this service implementation with the name justiceLeagueService within the spring context so that we can refer to the bean as a bean with an id of name justiceLeagueService. And we have auto wired the JusticeLeagueDAO and defined an @Qualifier so that it will be bound to the Hibernate implementation. The value of the Qualifier should be the same name we gave the class level Qualifier within the DAO Implementation class. And Lastly let us look at the Spring configuration which wires up all these together; Spring configuration for the application: com.justice.league.**.* org.hibernate.dialect.MySQLDialect com.mysql.jdbc.Driver jdbc:mysql://localhost:3306/my_test root password true org.hibernate.dialect.MySQLDialect Note that i have used the HibernateTransactionManager in this instance as i am running it stand alone. If you are running it within an application server you will almost always use a JTA Transaction manager. I have also used auto creation of tables by hibernate for simplicity purposes. The packagesToScan property instructs to scan through all sub packages(including nested packaged within them) under the root package com.justice.league.**.* to be scanned for @Entity annotated classes. We have also bounded the session factory to the justiceLeagueDAO so that we can work with the Hibernate Template. For testing purposes you can have the tag create initially if you want, and let hibernate create the tables for you. Ok so now that we have seen the building blocks of the application, lets see how this all works by first creating some super heroes within the Justice League A simple main class to show how it all works: As the first example lets see how we are going to persist the Justice League with a couple of Super Heroes; package com.test; import java.util.ArrayList; import java.util.List; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import com.justice.league.domain.JusticeLeague; import com.justice.league.domain.SuperHero; import com.justice.league.service.JusticeLeagureService; public class TestSpring { /** * @param args */ public static void main(String[] args) { ApplicationContext ctx = new ClassPathXmlApplicationContext( "spring-context.xml"); JusticeLeagureService service = (JusticeLeagureService) ctx .getBean("justiceLeagueService"); JusticeLeague league = new JusticeLeague(); List superHeroList = getSuperHeroList(); league.setSuperHeroList(superHeroList); league.setJusticeLeagueMoto("Guardians of the Galaxy"); league.setNumberOfMembers(superHeroList.size()); service.handleJusticeLeagureCreateUpdate(league); } private static List getSuperHeroList() { List superHeroList = new ArrayList(); SuperHero superMan = new SuperHero(); superMan.setAwesome(true); superMan.setName("Clark Kent"); superMan.setPowerDescription("Faster than a speeding bullet"); superHeroList.add(superMan); SuperHero batMan = new SuperHero(); batMan.setAwesome(true); batMan.setName("Bruce Wayne"); batMan.setPowerDescription("I just have some cool gadgets"); superHeroList.add(batMan); return superHeroList; } } And if we go to the database and check this we will see the following output; mysql> select * from superhero; +---------------+-----------+-----------------+-------------------------------+ | super_hero_id | isAwesome | super_hero_name | power_description | +---------------+-----------+-----------------+-------------------------------+ | 1 | Y | Clark Kent | Faster than a speeding bullet | | 2 | Y | Bruce Wayne | I just have some cool gadgets | +---------------+-----------+-----------------+-------------------------------+ mysql> select * from justiceleague; +-------------------+-------------------------+-------------------+ | justice_league_id | justice_league_moto | number_of_members | +-------------------+-------------------------+-------------------+ | 1 | Guardians of the Galaxy | 2 | +-------------------+-------------------------+-------------------+ So as you can see we have persisted two super heroes and linked them up with the Justice League. Now let us see how that delete orphan works with the below example; package com.test; import java.util.ArrayList; import java.util.List; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import com.justice.league.domain.JusticeLeague; import com.justice.league.domain.SuperHero; import com.justice.league.service.JusticeLeagureService; public class TestSpring { /** * @param args */ public static void main(String[] args) { ApplicationContext ctx = new ClassPathXmlApplicationContext( "spring-context.xml"); JusticeLeagureService service = (JusticeLeagureService) ctx .getBean("justiceLeagueService"); JusticeLeague league = service.retrieveJusticeLeagueById(1l); List superHeroList = league.getSuperHeroList(); /** * Here we remove Batman(a.k.a Bruce Wayne) out of the Justice League * cos he aint cool no more */ for (int i = 0; i < superHeroList.size(); i++) { SuperHero superHero = superHeroList.get(i); if (superHero.getName().equalsIgnoreCase("Bruce Wayne")) { superHeroList.remove(i); break; } } service.handleJusticeLeagureCreateUpdate(league); } } Here we first retrieve the Justice League record by its primary key. Then we loop through and remove Batman off the League and again call the createOrUpdate method. As we have the remove orphan defined, any Super Hero not in the list which is in the database will be deleted. Again if we query the database we will see that batman has been removed now as per the following; mysql> select * from superhero; +---------------+-----------+-----------------+-------------------------------+ | super_hero_id | isAwesome | super_hero_name | power_description | +---------------+-----------+-----------------+-------------------------------+ | 1 | Y | Clark Kent | Faster than a speeding bullet | +---------------+-----------+-----------------+-------------------------------+ So that's it. The story of how Justice League used Hibernate to remove Batman automatically without being bothered to do it themselves. Next up look forward to how Captain America used Hibernate Criteria to build flexible queries in order to locate possible enemies. Watch out!!!! Have a great day people and thank you for reading!!!! If you have any suggestions or comments pls do leave them by. From http://dinukaroshan.blogspot.com/2011/09/hibernate-by-example-part-1-orphan.html

while using lucene and solr we are used to a very high reliability. however, there may come a day when solr will inform us that our index is corrupted, and we need to do something about it. is the only way to repair the index to restore it from the backup or do full indexation? no – there is hope in the form of checkindex tool. what is checkindex ? checkindex is a tool available in the lucene library, which allows you to check the files and create new segments that do not contain problematic entries. this means that this tool, with little loss of data is able to repair a broken index, and thus save us from having to restore the index from the backup (of course if we have it) or do the full indexing of all documents that were stored in solr. where do i start? please note that, according to what we find in javadocs, this tool is experimental and may change in the future. therefore, before starting to work with it we should create a copy of the index. in addition, it is worth knowing that the tool analyzes the index byte by byte, and thus for large indexes the time of analysis and repair may be large. it is important not to run the tool with the -fix option at the moment when it is used by solr or other applications based on the lucene library. finally, be aware that the launch of the tool in repairing mode may result in removal of some or all documents that are stored in the index. how to run it to run the utility, go to the directory where the lucene library files are located and run the following command: java -ea:org.apache.lucene... org.apache.lucene.index.checkindex index_path -fix in my case, it looked as follows: java -cp lucene-core-2.9.3.jar -ea:org.apache.lucene... org.apache.lucene.index.checkindex e:\solr\solr\data\index\ -fix after a while i got the following information : opening index @ e:solrsolrdataindex segments file=segments_2 numsegments=1 version=format_diagnostics [lucene 2.9] 1 of 1: name=_0 doccount=19 compound=false hasprox=true numfiles=11 size (mb)=0,018 diagnostics = {os.version=6.1, os=windows 7, lucene.version=2.9.3 951790 - 2010-06-06 01:30:55, source=flush, os.arch=x86, java.version=1.6.0_23, java.vendor=sun microsystems inc.} no deletions test: open reader.........ok test: fields..............ok [15 fields] test: field norms.........ok [15 fields] test: terms, freq, prox...ok [900 terms; 1517 terms/docs pairs; 1707 tokens] test: stored fields.......ok [232 total field count; avg 12,211 fields per doc] test: term vectors........ok [3 total vector count; avg 0,158 term/freq vector fields per doc] no problems were detected with this index. it mean that the index is correct and there was no need for any corrective action. additionally, you can learn some interesting things about the index broken index but what happens in the case of the broken index? there is only one way to see it – let’s try. so, i broke one of the index files and ran the checkindex tool. the following appeared on the console after i’ve run the checkindex tool: opening index @ e:solrsolrdataindex segments file=segments_2 numsegments=1 version=format_diagnostics [lucene 2.9] 1 of 1: name=_0 doccount=19 compound=false hasprox=true numfiles=11 size (mb)=0,018 diagnostics = {os.version=6.1, os=windows 7, lucene.version=2.9.3 951790 - 2010-06-06 01:30:55, source=flush, os.arch=x86, java.version=1.6.0_23, java.vendor=sun microsystems inc.} no deletions test: open reader.........failed warning: fixindex() would remove reference to this segment; full exception: org.apache.lucene.index.corruptindexexception: did not read all bytes from file "_0.fnm": read 150 vs size 152 at org.apache.lucene.index.fieldinfos.read(fieldinfos.java:370) at org.apache.lucene.index.fieldinfos.(fieldinfos.java:71) at org.apache.lucene.index.segmentreader$corereaders.(segmentreader.java:119) at org.apache.lucene.index.segmentreader.get(segmentreader.java:652) at org.apache.lucene.index.segmentreader.get(segmentreader.java:605) at org.apache.lucene.index.checkindex.checkindex(checkindex.java:491) at org.apache.lucene.index.checkindex.main(checkindex.java:903) warning: 1 broken segments (containing 19 documents) detected warning: 19 documents will be lost note: will write new segments file in 5 seconds; this will remove 19 docs from the index. this is your last chance to ctrl+c! 5... 4... 3... 2... 1... writing... ok wrote new segments file "segments_3" as you can see, all the 19 documents that were in the index have been removed. this is an extreme case, but you should realize that this tool might work like this. the end if you remember about the basisc assumptions associated with the use of the checkindex tool you may find yourself in a situation when this tool will come in handy and you will not have to ask yourself a question like “when was the last backup was made?”

When developing JEE applications as part of the development cycle, it’s important to deploy to a development server before running end to end or integration tests. This blog demonstrates how to deploy your applications to your Weblogic server using Maven and, it transpires that there are at least two ways of doing this. The first way is using a the weblogic-maven-plugin available from Oracle and you can find an article on how to install this available on the Oracle website. The unfortunate thing about this is that Oracle don’t deploy their JARs to a Maven repository, which means that you have to do some jiggery-pokery messing around setting up your local repository. The Oracle article defines three steps: creating a plug-in jar using the wljarbuilder tool, extracting the pom.xml and then installing the results to your local repository using mvn install:install-file. In order to use this plug-in, you’ll need to add the following to your program’s POM file: com.oracle.weblogic weblogic-maven-plugin 10.3.4 t3://localhost:7001 weblogic your-password true deploy false true ../marin-tips-ear/target/marin-tips.ear ${project.build.finalName} install deploy The second method of deploying an application to your Weblogic server is by using a ‘good-ol’ Ant script. This method is simpler as you don’t need to bother creating plug-in JARs or installing them in your local repository. The POM file additions are: org.apache.maven.plugins maven-antrun-plugin 1.6 deploy-to-server pre-integration-test run I prefer the old fashioned Ant way more purely because it’s less hassle and adheres to the Keep It Simple Stupid rule of programming as, unless Oracle make their JAR files available on some repository, then extra set-up steps aren’t really an improvement. From http://www.captaindebug.com/2011/09/deploying-applications-to-weblogic.html