At work, I needed to create a very simple background job, without any concern about what I could get back, because mostly all the hard work was just batch processing and persistence, and all exceptions or roll-back concerns were already taking care of. At the beginning I used a very simple way to call my background job, using Java's: Executors.newSingleThreadExecutor() void myBackgroundJob() { Executors.newSingleThreadExecutor().submit(new Runnable() { @Override public void run() { //My Background Job } }); } And it worked great, just what I needed. Using Groovy facilitate even more the way to create a new Background job, as simple as: def myBackgroundJob() { Thread.start { //My Background Job } } Then, after this simple way to send something into the background, I decided to create a new AST in groovy, that remove the need to remember or copy and paste the same logic. I created two annotations that help to identify the class and the methods that are going to be put into a new Thread. One for the Class: package async import org.codehaus.groovy.transform.GroovyASTTransformationClass import java.lang.annotation.* import xml.ToXmlTransformation @Retention (RetentionPolicy.SOURCE) @Target ([ElementType.TYPE]) @GroovyASTTransformationClass (["async.AsyncTransformation"]) public @interface Asynchronous { } And the other for the Method: package async import org.codehaus.groovy.transform.GroovyASTTransformationClass import java.lang.annotation.* import async.AsyncTransformation @Retention (RetentionPolicy.SOURCE) @Target ([ElementType.METHOD]) @GroovyASTTransformationClass (["async.AsyncTransformation"]) public @interface Async { } then the Asynchronous Transformation, using the AstBuilder().buildFromString(). Here I combined a GroovyInterceptable to connect the method being call with the AST transformation to wrapped with the Thread logic. package async import org.codehaus.groovy.control.CompilePhase import org.codehaus.groovy.transform.* import org.codehaus.groovy.ast.* import org.codehaus.groovy.control.SourceUnit import org.codehaus.groovy.ast.builder.AstBuilder import org.codehaus.groovy.ast.stmt.ExpressionStatement import org.codehaus.groovy.ast.expr.MethodCallExpression import org.codehaus.groovy.ast.expr.ClosureExpression import org.codehaus.groovy.ast.expr.ConstantExpression import org.codehaus.groovy.ast.stmt.BlockStatement import org.codehaus.groovy.ast.expr.ClassExpression import org.codehaus.groovy.ast.expr.ArgumentListExpression @GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS) //CompilePhase.SEMANTIC_ANALYSIS class AsyncTransformation implements ASTTransformation{ void visit(ASTNode[] astNodes, SourceUnit sourceUnit) { if (!astNodes ) return if (!astNodes[0] || !astNodes[1]) return if (!(astNodes[0] instanceof AnnotationNode)) return if (astNodes[0].classNode?.name != Asynchronous.class.name) return def methods = makeMethods(astNodes[1]) if(methods){ astNodes[1]?.interfaces = [ ClassHelper.make(GroovyInterceptable, false), ] as ClassNode [] astNodes[1]?.addMethod(methods?.find { it.name == 'invokeMethod' }) } } def makeMethods(ClassNode source){ def methods = source.methods def annotatedMethods = methods.findAll { it?.annotations?.findAll { it?.classNode?.name == Async.class.name } } if(annotatedMethods){ def expression = annotatedMethods.collect { "name == \"${it.name}\"" }.join(" || ") def ast = new AstBuilder().buildFromString(CompilePhase.INSTRUCTION_SELECTION, false, """ package ${source.packageName} class ${source.nameWithoutPackage} implements GroovyInterceptable { def invokeMethod(String name, Object args){ if(${expression}){ Thread.start{ def calledMethod = ${source.nameWithoutPackage}.metaClass.getMetaMethod(name, args) calledMethod?.invoke(this, args) } }else{ def calledMethod = ${source.nameWithoutPackage}.metaClass.getMetaMethod(name, args)?.invoke(this,args) } } } """) ast[1].methods } } } The example: package async @Asynchronous class Sample{ String name String phone @Async def expensiveMethod(){ println "[${Thread.currentThread()}] Started expensiveMethod" sleep 15000 println "[${Thread.currentThread()}] Finished expensiveMethod..." } @Async def otherMethod(){ println "[${Thread.currentThread()}] Started otherMethod" sleep 5000 println "[${Thread.currentThread()}] Finished otherMethod" } } println "[${Thread.currentThread()}] Start" def sample = new Sample(name:"AST EXample",phone:"1800-GROOVY") sample.expensiveMethod() sample.otherMethod() println "[${Thread.currentThread()}] Finished" Final Notes: As you can see on the example I need to have the Asynchronous annotation on the class still. It could be better without it and just annotate the methods, something like the Groovy's SynchronizedASTTransformation. If you have any idea to complement this small example, please clone the source code [here], and let me know what you think. I could used the @javax.ejb.Asynchronous or the Spring's @org.springframework.scheduling.annotation.Async, but I only needed a very simple solution without any other configuration or library inclusion. The remain logic here could be play more with multi threading and expect some results like: java.util.concurrent.Future and its java.util.concurrent.Future.get() method or maybe integrated with another frameworks like Spring. Source: [Here]

DTOs, or data-transfer objects, are commonly used. What is not со commonly-known is that they originate from DDD (Domain-driven design). There it makes a lot of sense – domain objects have state, identity and business logic while DTOs have only state. But many projects today are using the anemic data model approach (my opinion) and still use DTOs. They are used whenever an object “leaves” the service layer or “leaves” the system (through web services, rmi, etc.). There are three approaches: every entity has at least one corresponding DTO. Usually more than one, for different scenarios in the view layer. When you display a user in a list you have one DTO, when you display it in a “user details” window you need a more extended DTO. I am not in favour of this approach because in too many cases the DTO and the domain structure have exactly the same structure and as a result there’s a lot of duplicated code + redundant mapping. Another thing is the variability of multiple DTOs. Even if they differ from the entity, they differ from one another with one or two fields. Why duplication is a bad thing? Because changes are to be made in two places, issues are traced harder when data passes through multiple objects, and because..it is duplication. Copy & paste within the same project is a sin. DTOs are only created when their structure significantly differs from the that of the entity. In all other cases the entity itself is used. The cases when you don’t want to show some fields (especially when exposing via web services to 3rd parties) exist, but are not that common. This can sometimes be handled via the serialization mechanism – mark them as @JsonIgnore or @XmlTransient for example – but in other cases the structures are just different. In these cases a DTO is due. For example you have a User and UserDetails, where UserDetails holds the details + the relations of the currently logged user to the given user. The latter has nothing to do with the entity, so you create a DTO. However in the case of a DirectMessage you have sender, recipient, text and datetime both in the DB and in the UI. No need to have a DTO. One caveat with this approach (as well as with the next one). Anemic entities usually come with an ORM (JPA in the case of Java). Whenever they exit the service layer they may be invalid, because of lazy collections that require an open session. You have two options here: use OpenSessionInView / OpenEntityManagerInView – thus your session stays open until you are finished preparing the response. This is easy to configure but is not my preferred option – it violates layer boundaries in a subtle way, and this sometimes leads to problems especially for novice developers Don’t use lazy collections. Lazy collections are unneeded. Either make them eager, if they are supposed to hold a small list of items (for example – the list of roles for a user), or if the data is likely to grow use queries. Yes, you are not going to show 1000 records at on go anyway, you will have to page it. Without lazy associations (@*ToOne are eager by default) you won’t have invalid objects when the session is closed Don’t use DTOs at all. Applicable a soon as there aren’t significantly varying structures. For smaller projects this is usually a good way to go. Everything mentioned in the above point applies here. So my preferred approach is the “middle way”. But it requires a lot of consideration in each case, which may not be applicable for bigger and/or less experienced teams. So one of the two “extremes” should be picked. Since the “no DTOs” approach also requires consideration – what to make @Transient, how does lazy collections affect the flow, etc, the “All DTOs” is usually chosen. But even though it is seemingly the safest approach, it has many pitfalls. First, how do you map from DTOs to entities and vice-versa? Three options: dedicated mapper classes constructors – the DTO constructor takes the entity and fills itself, and vice-versa (remember to also provide a default constructor) declarative mapping (e.g. Dozer). This is practically the same as the first option – it externalizes the mapping. It can even be used together with a dedicated mapper class map them in-line (whenever needed). This can generate unmaintainable code and is not preferred I prefer the constructor approach, at least because fewer classes are created. But they are essentially the same (DTOs are not famous for encapsulation, so all of your properties are exposed anyway). Here is a list of guidelines when using DTOs and either of the “mapping” approaches: Don’t generate too much redundant code. If two scenarios require slightly different DTOs, reuse. No need to create a new DTO for a difference of one or two fields Don’t put presentation logic in mappers/constructors. For example if (entity.isActive()) dto.setStatus("Active"); This should happen in the view layer Don’t sneak entities together with DTOs. DTOs should not have members which are entities. Generally, entities should not be used outside the service layer (this is a bit extreme, but if we use DTOs everywhere we should be consistent and stick to that practice) Don’t use the mappers/entity-to-dto constructors in controllers, use them in the service layer. The reason DTOs are used in the first place is that entities may be ORM-bound, and they may not valid outside a session (i.e. outside the service layer). If using mappers, prefer static mapper methods. Mappers don’t have state, so no need for them to be instantiated. (And they don’t have to be mocked, wrapped, etc). If using mappers, there’s no need for a separate mapper for each entity(+its multiple DTOs). Related entities can be grouped in one mapper. For example Company, CompanyProfile, CompanySubsidiary can use the same mapper class Just make sure you make all these decisions at the beginning of a project and figure out which is applicable in your scenario (team size and experience, project size, domain complexity). From http://techblog.bozho.net/?p=427

Are you building a Java web service based on EJB3? Do you need to return a more significant message to your web service clients other that just the exception message or even worst the recurring javax.transaction.TransactionRolledbackException? Well if the answer is YES to the above questions then keep reading... The code in this article has been tested with JBoss 5.1.0 but it should (!?) work on other EJB containers as well Create a base application exception that will be extended by all the other exception, I will refer to it as MyApplicationBaseException . This exception contains a list of UserMessage, again a class I created with some messages and locale information You need to create a javax.xml.ws.handler.soap.SOAPHandler < SOAPMessageContext > implementation. Mine looks like this import java.util.Set; import javax.xml.bind.JAXBContext; import javax.xml.bind.JAXBException; import javax.xml.bind.Marshaller; import javax.xml.namespace.QName; import javax.xml.soap.SOAPException; import javax.xml.soap.SOAPFault; import javax.xml.soap.SOAPMessage; import javax.xml.ws.handler.MessageContext; import javax.xml.ws.handler.soap.SOAPHandler; import javax.xml.ws.handler.soap.SOAPMessageContext; import org.apache.commons.lang.exception.ExceptionUtils; public class SoapExceptionHandler implements SOAPHandler { private transient Logger logger = ServiceLogFactory.getLogger(SoapExceptionHandler.class); @Override public void close(MessageContext context) { } @Override public boolean handleFault(SOAPMessageContext context) { try { boolean outbound = (Boolean) context.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY); if (outbound) { logger.info("Processing " + context + " for exceptions"); SOAPMessage msg = ((SOAPMessageContext) context).getMessage(); SOAPFault fault = msg.getSOAPBody().getFault(); // Retrives the exception from the context Exception ex = (Exception) context.get("exception"); if (ex != null) { // Add a fault to the body if not there already if (fault == null) { fault = msg.getSOAPBody().addFault(); } // Get my exception int indexOfType = ExceptionUtils.indexOfType(ex, MyApplicationBaseException.class); if (indexOfType != -1) { ex = (MyApplicationBaseException)ExceptionUtils.getThrowableList(ex).get(indexOfType); MyApplicationBaseException myEx = (AmsException) ex; fault.setFaultString(myEx.getMessage()); try { JAXBContext jaxContext = JAXBContext.newInstance(UserMessages.class); Marshaller marshaller = jaxContext.createMarshaller(); //Add the UserMessage xml as a fault detail. Detail interface extends Node marshaller.marshal(amsEx.getUserMessages(), fault.addDetail()); } catch (JAXBException e) { throw new RuntimeException("Can't marshall the user message ", e); } }else { logger.info("This is not an AmsException"); } }else { logger.warn("No exception found in the webServiceContext"); } } } catch (SOAPException e) { logger.warn("Error when trying to access the soap message", e); } return true; } @Override public boolean handleMessage(SOAPMessageContext context) { return true; } @Override public Set getHeaders() { return null; } } Now that you have the exception handler you need to register this SoapHandler with the EJB. To do that you'll need to create an Xml file in your class path and add an annotation to the EJB implementation class. The xml file : ExceptionHandler com.mycompany.utilities.ExceptionHandler and the EJB with annotation will be import javax.jws.HandlerChain; @Local(MyService.class) @Stateless @HandlerChain(file = "soapHandler.xml") @Interceptors( { MyApplicationInterceptor.class }) @SOAPBinding(style = SOAPBinding.Style.RPC) @WebService(endpointInterface = "com.mycompany.services.myservice", targetNamespace = "http://myservice.services.mycompany.com") public final class MyServiceImpl implements MyService { // service implementation } To make sure all my exceptions have proper messages and that the exception is set in the SOAPMessageContext I use an Interceptor to wrap all the service methods and transform any exception to an instance of MyApplicationException The interceptor has a single method @AroundInvoke private Object setException(InvocationContext ic) throws Exception { Object toReturn = null; try { toReturn = ic.proceed(); } catch (Exception e) { logger.error("Exception during the request processing.", e); //converts any exception to MyApplicationException e = MyApplicationExceptionHandler.getMyApplicationException(e); if (context != null && context.getMessageContext() != null) { context.getMessageContext().put("exception", e); } throw e; } return toReturn; } That's it! You're done. From http://www.devinprogress.info/2011/02/how-to-add-information-to-soap-fault.html

JSON data format is simple and still powerful. Nowadays you can encounter more and more web applications communicating using JSON format then a couple of years ago. It is simple for a developer to read the format, it is effective for a web browser to parse the format and there are databases using it as its primary data format. But what happens when the data structure changes? You need to migrate. And that is where acris-json-migration might help you! Example situation Let's shed a light into it and assume we have a data like this: { "firstName":"John" "secondName":"Doe" "street":"Over the rainbow" "streetNr":21 } Such data can be represented by following Java domain object: public class Person { String firstName; String secondName; String street; Integer streetNr; // ... and getters and setters... } Well, this seems like data about a person named John Doe. We stored it in a database and you can clearly see, that secondName is probably not the field name we really like to have. But a developer made a mistake and in second version of our domain model we are going to fix it: { "firstName":"John" "surname":"Doe" "street":"Over the rainbow" "streetNr":21 } Now you can see the point - thousands of data stored in the format defined by Person class in its version #1 but our program communicating in version #2 with changed secondName to surname in Person class. Clients can wonder why the don't see surnames, can't they? ;) One thing to remember (for the following context) - the class Person changed and there is only Person class in version #2. Simple migration script In this situation I would like to write a script: public class PersonV1toV2Script extends JacksonTransformationScript { @Override public void process(ObjectNode node) { rename(node, "secondName", "surname"); } } From the above example it is clear that the script will do the job. And you can do pretty anything with the whole tree of JSON data - adding new nodes, removing existing ones, transforming here and there - all thanks to Jackson's tree model. How can I execute it? There is a Transformer abstract class representing a transofmer responsible for passing JSON data to a script and writing it back. Currently there are tow kinds of transformers: Jackson-based JSONT-based Jackson-based is the preferred one and is more developed then JSONT-based. So to execute a transformation on a data set you have to specify only two lines of code: JacksonTransformer t = new JacksonTransformer(input, output); t.transform(PersonV1toV2Script.class.getName()); ... where input and output represent directories. In the input directory all files are treated as files containing JSON data and are transformed and written to the output directory. For a detailed test you can look into TransformerTest in the project. Conclusion The script's helper API is evolving and provides you with nice methods like removeIfExists or addNonExistent methods. We would like to hear about your use-cases which are not handled by acris-json-migration yet so the project can generally serve the purpose of JSON data migration.

Nowadays, pure CSS3 menus are still very popular. Usually these are UL-LI based menus. Today we will continue making nice menus for you. This tip will create a multi-level dropdown menu, but today submenus will appear not with the onhover action, but with the onclick action instead. Here is what the final result will look like: Here are samples and downloadable packages: Live Demo download in package Ok, download the example files and lets start coding ! Step 1. HTML As usual, we start with the HTML. Here is the full html code with our menu. As you can see - this is multi-level menu. I hope that you can easily understand it. The whole menu is built on UL-LI elements. index.html HomeTutorials HTML / CSSJS / jQuery jQueryJS PHPMySQLXSLTAjax Resources By category PHPMySQLMenu1 Menu1Menu2Menu3 Menu31Menu32Menu33Menu34 Menu4 Ajax By tag name captchagalleryanimation About Step 2. CSS Here are the CSS styles I used. First two selectors - a layout of our demo page. All rest belong to the menu. css/style.css /* demo page styles */ body { background:#eee; margin:0; padding:0; } .example { background:#fff url(../images/tech.jpg); width:770px; height:570px; border:1px #000 solid; margin:20px auto; padding:15px; border-radius:3px; -moz-border-radius:3px; -webkit-border-radius:3px; } /* main menu styles */ #nav,#nav ul { background-image:url(../images/tr75.png); list-style:none; margin:0; padding:0; } #nav { height:41px; padding-left:5px; padding-top:5px; position:relative; z-index:2; } #nav ul { left:-9999px; position:absolute; top:37px; width:auto; } #nav ul ul { left:-9999px; position:absolute; top:0; width:auto; } #nav li { float:left; margin-right:5px; position:relative; } #nav li a { background:#c1c1bf; color:#000; display:block; float:left; font-size:16px; padding:8px 10px; text-decoration:none; } #nav > li > a { -moz-border-radius:6px; -webkit-border-radius:6px; -o-border-radius:6px; border-radius:6px; overflow:hidden; } #nav li a.fly { background:#c1c1bf url(../images/arrow.gif) no-repeat right center; padding-right:15px; } #nav ul li { margin:0; } #nav ul li a { width:120px; } #nav ul li a.fly { padding-right:10px; } /*hover styles*/ #nav li:hover > a { background-color:#858180; color:#fff; } /*focus styles*/ #nav li a:focus { outline-width:0; } /*popups*/ #nav li a:active + ul.dd,#nav li a:focus + ul.dd,#nav li ul.dd:hover { left:0; } #nav ul.dd li a:active + ul,#nav ul.dd li a:focus + ul,#nav ul.dd li ul:hover { left:140px; } Step 3. Images Our menu is using only three images: arrow.gif, tech.jpg and tr75.png. I didn't include them into tutorial because two of them are very small (will be difficult to locate) and the last one is just background image. All images will be in the package. Conclusion Hope you enjoyed this tutorial and learned something new. Good luck! From Script-tutorials

here’s a quick tip for soapui users that i have been stumbling over recently. soapui allows you to refresh an already defined service definition from an updated wsdl file. but, it’s easy to blow away the data that’s in your soap test steps if you chose the wrong options. first, here’s the specific task i’m talking about. you have an existing soapui workspace that has interface definitions already imported and you have at least one test suite using that interface. now you’ve made a change to the wsdl and you need to refresh your interface definition in soapui and update the soap test steps in your test suite. clearly, you want to update the soap requests to match the new wsdl without losing the test data that already exists in the test steps. you reload a wsdl definition by right-clicking on an interface definition in soapui as shown below. here’s a rundown of the effect of some of the “update definition” parameter configurations. i haven’t gone through every permutation of the parameters, just the ones that seem most useful. i’ve portrayed the configurations in terms of actions you’re likely to want to take after changing your wsdl. you added a new operation to the wsdl and you want to add default requests for it. you made a change to an existing operation in your wsdl and you want to regenerate existing requests using the new schema without creating optional elements. this configuration will cause you to lose the data in your existing requests . you made a change to an existing operation in your wsdl and you want to regenerate existing requests using the new schema and create optional elements. this configuration will cause you to lose the data in your existing requests, unless the elements are optional . you made a change to an existing operation in your wsdl and you want to regenerate existing requests using the new schema and create optional elements. this configuration will keep the existing data in tact. you made a change to an existing operation in your wsdl and you want to regenerate existing requests and test steps using the new schema without creating optional elements. this configuration will keep the existing data in tact but will remove optional elements from the existing requests . you made a change to an existing operation in your wsdl and you want to regenerate existing requests and test steps using the new schema and maintain optional elements. this configuration will keep the existing data and optional elements in tact. the last configuration is the safest bet for most refreshes. it combines all of the above. that means that it picks up all wsdl changes and incorporates them into both default requests and your soap test steps. none of your existing data will be overwritten. it also opens up a window that lists exactly which items have been modified. i have not mentioned the “keep soap headers” and “create backups” parameters because i have not used them yet. i suggest playing around with the “update definition” feature a little before using it. soapui is a powerful tool, so it assumes you know what you want. it accomplishes that by exposing panels with lots of parameters on them, like the “update definiton” panel. if you get yourself into a bind, remember that you can just reload the project you’re working on by right-clicking on the project name and choosing “reload project”. this option will reload the project into the workspace without saving the changes you just made. from http://thewonggei.wordpress.com/2010/12/29/soapui-tip-options-for-refreshing-a-wsdl-definition/

Yesterday one of my reader asked me one question that How can I set or get values from Jquery of server side element for example. So I decided to write blog post for this. This blog post is for all this people who are learning Jquery and don’t know how to set or get value for a server like ASP.NET Textbox. I know most of people know about it as Jquery is very popular browser JavaScript framework. I believe jquery as framework because it’s a single file which has lots of functionality. For this I am going to take one simple example asp.net page which contain two textboxes txtName and txtCopyName and button called copyname. On click of that button it will get the value from txtName textbox and set value of another box. So following is HTML for this. As you can see in following code there are two textbox and one button which will call JavaScript function called to CopyName to copy text from one textbox from another textbox.Now we are going to use the Jquery for this. So first we need to include Jquery script file to accomplish the task. So I am going link that jquery.js file in my header section like following. Here I have used the ASP.NET Jquery CDN. If you want know more about Jquery CDN you can visit this link. http://www.asp.net/ajaxlibrary/cdn.ashx Now it’s time to write query code. Here I have used val function to set and get value for the element. Following is the code for CopyName function. function CopyName() { var name = $("#").val(); //get value $("#").val(name); //set value return false; } Here I have used val function of jquery to set and get value. As you can see in the above code, In first statement I have get value in name variable and in another statement it was set to txtCopy textbox. Some people might argue why you have used that ClientID but it’s a good practice to have that because when you use multiple user controls your id and client id will be different. From this I have came to know that there are lots of people still there who does not basic Jquery things so in future I am going to post more post on jquery basics.That’s it. Hope you like it.

I've just finished an iCalendar vCard parser for PHP. It's done almost completely with a 'natural' simplexml-like interface, so it should (hopefully) be just as easy to parse, and also modify iCalendar / vCard objects (ics/vcf files). To install using pear, run the following: pear channel-discover pear.sabredav.org pear install sabredav/Sabre_VObject-alpha Or download from pear.sabredav.org. For testing, I used this iCalendar file: icalendartest.ics. To load in an object, you use the Reader class: // Link to the correct path if you manually dowloaded the package include 'Sabre/VObject/includes.php'; // Reading an object $calendar = Sabre_VObject_Reader::read(file_get_contents('icalendartest.ics')); iCalendar objects consist of components (VEVENT, VTODO, VTIMEZONE, etc), properties (SUMMARY, DESCRIPTION, DTSTART, etc) and parameters, which are to properties what attributes are to elements in XML. To show a listing of all events in a calendar, this snippet would work: echo "There are ", count($calendar->vevent), " events in this calendar\n"; // Looping through events foreach($calendar->vevent as $event) { echo (string)$event->dtstart, ": ", $event->summary, "\n"; } You can easily modify properties: $calendar->vevent[0]->description = "It's a birthday party"; Creating new objects uses the following syntax: $todo = new Sabre_VObject_Component('vtodo'); $todo->summary = 'Take out the dog'; $calendar->add($todo); And to turn your newly modified calendar back into an ics file: file_put_contents('output.ics', $calendar->serialize()); Lastly, parameters are accessible through array-syntax: echo (string)$calendar->vevent[0]->dtstart['tzid'], "\n"; I had fun building this, I hope it's useful to you as well. It's 100% unittested, but bugs might still appear due to the complex nature of API. Use at your own risk :). This library will be part of the SabreDAV project, which is also where you can go for the source, report bugs or make suggestions.

There is little support for testing the database of your enterprise application. We will describe some problems and possible solutions based on Hibernate and JUnit.

What happens in the debate for a native application vs. web application in the case offline usage comes into play? Surely you can't use a web application if you do not have an active connection for loading it. With the boatload of innovations in the HTML5 and related specifications, this has changed. In a few Pomodoros (3 half hours) I could learn how to code a web application able to run in a stable browser like Firefox 5 without a network connection. What this application does? Since it serves as an example for this post, it simply loads a remote .php file which changes often, and that represents for example a stream of data such as Twitter's newsfeed. When the connection is not available, the application displays the data from the last time it was able to connect to the server. Of course, while it is offline it could do anything from slicing bread to executing arbitrary JavaScript code: the point was showing a web application working from the cache and able to recognize that it is currently offline. This is the first time you load it: index.html is downloaded and it makes an Ajax request for the news feed. If you refresh the page, the browser caches some content I have chosen (index.html), but still the news feed can be downloaded via Ajax: If now you cut the network cable, and refresh again, the application becomes aware that it's offline and retrieves a saved copy of the news (of course it can do anything you can code in JavaScript): What technologies you have used? First of all, the cache manifest from the Offline Web applications part of the HTML5 specification. With this manifest, I was able to specify that some file should have been cached for offline usage (my page containing JavaScript code and its style sheet). At the same time, other files were listed as always to be loaded from the network (my news feed and some other machinery). Another crucial technology is the Web Storage, in this case the localStorage JavaScript object which serves as a database for storing the last newsfeed retrieved via Ajax. And of course I've used Ajax both to retrieve the news feed from my cached main page (that is never reloaded), and to ping the server to check if the connection is really available or if we are just in a LAN. Which browser you used? I used Firefox 5, and you need to be aware of a few issues if you want to develop an offline web application. The cache you have to clear in case you want to reset the application after some changes to the code is in Options (or Preferences on Linux) -> Advanced -> Network tab, at the label "The following websites have stored data for offline use". Firebug will continue to show HTTP requests as if you were connected to a real server, a value in the Remote IP column; even with localhost, an IP address is usually shown when the connection is really established. You should check File -> Work offline to simulate being offline, since interrupting the connection won't work while loading from localhost or another hostname pointing to 127.0.0.1. Which resources you have read? The wonderful online (and not) book Dive into HTML5 by Mark Pilgrim, which has an introductory chapter on the topic. A guide from Mozilla about offline resources, which has more examples of cache manifest lines. This Ed Norton's article about detecting the switch between online and offline status with a real ping, since the JavaScript API cannot be trusted in many implementations. Show us the code! Yep, I was going to do that. However there's a repository containing it all if you want to play with it. First of all, I have an .htaccess file which will work on Apache webservers: AddType text/cache-manifest .manifest ExpiresActive On ExpiresDefault "access" The first line is necessary to serve the manifest with the right MIME type. The other two are to avoid any caching on the web server (it's already done in the browser), and develop without hassles. It's not a production setting: in that case it should target only the manifest. Here is my manifest: CACHE MANIFEST NETWORK: /news.php /ping.js CACHE: /style.css http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js I only used 2 of the 3 available sections. What is in NETWORK will always be loaded if requested, what is in CACHE will always be cached. My main page: Hello, world! The element has an attribute pointing to the manifest, and the file loads other resources from the cache like jQuery and the style sheet. The checkNetworkStatus function is called upon relevant events from the JavaScript API and at the startup: it makes a ping to the web server to verify the connection is open. The showNetworkStatus function instead executes an action in case the status of the connection changes: it updates the Offline/Online label at the bottom of the page. In the online case, fills the #news div with updated text retrieved by Ajax (saving it). In the offline case, fills the #news div with the last text saved. ping.js is an empty file: {} news.php constantly updates to show the effects of being again online. Last update: '. date('Y-m-d H:i:s') . "\n"; ?> Lorem ipsum dolor amet... Conclusion We're finished. It's feasible and easy, at least for the base case, to deploy to a desktop or mobile platform which can be used even while offline (but of course it would sync with the server only when the connection is available.) Not good times for native applications...

In one of my previous lives, I used to work in Microsoft and there this word – NTLM (NT Lan Manager) was something that came to us whenever we used to work on applications. Microsoft OS have always provided us with an inbuilt security systems that can be effectively used to offer authentication (and even authorization to web applications). Many years back, I moved over into Java world and when I was asked to carry out my very first security implementation, I realized that there was no easy way to do this and many clients would actually want us to use LDAP for authentication and authorization. For many years, I continued to use that. And, then one day in a discussion with a client, we were asked to offer SSO implementation and client did not have an existing setup like SiteMinder. I started to think about if we can go about using NTLM based authentication. The reason that was possible was because the application we were asked to build was to be used within the organization itself and all the people were required to login into a domain. After some research, I was able to find out a way we could do this. We did a POC and showed it to the client and they were happy about it. What we did has been explained below: Wrote a Servlet which was the first one to be loaded (like Authentication Interceptor). This servlet was responsible for reading the header attributes and identify the user’s Domain and NTID Once we had the details; we sent a request to our Database to see if that user is registered under the same domain/NTID If the user was found in our user-database we allowed him to pass through And then roles and authorization for user was loaded Basically, we bypassed the “Login Screen” where the user was entering the password and used Domain information. Please note that it was possible for us because the Client guaranteed that there was this domain always and all users had unique NTIDs. Also, that it was their responsibility to shield the application from any external entry points where someone may impersonate the Domain/ID. If you are interested, you can refer to the code below: From http://scrtchpad.wordpress.com/2011/08/04/ntml-authentication-in-java/

Memory is stored within the cache system in units know as cache lines. Cache lines are a power of 2 of contiguous bytes which are typically 32-256 in size. The most common cache line size is 64 bytes. False sharing is a term which applies when threads unwittingly impact the performance of each other while modifying independent variables sharing the same cache line. Write contention on cache lines is the single most limiting factor on achieving scalability for parallel threads of execution in an SMP system. I’ve heard false sharing described as the silent performance killer because it is far from obvious when looking at code. To achieve linear scalability with number of threads, we must ensure no two threads write to the same variable or cache line. Two threads writing to the same variable can be tracked down at a code level. To be able to know if independent variables share the same cache line we need to know the memory layout, or we can get a tool to tell us. Intel VTune is such a profiling tool. In this article I’ll explain how memory is laid out for Java objects and how we can pad out our cache lines to avoid false sharing. Figure 1. Figure 1. above illustrates the issue of false sharing. A thread running on core 1 wants to update variable X while a thread on core 2 wants to update variable Y. Unfortunately these two hot variables reside in the same cache line. Each thread will race for ownership of the cache line so they can update it. If core 1 gets ownership then the cache sub-system will need to invalidate the corresponding cache line for core 2. When Core 2 gets ownership and performs its update, then core 1 will be told to invalidate its copy of the cache line. This will ping pong back and forth via the L3 cache greatly impacting performance. The issue would be further exacerbated if competing cores are on different sockets and additionally have to cross the socket interconnect. Java Memory Layout For the Hotspot JVM, all objects have a 2-word header. First is the “mark” word which is made up of 24-bits for the hash code and 8-bits for flags such as the lock state, or it can be swapped for lock objects. The second is a reference to the class of the object. Arrays have an additional word for the size of the array. Every object is aligned to an 8-byte granularity boundary for performance. Therefore to be efficient when packing, the object fields are re-ordered from declaration order to the following order based on size in bytes: doubles (8) and longs (8) ints (4) and floats (4) shorts (2) and chars (2) booleans (1) and bytes (1) references (4/8) With this knowledge we can pad a cache line between any fields with 7 longs. Within the Disruptor we pad cache lines around the RingBuffer cursor and BatchEventProcessor sequences. To show the performance impact let’s take a few threads each updating their own independent counters. These counters will be volatile longs so the world can see their progress. public final class FalseSharing implements Runnable { public final static int NUM_THREADS = 4; // change public final static long ITERATIONS = 500L * 1000L * 1000L; private final int arrayIndex; private static VolatileLong[] longs = new VolatileLong[NUM_THREADS]; static { for (int i = 0; i < longs.length; i++) { longs[i] = new VolatileLong(); } } public FalseSharing(final int arrayIndex) { this.arrayIndex = arrayIndex; } public static void main(final String[] args) throws Exception { final long start = System.nanoTime(); runTest(); System.out.println("duration = " + (System.nanoTime() - start)); } private static void runTest() throws InterruptedException { Thread[] threads = new Thread[NUM_THREADS]; for (int i = 0; i < threads.length; i++) { threads[i] = new Thread(new FalseSharing(i)); } for (Thread t : threads) { t.start(); } for (Thread t : threads) { t.join(); } } public void run() { long i = ITERATIONS + 1; while (0 != --i) { longs[arrayIndex].value = i; } } public final static class VolatileLong { public volatile long value = 0L; public long p1, p2, p3, p4, p5, p6; // comment out } } Results Running the above code while ramping the number of threads and adding/removing the cache line padding, I get the results depicted in Figure 2. below. This is measuring the duration of test runs on my 4-core Nehalem at home. Figure 2. The impact of false sharing can clearly be seen by the increased execution time required to complete the test. Without the cache line contention we achieve near linear scale up with threads. This is not a perfect test because we cannot be sure where the VolatileLongs will be laid out in memory. They are independent objects. However experience shows that objects allocated at the same time tend to be co-located. So there you have it. False sharing can be a silent performance killer. From http://mechanical-sympathy.blogspot.com/2011/07/false-sharing.html

The integration framework Apache Camel already supports several important cloud services (see my overview article at http://www.kai-waehner.de/blog/2011/07/09/cloud-computing-heterogeneity-will-require-cloud-integration-apache-camel-is-already-prepared for more details). This article describes the combination of Apache Camel and the Amazon Web Services (AWS) interfaces of Simple Storage Service (S3), Simple Queue Service (SQS) and Simple Notification Service (SNS). Thus, The concept of Infrastructure as a Service (IaaS) is used to access messaging systems and data storage without any need for configuration. Registration to AWS and Setup of Camel First, you have to register to the Amazon Web Services (for free). Most AWS services include a free monthly quota, which is absolutely sufficient to play around and develop some simple applications. As its name states, AWS uses technology-independent web services. Besides, APIs for several different programming languages are available to ease development. By the way, Camel uses the AWS SDK for Java (http://aws.amazon.com/sdkforjava), of course. The documentation is detailed and easy to understand, including tutorials, screenshots and code examples . Hint 1: You should read the introductions to S3, SQS and SNS (go to http://aws.amazon.com and click on „products“) and play around with the AWS Management Console (http://aws.amazon.com/console) before you continue. This step is very easy and takes less than one hour. Then, you will have a much better understanding about AWS and where Camel can help you! Hint 2: It really helps to look at the source code of the camel-aws component, It helps you to understand how Camel uses the AWS Java API internally. If you want to write tests, you can do it the same way. In the past, I was afraid of looking at „complex“ source code of open source frameworks. But there is no need to be scared! The camel-aws component (and most other camel components) contain only of a few classes. Everything is easy to understand. It helps you to understand Camel internals, the AWS API, and to spot and solve errors due to exceptions in your code. In the meanwhile, the current Camel version 2.8 supports three AWS services: S3, SQS and SNS. All of them use similar concepts. Therefore, they are included in one single camel component: „camel-aws“. You have to add the libraries to your existing Camel project. As always, the simplest way is to use Maven and add the following dependency to the pom.xml: org.apache.camel camel-aws ${camel-version} Configuration of the Camel Endpoint The implementation and configuration of all three services is very similar. The URI looks like this (the code shows the SQS service): aws-sqs://queue-name[?options] There are two alternatives to configure your endpoint. Using Parameters The easy way is to use two paramters in the URI of your endpoint: „accessKey“ and „secretKey“ (you receive both after your AWS registration). “aws-sqs://unique-queue-name?accessKey=“INSERT_ME“&secretKey=INSERT_ME” Be aware of the following problem, which can result in a strange, non-speaking exception (thanks to Brendan Long): You’ll need to URL encode any +’s in your secret key (otherwise, they’ll be treated as spaces). + = %2B, so if your secretkey was “my+secret\key”, your Camel URL should have “secretKey=my%2Bsecret\key”. “Within the query string, the plus sign is reserved as shorthand notation for a space. Therefore, real plus signs must be encoded. This method was used to make query URIs easier to pass in systems which did not allow spaces.” Source: WC3 URI Recommendations Adding a configured AmazonClient to the Registry If you need to do more configuration (e.g. because your system is behind a firewall), you have to add an AmazonClient object to your registry. The following code shows an example using SQS, but SNS and S3 use exactly the same concept. @Override protected JndiRegistry createRegistry() throws Exception { JndiRegistry registry = super.createRegistry(); AWSCredentials awsCredentials = new BasicAWSCredentials(“INSERT_ME”, “INSERT_ME”); ClientConfiguration clientConfiguration = new ClientConfiguration(); clientConfiguration.setProxyHost(“http://myProxyHost”); clientConfiguration.setProxyPort(8080); AmazonSQSClient client = new AmazonSQSClient(awsCredentials, clientConfiguration); registry.bind(“amazonSQSClient”, client); return registry; } This example overwrites the createRegistry() method of a JUnit test (extending CamelTestSupport). You can also add this information to your runtime Camel application, of course. Apache Camel and the Simple Storage Service (S3) Simple Storage Service (S3) is a key-value-store. You can store small to very large data. The usage is very easy. You create buckets and put key-value data into these buckets. You can also create folders within buckets to organize your data. That’s it. You can monitor your buckets using the AWS Management Console – an intuitive GUI supporting most AWS services. The following example shows both alternatives for accessing the Amazon services (as described above): Paramenters and the AmazonClient. // Transfer data from your file inbox to the AWS S3 service from(“file:files/inbox”) // This is the key of your key-value data .setHeader(S3Constants.KEY, simple(“This is a static key”)) // Using parameters for accessing the AWS service .to(“aws-s3://camel-integration-bucket-mwea-kw?accessKey=INSERT_ME&secretKey=INSERT_ME&region=eu-west-1″); // Transfer data from the AWS S3 service to your file outbox from(“aws-s3://camel-integration-bucket-mwea-kw?amazonS3Client=#amazonS3Client&region=eu-wes”) .to(“file:files/outbox”); There are some additional parameters, for instance you can submit the desired AWS region or delete data after receiving it (see http://camel.apache.org/aws-s3.html and the corresponding SQS and SNS sites for more details about parameters and message headers). As you see in the code, you can use the AWS-S3 endpoint for producing and for consuming messages. Each bucket must be unique, thus you have to add some specific information such as your company to its name. Hint: If a bucket does not exist, Camel is creating it automatically (as the AWS API does). This concept is also used for SQS queues and SNS topics. Apache Camel and the Simple Queue Service (SQS) The Simple Queue Service (SQS) is similar to a JMS provider such as WebSphere MQ or ActiveMQ (but with some differences). You create queues and send messages to them. Consumers receive the messages. Contrary to most other AWS services, you cannot monitor queues by using the AWS management console directly. You have to use the service „Cloudwatch“ (http://aws.amazon.com/cloudwatch) and start an EC2 instance to monitor queues and its content. As you can see in the following code example, the syntax and concepts are almost the same as for the S3 service: from(“file:inbox”) .to(“aws-sqs://camel-integration-queue-mwea-kw?accessKey=INSERT_ME&secretKey=INSERT_ME”); from(“aws-sqs://camel-integration-queue-mwea-kw?amazonSQSClient=#amazonSQSClient”) .to(“file:outbox?fileName=sqs-${date:now:yyyy.MM.dd-hh:mm:ss:SS}”); Again, you can use the AWS-SQS endpoint for producing and for consuming messages. Each queue name must be unique. There exist two important differences to JMS (copy & paste from the AWS documentation): Q: How many times will I receive each message? Amazon SQS is engineered to provide “at least once” delivery of all messages in its queues. Although most of the time each message will be delivered to your application exactly once, you should design your system so that processing a message more than once does not create any errors or inconsistencies. Q: Why are there separate ReceiveMessage and DeleteMessage operations? When Amazon SQS returns a message to you, that message stays in the queue, whether or not you actually received the message. You are responsible for deleting the message; the delete request acknowledges that you’re done processing the message. If you don’t delete the message, Amazon SQS will deliver it again on another receive request. Apache Camel and the Simple Notification Service (SNS) The Simple Notification Service (SNS) acts like JMS topics. You create a topic, consumers subscribe to the topic and then receive notifications. Several transport protocols are supported: HTTP(S), Email and SQS. Further interfaces will be added in the future, e.g. the Short Message Service (SMS) for mobile phones. Contrary to S3 and SQS, Camel only offers a producer endpoint for this AWS service. You can only create topics and send messages via Camel. The reason is simple: Camel already offers endpoints for consuming these messages: HTTP, Email and SQS are already available. There is one tradeoff: A consumer cannot subscribe to topics using Camel – at the moment. The AWS Management Console has to be used. A very interesting discussion can be read on the Camel JIRA issue regarding the following questions: Should Camel be able to subscribe to topics? Should the producer contain this feature or should there be a consumer? In my opinion, there should be a consumer which is able to subscribe to topics, otherwise Camel is missing a key part of the AWS SNS service! Please read the discussion and contribute your opinion: https://issues.apache.org/jira/browse/CAMEL-3476. Apache Camel is already ready for the Cloud Computing Era AWS offers many more services for the cloud. Probably, it does not make sense to integrate everyone into Camel, but more AWS services will be supported in the future. For instance, SimpleDB and the Relational Database Service (RDS) are already planned and make sende, too: http://camel.apache.org/aws.html. The conclusion is easy: Apache Camel is already ready for the cloud computing era. Several important cloud services are already supported. Cloud integration will become very important in the future. Thus, Camel is on a very good way. Hopefully, we will see more cloud components, soon. I will continue to write articles about other Camel cloud components (and new AWS addons, ouf course). For instance, a component for the Platform as a Service (PaaS) product Google App Engine (GAE) is already available. If you have any additional important information, questions or other feedback, please write a comment. Thank you in advance… Best regards, Kai Wähner (Twitter: @KaiWaehner) [Content from my Blog: Cloud Integration with Apache Camel and Amazon Web Services (AWS): S3, SQS and SNS]

In my career spanning 15 years, the problem of Producer and Consumer is one that I have come across only a few times. In most programming cases, what we are doing is performing functions in a synchronous fashion where the JVM or the web container handles the complexities of multi-threading on its own. However, when writing certain kinds of use cases where we need this. Last week, I came acros one such use case that sent me 3 years back when I last did it. However, the way it was done last time was very different. When I first heard the problem statement, I knew instantly what was needed. However, my approach to doing it this time was going to be different from last time. It had simply to do with how I am viewing technology in my life today. I will not go into any non-technical side and will jump straight into the problem and its solution. I started to look at what existed in the market and did come across a couple of posts that helped me in channelizing my thoughts in the right way. Problem Statement We need a solution for a batch migration. We are migrating data form System 1 to System 2 and in the process we need to do three tasks: Load data from Database based on groups Process the data Update the records loaded in step#1 with modifications We have to handle 100s of groups and each group will have around 40K records. You can imagine the amount of time it would take if we were to perform this exercise in a synchronous fashion. Image here explains this problem in an effective way. Producer Consumer: The Problem Producer and Consumer Pattern Let us take a look at the Producer Consumer pattern to begin with. If you refer to the problem statement above and look at the image, we see that there are so many entities who are ready with their part of data. However, there are not enough workers who can process all the data. Hence, as the producers continue to line-up in a queue it just continues to grow. We see that the systems start to hog up threads and take a lot of time. Intermediate Solution Producer Consumer: The Intermediate approch We do have an intermediate solution. Refer to the image and you will immediately notice that the producers are piling up their work in a filing cabinet and the worker continues to pick it up as they get done with the previous task. However, this approach does have some glaring shortcomings: There is still one worker who has to do all the work. The external systems may be happy, but the task will still continue to exist until the worker has completed all of the tasks The producers will pile up their data in a queue and it needs resources to hold the same. Just as in this example the cabinet can fill up, the same can happen with the JVM resources too. We need to be careful how much data we are going to place in memory and in some cases it may not be much. The Solution Producer Consumer: The Solution The solution is what we see everyday in many places – like the cinema hall queue, Petrol Pumps etc. There are so many people who come in to book a ticket and based on how many people come in, the more people are added to issue tickets. Essentially, refer to image here and you will notice that Producers will keep adding their jobs to the cabinet and we have more workers to handle the work load. Java provided concurrency package to solve this issue. Till now, I have always worked on threading at a much lower level and this was first time I was going to work with this package. As I started to explore the web and read fellow bloggers with what they have to say, I came across one very good article. It helped in understanding the use of BlockingQueue in a very effective manner. However, the solutions provided by Dhruba would not have helped me in achieving the high throughput which is needed. So, I started to explore the use of ArrayBlockingQueue for the same. The Controller This is the first class where the contract between the producers and consumers are managed. The controller will setup 1 thread for the Producer and 2 threads for the consumer. Based on the needs we can create as many threads as we need; and even can even read the data from a properties or do some dynamic magic. For now, we will keep this simple. package com.kapil.techieforever.producerconsumer; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; public class TestProducerConsumer { public static void main(String args[]) { try { Broker broker = new Broker(); ExecutorService threadPool = Executors.newFixedThreadPool(3); threadPool.execute(new Consumer("1", broker)); threadPool.execute(new Consumer("2", broker)); Future producerStatus = threadPool.submit(new Producer(broker)); // this will wait for the producer to finish its execution. producerStatus.get(); threadPool.shutdown(); } catch (Exception e) { e.printStackTrace(); } } } I am using ExecuteService to create a thread pool and manage it. Instead of using the basic Thread implementation, this is a more effective way as it will handle the exiting and restarting the threads as needed. You will also notice that I am using Future class to get the status of the producer thread. This class is very effective and will halt my program from further execution. This is a nice way of replacing the “.join” method on the threads. Note: I am not using Future very effectively in this example; so you may have to try a few things as you feel fit. Also, you should note the Broker class which is being used as filing cabinet between the producers and consumers. We will see its implementation in just a little while. The Producer This class is responsible for producing the data that needs to be worked upon. package com.kapil.techieforever.producerconsumer; public class Producer implements Runnable { private Broker broker; public Producer(Broker broker) { this.broker = broker; } @Override public void run() { try { for (Integer i = 1; i < 5 + 1; ++i) { System.out.println("Producer produced: " + i); Thread.sleep(100); broker.put(i); } this.broker.continueProducing = Boolean.FALSE; System.out.println("Producer finished its job; terminating."); } catch (InterruptedException ex) { ex.printStackTrace(); } } } This class is doing the most simplest of things that it can do – adding an integer to the broker. Some key areas to note are: 1. There is a property on Broker which is updated in the end by the producer when its done producing. This is also known as the “final” or “poison” entry. This is used by the consumers to know that there are no more data coming up 2. I have used Thread.sleep to simulate that some producers may take more time to produce the data. You can tweak this value and see the consumers act The Consumer This class is responsible for reading the data from the broker and doing its job package com.kapil.techieforever.producerconsumer; public class Consumer implements Runnable { private String name; private Broker broker; public Consumer(String name, Broker broker) { this.name = name; this.broker = broker; } @Override public void run() { try { Integer data = broker.get(); while (broker.continueProducing || data != null) { Thread.sleep(1000); System.out.println("Consumer " + this.name + " processed data from broker: " + data); data = broker.get(); } System.out.println("Comsumer " + this.name + " finished its job; terminating."); } catch (InterruptedException ex) { ex.printStackTrace(); } } } This is again a simple class that reads the Integer and prints it on the console. However, key points to note are: 1. The loop to process data is an endless loop, that runs on two conditions – until the producer is consuming and there is some data with the broker 2. Again, the Thread.sleep is used to create effective and different scenarios The Broker package com.kapil.techieforever.producerconsumer; import java.util.concurrent.ArrayBlockingQueue; import java.util.concurrent.TimeUnit; public class Broker { public ArrayBlockingQueue queue = new ArrayBlockingQueue(100); public Boolean continueProducing = Boolean.TRUE; public void put(Integer data) throws InterruptedException { this.queue.put(data); } public Integer get() throws InterruptedException { return this.queue.poll(1, TimeUnit.SECONDS); } } The very first thing to note is that we are using ArrayBlockingQueue as the data holder. I am not going to say what this does, but insist you to read it on the JavaDocs here. however, I will explain that the producers are going to place the data in the queue and the consumers will fetch from the queue in FIFO format. But, if the producers are slow, the consumers will wait for data to come in and if the array is full, the producers will wait for it to fill up. Also, note that I am using the ‘poll’ function instead of get in the queue. This is to ensure that the consumers will not keep waiting for ever and the waiting will time out after a few seconds. This helps us in inter-communication and kill the consumers when all the data is processed. (Note: try replacing poll with get and you will see some interesting outputs). Code I have the code sitting on Google project hosting. Feel free to go across and download it from there. It is essentially an eclipse (Spring STS) project. You may also get additional packages and classes when you download it based on when you are downloading it. Feel free to look into those too and share your comments - You can browse the source code on the SVN browser or; - You can download it from the project itself From http://scratchpad101.com/2011/08/22/concurrency-pattern-producer-consumer/

introduction so you want to handle somehow all the unhandled exceptions in your application. usually you want to accomplish one of the following: log the exception for later diagnostics present the user an unhandled exception ui, nicer than the default you heard there’s an event you should register, or maybe you find one by mistake, but is it the correct one? did you know there are four (!) different events for handling unhandled exceptions in the .net framework? so what is the difference between them and when should we use each one? this post will hopefully answer these questions. note: this is not a replacement for try-catch blocks! summary i’ll begin with the summary to avoid boring busy developers. in a typical wpf application you should use application.current.dispatcherunhandledexception for exceptions generated on the ui thread and appdomain.currentdomain.unhandledexception for all the other exceptions. now, for the details.. system.windows.forms.application.threadexception this event is used for catching unhandled exceptions only on ui threads created by winforms . other exceptions, generated on non-ui threads won’t arrive to this event. use appdomain.current.unhandledexception to catch them. the default behavior of a winforms application with an unhandled exception is to present the following error dialog: if you register to this event, the application will not show the error dialog and will automatically extend the application life (i.e. the application won’t be killed). if you do want to end your application, maybe after logging the exception or asking the user with a personalized dialog, you must do it yourself using application.exit() . you can find the msdn documentation on this event here . system.windows.application.current.dispatcherunhandledexception this event is used for catching unhandled exceptions only from the main ui thread created by wpf . the default behavior of a wpf application with an unhandled exception is to present the following error dialog and end the application: if you register to this event, you will get the chance to log the exception, but the application will still end, unless you set e.handled = true , on the event’s eventargs parameter. you can find the msdn documentation on this event here . dispatcher.unhandledexception this event is used for catching unhandled exceptions on the thread attached to the specific dispatcher (wpf only). note, that in wpf two threads can have two different dispatcher object attached. this event is only useful if you have several ui threads in your wpf application, which is quite rare. if you’re not sure, you probably need to handle only the previous event: application.current.dispatcherunhandledexception. as in the previous event, if you register, you will get the change to log the exception. to prevent the exception internal handling from being called set e.handled = true . you can find the msdn documentation on this event here . appdomain.currentdomain.unhandledexception this event is used for catching unhandled exceptions generated from all threads running under the context of a specific application domain. you can find the msdn documentation on this event here . bonus: appdomain.currentdomain.firstchanceexception this event which exists only from .net 4, is raised on any exception, if the handled one. in fact, the event is raised before the search for the catch blocks. you can’t handle the exception using this event. you can use it if you need to log exceptions that are caught. you can find the msdn documentation on this event here .

when studying both the java nio and io api's, a question quickly pops into mind: when should i use io and when should i use nio? in this text i will try to shed some light on the differences between java nio and io, their use cases, and how they affect the design of your code. main differences of java nio and io the table below summarizes the main differences between java nio and io. i will get into more detail about each difference in the sections following the table. io nio stream oriented buffer oriented blocking io non blocking io selectors stream oriented vs. buffer oriented the first big difference between java nio and io is that io is stream oriented, where nio is buffer oriented. so, what does that mean? java io being stream oriented means that you read one or more bytes at a time, from a stream. what you do with the read bytes is up to you. they are not cached anywhere. furthermore, you cannot move forth and back in the data in a stream. if you need to move forth and back in the data read from a stream, you will need to cache it in a buffer first. java nio's buffer oriented approach is slightly different. data is read into a buffer from which it is later processed. you can move forth and back in the buffer as you need to. this gives you a bit more flexibility during processing. however, you also need to check if the buffer contains all the data you need in order to fully process it. and, you need to make sure that when reading more data into the buffer, you do not overwrite data in the buffer you have not yet processed. blocking vs. non-blocking io java io's various streams are blocking. that means, that when a thread invokes a read() or write(), that thread is blocked until there is some data to read, or the data is fully written. the thread can do nothing else in the meantime. java nio's non-blocking mode enables a thread to request reading data from a channel, and only get what is currently available, or nothing at all, if no data is currently available. rather than remain blocked until data becomes available for reading, the thread can go on with something else. the same is true for non-blocking writing. a thread can request that some data be written to a channel, but not wait for it to be fully written. the thread can then go on and do something else in the mean time. what threads spend their idle time on when not blocked in io calls, is usually performing io on other channels in the meantime. that is, a single thread can now manage multiple channels of input and output. selectors java nio's selectors allow a single thread to monitor multiple channels of input. you can register multiple channels with a selector, then use a single thread to "select" the channels that have input available for processing, or select the channels that are ready for writing. this selector mechanism makes it easy for a single thread to manage multiple channels. how nio and io influences application design whether you choose nio or io as your io toolkit may impact the following aspects of your application design: the api calls to the nio or io classes. the processing of data. the number of thread used to process the data. the api calls of course the api calls when using nio look different than when using io. this is no surprise. rather than just read the data byte for byte from e.g. an inputstream, the data must first be read into a buffer, and then be processed from there. the processing of data the processing of the data is also affected when using a pure nio design, vs. an io design. in an io design you read the data byte for byte from an inputstream or a reader. imagine you were processing a stream of line based textual data. for instance: name: anna age: 25 email: [email protected] phone: 1234567890 this stream of text lines could be processed like this: inputstream input = ... ; // get the inputstream from the client socket bufferedreader reader = new bufferedreader(new inputstreamreader(input)); string nameline = reader.readline(); string ageline = reader.readline(); string emailline = reader.readline(); string phoneline = reader.readline(); notice how the processing state is determined by how far the program has executed. in other words, once the first reader.readline() method returns, you know for sure that a full line of text has been read. the readline() blocks until a full line is read, that's why. you also know that this line contains the name. similarly, when the second readline() call returns, you know that this line contains the age etc. as you can see, the program progresses only when there is new data to read, and for each step you know what that data is. once the executing thread have progressed past reading a certain piece of data in the code, the thread is not going backwards in the data (mostly not). this principle is also illustrated in this diagram: java io: reading data from a blocking stream. a nio implementation would look different. here is a simplified example: bytebuffer buffer = bytebuffer.allocate(48); int bytesread = inchannel.read(buffer); notice the second line which reads bytes from the channel into the bytebuffer. when that method call returns you don't know if all the data you need is inside the buffer. all you know is that the buffer contains some bytes. this makes processing somewhat harder. imagine if, after the first read(buffer) call, that all what was read into the buffer was half a line. for instance, "name: an". can you process that data? not really. you need to wait until at leas a full line of data has been into the buffer, before it makes sense to process any of the data at all. so how do you know if the buffer contains enough data for it to make sense to be processed? well, you don't. the only way to find out, is to look at the data in the buffer. the result is, that you may have to inspect the data in the buffer several times before you know if all the data is inthere. this is both inefficient, and can become messy in terms of program design. for instance: bytebuffer buffer = bytebuffer.allocate(48); int bytesread = inchannel.read(buffer); while(! bufferfull(bytesread) ) { bytesread = inchannel.read(buffer); } the bufferfull() method has to keep track of how much data is read into the buffer, and return either true or false, depending on whether the buffer is full. in other words, if the buffer is ready for processing, it is considered full. the bufferfull() method scans through the buffer, but must leave the buffer in the same state as before the bufferfull() method was called. if not, the next data read into the buffer might not be read in at the correct location. this is not impossible, but it is yet another issue to watch out for. if the buffer is full, it can be processed. if it is not full, you might be able to partially process whatever data is there, if that makes sense in your particular case. in many cases it doesn't. the is-data-in-buffer-ready loop is illustrated in this diagram: java nio: reading data from a channel until all needed data is in buffer. summary nio allows you to manage multiple channels (network connections or files) using only a single (or few) threads, but the cost is that parsing the data might be somewhat more complicated than when reading data from a blocking stream. if you need to manage thousands of open connections simultanously, which each only send a little data, for instance a chat server, implementing the server in nio is probably an advantage. similarly, if you need to keep a lot of open connections to other computers, e.g. in a p2p network, using a single thread to manage all of your outbound connections might be an advantage. this one thread, multiple connections design is illustrated in this diagram: java nio: a single thread managing multiple connections. if you have fewer connections with very high bandwidth, sending a lot of data at a time, perhaps a classic io server implementation might be the best fit. this diagram illustrates a classic io server design: java io: a classic io server design - one connection handled by one thread. from http://tutorials.jenkov.com/java-nio/nio-vs-io.html

What happens when you have no product owner at all? How does a team know what features to develop in what order? Several teams I know encountered this. They all had product managers. Most of them had BAs. All of them had a technical manager who was willing to be their product owner, but they had no real product owner. They called themselves Scrum-but. I used to think this was ok. I now think Scrum-but is a bad label. That’s because agile needs a responsible person who is not part of the cross-functional technical team to rank the backlog so the team knows the order of the work. Without that person, the team does not know what to do. So why is it so bad for a team to call itself Scrum-but? Because it’s not Scrum-but. It’s not Scrum. It’s iterative and incremental, but it’s not even close to Scrum. It’s not agile. Johanna's General Agile Picture When you have no product owner who is not outside the team, or outside the hierarchy of the team, you lose something very precious to agility, the notion of the customer or customer surrogate. You lose the person who could be helping the team understand what the customer really wants. You lose the back-and-forth about the product that the customer helps the team understand. The manager can help the team understand the requirements, but the manager is not the customer. The manager is not the person who can set the real acceptance criteria. The manager can see the demo, but the manager cannot say for sure that the team is developing the correct requirements in the correct order. So why am I so insistent that we stop calling this Scrum-but, and even stop calling this agile? Because it breaks down the separation when-and-what-to-build (responsible person responsibility from ongoing incremental delivery of product on a regular basis (the cross-functional team responsibility). The customer or responsible person explains when-to-build in my little picture. The team decides how to build it. When the team manager gets involved, that allows the “business” to be unaccountable for developing the system. How do you know what is shippable product without the responsible person? The problem is this: System development, product development is a joint venture between the business people and the technical people. We need the legal, marketing, sales, and anyone else on the “business” side of the house to help us with the what-and-when to build decisions. That’s why we need a responsible person. In Scrum, that person is called a product owner. And, we need a technical project team to deliver the value. We use agile as an approach and use the demo because it shows business value every iteration. When the business is unaccountable, the agile ecosystem breaks down. We no longer have ideas coming into that funnel, being evaluated by that responsible person. Sure that responsible person has a lot to do. And, that responsible person should develop product roadmaps and make the potential product direction transparent to the rest of the organization. That way, the next iteration or two is clear for the team, and everyone can fight discuss the product direction. But when all the discussion is in the technical organization, those discussions tend to not happen. Or the discussions go off in a different direction than the product needs to go. And, that’s a Very Bad Thing. Because, when the discussions don’t occur, the technical group takes all the responsibility for the product: for what to build, when to build it, and for how to build it. And that means we have let the rest of the business abdicate all of their responsibility for their part of the product. That’s not the partnership agile promises us, nor is the transparency agile promises us. So, when you hear Scrum-but because you have no product owner, substitute “On the road to agile.” You’re actually iterative and incremental, but not agile. You have not made one of the necessary cultural changes for transitioning to agile. Can you keep doing what you are doing? Sure, if it’s working for you. And, that’s the million dollar question: How is this working for you? (If you would like more hints as to what else to do, consider my project management book, Manage It! Your Guide to Modern, Pragmatic Project Management. You have other options, if you cannot manage the agile cultural change right now. Those other options will help you move closer to agile than trying Scrum-but and failing.) This is one of the points—the agile ecosystem and making it succeed—I’m working on for my keynote at the Agile Vancouver conference in late October.

This refactoring is a specialization of Replace Data Value with Object: its goal is to replace a scalar or primitive structure (in this case, an ever-present array) with an object where we can host methods that act on those data. We have already seen a lightweight version of this refactoring in the code sample of that article: this time we go all the way to a real object, which has private fields representing the elements of the array. Usually the target of the refactoring is an associative array, but it may also be a numeric one, with a limited number of elements. When to introduce an object where a simple array already works? A clue that points to the need for this refactoring in numerical arrays is the fact that the elements are not homogeneous: they may be in type (all strings or integers) but not in meaning. For example, if two or them are flipped the array loses meaning or becomes very strange: array( 'FirstName LastName', '[email protected]' ) For associative arrays, the refactoring is viable everytime the number of elements is strictly fixed: array( 'name' => ... 'email' => ... ) Private fields are self-documenting, and they're easier to understand and maintain that the documentation of the keys of an array. Documentation on array structures always gets repeated in docblocks and doesn't have a real place to live in without a class; moreover, it's the death of encapsulation as nothing stops client code (even in the parts that should only pass the array to other methods) from accessing every single element of the array. And of course, a class is a place where to put methods, while an array cannot host them. Steps The technique described by Fowler for this refactoring is composed of many little steps: create a new class: it should contain only a public field encapsulating a little the array. Change the client code to use this new class in place of the primitive variable. In an iterative cycle, add a getter and a setter for each field and change client code. At each step, the relevant tests should be run. The methods should still use internally the elements of the array. When this phase has been completed, make the array private and see if the code still works. Add private fields to substitute the elements of the array, and change getters and setters accordingly. This change now ripples only into the source code of the new class. When you're finished, delete the field storing the array. Many little steps are often appropriate as the usage of the array spans over dozens of differente classes, and raises the risk of reaching an irreparably broken build. After you have reached the final state, an object with getters and setters, you can go on and remove methods accordingly for immutability or encapsulation; or move Foreign Methods to the new class now that it has become a first class citizen. Note that tests may encompass even end-to-end ones if the array was used on a large scale. For example, we replaced arrays with objects in the two upper layers of the application, forcing us to run tests at the end-to-end scale. Example In the initial state, a response is created by putting together an array. Client code is omitted for brevity, and only the creation part will be our target. true, 'content' => '{someJson:"ok"}' ); } } The array is moved onto a public field of a new class. true, 'content' => '{someJson:"ok"}' )); } } class HttpResponse { public $data; public function __construct(array $data) { $this->data = $data; } } We add setters (also getters in case we need them.) class HttpResponse { public $data; public function __construct(array $data) { $this->data = $data; } public function setSuccess($boolean) { $this->data['success'] = $boolean; } public function setContent($content) { $this->data['content'] = $content; } } The array becomes private, to check that only getters, setters and methods are really used externally. true, 'content' => '{someJson:"ok"}' )); $response->setSuccess(false); $response->setContent('{}'); $this->assertEquals(new HttpResponse(array( 'success' => false, 'content' => '{}' )), $response); } } class HttpResponse { private $data; public function __construct(array $data) { $this->setSuccess($data['success']); $this->setContent($data['content']); } public function setSuccess($boolean) { $this->data['success'] = $boolean; } public function setContent($content) { $this->data['content'] = $content; } } Private fields replace the array elements. We can start move logic into methods on the new class. class HttpResponse { private $success; private $content; public function __construct(array $data) { $this->setSuccess($data['success']); $this->setContent($data['content']); } public function setSuccess($boolean) { $this->success = $boolean; } public function setContent($content) { $this->content = $content; } }

Today I ran into a common situation: I needed to split a list into 2 sublists - elements that passed a predicate and elements that failed a predicate. I'm sure I've run into this problem several times, but it's been awhile and I'd forgotten what options were available to me. A quick look at http://clojure.github.com/clojure/ reveals several potential functions: partition-by, split-with, and group-by. partition-by From the docs: Usage: (partition-by f coll) Applies f to each value in coll, splitting it each time f returns a new value. Returns a lazy seq of partitions. Let's assume we have a collection of ints and we want to split them into a list of evens and a list of odds. The following REPL session shows the result of calling partition-by with our list of ints. user=> (partition-by even? [1 2 4 3 5 6]) ((1) (2 4) (3 5) (6)) The partition-by function works as described; unfortunately, it's not exactly what I'm looking for. I need a function that returns ((1 3 5) (2 4 6)). split-with From the docs: Usage: (split-with pred coll) Returns a vector of [(take-while pred coll) (drop-while pred coll)] The split-with function sounds promising, but a quick REPL session shows it's not what we're looking for. user=> (split-with even? [1 2 4 3 5 6]) [() (1 2 4 3 5 6)] As the docs state, the collection is split on the first item that fails the predicate - (even? 1). group-by From the docs: Usage: (group-by f coll) Returns a map of the elements of coll keyed by the result of f on each element. The value at each key will be a vector of the corresponding elements, in the order they appeared in coll. The group-by function works, but it gives us a bit more than we're looking for. user=> (group-by even? [1 2 4 3 5 6]) {false [1 3 5], true [2 4 6]} The result as a map isn't exactly what we desire, but using a bit of destructuring allows us to grab the values we're looking for. user=> (let [{evens true odds false} (group-by even? [1 2 4 3 5 6])] [evens odds]) [[2 4 6] [1 3 5]] The group-by results mixed with destructuring do the trick, but there's another option. juxt From the docs: Usage: (juxt f) (juxt f g) (juxt f g h) (juxt f g h & fs) Alpha - name subject to change. Takes a set of functions and returns a fn that is the juxtaposition of those fns. The returned fn takes a variable number of args, and returns a vector containing the result of applying each fn to the args (left-to-right). ((juxt a b c) x) => [(a x) (b x) (c x)] The first time I ran into juxt I found it a bit intimidating. I couldn't tell you why, but if you feel the same way - don't feel bad. It turns out, juxt is exactly what we're looking for. The following REPL session shows how to combine juxt with filter and remove to produce the desired results. user=> ((juxt filter remove) even? [1 2 4 3 5 6]) [(2 4 6) (1 3 5)] There's one catch to using juxt in this way, the entire list is processed with filter and remove. In general this is acceptable; however, it's something worth considering when writing performance sensitive code. From http://blog.jayfields.com/2011/08/clojure-partition-by-split-with-group.html

Varnish is a tool built to be an intermediate server in the HTTP chain, not an origin one like Apache or IIS. You can outsource caching, logging, zipping and other filters to Varnish, since they are not the main feature of an HTTP server like Apache. What we'll see today is how to work with Edge Side Includes in Varnish, as a way to compose dynamic pages from independently generated and cached fragments; we won't encounter logging or other features. If you are familiar with PHP, ESI is an (almost) standard for executing include()-like statements on a front end server like Varnish; the proxy is able not only to assembly pages but also to cache them according to different policies: a certain time, for a single user, and so on. Thijs Feryn and Alessandro Nadalin introduced me to Varnish and ESI respectively, for the first time. I recommend you to consider their blogs and talks as additional sources on these topics. Installation The default version of Varnish in Ubuntu 11.04 is instead 2.1, and apparently does not support ESI very much. Installation via packages means adding a public key and a repository to your list of software sources, and install the varnish package via apt-get or an equivalent command. You can install version 3.0.0 via packages, but only in Ubuntu LTS (10.04). A way that always works in these cases is the installation from sources. The linked page will list the package dependencies and give you a sequence of 3-4 commands to seamlessly compile varnish. I used checkinstall instead of make install to get a binary package that I can reuse later: $ sudo checkinstall -D --install=no --fstrans=no [email protected] --reset-uids=yes --nodoc --pkgname=varnish --pkgversion=3.0.0 --pkgrelease=201108231000 --arch=i386 After installation with dpkg, check that varnishd is available and of the right version: [10:18:17][giorgio@Desmond:~]$ varnishd -V varnishd (varnish-3.0.0 revision 3bd5997) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS Varnish needs minimal configuration: a server to point at. For our tests you can edit /etc/varnish/default.vcl and check (or add) the following: backend default { .host = "127.0.0.1"; .port = "80"; } You can execute ps -A | grep varnishd at any time to see if varnish is already in execution. Execution [09:55:18][giorgio@Desmond:~]$ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:8080 storage_malloc: max size 1024 MB. 1 gigabyte of memory is allocated for keeping fragments in RAM. An administrative interface will respond on port 2000, and only be accessible from localhost. http://localhost:8080/ is the exposed HTTP server, and will point to http://localhost:80 as defined in the configuration. Look at man varnishd for more switched and to man vcl for additional explanations on the configuration language. A bit of ESI ESI is a technique for leveraging HTTP cache and at the same time build dynamic pages. The problem with today's pages is that they are highly dynamic: some sections change very often or according to the current user (Welcome, John Doe or the current posts timeline); some sections do not change at all for days (the navigation bar and the layout structure); some sections change in response to external events (the list of incoming messages only when a new message arrives). It would be ideal to set different caching configurations for all the page's fragments. But implementing this strategy in the application code is error-prone and means reinventing the wheel. To use HTTP cache you will be forced to load with Ajax every single fragment of the page, even a single paragraph. With ESI, your application produces only the pieces, and lets an implementor of the Edge Side Include specification like Varnish assemble the whole thing. Example HTML page (very static): Varnish will work on this page: . PHP page (really dynamic, can change at any time): Varnish will work on this page: 2011-08-23. No sign of Varnish interventions, and totally transparent for the client. And sometimes you can also throw away Zend_Layout and similar components to assemble HTML on the PHP side.