At work, I needed to create a very simple background job, without any concern about what I could get back, because mostly all the hard work was just batch processing and persistence, and all exceptions or roll-back concerns were already taking care of. At the beginning I used a very simple way to call my background job, using Java's: Executors.newSingleThreadExecutor() void myBackgroundJob() { Executors.newSingleThreadExecutor().submit(new Runnable() { @Override public void run() { //My Background Job } }); } And it worked great, just what I needed. Using Groovy facilitate even more the way to create a new Background job, as simple as: def myBackgroundJob() { Thread.start { //My Background Job } } Then, after this simple way to send something into the background, I decided to create a new AST in groovy, that remove the need to remember or copy and paste the same logic. I created two annotations that help to identify the class and the methods that are going to be put into a new Thread. One for the Class: package async import org.codehaus.groovy.transform.GroovyASTTransformationClass import java.lang.annotation.* import xml.ToXmlTransformation @Retention (RetentionPolicy.SOURCE) @Target ([ElementType.TYPE]) @GroovyASTTransformationClass (["async.AsyncTransformation"]) public @interface Asynchronous { } And the other for the Method: package async import org.codehaus.groovy.transform.GroovyASTTransformationClass import java.lang.annotation.* import async.AsyncTransformation @Retention (RetentionPolicy.SOURCE) @Target ([ElementType.METHOD]) @GroovyASTTransformationClass (["async.AsyncTransformation"]) public @interface Async { } then the Asynchronous Transformation, using the AstBuilder().buildFromString(). Here I combined a GroovyInterceptable to connect the method being call with the AST transformation to wrapped with the Thread logic. package async import org.codehaus.groovy.control.CompilePhase import org.codehaus.groovy.transform.* import org.codehaus.groovy.ast.* import org.codehaus.groovy.control.SourceUnit import org.codehaus.groovy.ast.builder.AstBuilder import org.codehaus.groovy.ast.stmt.ExpressionStatement import org.codehaus.groovy.ast.expr.MethodCallExpression import org.codehaus.groovy.ast.expr.ClosureExpression import org.codehaus.groovy.ast.expr.ConstantExpression import org.codehaus.groovy.ast.stmt.BlockStatement import org.codehaus.groovy.ast.expr.ClassExpression import org.codehaus.groovy.ast.expr.ArgumentListExpression @GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS) //CompilePhase.SEMANTIC_ANALYSIS class AsyncTransformation implements ASTTransformation{ void visit(ASTNode[] astNodes, SourceUnit sourceUnit) { if (!astNodes ) return if (!astNodes[0] || !astNodes[1]) return if (!(astNodes[0] instanceof AnnotationNode)) return if (astNodes[0].classNode?.name != Asynchronous.class.name) return def methods = makeMethods(astNodes[1]) if(methods){ astNodes[1]?.interfaces = [ ClassHelper.make(GroovyInterceptable, false), ] as ClassNode [] astNodes[1]?.addMethod(methods?.find { it.name == 'invokeMethod' }) } } def makeMethods(ClassNode source){ def methods = source.methods def annotatedMethods = methods.findAll { it?.annotations?.findAll { it?.classNode?.name == Async.class.name } } if(annotatedMethods){ def expression = annotatedMethods.collect { "name == \"${it.name}\"" }.join(" || ") def ast = new AstBuilder().buildFromString(CompilePhase.INSTRUCTION_SELECTION, false, """ package ${source.packageName} class ${source.nameWithoutPackage} implements GroovyInterceptable { def invokeMethod(String name, Object args){ if(${expression}){ Thread.start{ def calledMethod = ${source.nameWithoutPackage}.metaClass.getMetaMethod(name, args) calledMethod?.invoke(this, args) } }else{ def calledMethod = ${source.nameWithoutPackage}.metaClass.getMetaMethod(name, args)?.invoke(this,args) } } } """) ast[1].methods } } } The example: package async @Asynchronous class Sample{ String name String phone @Async def expensiveMethod(){ println "[${Thread.currentThread()}] Started expensiveMethod" sleep 15000 println "[${Thread.currentThread()}] Finished expensiveMethod..." } @Async def otherMethod(){ println "[${Thread.currentThread()}] Started otherMethod" sleep 5000 println "[${Thread.currentThread()}] Finished otherMethod" } } println "[${Thread.currentThread()}] Start" def sample = new Sample(name:"AST EXample",phone:"1800-GROOVY") sample.expensiveMethod() sample.otherMethod() println "[${Thread.currentThread()}] Finished" Final Notes: As you can see on the example I need to have the Asynchronous annotation on the class still. It could be better without it and just annotate the methods, something like the Groovy's SynchronizedASTTransformation. If you have any idea to complement this small example, please clone the source code [here], and let me know what you think. I could used the @javax.ejb.Asynchronous or the Spring's @org.springframework.scheduling.annotation.Async, but I only needed a very simple solution without any other configuration or library inclusion. The remain logic here could be play more with multi threading and expect some results like: java.util.concurrent.Future and its java.util.concurrent.Future.get() method or maybe integrated with another frameworks like Spring. Source: [Here]

Nowadays, pure CSS3 menus are still very popular. Usually these are UL-LI based menus. Today we will continue making nice menus for you. This tip will create a multi-level dropdown menu, but today submenus will appear not with the onhover action, but with the onclick action instead. Here is what the final result will look like: Here are samples and downloadable packages: Live Demo download in package Ok, download the example files and lets start coding ! Step 1. HTML As usual, we start with the HTML. Here is the full html code with our menu. As you can see - this is multi-level menu. I hope that you can easily understand it. The whole menu is built on UL-LI elements. index.html HomeTutorials HTML / CSSJS / jQuery jQueryJS PHPMySQLXSLTAjax Resources By category PHPMySQLMenu1 Menu1Menu2Menu3 Menu31Menu32Menu33Menu34 Menu4 Ajax By tag name captchagalleryanimation About Step 2. CSS Here are the CSS styles I used. First two selectors - a layout of our demo page. All rest belong to the menu. css/style.css /* demo page styles */ body { background:#eee; margin:0; padding:0; } .example { background:#fff url(../images/tech.jpg); width:770px; height:570px; border:1px #000 solid; margin:20px auto; padding:15px; border-radius:3px; -moz-border-radius:3px; -webkit-border-radius:3px; } /* main menu styles */ #nav,#nav ul { background-image:url(../images/tr75.png); list-style:none; margin:0; padding:0; } #nav { height:41px; padding-left:5px; padding-top:5px; position:relative; z-index:2; } #nav ul { left:-9999px; position:absolute; top:37px; width:auto; } #nav ul ul { left:-9999px; position:absolute; top:0; width:auto; } #nav li { float:left; margin-right:5px; position:relative; } #nav li a { background:#c1c1bf; color:#000; display:block; float:left; font-size:16px; padding:8px 10px; text-decoration:none; } #nav > li > a { -moz-border-radius:6px; -webkit-border-radius:6px; -o-border-radius:6px; border-radius:6px; overflow:hidden; } #nav li a.fly { background:#c1c1bf url(../images/arrow.gif) no-repeat right center; padding-right:15px; } #nav ul li { margin:0; } #nav ul li a { width:120px; } #nav ul li a.fly { padding-right:10px; } /*hover styles*/ #nav li:hover > a { background-color:#858180; color:#fff; } /*focus styles*/ #nav li a:focus { outline-width:0; } /*popups*/ #nav li a:active + ul.dd,#nav li a:focus + ul.dd,#nav li ul.dd:hover { left:0; } #nav ul.dd li a:active + ul,#nav ul.dd li a:focus + ul,#nav ul.dd li ul:hover { left:140px; } Step 3. Images Our menu is using only three images: arrow.gif, tech.jpg and tr75.png. I didn't include them into tutorial because two of them are very small (will be difficult to locate) and the last one is just background image. All images will be in the package. Conclusion Hope you enjoyed this tutorial and learned something new. Good luck! From Script-tutorials

I've just finished an iCalendar vCard parser for PHP. It's done almost completely with a 'natural' simplexml-like interface, so it should (hopefully) be just as easy to parse, and also modify iCalendar / vCard objects (ics/vcf files). To install using pear, run the following: pear channel-discover pear.sabredav.org pear install sabredav/Sabre_VObject-alpha Or download from pear.sabredav.org. For testing, I used this iCalendar file: icalendartest.ics. To load in an object, you use the Reader class: // Link to the correct path if you manually dowloaded the package include 'Sabre/VObject/includes.php'; // Reading an object $calendar = Sabre_VObject_Reader::read(file_get_contents('icalendartest.ics')); iCalendar objects consist of components (VEVENT, VTODO, VTIMEZONE, etc), properties (SUMMARY, DESCRIPTION, DTSTART, etc) and parameters, which are to properties what attributes are to elements in XML. To show a listing of all events in a calendar, this snippet would work: echo "There are ", count($calendar->vevent), " events in this calendar\n"; // Looping through events foreach($calendar->vevent as $event) { echo (string)$event->dtstart, ": ", $event->summary, "\n"; } You can easily modify properties: $calendar->vevent[0]->description = "It's a birthday party"; Creating new objects uses the following syntax: $todo = new Sabre_VObject_Component('vtodo'); $todo->summary = 'Take out the dog'; $calendar->add($todo); And to turn your newly modified calendar back into an ics file: file_put_contents('output.ics', $calendar->serialize()); Lastly, parameters are accessible through array-syntax: echo (string)$calendar->vevent[0]->dtstart['tzid'], "\n"; I had fun building this, I hope it's useful to you as well. It's 100% unittested, but bugs might still appear due to the complex nature of API. Use at your own risk :). This library will be part of the SabreDAV project, which is also where you can go for the source, report bugs or make suggestions.

In one of my previous lives, I used to work in Microsoft and there this word – NTLM (NT Lan Manager) was something that came to us whenever we used to work on applications. Microsoft OS have always provided us with an inbuilt security systems that can be effectively used to offer authentication (and even authorization to web applications). Many years back, I moved over into Java world and when I was asked to carry out my very first security implementation, I realized that there was no easy way to do this and many clients would actually want us to use LDAP for authentication and authorization. For many years, I continued to use that. And, then one day in a discussion with a client, we were asked to offer SSO implementation and client did not have an existing setup like SiteMinder. I started to think about if we can go about using NTLM based authentication. The reason that was possible was because the application we were asked to build was to be used within the organization itself and all the people were required to login into a domain. After some research, I was able to find out a way we could do this. We did a POC and showed it to the client and they were happy about it. What we did has been explained below: Wrote a Servlet which was the first one to be loaded (like Authentication Interceptor). This servlet was responsible for reading the header attributes and identify the user’s Domain and NTID Once we had the details; we sent a request to our Database to see if that user is registered under the same domain/NTID If the user was found in our user-database we allowed him to pass through And then roles and authorization for user was loaded Basically, we bypassed the “Login Screen” where the user was entering the password and used Domain information. Please note that it was possible for us because the Client guaranteed that there was this domain always and all users had unique NTIDs. Also, that it was their responsibility to shield the application from any external entry points where someone may impersonate the Domain/ID. If you are interested, you can refer to the code below: From http://scrtchpad.wordpress.com/2011/08/04/ntml-authentication-in-java/

when studying both the java nio and io api's, a question quickly pops into mind: when should i use io and when should i use nio? in this text i will try to shed some light on the differences between java nio and io, their use cases, and how they affect the design of your code. main differences of java nio and io the table below summarizes the main differences between java nio and io. i will get into more detail about each difference in the sections following the table. io nio stream oriented buffer oriented blocking io non blocking io selectors stream oriented vs. buffer oriented the first big difference between java nio and io is that io is stream oriented, where nio is buffer oriented. so, what does that mean? java io being stream oriented means that you read one or more bytes at a time, from a stream. what you do with the read bytes is up to you. they are not cached anywhere. furthermore, you cannot move forth and back in the data in a stream. if you need to move forth and back in the data read from a stream, you will need to cache it in a buffer first. java nio's buffer oriented approach is slightly different. data is read into a buffer from which it is later processed. you can move forth and back in the buffer as you need to. this gives you a bit more flexibility during processing. however, you also need to check if the buffer contains all the data you need in order to fully process it. and, you need to make sure that when reading more data into the buffer, you do not overwrite data in the buffer you have not yet processed. blocking vs. non-blocking io java io's various streams are blocking. that means, that when a thread invokes a read() or write(), that thread is blocked until there is some data to read, or the data is fully written. the thread can do nothing else in the meantime. java nio's non-blocking mode enables a thread to request reading data from a channel, and only get what is currently available, or nothing at all, if no data is currently available. rather than remain blocked until data becomes available for reading, the thread can go on with something else. the same is true for non-blocking writing. a thread can request that some data be written to a channel, but not wait for it to be fully written. the thread can then go on and do something else in the mean time. what threads spend their idle time on when not blocked in io calls, is usually performing io on other channels in the meantime. that is, a single thread can now manage multiple channels of input and output. selectors java nio's selectors allow a single thread to monitor multiple channels of input. you can register multiple channels with a selector, then use a single thread to "select" the channels that have input available for processing, or select the channels that are ready for writing. this selector mechanism makes it easy for a single thread to manage multiple channels. how nio and io influences application design whether you choose nio or io as your io toolkit may impact the following aspects of your application design: the api calls to the nio or io classes. the processing of data. the number of thread used to process the data. the api calls of course the api calls when using nio look different than when using io. this is no surprise. rather than just read the data byte for byte from e.g. an inputstream, the data must first be read into a buffer, and then be processed from there. the processing of data the processing of the data is also affected when using a pure nio design, vs. an io design. in an io design you read the data byte for byte from an inputstream or a reader. imagine you were processing a stream of line based textual data. for instance: name: anna age: 25 email: [email protected] phone: 1234567890 this stream of text lines could be processed like this: inputstream input = ... ; // get the inputstream from the client socket bufferedreader reader = new bufferedreader(new inputstreamreader(input)); string nameline = reader.readline(); string ageline = reader.readline(); string emailline = reader.readline(); string phoneline = reader.readline(); notice how the processing state is determined by how far the program has executed. in other words, once the first reader.readline() method returns, you know for sure that a full line of text has been read. the readline() blocks until a full line is read, that's why. you also know that this line contains the name. similarly, when the second readline() call returns, you know that this line contains the age etc. as you can see, the program progresses only when there is new data to read, and for each step you know what that data is. once the executing thread have progressed past reading a certain piece of data in the code, the thread is not going backwards in the data (mostly not). this principle is also illustrated in this diagram: java io: reading data from a blocking stream. a nio implementation would look different. here is a simplified example: bytebuffer buffer = bytebuffer.allocate(48); int bytesread = inchannel.read(buffer); notice the second line which reads bytes from the channel into the bytebuffer. when that method call returns you don't know if all the data you need is inside the buffer. all you know is that the buffer contains some bytes. this makes processing somewhat harder. imagine if, after the first read(buffer) call, that all what was read into the buffer was half a line. for instance, "name: an". can you process that data? not really. you need to wait until at leas a full line of data has been into the buffer, before it makes sense to process any of the data at all. so how do you know if the buffer contains enough data for it to make sense to be processed? well, you don't. the only way to find out, is to look at the data in the buffer. the result is, that you may have to inspect the data in the buffer several times before you know if all the data is inthere. this is both inefficient, and can become messy in terms of program design. for instance: bytebuffer buffer = bytebuffer.allocate(48); int bytesread = inchannel.read(buffer); while(! bufferfull(bytesread) ) { bytesread = inchannel.read(buffer); } the bufferfull() method has to keep track of how much data is read into the buffer, and return either true or false, depending on whether the buffer is full. in other words, if the buffer is ready for processing, it is considered full. the bufferfull() method scans through the buffer, but must leave the buffer in the same state as before the bufferfull() method was called. if not, the next data read into the buffer might not be read in at the correct location. this is not impossible, but it is yet another issue to watch out for. if the buffer is full, it can be processed. if it is not full, you might be able to partially process whatever data is there, if that makes sense in your particular case. in many cases it doesn't. the is-data-in-buffer-ready loop is illustrated in this diagram: java nio: reading data from a channel until all needed data is in buffer. summary nio allows you to manage multiple channels (network connections or files) using only a single (or few) threads, but the cost is that parsing the data might be somewhat more complicated than when reading data from a blocking stream. if you need to manage thousands of open connections simultanously, which each only send a little data, for instance a chat server, implementing the server in nio is probably an advantage. similarly, if you need to keep a lot of open connections to other computers, e.g. in a p2p network, using a single thread to manage all of your outbound connections might be an advantage. this one thread, multiple connections design is illustrated in this diagram: java nio: a single thread managing multiple connections. if you have fewer connections with very high bandwidth, sending a lot of data at a time, perhaps a classic io server implementation might be the best fit. this diagram illustrates a classic io server design: java io: a classic io server design - one connection handled by one thread. from http://tutorials.jenkov.com/java-nio/nio-vs-io.html

This refactoring is a specialization of Replace Data Value with Object: its goal is to replace a scalar or primitive structure (in this case, an ever-present array) with an object where we can host methods that act on those data. We have already seen a lightweight version of this refactoring in the code sample of that article: this time we go all the way to a real object, which has private fields representing the elements of the array. Usually the target of the refactoring is an associative array, but it may also be a numeric one, with a limited number of elements. When to introduce an object where a simple array already works? A clue that points to the need for this refactoring in numerical arrays is the fact that the elements are not homogeneous: they may be in type (all strings or integers) but not in meaning. For example, if two or them are flipped the array loses meaning or becomes very strange: array( 'FirstName LastName', '[email protected]' ) For associative arrays, the refactoring is viable everytime the number of elements is strictly fixed: array( 'name' => ... 'email' => ... ) Private fields are self-documenting, and they're easier to understand and maintain that the documentation of the keys of an array. Documentation on array structures always gets repeated in docblocks and doesn't have a real place to live in without a class; moreover, it's the death of encapsulation as nothing stops client code (even in the parts that should only pass the array to other methods) from accessing every single element of the array. And of course, a class is a place where to put methods, while an array cannot host them. Steps The technique described by Fowler for this refactoring is composed of many little steps: create a new class: it should contain only a public field encapsulating a little the array. Change the client code to use this new class in place of the primitive variable. In an iterative cycle, add a getter and a setter for each field and change client code. At each step, the relevant tests should be run. The methods should still use internally the elements of the array. When this phase has been completed, make the array private and see if the code still works. Add private fields to substitute the elements of the array, and change getters and setters accordingly. This change now ripples only into the source code of the new class. When you're finished, delete the field storing the array. Many little steps are often appropriate as the usage of the array spans over dozens of differente classes, and raises the risk of reaching an irreparably broken build. After you have reached the final state, an object with getters and setters, you can go on and remove methods accordingly for immutability or encapsulation; or move Foreign Methods to the new class now that it has become a first class citizen. Note that tests may encompass even end-to-end ones if the array was used on a large scale. For example, we replaced arrays with objects in the two upper layers of the application, forcing us to run tests at the end-to-end scale. Example In the initial state, a response is created by putting together an array. Client code is omitted for brevity, and only the creation part will be our target. true, 'content' => '{someJson:"ok"}' ); } } The array is moved onto a public field of a new class. true, 'content' => '{someJson:"ok"}' )); } } class HttpResponse { public $data; public function __construct(array $data) { $this->data = $data; } } We add setters (also getters in case we need them.) class HttpResponse { public $data; public function __construct(array $data) { $this->data = $data; } public function setSuccess($boolean) { $this->data['success'] = $boolean; } public function setContent($content) { $this->data['content'] = $content; } } The array becomes private, to check that only getters, setters and methods are really used externally. true, 'content' => '{someJson:"ok"}' )); $response->setSuccess(false); $response->setContent('{}'); $this->assertEquals(new HttpResponse(array( 'success' => false, 'content' => '{}' )), $response); } } class HttpResponse { private $data; public function __construct(array $data) { $this->setSuccess($data['success']); $this->setContent($data['content']); } public function setSuccess($boolean) { $this->data['success'] = $boolean; } public function setContent($content) { $this->data['content'] = $content; } } Private fields replace the array elements. We can start move logic into methods on the new class. class HttpResponse { private $success; private $content; public function __construct(array $data) { $this->setSuccess($data['success']); $this->setContent($data['content']); } public function setSuccess($boolean) { $this->success = $boolean; } public function setContent($content) { $this->content = $content; } }

Today I ran into a common situation: I needed to split a list into 2 sublists - elements that passed a predicate and elements that failed a predicate. I'm sure I've run into this problem several times, but it's been awhile and I'd forgotten what options were available to me. A quick look at http://clojure.github.com/clojure/ reveals several potential functions: partition-by, split-with, and group-by. partition-by From the docs: Usage: (partition-by f coll) Applies f to each value in coll, splitting it each time f returns a new value. Returns a lazy seq of partitions. Let's assume we have a collection of ints and we want to split them into a list of evens and a list of odds. The following REPL session shows the result of calling partition-by with our list of ints. user=> (partition-by even? [1 2 4 3 5 6]) ((1) (2 4) (3 5) (6)) The partition-by function works as described; unfortunately, it's not exactly what I'm looking for. I need a function that returns ((1 3 5) (2 4 6)). split-with From the docs: Usage: (split-with pred coll) Returns a vector of [(take-while pred coll) (drop-while pred coll)] The split-with function sounds promising, but a quick REPL session shows it's not what we're looking for. user=> (split-with even? [1 2 4 3 5 6]) [() (1 2 4 3 5 6)] As the docs state, the collection is split on the first item that fails the predicate - (even? 1). group-by From the docs: Usage: (group-by f coll) Returns a map of the elements of coll keyed by the result of f on each element. The value at each key will be a vector of the corresponding elements, in the order they appeared in coll. The group-by function works, but it gives us a bit more than we're looking for. user=> (group-by even? [1 2 4 3 5 6]) {false [1 3 5], true [2 4 6]} The result as a map isn't exactly what we desire, but using a bit of destructuring allows us to grab the values we're looking for. user=> (let [{evens true odds false} (group-by even? [1 2 4 3 5 6])] [evens odds]) [[2 4 6] [1 3 5]] The group-by results mixed with destructuring do the trick, but there's another option. juxt From the docs: Usage: (juxt f) (juxt f g) (juxt f g h) (juxt f g h & fs) Alpha - name subject to change. Takes a set of functions and returns a fn that is the juxtaposition of those fns. The returned fn takes a variable number of args, and returns a vector containing the result of applying each fn to the args (left-to-right). ((juxt a b c) x) => [(a x) (b x) (c x)] The first time I ran into juxt I found it a bit intimidating. I couldn't tell you why, but if you feel the same way - don't feel bad. It turns out, juxt is exactly what we're looking for. The following REPL session shows how to combine juxt with filter and remove to produce the desired results. user=> ((juxt filter remove) even? [1 2 4 3 5 6]) [(2 4 6) (1 3 5)] There's one catch to using juxt in this way, the entire list is processed with filter and remove. In general this is acceptable; however, it's something worth considering when writing performance sensitive code. From http://blog.jayfields.com/2011/08/clojure-partition-by-split-with-group.html

Varnish is a tool built to be an intermediate server in the HTTP chain, not an origin one like Apache or IIS. You can outsource caching, logging, zipping and other filters to Varnish, since they are not the main feature of an HTTP server like Apache. What we'll see today is how to work with Edge Side Includes in Varnish, as a way to compose dynamic pages from independently generated and cached fragments; we won't encounter logging or other features. If you are familiar with PHP, ESI is an (almost) standard for executing include()-like statements on a front end server like Varnish; the proxy is able not only to assembly pages but also to cache them according to different policies: a certain time, for a single user, and so on. Thijs Feryn and Alessandro Nadalin introduced me to Varnish and ESI respectively, for the first time. I recommend you to consider their blogs and talks as additional sources on these topics. Installation The default version of Varnish in Ubuntu 11.04 is instead 2.1, and apparently does not support ESI very much. Installation via packages means adding a public key and a repository to your list of software sources, and install the varnish package via apt-get or an equivalent command. You can install version 3.0.0 via packages, but only in Ubuntu LTS (10.04). A way that always works in these cases is the installation from sources. The linked page will list the package dependencies and give you a sequence of 3-4 commands to seamlessly compile varnish. I used checkinstall instead of make install to get a binary package that I can reuse later: $ sudo checkinstall -D --install=no --fstrans=no [email protected] --reset-uids=yes --nodoc --pkgname=varnish --pkgversion=3.0.0 --pkgrelease=201108231000 --arch=i386 After installation with dpkg, check that varnishd is available and of the right version: [10:18:17][giorgio@Desmond:~]$ varnishd -V varnishd (varnish-3.0.0 revision 3bd5997) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS Varnish needs minimal configuration: a server to point at. For our tests you can edit /etc/varnish/default.vcl and check (or add) the following: backend default { .host = "127.0.0.1"; .port = "80"; } You can execute ps -A | grep varnishd at any time to see if varnish is already in execution. Execution [09:55:18][giorgio@Desmond:~]$ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:8080 storage_malloc: max size 1024 MB. 1 gigabyte of memory is allocated for keeping fragments in RAM. An administrative interface will respond on port 2000, and only be accessible from localhost. http://localhost:8080/ is the exposed HTTP server, and will point to http://localhost:80 as defined in the configuration. Look at man varnishd for more switched and to man vcl for additional explanations on the configuration language. A bit of ESI ESI is a technique for leveraging HTTP cache and at the same time build dynamic pages. The problem with today's pages is that they are highly dynamic: some sections change very often or according to the current user (Welcome, John Doe or the current posts timeline); some sections do not change at all for days (the navigation bar and the layout structure); some sections change in response to external events (the list of incoming messages only when a new message arrives). It would be ideal to set different caching configurations for all the page's fragments. But implementing this strategy in the application code is error-prone and means reinventing the wheel. To use HTTP cache you will be forced to load with Ajax every single fragment of the page, even a single paragraph. With ESI, your application produces only the pieces, and lets an implementor of the Edge Side Include specification like Varnish assemble the whole thing. Example HTML page (very static): Varnish will work on this page: . PHP page (really dynamic, can change at any time): Varnish will work on this page: 2011-08-23. No sign of Varnish interventions, and totally transparent for the client. And sometimes you can also throw away Zend_Layout and similar components to assemble HTML on the PHP side.

Many frameworks for storing objects in an off-line or cached manner, use standard Java Serialization to encode the object as bytes which can be turned back into the original object. Java Serialization is generic and can serialise just about any type of object. Why avoid it The main problem with Java Serialization is performance and efficiency. Java serialization is much slower than using in memory stores and tends to significantly expand the size of the object. Java Serialization also creates a lot of garbage. Access performance Say you have a collection and you want to update a field of many elements. Something like for (MutableTypes mt : mts) { mt.setInt(mt.getInt()); } If you update one million elements for about five seconds how long does each one take. Huge Collection update one field, took an average 5.1 ns. List update one field took an average 6.5 ns. List with Externalizable update one field took an average 5,841 ns. List update one field took an average 23,217 ns. If you update ten million elements for five seconds or more Huge Collection update one field, took an average 5.4 ns. List, update one field took an average 6.6 ns. List with readObject/writeObject update one field took an average 6,073 ns. List update one field took an average 22,943 ns. Huge Collection stores information in a column based based, so accessing just one field is much more CPU cache efficient than using JavaBeans. If you were to update every field, it would be about 2x or more times slower. Using an optimised Externalizable is much faster than the default Serializable, however is it 400x slower than using a a JavaBean Memory efficiency The per object memory used is also important as it impacts how many object you can store and the performance of accessing those objects. Collection type Heap used per million Direct memory per million Garbage produced per million Huge Collection 0.09 MB 34 MB 80 bytes List 68 MB none 30 bytes List using Externalizable 140 MB none 5,941 MB List 506 MB none 16,746 MB This test was performed on a collection of one million elements. To test the amount of garbage produced I set the Eden size target than 15 GB so no GC would be performed. -mx22g -XX:NewSize=20g -XX:-UseTLAB -verbosegc Conclusion Having an optimised readExternal/writeExternal can improve performance and the size of a serialised object by 2-4 times, however if you need to maximise performance and efficiency you can gain much more by not using it. From http://vanillajava.blogspot.com/2011/08/avoiding-java-serialization-to-increase.html

In Eclipse, when you press Ctrl button and click on any Class names, the IDE will take you to the source file for that class. This is the normal behavior for the classes you have in your project. But, in case you want the same behavior for Java’s core classes too, you can have it by attaching the Java source with the Eclipse IDE. Once you attach the source, thereafter when you Ctrl+Click any Java class names (String for example), Eclipse will open the source code of that class. To attach the Java source code with Eclipse, When you install the JDK, you must have selected the option to install the Java source files too. This will copy the src.zip file in the installation directory. In Eclipse, go to Window -> Preferences -> Java -> Installed JREs -> Add and choose the JDK you have in your system. Eclipse will now list the JARs found in the dialog box. There, select the rt.jar and choose Source Attachment. By default, this will be pointing to the correct src.zip. If not, choose the src.zip file which you have in your java installation directory. Similarly, if you have the javadoc downloaded in your machine, you can configure that too in this dialog box. Done! Here after, for all the projects for which you are using the above JDK, you’ll be able to browse the Java’s source code just like how you browse your own code. From http://veerasundar.com/blog/2011/08/attaching-java-source-with-eclipse-ide

JSDoc is the de facto standard for documenting JavaScript code. You need to know at least its syntax (which is also used by many other tools) if you publish code. Alas, documentation is still scarce, but this post can help – it shows you how to run JSDoc and how its syntax works. (The JSDoc wiki [2] is the main source of this post, some examples are borrowed from it.) As a tool, JSDoc takes JavaScript code with special /** */ comments and produces HTML documentation for it. For example: Given the following code. /** @namespace */ var util = { /** * Repeat str several times. * @param {string} str The string to repeat. * @param {number} [times=1] How many times to repeat the string. * @returns {string} */ repeat: function(str, times) { if (times === undefined || times < 1) { times = 1; } return new Array(times+1).join(str); } }; The generated HTML looks as follows in a web browser: This post begins with a quick start, so can try out JSDoc immediately if you are impatient. Afterwards, more background information is given. 1. Quick start For the steps described below, you need to have Java installed. JSDoc includes the shell script jsrun.sh that requires Unix (including OS X and Linux) to run. But it should be easy to translate that script to a Windows batch file. Download the latest jsdoc_toolkit. Unpack the archive into, say, $HOME/jsdoc-toolkit. Make the script $HOME/jsdoc-toolkit/jsrun.sh executable and tell it where to look for the JSDoc binary and the template (which controls what the result looks like). JSDOCDIR="$HOME/local/jsdoc-toolkit" JSDOCTEMPLATEDIR="$JSDOCDIR/templates/jsdoc" Now you can move the script anywhere you want to, e.g. a bin/ directory. For the purpose of this demonstration, we don’t move the script. Use jsrun.sh on a directory of JavaScript files: $HOME/jsdoc-toolkit/jsrun.sh -d=$HOME/doc $HOME/js Input: $HOME/js – a directory of JavaScript files (see below for an example). Output: $HOME/doc – where to write the generated files. If you put the JavaScript code at the beginning of this post into a file $HOME/js/util.js then JSDoc produces the following files: $HOME/doc +-- files.html +-- index.html +-- symbols +-- _global_.html +-- src ¦ +-- util.js.html +-- util.html 2. Introduction: What is JSDoc? It’s a common programming problem: You have written JavaScript code that is to be used by others and need a nice-looking HTML documentation of its API. Java has pioneered this domain via its JavaDoc tool. The quasi-standard in the JavaScript world is JSDoc. As seen above, you document an entity by putting before it a special comment that starts with two asterisks. Templates. In order to output anything, JSDoc always needs a template, a mix of JavaScript and specially marked-up HTML that tells it how to translate the parsed documentation to HTML. JSDoc comes with a built-in template, but there are others that you can download [3]. 2.1. Terminology and conventions of JSDoc Doclet: JSDoc calls its comments doclets which clashes with JavaDoc terminology where such comments are called doc comments and a doclet is similar to a JSDoc template, but written in Java. Variable: The term variable in JSDoc often refers to all documentable entities which include global variables, object properties, and inner members. Instance properties: In JavaScript one typically puts methods into a prototype to share them with all instances of a class, while fields (non-function-valued properties) are put into each instance. JSDoc conflates shared properties and per-instance properties and calls them instance properties. Class properties, static properties: are properties of classes, usually of constructor functions. For example, Object.create is a class property of Object. Inner members: An inner member is data nested inside a function. Most relevant for documentation is instance-private data nested inside a constructor function. function MyClass() { var privateCounter = 0; // an inner member this.inc = function() { // an instance property privateCounter++; }; } 2.2. Syntax Let’s review the comment shown at the beginning: /** * Repeat str several times. * @param {string} str The string to repeat. * @param {number} [times=1] How many times to repeat the string. * @returns {string} */ This demonstrates some of the JSDoc syntax which consists of the following pieces. JSDoc comment: is a JavaScript block comment whose first character is an asterisk. This creates the illusion that the token /** starts such a comment. Tags: Comments are structured by starting lines with tags, keywords that are prefixed with an @ symbol. @param is an example above. HTML: You can freely use HTML in JSDoc comments; for example, to display a word in a monospaced font. Type annotations: You can document the type of a value by putting the type name in braces after the appropriate tags. Variations: Single type: @param {string} name Multiple types: @param {string|number} idCode Arrays of a type: @param {string[]} names Name paths: are used to refer to variables inside JSDoc comments. The syntax of such paths is as follows. myFunction MyConstructor MyConstructor.classProperty MyConstructor#instanceProperty MyConstructor-innerMember 2.3. A word on types There are two kinds of values in JavaScript: primitives and objects [7]. Primitive types: boolean, number, string. The values undefined and null are also considered primitive. Object types: All other types are object types, including arrays and functions. Watch out: the names of primitive types start with a lowercase letter. Each primitive type has a corresponding wrapper type, an object type with a capital name whose instances are objects: The wrapper type of boolean is Boolean. The wrapper type of number is Number. The wrapper type of string is String. Getting the name of the type of a value: Primitive value p: via typeof p. > typeof "" 'string' Compare: an instance of the wrapper type is an object. > typeof new String() 'object' Object value o: via o.constructor.name. Example: > new String().constructor.name 'String' 3. Basic tags Meta-data: @fileOverview: marks a JSDoc comment that describes the whole file. @author: Who has written the variable being documented? @deprecated: indicates that the variable is not supported, any more. It is a good practice to document what to use instead. @example: contains a code example, illustrating how the given entity should be used. /** * @example * var str = "abc"; * console.log(repeat(str, 3)); // abcabcabc */ Linking: @see: points to a related resource. /** * @see MyClass#myInstanceMethod * @see The Example Project. */ {@link ...}: works like @see, but can be used inside other tags. @requires resourceDescription: a resource that the documented entity needs. The resource description is either a name path or a natural language description. Versioning: @version versionNumber: indicates the version of the documented entity. Example: @version 10.3.1 @since versionNumber: indicates since which version the documented entity has been available. Example: @since 10.2.0 4. Documenting functions and methods For functions and methods, one can document parameters, return values, and exceptions they might throw. @param {paramType} paramName description: describes the parameter whose name is paramName. Type and description are optional. Examples: @param str @param str The string to repeat. @param {string} str @param {string} str The string to repeat. Advanced features: Optional parameter: @param {number} [times] The number of times is optional. Optional parameter with default value: @param {number} [times=1] The number of times is optional. @returns {returnType} description: describes the return value of the function or method. Either type or description can be omitted. @throws {exceptionType} description: describes an exception that might be thrown during the execution of the function or method. Either type or description can be omitted. 4.1. Inline type information (“inline doc comments”) There are two ways of providing type information for parameters and return values. First, you can add a type annotation to @param and @returns. /** * @param {String} name * @returns {Object} */ function getPerson(name) { } Second, you can inline the type information: function getPerson(/**String*/ name) /**Object*/ { } 5. Documenting variables and fields Fields are properties with non-function values. Because instance fields are often created inside a constructor, you have to document them there. @type {typeName}: What type does the documented variable have? Example: /** @constructor */ function Car(make, owner) { /** @type {string} */ this.make = make; /** @type {Person} */ this.owner = owner; } @type {Person} This tag can also be used to document the return type of functions, but @returns is preferable in this case. @constant: A flag that indicates that the documented variable has a constant value. @default defaultValue: What is the default value of a variable? Example: /** @constructor */ function Page(title) { /** * @default "Untitled" */ this.title = title || "Untitled"; } @property {propType} propName description: Document an instance property in the class comment. Example: /** * @class * @property {string} name The name of the person. */ function Person(name) { this.name = name; } Without this tag, instance properties are documented as follows. /** * @class */ function Person(name) { /** * The name of the person. * @type {string} */ this.name = name; } Which one of those styles to use is a matter of taste. @property does introduce redundancies, though. 6. Documenting classes JavaScript’s built-in means for defining classes are weak, which is why there are many APIs that help with this task [5]. These APIs differ, often radically, so you have to help JSDoc with figuring out what is going on. There are three basic ways of defining a class: Constructor function: You must mark a constructor function, otherwise it will not be documented as a class. That is, capitalization alone does not mark a function as a constructor. /** * @constructor */ function Person(name) { } @class is a synonym for @constructor, but it also allows you to describe the class – as opposed to the function setting up an instance (see tag documentation below for an example). API call and object literal: You need two markers. First, you need to tell JSDoc that a given variable holds a class. Second, you need to mark an object literal as defining a class. The latter is done via the @lends tag. /** @class */ var Person = makeClass( /** @lends Person# */ { say: function(message) { return "This person says: " + message; } } ); API call and object literal with a constructor method: If one of the methods in an object literal performs the task of a constructor (setting up instance data, [8]), you need to mark it as such so that fields are found by JSDoc. Then the documentation of the class moves to that method. var Person = makeClass( /** @lends Person# */ { /** * A class for managing persons. * @constructs */ initialize: function(name) { this.name = name; }, say: function(message) { return this.name + " says: " + message; } } ); Tags: @constructor: marks a function as a constructor. @class: marks a variable as a class or a function as a constructor. Can be used in a constructor comment to separate the description of the constructor (first line below) from the description of the class (second line below). /** * Creates a new instance of class Person. * @class Represents a person. */ Person = function() { } @constructs: marks a method in an object literal as taking up the duties of a constructor. That is, setting up instance data. In such a case, the class must be documented there. Works in tandem with @lends. @lends namePath: specifies to which class the following object literal contributes. There are two ways of contributing. @lends Person# – the object literal contributes instance properties to Person. @lends Person – the object literal contributes class properties to Person. 6.1. Inheritance, namespacing JavaScript has no simple support for subclassing and no real namespaces [6]. You thus have to help JSDoc see what is going on when you are using work-arounds. @extends namePath: indicates that the documented class is the subclass of another one. Example: /** * @constructor * @extends Person */ function Programmer(name) { Person.call(this, name); ... } // Remaining code for subclassing omitted @augments: a synonym for @extends. @namespace: One can use objects to simulate namespaces in JavaScript. This tag marks such objects. Example: /** @namespace */ var util = { ... }; 7. Meta-tags Meta-tags are tags that are added to several variables. You put them in a comment that starts with “/**#@+”. They are then added to all variables until JSDoc encounters the closing comment “/**#@-*/”. Example: /**#@+ * @private * @memberOf Foo */ function baz() {} function zop() {} function pez() {} /**#@-*/ 8. Rarely used tags @ignore: ignore a variable. Note that variables without /** comments are ignored, anyway. @borrows otherNamePath as this.propName: A variable is just a reference to somewhere else; it is documented there. Example: /** * @constructor * @borrows Remote#transfer as this.send */ function SpecialWriter() { this.send = Remote.prototype.transfer; } @description text: Provide a description, the same as all of the text before the first tag. Manual categorization. Sometimes JSDoc misinterprets what a variable is. Then you can help it via one of the following tags: Tag Mark variable as @function function @field non-function value @public public (especially inner variables) @private private @inner inner and thus also private @static accessible without instantiation Name and membership: @name namePath: override the parsed name and use the given name, instead. @memberOf parentNamePath: the documented variable is a member of the specified object. Not explained here: @event, see [2]. 9. Related reading jsdoc-toolkit - A documentation generator for JavaScript: JSDoc homepage on Google Code. Includes a link to the downloads. JSDoc wiki: the official documentation of JSDoc and source of this post. JSDoc wiki – TemplateGallery: lists available JSDoc templates. JSDoc wiki – TagReference: a handy cheat-sheet for JSDoc tags. Lightweight JavaScript inheritance APIs Modules and namespaces in JavaScript JavaScript values: not everything is an object Prototypes as classes – an introduction to JavaScript inheritance From http://www.2ality.com/2011/08/jsdoc-intro.html

One of the rules of simple design is the necessity to minimize the number of moving parts, like classes and methods, as long as the tests are satisfied and we are not accepting duplication or feeling the lack of an explicit concept. Thus, a rule that aids simple design is to use primitive types unless a field has already some behavior attached: we don't create a class for the user's name or the user's password; we just use some strings. As we make progress, however, we must be able to revise our decisions via refactoring: if a field gains some logic, this behavior shouldn't be modelled by methods in the containing class, but by a new object. The code in this new class can be reused, while the containing object will change from case to case and you will end up duplicating the same methods. Transforming a scalar value into an object is the essence of the Replace Data Value with Object refactoring. In most of the cases, a Value Object or a Parameter Object come out as a result: while DDD pursue Value Objects as concepts in the domain layer, this refactoring is more general and can be applied anywhere. For instance, in a project we started introducing Data Transfer Objects to model the data sent by the controller to a Service Layer. Data values in PHP In PHP, all scalar values are by nature data values as they cannot host methods: string, integers, and booleans are proper scalar. arrays are not scalar in the Perl or mathematical sense, but they are still a primitive type. On the borderline, we find some simple objects used as data containers in PHP: ArrayObjects. SplHeap and other SPL data structures. The classes on the borderline may host methods, but the original class is out of reach for modification, and an indirection has to be introduced."Local Extension" Steps Create the new class: it should contain as a private field just the value you want to substitute. The methods you immediately need have to be chosen between a constructor, getters, and setters (where needed). Change the field in the containing class. Update the constructor to also create the new object and populate the field, or accept injection (a rarer case). Update the original getter to delegate to the new one. Update the original setter to delegate to the new one (where present) or to create a new object. Run tests at the functional level; the changes should be propagated to the construction phases, while the external usage should not change very much. Example In the initial state, magic arrays are passed around. It's very easy to build an array where a key is missing or is called incorrectly. newPassword(array( 'userId' => 42, 'oldPassword' => 'gismo', 'newPassword' => 'supersecret', 'repeatNewPassword' => 'supersecret' )); $this->markTestIncomplete('This refactoring is about the introduction of an object; it suffices that the test does not explode.'); } } class UserService { public function newPassword($changePasswordData) { /* it's not interesting to do something here */ } } After the introduction of an ArrayObject extension, a little type safety is ensure and we gained a place to put methods at a little cost. newPassword(new ChangePasswordCommand(array( 'userId' => 42, 'oldPassword' => 'gismo', 'newPassword' => 'supersecret', 'repeatNewPassword' => 'supersecret' ))); $this->markTestIncomplete('This refactoring is about the introduction of an object; it suffices that the test does not explode.'); } } class UserService { public function newPassword(ChangePasswordCommand $changePasswordData) { /* it's not interesting to do something here */ } } class ChangePasswordCommand extends ArrayObject { } We add methods to implement logic on this object; in this case, validation logic; in general cases, any kind of code that should not be duplicated by the different clients. For a stricter implementation, wrap an array or another data structure (scalars, SPL objects) instead of extending ArrayObject as you gain immutability and encapsulation (but this kind of objects need little encapsulation.) class ChangePasswordCommand extends ArrayObject { public function __construct($data) { if (!isset($data['userId'])) { throw new Exception('User id is missing.'); } parent::__construct($data); } public function getPassword() { if ($this['newPassword'] != $this['repeatNewPassword']) { throw new Exception('Password do not match.'); } return $this['newPassword']; } } Being this a refactoring however, this is the less invasive kind of introduction of objects you can make as the client code can still use the ArrayAccess interface and treat the object as a scalar array.

About one year ago I wrote a blog post about JavaScriptSerializer and the Serialize and Deserialize methods it supports. Note: This blog post has been in draft for sometime now, so I decided to complete it and publish it. There might be situation when you want to serialize to JSON string only specific properties of a given class. You can do that using JavaScriptSerializer in combination with LINQ. Let’s say we have the following class definition public class Customer { public string Name { get; set; } public string Surname { get; set; } public string Email { get; set; } public int Age { get; set; } public bool Drinker { get; set; } public bool Smoker { get; set; } public bool Single { get; set; } } Next, lets create method that will create sample data for our demo private List GetListOfCustomers() { List customers = new List(); customers.Add(new Customer() { Name = "Hajan", Surname = "Selmani", Age = 25, Drinker = false, Smoker = false, Single = false, Email = "[email protected]" }); customers.Add(new Customer() { Name = "John", Surname = "Doe", Age = 29, Drinker = false, Smoker = true, Single = false, Email = "[email protected]" }); customers.Add(new Customer() { Name = "Mark", Surname = "Moris", Age = 34, Drinker = true, Smoker = true, Single = true, Email = "[email protected]" }); return customers; } So, we have three customers with some property values for each of them. Now, lets serialize some of their properties using JavaScriptSerializer. First, you must put the following directive: using System.Web.Script.Serialization; Next, we create list of customers that will get the returned value from GetListOfCustomers method and we create instance of JavaScriptSerializer class List customers = GetListOfCustomers(); JavaScriptSerializer serializer = new JavaScriptSerializer(); Now, lets say we want to serialize as JSON string and retrieve only the Age property data… We do that with only one simple line of code: //this will serialize only the 'Age' property string jsonString = serializer.Serialize(customers.Select(x => x.Age)); The result will be: Nice! Now, what if we want to serialize multiple properties at once, but not all class properties? string jsonStringMultiple = serializer.Serialize(customers.Select(x => new { x.Name, x.Surname, x.Age })); The result will be: You see, the result is an array of objects with the four properties and their corresponding values we have selected using the LINQ query above. You can see that integer and boolean values are without quotes, which is correct way of serialization. Now, you probably saw a difference somewhere? Namely, in the first example where we have selected only one property, there are only the values of the property (no property name), while in the second example we have the property name and it’s corresponding value… Why is it like that? It’s because in the second query, we use new { … } to specify multiple properties in the select statement. Therefore, the anonymous new { … } creates an object of each found item. So, if you are interested to make some more tests, run the following two lines of code: var customers1 = customers.Select(x => x.Name).ToList(); var customers2 = customers.Select(x=> new { x.Name } ).ToList(); and you will obviously see the difference. If we use the new { } way for single property selection, like in the following example string jsonString2 = serializer.Serialize(customers.Select(x => new { x.Age })); the result will be: The complete demo code used for this blog post: List customers = GetListOfCustomers(); JavaScriptSerializer serializer = new JavaScriptSerializer(); //this will serialize only the 'Age' property string jsonString = serializer.Serialize(customers.Select(x => x.Age )); string jsonStringMultiple = serializer.Serialize(customers.Select(x => new { x.Name, x.Surname, x.Age, x.Drinker })); var customers1 = customers.Select(x => x.Name).ToList(); var customers2 = customers.Select(x=> new { x.Name } ).ToList(); string jsonString2 = serializer.Serialize(customers.Select(x => new { x.Age })); You can download the demo project here.

There are a number of problems with having a large number of records in memory. One way around this is to use direct memory, but this is too low level for most developers. Is there a way to make this more friendly? Limitations of large numbers of objects The overhead per object is between 12 and 16 bytes for 64-bit JVMs. If the object is relatively small, this is significant and could be more than the data itself. The GC pause time increases with the number of objects. Pause times can be around one second per GB of objects. Collections and arrays only support two billion elements Huge collections One way to store more data and still follow object orientated principles is have wrappers for direct ByteBuffers. This can be tedious to write, but very efficient. What would be ideal is to have these wrappers generated automatically. Small JavaBean Example This is an example of JavaBean which would have far more overhead than actual data contained. interface MutableByte { public void setByte(byte b); public byte getByte(); } It is also small enough that I can create billions of these on my machine. This example creates a List with 16 billion elements. final long length = 16_000_000_000L; HugeArrayList hugeList = new HugeArrayBuilder() {{ allocationSize = 4 * 1024 * 1024; capacity = length; }.create(); List list = hugeList; assertEquals(0, list.size()); hugeList.setSize(length); // add a GC to see what the GC times are like. System.gc(); assertEquals(Integer.MAX_VALUE, list.size()); assertEquals(length, hugeList.longSize()); byte b = 0; for (MutableByte mb : list) mb.setByte(b++); b = 0; for (MutableByte mb : list) { byte b2 = mb.getByte(); byte expected = b++; if (b2 != expected) assertEquals(expected, b2); } From start to finish, the heap memory used is as follows. with -verbosegc 0 sec - 3100 KB used [GC 9671K->1520K(370496K), 0.0020330 secs] [Full GC 1520K->1407K(370496K), 0.0063500 secs] 10 sec - 3885 KB used 20 sec - 4428 KB used 30 sec - 4428 KB used ... deleted ... 1380 sec - 4475 KB used 1390 sec - 4476 KB used 1400 sec - 4476 KB used 1410 sec - 4476 KB used The only GC is one triggered explicitly. Without the System.gc(); no GC logs appear. After 20 sec, the increase in memory used is from logging how much memory was used. Conclusion The library is relatively slow. Each get or set takes about 40 ns which really adds up when there are so many calls to make. I plan to work on it so it is much faster. ;) On the upside, it wouldn't be possible to create 16 billion objects with the memory I have, nor could it be put in an ArrayList, so having it a little slow is still better than not working at all. From http://vanillajava.blogspot.com/2011/08/collection-with-billions-of-entries.html

i often get asked about ‘rest to soap’ transformation use cases these days. using an soa gateway like securespan to perform this type of transformation at runtime is trivial to setup. with securespan in front of any existing web service (in the dmz for example), you can virtualize a rest version of this same service. using an example, here is a description of the steps to perform this conversion. imagine the geoloc web service for recording geographical locations. it has two methods, one for setting a location and one for getting a location. see below what this would look like in soap. request: 34802398402 response: 52.37706 4.889721 request: 34802398402 52.37706 4.889721 response: ok here is the equivalent rest target that i want to support at the edge. payloads could be xml, but let’s use json to make it more interesting. get /position/34802398402 http 200 ok content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } post /position/34802398402 content-type: text/json { 'latitude' : 52.37706 'longitude' : 4.889721 } http 200 ok ok now let’s implement this rest version of the service using securespan. i’m assuming that you already have a securespan gateway deployed between the potential rest requesters and the existing soap web service. first, i will create a new service endpoint on the gateway for this service and assign anything that comes at the uri pattern /position/* to this service. i will also allow the http verbs get and post for this service. rest geoloc service properties next, let’s isolate the resource id from the uri and save this as a context variable named ‘trackerid’. we can use a simple regex assertion to accomplish this. also, i will branch on the incoming http verb using an or statement. i am just focusing on get and post for this example but you could add additional logic for other http verbs that you want to support for this rest service. regex for rest service resource identification policy branching for get vs post for get requests, the transformation is very simple, we just declare a message variable using a soap skeleton into which we refer to the trackerid variable. soap request template this soap message is routed to the existing web service and the essential elements are isolated using xpath assertions. processing soap response the rest response is then constructed back using a template response. template json response a similar logic is performed for the post message. see below for the full policy logic. complete policy you’re done for virtualizing the rest service. setting this up with securespan took less than an hour, did not require any change on the existing soap web service and did not require the deployment of an additional component. from there, you would probably enrich the policy to perform some json schema validation , some url and query parameter validation, perhaps some authentication, authorization , etc.

A new method is needed, as we are factoring out some lines of code. It would be nice to have it available on a class which already has all the fields to execute it, but unfortunately we cannot modify original code (because it's part of a library, or it's bundled with PHP.) A classic example of Introduce Foreign Method is cause by a missing method on ArrayObject, or SplQueue, or other external classes. The first place to place a new method is the object the code works with, since it's cohesive with the rest of the data. The second best place is the client object, although the method could get duplicated in different clients. A Foreign Method is a method created in the client code to complete the functionality of a source object. How it works The source object becomes a first additional argument of the Foreign Method: Python and some other languages reflects this with the self first argument, which makes refactoring Foreign Methods into normal ones easier. For a PHP- example, consider PHPUnit: I use Foreign Methods in my test cases to wrap $this->getMock() multiple usages. I factor away the adaptation to the Api (use of false arguments or MockBuilder) and leave as arguments the variability of the mocks, like the number of calls or the expected parameters. A Foreign Method is a work-around: if you have the possibility to change the source class, definitely go for that. Another alternative is to introduce a wrapper, when you can modify the lifecycle in order to inject a wrapper in the client code instead of the original object. It's not the case for PHPUnit (you extend the class), but it is for SplQueue and other collection objects; wrapping has also other benefits which will be shown in the next articles. Steps Create the method in the client class. Make the server object the first parameter of the method, if it cannot be passed already through $this (in case it's referenced by a field). Substitute the duplicated code with method calls. Fowler suggests also to comment the method calling it Foreign Method and saying it should be moved onto the server object when it will become possible. Example Initially, we're using an ArrayObject and continuing to ordering it after each addition of an element: addUrl('twitter.com'); $links->add('plus.google.com', 'Google+'); $links->add('facebook.com', 'Facebook'); $expected = "Facebook\n" . "Google+\n" . "twitter.com"; $this->assertEquals($expected, $links->__toString()); } } class LinkGroup { private $links; public function __construct() { $this->links = new ArrayObject(); } public function add($url, $text) { $this->links[$url] = $text; $this->links->asort(); } public function addUrl($url) { $this->links[$url] = $url; $this->links->asort(); } public function __toString() { $links = array(); foreach ($this->links as $url => $text) { $links[] = "$text"; } return implode("\n", $links); } } We introduce a Foreign Method, newLink(): class LinkGroup { private $links; public function __construct() { $this->links = new ArrayObject(); } public function add($url, $text) { $this->newLink($url, $text); } public function addUrl($url) { $this->newLink($url, $url); } public function __toString() { $links = array(); foreach ($this->links as $url => $text) { $links[] = "$text"; } return implode("\n", $links); } private function newLink($url, $text) { $this->links[$url] = $text; $this->links->asort(); } } We also add some comments noting that if more and more Foreign Methods pop out, a radical solution would be needed: /** * Foreign Method of the ArrayObject. Should be moved onto a newly extracted * collaborator which wraps the ArrayObject, or an heap-like data structure * should be used. */ private function newLink($url, $text) { $this->links[$url] = $text; $this->links->asort(); }

Below is a list of some tools that can help you examine your Java source code for potential problems: 1. PMD from http://pmd.sourceforge.net/ License: PMD is licensed under a “BSD-style” license PMD scans Java source code and looks for potential problems like: * Possible bugs – empty try/catch/finally/switch statements * Dead code – unused local variables, parameters and private methods * Suboptimal code – wasteful String/StringBuffer usage * Overcomplicated expressions – unnecessary if statements, for loops that could be while loops * Duplicate code – copied/pasted code means copied/pasted bugs You can download everything from here, and you can get an overview of all the rules at the rulesets index page. PMD is integrated with JDeveloper, Eclipse, JEdit, JBuilder, BlueJ, CodeGuide, NetBeans/Sun Java Studio Enterprise/Creator, IntelliJ IDEA, TextPad, Maven, Ant, Gel, JCreator, and Emacs. 2. FindBug from http://findbugs.sourceforge.net License: L-GPL FindBugs, a program which uses static analysis to look for bugs in Java code. And since this is a project from my alumni university (IEEE – University of Maryland, College Park – Bill Pugh) , I have to definitely add this contribution to this list. 3. Clover from http://www.cenqua.com/clover/ License: Free for Open Source (more like a GPL) Measures statement, method, and branch coverage and has XML, HTML, and GUI reporting. and comprehensive plug-ins for major IDEs. * Improve Test Quality * Increase Testing Productivity * Keep Team on Track Fully integrated plugins for NetBeans, Eclipse , IntelliJ IDEA, JBuilder and JDeveloper. These plugins allow you to measure and inspect coverage results without leaving the IDE. Seamless Integration with projects using Apache Ant and Maven. * Easy integration into legacy build systems with command line interface and API. Fast, accurate, configurable, detailed coverage reporting of Method, Statement, and Branch coverage. Rich reporting in HTML, PDF, XML or a Swing GUI Precise control over the coverage gathering with source-level filtering. Historical charting of code coverage and other metrics. Fully compatible with JUnit 3.x & 4.x, TestNG, JTiger and other testing frameworks. Can also be used with manual, functional or integration testing. 4. Macker from http://innig.net/macker/ License: GPL Macker is a build-time architectural rule checking utility for Java developers. It’s meant to model the architectural ideals programmers always dream up for their projects, and then break — it helps keep code clean and consistent. You can tailor a rules file to suit a specific project’s structure, or write some general “good practice” rules for your code. Macker doesn’t try to shove anybody else’s rules down your throat; it’s flexible, and writing a rules file is part of the development process for each unique project. 5 EMMA from http://emma.sourceforge.net/ License: EMMA is distributed under the terms of Common Public License v1.0 and is thus free for both open-source and commercial development. Reports on class, method, basic block, and line coverage (text, HTML, and XML). EMMA can instrument classes for coverage either offline (before they are loaded) or on the fly (using an instrumenting application classloader). Supported coverage types: class, method, line, basic block. EMMA can detect when a single source code line is covered only partially. Coverage stats are aggregated at method, class, package, and “all classes” levels. Output report types: plain text, HTML, XML. All report types support drill-down, to a user-controlled detail depth. The HTML report supports source code linking. Output reports can highlight items with coverage levels below user-provided thresholds. Coverage data obtained in different instrumentation or test runs can be merged together. EMMA does not require access to the source code and degrades gracefully with decreasing amount of debug information available in the input classes. EMMA can instrument individial .class files or entire .jars (in place, if desired). Efficient coverage subset filtering is possible, too. Makefile and ANT build integration are supported on equal footing. EMMA is quite fast: the runtime overhead of added instrumentation is small (5-20%) and the bytecode instrumentor itself is very fast (mostly limited by file I/O speed). Memory overhead is a few hundred bytes per Java class. EMMA is 100% pure Java, has no external library dependencies, and works in any Java 2 JVM (even 1.2.x). 6. XRadar from http://xradar.sourceforge.net/ License: BSD (me thinks) The XRadar is an open extensible code report tool currently supporting all Java based systems. The batch-processing framework produces HTML/SVG reports of the systems current state and the development over time – all presented in sexy tables and graphs. The XRadar gives measurements on standard software metrics such as package metrics and dependencies, code size and complexity, code duplications, coding violations and code-style violations. 7. Hammurapi from Hammurapi Group License: (if anyone knows the license for this email me Venkatt.Guhesan at Y! dot com) Hammurapi is a tool for execution of automated inspection of Java program code. Following the example of 282 rules of Hammurabi’s code, we are offered over 120 Java classes, the so-called inspectors, which can, at three levels (source code, packages, repository of Java files), state whether the analysed source code contains violations of commonly accepted standards of coding. Relevant Links: http://en.sdjournal.org/products/articleInfo/93 http://wiki.hammurapi.biz/index.php?title=Hammurapi_4_Quick_Start 8. Relief from http://www.workingfrog.org/ License: GPL Relief is a design tool providing a new look on Java projects. Relying on our ability to deal with real objects by examining their shape, size or relative place in space it gives a “physical” view on java packages, types and fields and their relationships, making them easier to handle. 9. Hudson from http://hudson-ci.org/ License: MIT Hudson is a continuous integration (CI) tool written in Java, which runs in a servlet container, such as Apache Tomcat or the GlassFish application server. It supports SCM tools including CVS, Subversion, Git and Clearcase and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands. 10. Cobertura from http://cobertura.sourceforge.net/ License: GNU GPL Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage. 11. SonarSource from http://www.sonarsource.org/ (recommended by Vishwanath Krishnamurthi – thanks) License: LGPL Sonar is an open platform to manage code quality. As such, it covers the 7 axes of code quality: Architecture & Design, Duplications, Unit Tests, Complexity, Potential bugs, Coding rules, Comments. From http://mythinkpond.wordpress.com/2011/07/14/java-tools-for-source-code-optimization-and-analysis/

I have seen a number of tests where a JVM has 10K threads. However, what happens if you go beyond this? My recommendation is to consider having more servers once your total reaches 10K. You can get a decent server for $2K and a powerful one for $10K. Creating threads gets slower The time it takes to create a thread increases as you create more thread. For the 32-bit JVM, the stack size appears to limit the number of threads you can create. This may be due to the limited address space. In any case, the memory used by each thread's stack add up. If you have a stack of 128KB and you have 20K threads it will use 2.5 GB of virtual memory. Bitness Stack Size Max threads 32-bit 64K 32,073 32-bit 128K 20,549 32-bit 256K 11,216 64-bit 64K stack too small 64-bit 128K 32,072 64-bit 512K 32,072 Note: in the last case, the thread stacks total 16 GB of virtual memory. Java 6 update 26 32-bit,-XX:ThreadStackSize=64 4,000 threads: Time to create 4,000 threads was 0.522 seconds 8,000 threads: Time to create 4,000 threads was 1.281 seconds 12,000 threads: Time to create 4,000 threads was 1.874 seconds 16,000 threads: Time to create 4,000 threads was 2.725 seconds 20,000 threads: Time to create 4,000 threads was 3.333 seconds 24,000 threads: Time to create 4,000 threads was 4.151 seconds 28,000 threads: Time to create 4,000 threads was 5.293 seconds 32,000 threads: Time to create 4,000 threads was 6.636 seconds After creating 32,073 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 32-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.525 seconds 8,000 threads: Time to create 4,000 threads was 1.239 seconds 12,000 threads: Time to create 4,000 threads was 1.902 seconds 16,000 threads: Time to create 4,000 threads was 2.529 seconds 20,000 threads: Time to create 4,000 threads was 3.165 seconds After creating 20,549 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 32-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.526 seconds 8,000 threads: Time to create 4,000 threads was 1.212 seconds After creating 11,216 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 64-bit,-XX:ThreadStackSize=128 4,000 threads: Time to create 4,000 threads was 0.577 seconds 8,000 threads: Time to create 4,000 threads was 1.292 seconds 12,000 threads: Time to create 4,000 threads was 1.995 seconds 16,000 threads: Time to create 4,000 threads was 2.653 seconds 20,000 threads: Time to create 4,000 threads was 3.456 seconds 24,000 threads: Time to create 4,000 threads was 4.663 seconds 28,000 threads: Time to create 4,000 threads was 5.818 seconds 32,000 threads: Time to create 4,000 threads was 6.792 seconds After creating 32,072 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) Java 6 update 26 64-bit,-XX:ThreadStackSize=512 4,000 threads: Time to create 4,000 threads was 0.577 seconds 8,000 threads: Time to create 4,000 threads was 1.292 seconds 12,000 threads: Time to create 4,000 threads was 1.995 seconds 16,000 threads: Time to create 4,000 threads was 2.653 seconds 20,000 threads: Time to create 4,000 threads was 3.456 seconds 24,000 threads: Time to create 4,000 threads was 4.663 seconds 28,000 threads: Time to create 4,000 threads was 5.818 seconds 32,000 threads: Time to create 4,000 threads was 6.792 seconds After creating 32,072 threads, java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at com.google.code.java.core.threads.MaxThreadsMain.addThread(MaxThreadsMain.java:46) at com.google.code.java.core.threads.MaxThreadsMain.main(MaxThreadsMain.java:16) The Code MaxThreadsMain.java From http://vanillajava.blogspot.com/2011/07/java-what-is-limit-to-number-of-threads.html

as i'm sure you know python doesn't have a concept of private members. one trick that is sometimes used is to hide an object inside a python closure, and provide a proxy object that only permits limited access to the original object. here's a simple example of a hide function that takes an object and returns a proxy. the proxy allows you to access any attribute of the original, but not to set or change any attributes. def hide(obj): class proxy(object): __slots__ = () def __getattr__(self, name): return getattr(obj, name) return proxy() here it is in action: >>> class foo(object): ... def __init__(self, a, b): ... self.a = a ... self.b = b ... >>> f = foo(1, 2) >>> p = hide(f) >>> p.a, p.b (1, 2) >>> p.a = 3 traceback (most recent call last): ... attributeerror: 'proxy' object has no attribute 'a' after the hide function has returned the proxy object the __getattr__ method is able to access the original object through the closure . this is stored on the __getattr__ method as the func_closure attribute (python 2) or the __closure__ attribute (python 3). this is a "cell object" and you can access the contents of the cell using the cell_contents attribute: >>> cell_obj = p.__getattr__.func_closure[0] >>> cell_obj.cell_contents <__main__.foo object at 0x...> this makes hide useless for actually preventing access to the original object. anyone who wants access to it can just fish it out of the cell_contents . what we can't do from pure-python is*set* the contents of the cell, but nothing is really private in python - or at least not in cpython. there are two python c api functions, pycell_get and pycell_set , that provide access to the contents of closures. from ctypes we can call these functions and both introspect and modify values inside the cell object: >>> import ctypes >>> ctypes.pythonapi.pycell_get.restype = ctypes.py_object >>> py_obj = ctypes.py_object(cell_obj) >>> f2 = ctypes.pythonapi.pycell_get(py_obj) >>> f2 is f true >>> new_py_obj = ctypes.py_object(foo(5, 6)) >>> ctypes.pythonapi.pycell_set(py_obj, new_py_obj) 0 >>> p.a, p.b (5, 6) as you can see, after the call to pycell_set the proxy object is using the new object we put in the closure instead of the original. using ctypes may seem like cheating, but it would only take a trivial amount of c code to do the same. two notes about this code. it isn't (of course) portable across different python implementations don't ever do this, it's for illustration purposes only! still, an interesting poke around the cpython internals with ctypes. interestingly i have heard of one potential use case for code like this. it is alleged that at some point armin ronacher was using a similar technique in jinja2 for improving tracebacks. (tracebacks from templating languages can be very tricky because the compiled python code usually bears a quite distant relationship to the original text based template.) just because armin does it doesn't mean you can though...

As you probably saw by now, JetBrains just announced that they are working on a brand new statically typed JVM language called Kotlin. I am planning to write a post to evaluate how Kotlin compares to the other existing languages, but first, I’d like to take a slightly different angle and try to answer a question I have already seen asked several times: what’s the point? We already have quite a few JVM languages, do we need any more? Here are a few reasons that come to mind. 1) Coolness New languages are exciting! They really are. I’m always looking forward to learning new languages. The more foreign they are, the more curious I am, but the languages I really look forward to discovering are the ones that are close to what I already know but not identical, just to find out what they did differently that I didn’t think of. Allow me to make a small digression in order to clarify my point. Some time ago, I started learning Japanese and it turned out to be the hardest and, more importantly, the most foreign natural language I ever studied. Everything is different from what I’m used to in Japanese. It’s not just that the grammar, the syntax and the alphabets are odd, it’s that they came up with things that I didn’t even think would make any sense. For example, in English (and many other languages), numbers are pretty straightforward and unique: one bag, two cars, three tickets, etc… Now, did it ever occur to you that a language could allow several words to mean “one”, and “two”, and “three”, etc…? And that these words are actually not arbitrary, their usage follows some very specific rules. What could these rules be? Well, in Japanese, what governs the word that you pick is… the shape of the object that you are counting. That’s right, you will use a different way to count if the object is long, flat, a liquid or a building. Mind-boggling, isn’t it? Here is another quick example: in Russian, each verb exists in two different forms, which I’ll call A and B to simplify. Russian doesn’t have a future tense, so when you want to speak at the present tense, you’ll conjugate verb A in the present, and when you want the future, you will use the B form… in the present tense. It’s not just that you need to learn two verbs per verb, you also need to know which one is which if you want to get your tenses right. Oh and these forms also have different meanings when you conjugate them in past tenses. End of digression. The reason why I am mentioning this is because this kind of construct bends your mind, and this goes for natural languages as much as programming languages. It’s tremendously exciting to read new syntaxes this way. For that reason alone, the arrival of new languages should be applauded and welcome. Kotlin comes with a few interesting syntactic innovations of its own, which I’ll try to cover in a separate post, but for now, I’d like to come back to my original point, which was to give you reasons why you should be excited about Kotlin, so let’s keep going down the list. 2) IDE support None of the existing JVM languages (Groovy, Scala, Fantom, Gosu, Ceylon) have really focused much on the tooling aspect. IDE plug-ins exist for each of them, all with varying quality, but they are all an afterthought, and they suffer from this oversight. The plug-ins are very slow to mature, they have to keep up with the internals of a compiler that’s always evolving and which, very often, doesn’t have much regards for the tools built on top of it. It’s a painful and frustrating process for tool creators and tool users alike. With Kotlin, we have good reasons to think that the IDE support will be top notch. JetBrains is basically announcing that they are building the compiler and the IDEA support in lockstep, which is a great way to guarantee that the language will work tremendously well inside IDEA, but also that other tools should integrate nicely with it as well (I’m rooting for a speedy Eclipse plug-in, obviously). 3) Reified generics This is a pretty big deal. Not so much for the functionality (I’ll get back to this in the next paragraph) but because this is probably the very first time that we see a JVM language with true support for reified generics. This innovation needs to be saluted. Correction from the comments: Gosu has reified generics Having said that, I don’t feel extremely excited by this feature because overall, I think that reified generics come at too high a price. I’ll try to dedicate a full blog post to this topic alone, because it deserves a more thorough treatment. 4) Commercial support JetBrains has a very clear financial interest in seeing Kotlin succeed. It’s not just that they are a commercial entity that can put money behind the development of the language, it’s also that the success of the language would most likely mean that they will sell more IDEA licenses, and this can also turn into an additional revenue stream derived from whatever other tools they might come up with that would be part of the Kotlin ecosystem. This kind of commercial support for a language was completely unheard of in the JVM world for fifteen years, and suddenly, we have two instances of it (Typesafe and now JetBrains). This is a good sign for the JVM community. 5) Still no Java successors Finally, the simple truth is that we still haven’t found any credible Java successor. Java still reigns supreme and is showing no sign of giving away any mindshare. Pulling numbers out of thin air, I would say that out of all the code currently running on the JVM today, maybe 94% of it is in Java, 3% is in Groovy and 1% is in Scala. This 94% figure needs to go down, but so far, no language has stepped up to whittle it down significantly. What will it take? Obviously, nothing that the current candidates are offering (closures, modularity, functional features, more concise syntax, etc…) has been enough to move that needle. Something is still missing. Could the missing piece be “stellar IDE support” or “reified generics”? We don’t know yet because no contender offers any of these features, but Kotlin will, so we will soon know. Either way, I am predicting that we will keep seeing new JVM languages pop up at a regular pace until one finally claims the prize. And this should be cause for rejoicing for everyone interested in the JVM ecosystem. So let’s cheer for Kotlin and wish JetBrains the best of luck with their endeavor. I can’t wait to see what will come out of this. Oh, and secretly, I am rooting for Eclipse to start working on their own JVM language too, obviously. From http://beust.com/weblog/2011/07/20/five-reasons-why-should-rejoice-about-kotlin/