Innodb can indexes built by sort since Innodb Plugin for MySQL 5.1 which is a lot faster than building them through insertion, especially for tables much larger than memory and large uncorrelated indexes you might be looking at 10x difference or more. Yet for some reason Innodb team has chosen to use very small (just 1MB) and hard coded buffer for this operation, which means almost any such index build operation has to use excessive sort merge passes significantly slowing down index built process. Mark Callaghan and Facebook Team has fixed this in their tree back in early 2011 adding innodb_merge_sort_block_size variable and I was thinking this small patch will be merged to MySQL 5.5 promptly, yet it has not happen to date. Here is example of gains you can expect (courtesy of Alexey Kopytov), using 1Mil rows Sysbench table. Buffer Length | alter table sbtest add key(c) 1MB 34 sec 8MB 26 sec 100MB 21 sec 128MB 17 sec REBUILD 37 sec REBUILD in this table is using “fast_index_creation=0″ which allows to disable fast index creation in Percona Server and force complete table to be rebuilt instead. Looking at this data we can see even for such small table there is possible to improve index creation time 2x by using large buffer. Also we can see we can substantially improve performance even increasing it from 1MB to 8MB, which might be sensible as default as even small systems should be able to allocate 8MB to do alter table. You may be wondering why in this case table rebuild is so close in performance to building index by sort with small buffer – this comes from building index on long character field with very short length, Innodb would use fixed size records for sort space which results in a lot more work done than you would otherwise need. Having some optimization to better deal with this case also would be nice. The table also was fitting in buffer pool completely in this case which means table rebuild could have done fast too. Results are from Percona Server 5.5.24

The immediate attribute in JSF is commonly misunderstood. If you don't believe me, check out Stack Overflow. Part of the confusion is likely due to immediate being available on both input (i.e.. ) and command (i.e. ) components, each of which affects the JSF lifecycle differently. Here is the standard JSF lifecycle: For the purposes of this article, I'll assume you are familiar with the basics of the JSF lifecycle. If you need an introduction or a memory refresher, check out the Java EE 6 Tutorial - The Lifecycle of a JavaServer Faces Application. Note: the code examples in this article are for JSF 2 (Java EE 6), but the principals are the same for JSF 1.2 (Java EE 5). immediate=true on Command components In the standard JSF lifecycle, the action attribute on an Command component is evaluated in the Invoke Application phase. For example, say we have a User entity/bean: public class User implements Serializable { @NotBlank @Length(max = 50) private String firstName; @NotBlank @Length(max = 50) private String lastName; /* Snip constructors, getters/setters, a nice toString() method, etc */ } And a UserManager to serve as our managed bean: @SessionScoped @ManagedBean public class UserManager { private User newUser; /* Snip some general page logic... */ public String addUser() { //Snip logic to persist newUser FacesContext.getCurrentInstance().addMessage(null, new FacesMessage("User " + newUser.toString() + " added")); return "/home.xhtml"; } And a basic Facelets page, newUser.xhtml, to render the view: Which all combine to produce this lovely form: When the user clicks on the Add User button, #{userManager.addUser} will be called in the Invoke Application phase; this makes sense, because we want the input fields to be validated, converted, and applied to newUser before it is persisted. Now let's add a "cancel" button to the page, in case the user changes his/her mind. We'll add another to the page: And the cancel() method to UserManager: public String cancel() { newUser = new User(); FacesContext.getCurrentInstance().addMessage(null, new FacesMessage("Cancelled new user")); return "/home.xhtml"; } Looks good, right? But when we actually try to use the cancel button, we get errors complaining that first and last name are required: This is because #{userManager.cancel} isn't called until the Invoke Application phase, which occurs after the Process Validations phase; since we didn't enter a first and last name, the validations failed before #{userManager.cancel} is called, and the response is rendered after the Process Validations phase. We certainly don't want to require the end user to enter a valid user before cancelling! Fortunately, JSF provides the immediate attribute on Command components. When immediate is set to true on an Command component, the action is invoked in the Apply Request Values phase: This is perfect for our Cancel use case. If we add immediate=true to the Cancel , #{userManager.cancel} will be called in the Apply Request Values phase, before any validation occurs. So now when we click cancel, #{userManager.cancel} is called in the Apply Request Values phase, and we are directed back to the home page with the expected cancellation message; no validation errors! What about Input components? Input components have the immediate attribute as well, which also moves all their logic into the Apply Request Values phase. However, the behavior is slightly different from Command components, especially depending on whether or not the validation on the Input component succeeds. My next article will address immediate=true on Input components. For now, here's a preview of how the JSF lifecycle is affected:

Usually we have some required fields on our forms and it would be nice if ASP.NET MVC views can detect those fields automatically and display nice red asterisk after field label. As this functionality is not built in I built my own solution based on data annotations. In this posting I will show you how to show red asterisk after label of required fields. Here are the main information sources I used when working out my own solution: How can I modify LabelFor to display an asterisk on required fields? (stackoverflow) ASP.NET MVC – Display visual hints for the required fields in your model (Radu Enucă) Although my code was first written for completely different situation I needed it later and I modified it to work with models that use data annotations. If data member of model has Required attribute set then asterisk is rendered after field. If Required attribute is missing then there will be no asterisk. Here’s my code. You can take just LabelForRequired() methods and paste them to your own HTML extension class. public static class HtmlExtensions { [SuppressMessage("Microsoft.Design", "CA1006:DoNotNestGenericTypesInMemberSignatures", Justification = "This is an appropriate nesting of generic types")] public static MvcHtmlString LabelForRequired(this HtmlHelper html, Expression> expression, string labelText = "") { return LabelHelper(html, ModelMetadata.FromLambdaExpression(expression, html.ViewData), ExpressionHelper.GetExpressionText(expression), labelText); } private static MvcHtmlString LabelHelper(HtmlHelper html, ModelMetadata metadata, string htmlFieldName, string labelText) { if (string.IsNullOrEmpty(labelText)) { labelText = metadata.DisplayName ?? metadata.PropertyName ?? htmlFieldName.Split('.').Last(); } if (string.IsNullOrEmpty(labelText)) { return MvcHtmlString.Empty; } bool isRequired = false; if (metadata.ContainerType != null) { isRequired = metadata.ContainerType.GetProperty(metadata.PropertyName) .GetCustomAttributes(typeof(RequiredAttribute), false) .Length == 1; } TagBuilder tag = new TagBuilder("label"); tag.Attributes.Add( "for", TagBuilder.CreateSanitizedId( html.ViewContext.ViewData.TemplateInfo.GetFullHtmlFieldName(htmlFieldName) ) ); if (isRequired) tag.Attributes.Add("class", "label-required"); tag.SetInnerText(labelText); var output = tag.ToString(TagRenderMode.Normal); if (isRequired) { var asteriskTag = new TagBuilder("span"); asteriskTag.Attributes.Add("class", "required"); asteriskTag.SetInnerText("*"); output += asteriskTag.ToString(TagRenderMode.Normal); } return MvcHtmlString.Create(output); } } And here’s how to use LabelForRequired extension method in your view: @Html.LabelForRequired(m => m.Name) @Html.TextBoxFor(m => m.Name) @Html.ValidationMessageFor(m => m.Name) After playing with CSS style called .required my example form looks like this: These red asterisks are not part of original view mark-up. LabelForRequired method detected that these properties have Required attribute set and rendered out asterisks after field names. NB! By default asterisks are not red. You have to define CSS class called “required” to modify how asterisk looks like and how it is positioned.

Apache Camel has a very powerful bean injection framework which allows developers to focus only on solving business problems. However there are situations when you need to do a little bit more. Read below to see how easy it is to setup aspects (AspectJ) in Apache Camel. Use case In Qualitas I have an installation route which consists of 10 mandatory and 2 optional processors. Some processors like property resolvers or validators don't modify contents of message's body so I have to always copy body from the in message to out message. Also, all my processors require some headers to function properly. Finally, I would like to get a status updated after each of my processors either finishes processing successfully or fails. Setting up AspectJ This is just a plain Spring configuration frankly. All you have to do is: Apache Camel processor aspect To define aspect in AspectJ I used AspectJ-specific @Aspect and @Around annotations: @Aspect @Component @Order(Ordered.LOWEST) public class HeadersAndBodyCopierAspect { @Around("execution(* com.googlecode.qualitas.internal.installation..*.process(org.apache.camel.Exchange)) && args(exchange) && target(org.apache.camel.Processor)") public Object copyHeadersAndBody(ProceedingJoinPoint pjp, Exchange exchange) throws Throwable { Object retValue = pjp.proceed(); Message in = exchange.getIn(); Message out = exchange.getOut(); // always copy headers out.setHeaders(in.getHeaders()); // if output body is empty copy it from input if (out.getBody() == null) { out.setBody(in.getBody()); } return retValue; } } I also used @Order Spring-specific annotation to control the order of execution of my aspects and @Component for automatic context scanning. Now, the join point is defined as execution(* com.googlecode.qualitas.internal.installation..*.process(org.apache.camel.Exchange)) && args(exchange) && target(org.apache.camel.Processor) which basically means: apply this aspect to all process methods which are defined in all classes in com.googlecode.qualitas.internal.installation or subpackages and which take Exchange object as an argument there can be many custom methods whose names may be process and whose argument may be Exchange, so I added one more constraint, this class has to be an instance of Processor args(exchange) allows me to add Exchange object as an argument to my aspect More complex aspects and source code Of course in Spring you can inject other beans directly into your aspects. I used it in my ProcessStatusUpdaterAspect aspect which you can find in Qualitas repo on GoogleCode or GitHub. If you are interested in trying out the whole Qualitas system take a look at the following two links: BuildingTheProject and RunningTheProject. cheers, Łukasz

News flash: concurrency is hard. Any time you have mutable data and multiple threads, you are just asking for abuse, and synchronized is simply not going to cut it. I was recently contacted by a client who was load testing their Tapestry 5.3.3 application; they were using Tomcat 6.0.32 with 500 worker threads, on a pretty beefy machine: Intel Xeon X7460 @ 2.66Ghz, OpenJDK 64-Bit Server VM (14.0-b16, mixed mode). That's a machine with six cores, and 16 MB of L2 cache. For all that power, they were tapping out at 450 requests per second. That's not very good when you have 500 worker threads ... it means that you've purchased memory and processing power just to see all those worker threads block, and you get to see your CPU utilization stay low. When synchronization is done properly, increasing the load on the server should push CPU utilization to 100%, and response time should be close to linear with load (that is to say, all the threads should be equally sharing the available processing resources) until the hard limit is reached. Fortunately, these people approached me not with a vague performance complaint, but with a detailed listing of thread contention hotspots. The goal with Tapestry has always been to build the code right initially, and optimize the code later if needed. I've gone through several cycles of this over the past couple of years, optimizing page construction time, or memory usage, or throughput performance (as here). In general, I follow Brian Goetz's advice: write simple, clean, code and let the compiler and Hotspot figure out the rest. Another piece of advice from Brian is that "uncontested synchronized calls are very cheap". Many of the hotspots located by my client were, in fact, simple synchronized methods that did some lazy initialization. Here's an example: public class InternalComponentResourcesImpl ... private Messages messages; public synchronized Messages getMessages() { if (messages == null) messages = elementResources.getMessages(componentModel); return messages; } } In this example, getting the messages can be relatively time consuming and expensive, and is often not necessary at all. That is, in most instances of the class, the getMessages() method is never invoked. There were a bunch of similar examples of optional things that are often not needed ... but can be heavily used in the cases where they are used. It turns out that "uncontested" really means virtually no thread contention whatsoever. I chatted with Brian at the Hacker Bed & Breakfast about this, and he explained that you can quickly go from "extremely cheap" to "asymptotically expensive" when there's any potential for contention. The synchronized keyword is very limited in one area: when exiting a synchronized block, all threads that are waiting for that lock must be unblocked, but only one of those threads gets to take the lock; all the others see that the lock is taken and go back to the blocked state. That's not just a lot of wasted processing cycles: often the context switch to unblock a thread also involves paging memory off the disk, and that's very, very, expensive. Enter ReentrantReadWriteLock: this is an alternative that allows any number of readers to share a lock, but only a single writer. When a thread attempts to acquire the write lock, the thread blocks until all reader threads have released the read lock. The cost of managing the ReentrantReadWriteLock's state is somewhat higher than synchronized, but has the huge advantage of letting multiple reader threads operate simultaneously. That means much, much higher throughput. In practice, this means you must acquire the shared read lock to look at a field, and acquire the write lock in order to change the field. ReentrantReadWriteLock is smart about only waking the right thread or threads when either the read lock or the write lock is released. You don't see the same thrash you would with synchronized: if a thread is waiting for the write lock, and another thread releases it, ReentrantReadWriteLock will (likely) just unblock the one waiting thread. Using synchronized is easy; with an explicit ReentrantReadWriteLock there's a lot more code to manage: public class InternalComponentResourcesImpl ... private final ReadWriteLock lazyCreationLock = new ReentrantReadWriteLock(); private Messages messages; public Messages getMessages() { try { lazyCreationLock.readLock().lock(); if (messages == null) { obtainComponentMessages(); } return messages; } finally { lazyCreationLock.readLock().unlock(); } } private void obtainComponentMessages() { try { lazyCreationLock.readLock().unlock(); lazyCreationLock.writeLock().lock(); if (messages == null) { messages = elementResources.getMessages(componentModel); } } finally { lazyCreationLock.readLock().lock(); lazyCreationLock.writeLock().unlock(); } } } I like to avoid nested try ... finally blocks, so I broke it out into seperate methods. Notice the "lock dance": it is not possible to acquire the write lock if any thread, even the current thread, has the read lock. This opens up a tiny window where some other thread might pop in, grab the write lock and initialize the messages field. That's why it is desirable to double check, once the write lock has been acquired, that the work has not already been done. Also notice that things aren't quite symmetrical: with ReentrantReadWriteLock it is allowable for the current thread to acquire the read lock before releasing the write lock. This helps to minimize context switches when the write lock is released, though it isn't expressly necessary. Is the conversion effort worth it? Well, so far, simply by converting synchronized to ReentrantReadWriteLock, and adding a couple of additional caches (also using ReentrantReadWriteLock), we've seen some significant improvements; from 450 req/sec to 2000 req/sec ... and there's still a few minor hotspots to address. I think that's been worth a few hours of work!

This article is part 2 of our NoClassDefFoundError troubleshooting series. It will focus and describe the simplest type of NoClassDefFoundError problem. This article is ideal for Java beginners and I highly recommend that you compile and run the sample Java program yourself. The following writing format will be used going forward and will provide you with: - Description of the problem case and type of NoClassDefFoundError - Sample Java program “simulating” the problem case - ClassLoader chain view - Recommendations and resolution strategies NoClassDefFoundError problem case 1 – missing JAR file The first problem case we will cover is related to a Java program packaging and / or classpath problem. A typical Java program can include one or many JAR files created at compile time. NoClassDefFoundError can often be observed when you forget to add JAR file(s) containing Java classes referenced by your Java or Java EE application. This type of problem is normally not hard to resolve once you analyze the Java Exception and missing Java class name. Sample Java program The following simple Java program is split as per below: - The main Java program NoClassDefFoundErrorSimulator - The caller Java class CallerClassA - The referencing Java class ReferencingClassA - A util class for ClassLoader and logging related facilities JavaEETrainingUtil This program is simple attempting to create a new instance and execute a method of the Java class CallerClassA which is referencing the class ReferencingClassA.It will demonstrate how a simple classpath problem can trigger NoClassDefFoundError. The program is also displaying detail on the current class loader chain at class loading time in order to help you keep track of this process. This will be especially useful for future and more complex problem cases when dealing with larger class loader chains. #### NoClassDefFoundErrorSimulator.java package org.ph.javaee.training1; import org.ph.javaee.training.util.JavaEETrainingUtil; /** * NoClassDefFoundErrorTraining1 * @author Pierre-Hugues Charbonneau * */ public class NoClassDefFoundErrorSimulator { /** * @param args */ public static void main(String[] args) { System.out.println("java.lang.NoClassDefFoundError Simulator - Training 1"); System.out.println("Author: Pierre-Hugues Charbonneau"); System.out.println("http://javaeesupportpatterns.blogspot.com"); // Print current Classloader context System.out.println("\nCurrent ClassLoader chain: "+JavaEETrainingUtil.getCurrentClassloaderDetail()); // 1. Create a new instance of CallerClassA CallerClassA caller = new CallerClassA(); // 2. Execute method of the caller caller.doSomething(); System.out.println("done!"); } } #### CallerClassA.java package org.ph.javaee.training1; import org.ph.javaee.training.util.JavaEETrainingUtil; /** * CallerClassA * @author Pierre-Hugues Charbonneau * */ public class CallerClassA { private final static String CLAZZ = CallerClassA.class.getName(); static { System.out.println("Classloading of "+CLAZZ+" in progress..."+JavaEETrainingUtil.getCurrentClassloaderDetail()); } public CallerClassA() { System.out.println("Creating a new instance of "+CallerClassA.class.getName()+"..."); } public void doSomething() { // Create a new instance of ReferencingClassA ReferencingClassA referencingClass = new ReferencingClassA(); } } #### ReferencingClassA.java package org.ph.javaee.training1; import org.ph.javaee.training.util.JavaEETrainingUtil; /** * ReferencingClassA * @author Pierre-Hugues Charbonneau * */ public class ReferencingClassA { private final static String CLAZZ = ReferencingClassA.class.getName(); static { System.out.println("Classloading of "+CLAZZ+" in progress..."+JavaEETrainingUtil.getCurrentClassloaderDetail()); } public ReferencingClassA() { System.out.println("Creating a new instance of "+ReferencingClassA.class.getName()+"..."); } public void doSomething() { //nothing to do... } } #### JavaEETrainingUtil.java package org.ph.javaee.training.util; import java.util.Stack; import java.lang.ClassLoader; /** * JavaEETrainingUtil * @author Pierre-Hugues Charbonneau * */ public class JavaEETrainingUtil { /** * getCurrentClassloaderDetail * @return */ public static String getCurrentClassloaderDetail() { StringBuffer classLoaderDetail = new StringBuffer(); Stack classLoaderStack = new Stack(); ClassLoader currentClassLoader = Thread.currentThread().getContextClassLoader(); classLoaderDetail.append("\n-----------------------------------------------------------------\n"); // Build a Stack of the current ClassLoader chain while (currentClassLoader != null) { classLoaderStack.push(currentClassLoader); currentClassLoader = currentClassLoader.getParent(); } // Print ClassLoader parent chain while(classLoaderStack.size() > 0) { ClassLoader classLoader = classLoaderStack.pop(); // Print current classLoaderDetail.append(classLoader); if (classLoaderStack.size() > 0) { classLoaderDetail.append("\n--- delegation ---\n"); } else { classLoaderDetail.append(" **Current ClassLoader**"); } } classLoaderDetail.append("\n-----------------------------------------------------------------\n"); return classLoaderDetail.toString(); } } Problem reproduction In order to replicate the problem, we will simply “voluntary” omit one of the JAR files from the classpath that contains the referencing Java class ReferencingClassA. The Java program is packaged as per below: - MainProgram.jar (contains NoClassDefFoundErrorSimulator.class and JavaEETrainingUtil.class) - CallerClassA.jar (contains CallerClassA.class) - ReferencingClassA.jar (contains ReferencingClassA.class) Now, let’s run the program as is: ## Baseline (normal execution) .\bin>java -classpath CallerClassA.jar;ReferencingClassA.jar;MainProgram.jar org.ph.javaee.training1.NoClassDefFoundErrorSimulator java.lang.NoClassDefFoundError Simulator - Training 1 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com Current ClassLoader chain: ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Creating a new instance of org.ph.javaee.training1.CallerClassA... Classloading of org.ph.javaee.training1.ReferencingClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Creating a new instance of org.ph.javaee.training1.ReferencingClassA... done! For the initial run (baseline), the main program was able to create a new instance of CallerClassA and execute its method successfully; including successful class loading of the referencing class ReferencingClassA. ## Problem reproduction run (with removal of ReferencingClassA.jar) ../bin>java -classpath CallerClassA.jar;MainProgram.jar org.ph.javaee.training1.NoClassDefFoundErrorSimulator java.lang.NoClassDefFoundError Simulator - Training 1 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com Current ClassLoader chain: ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Creating a new instance of org.ph.javaee.training1.CallerClassA... Exception in thread "main" java.lang.NoClassDefFoundError: org/ph/javaee/training1/ReferencingClassA at org.ph.javaee.training1.CallerClassA.doSomething(CallerClassA.java:25) at org.ph.javaee.training1.NoClassDefFoundErrorSimulator.main(NoClassDefFoundErrorSimulator.java:28) Caused by: java.lang.ClassNotFoundException: org.ph.javaee.training1.ReferencingClassA at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 2 more What happened? The removal of the ReferencingClassA.jar, containing ReferencingClassA, did prevent the current class loader to locate this referencing Java class at runtime leading to ClassNotFoundException and NoClassDefFoundError. This is the typical Exception that you will get if you omit JAR file(s) from your Java start-up classpath or within an EAR / WAR for Java EE related applications. ClassLoader view Now let’s review the ClassLoader chain so you can properly understand this problem case. As you saw from the Java program output logging, the following Java ClassLoaders were found: Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- ** Please note that the Java bootstrap class loader is responsible to load the core JDK classes and is written in native code ** ## sun.misc.Launcher$AppClassLoader This is the system class loader responsible to load our application code found from the Java classpath specified at start-up. ##sun.misc.Launcher$ExtClassLoader This is the extension class loader responsible to load code in the extensions directories (/lib/ext, or any other directory specified by the java.ext.dirs system property). As you can see from the Java program logging output, the extension class loader is the actual super parent of the system class loader. Our sample Java program was loaded at the system class loader level. Please note that this class loader chain is very simple for this problem case since we did not create child class loaders at this point. This will be covered in future articles. Recommendations and resolution strategies Now find below my recommendations and resolution strategies for NoClassDefFoundError problem case 1: - Review the java.lang.NoClassDefFoundError error and identify the missing Java class - Verify and locate the missing Java class from your compile / build environment - Determine if the missing Java class is from your application code, third part API or even the Java EE container itself. Verify where the missing JAR file(s) is / are expected to be found - Once found, verify your runtime environment Java classpath for any typo or missing JAR file(s) - If the problem is triggered from a Java EE application, perform the same above steps but verify the packaging of your EAR / WAR file for missing JAR and other library file dependencies such as MANIFEST Please feel free to post any question or comment. The part 3 will be available shortly.

In this tutorial we'll look at how you can use three.js to render a 3D map of an image using webgl on a canvas element. In this example we'll rasterize an image (make it like an old 8-bit image), and use this rasterized image as input for our 3D model. Each raster element is rendered as a cube using three.js. The height of the cube is defined by the brightness of the raster element. Since an image usually better explains what we're aiming for, lets look at what we're going to create: This article uses a couple of examples from previous articles: It uses the web worker threadpool shown in this article. And it rasterizes the image based on info from here. And brightness is calculated as explained here. You don't really need to dive into those articles to learn about three.js, but if you like some background information, those articles are the places to look at. Now, what are we going to show in this article. Create HTML layout: We'll create a very simple gallery, where you can select the image you want to render. We also need to setup a hidden canvas, we can use for rasterizing. Initialize the three.js scene: We create a simple three.js scene, with a rotating camera. To this scene we'll add a couple of hundred cubes. One for each part of our rasterized image. Add the cubes to the scene: When an image is selected we, rasterize the image (in a number of background web worker threads) and based on the brightness add a cube at a specific position to the three.js scene. Create HTML layout The HTML is very simple. We just got a couple of divs, include some javascript libraries and style some of the elements. The complete html is shown here: As you can see from this HTML, everything we do here is rather straightforward. We define a hidden div that we use for rasterizing, a div that contains a couple of images for our gallery and finally a div that is going to be used to render the result in. We also use a simple JQuery $(document) and $("#f3") to make sure the document is ready and the image is loaded before we start rendering. Once the first image is loaded, we pass that to renderImage function. This function will rasterize the image and show the output in the webglcontainer div. Initialize the three.js scene Before the image can be rendered we first need to correctly setup the scene for three.js. We do this in thie init method. // some global variables var camera, scene, renderer; var elements = []; // some default values var bulletSize = 10; var offset = 300; var defPos = 800; // Initialize the scene and threadpool function init() { // create a queuepool with 6 queues queuepool = new Pool(3); queuepool.init(); // create a scene and a camera scene = new THREE.Scene(); //Three.PerspectiveCamera() camera = new THREE.PerspectiveCamera( 55, 1, 0.1, 10000, -2000, 10000 ); // position the camera camera.position.y = defPos+200; camera.position.z = defPos; camera.position.x = defPos; // and add to the scene scene.add(camera); // setup the renderer and attach to canvas renderer = new THREE.WebGLRenderer(); renderer.setSize( 600, 600 ); $("#webglcontainer").append(renderer.domElement); animate(); } In this method we first create a queuepool, this queuepool is used to run jobs. For this example we use jobs to calculate the most dominant color of a specific part of an image (more info can be found in this article). Next we create the scene, the camera and add the camera to the scene. Finally, in this code fragment, we create the renderer for this scene and append it to the canvas. Now that the scene is created we can render it. This is done in the animate function: // the animation loop. This rotates the camera around the central point. function animate() { var timer = Date.now() * 0.0008; camera.position.x = (Math.cos( timer ) * defPos); camera.position.z = (Math.sin( timer ) * defPos) ; camera.lookAt( scene.position ); renderer.render( scene, camera ); requestAnimationFrame( animate ); } This animate function uses the requestAnimationFrame functionality to get a callback when the animation needs to be updated. We supply the animate method itself as it's callback, so the animation will keep on running. In this animate function we also rotate the camera around the scene. For this we alter the X and the Z position of the camera, while keeping it focussed on our scene. Without going into the math behind this, if you alternate the X-pos using a Math.cos(t) and the Z-pos simultaniously using Math.sin(t) your camera will smoothly rotate around the scene. Now we can call the render operation on the renderer to render the scene. Add the cubes to the scene What is left is adding the cubes to the scene. We do this in the following function called addElement // add a cube to the grid. The cube is positioned base on the x,y values. The color // is used to define the material, and the luminance is used for the height of the element. function addElement(x,y, color, lumin) { var voxelPosition2 = voxelPosition = new THREE.Vector3(); voxelPosition2.x = bulletSize*x -offset ; voxelPosition2.z = bulletSize*y -offset ; voxelPosition2.y = 200 + ((lumin/(255))*200)/2; var geometry = new THREE.CubeGeometry( bulletSize, (lumin/(255))*200, bulletSize ); var mat = new THREE.MeshBasicMaterial( { color: color, shading: THREE.NoShading, wireframe: false, transparent: false }) var cube = new THREE.Mesh(geometry,mat); cube.position=voxelPosition2; // add to elements list and to scene elements.push(cube); scene.add(cube); } This function takes as parameters the position of the element, the color in which we need to render the element and the luminance of the element. Based on this information we determine the position where we need to render the cube, we create a cube whose height is based on the luminance, and make a material for this cube based on the most dominant color. With all these parts we can add the cube at the correct position to the scene. And since we already started the animate function, the scene will be updated continuously. One thing missing we haven't talked about is what the renderImage operation looks like that we call from the HTML page whenever you click on an image. This function is shown here: function callback(event) { var wp = event.data; // get the colors var colors = wp.result; var color = "0x" + ("0" + parseInt(colors[0][0],10).toString(16)).slice(-2) + ("0" + parseInt(colors[0][1],10).toString(16)).slice(-2) + ("0" + parseInt(colors[0][2],10).toString(16)).slice(-2); var lumin = colors[0][0] * .3 + colors[0][1] * .59 + colors[0][2] * .11; addElement(wp.x,wp.y, color, lumin); } What we do here, is that we clear the queue and stop any running tasks (this isn't perfect at the moment, so you might see some cubes from the previous image). Next we remove all the current cubes from the scene and finally we rasterize the selected image. In this rasterize function we split the image in parts. I won't show the details for rasterizing here, but I'll show the callback that is called after determining the dominant color (see here for more info on the rasterizing part). The event we receive here contains information about what the dominant color of a specific part of the image is. We convert this color to the format used by three.js and calculate the luminance of this color. all this information is passed to the addElement function we saw earlier and it is added to the scene. That's it. If you run this, you'll slowly see the scene being filled by colored cubes with different heights like this: This example was tested using the latest chrome build and the latest firefox beta. I noticed that chrome, even though it was quicker, sometimes crashed, but both browsers should be able to render this.

There are several code analysis tools for Python. The most well known is pylint. Then there’s pychecker and now we’re moving on to pyflakes. The pyflakes project is a part of something known as the Divmod Project. Pyflakes doesn’t actually execute the code it checks, unlike pychecker. Of course, pylint also doesn’t execute the code. Regardless, we’ll take a quick look at it and see how pyflakes works and if it’s better than the competition. Getting Started As you have probably guessed, pyflakes is not a part of the Python distribution. You will need to download it from PyPI or from the project’s launchpad page. Once you have it installed, you can run it against some of your own code. Or you can follow along and see how it works with our test script. Running pyflakes We’ll be using a super simple and pretty silly example script. In fact, it’s the same one we used for the pylint and pychecker articles. Here it is again for your viewing pleasure: import sys ######################################################################## class CarClass: """""" #---------------------------------------------------------------------- def __init__(self, color, make, model, year): """Constructor""" self.color = color self.make = make self.model = model self.year = year if "Windows" in platform.platform(): print "You're using Windows!" self.weight = self.getWeight(1, 2, 3) #---------------------------------------------------------------------- def getWeight(this): """""" return "2000 lbs" As was noted in the other articles, this dumb code has 4 issues, 3 of which would stop the programming from running. Let’s see what pyflakes can find! Try running the following command and you’ll see the following output: C:\Users\mdriscoll\Desktop>pyflakes crummy_code.py crummy_code.py:1: 'sys' imported but unused crummy_code.py:15: undefined name 'platform' While pyflakes was super fast at returning this output, it didn’t find all the errors. The getWeight method call is passing too many arguments and getWeight method itself is defined incorrectly as it doesn’t have a “self” argument. If you fixed your code according to what pyflakes told you, you’re code still wouldn’t work. Wrapping Up The pyflakes website claims that pyflakes is faster than pychecker and pylint. I didn’t test this, but anyone who wants to can do so pretty easily by just running it against some big files. Maybe grab the BeautifulSoup file or run it (and the others) against something complex like PySide or SQLAlchemy and see how they compare. I personally am disappointed that it didn’t catch all the issues I was looking for. I think for my purposes, I’ll be sticking with pylint. This might be a handy tool for a quick and dirty test or just to make you feel better after a particularly poor result from a pylint scan.

Here are the 10 important differences between WCF Services and ASP.NET Web Services.

The model contains the business logic and interact with the persistance storage to store, retrive and manipulate data. The view is responsible for dispalying the results back to the user. In Struts the view layer is implemented using JSP. The controller handles all the request from the user and selects the appropriate view to return. In Sruts the controller's job is done by the ActionServlet. The following events happen when the Client browser issues an HTTP request. The ActionServlet receives the request. The struts-config.xml file contains the details regarding the Actions, ActionForms, ActionMappings and ActionForwards. During the startup the ActionServelet reads the struts-config.xml file and creates a database of configuration objects. Later while processing the request the ActionServlet makes decision by refering to this object. When the ActionServlet receives the request it does the following tasks. Bundles all the request values into a JavaBean class which extends Struts ActionForm class. Decides which action class to invoke to process the request. Validate the data entered by the user. The action class process the request with the help of the model component. The model interacts with the database and process the request. After completing the request processing the Action class returns an ActionForward to the controller. Based on the ActionForward the controller will invoke the appropriate view. The HTTP response is rendered back to the user by the view component.

Let’s assume that you’re writing code that’d track the price of mobile phones. Now, let’s say you have a collection of objects representing different Mobile phone vendors (MobileVendor), and each vendor has a collection of objects representing the PhoneModels they offer. To put it simple, there’s exists a one-to-many relationship between MobileVendor:PhoneModel. MobileVendor Class Class MobileVendor{ long vendor_id; PhoneModel[] phoneModels; ... } Okay, so you want to print out all the details of phone models. A naive O/R implementation would SELECT all mobile vendors and then do N additional SELECTs for getting the information of PhoneModel for each vendor. -- Get all Mobile Vendors SELECT * FROM MobileVendor; -- For each MobileVendor, get PhoneModel details SELECT * FROM PhoneModel WHERE MobileVendor.vendorId=? As you see, the N+1 problem can happen if the first query populates the primary object and the second query populates all the child objects for each of the unique primary objects returned. Resolve N+1 SELECTs problem (i) HQL fetch join "from MobileVendor mobileVendor join fetch mobileVendor.phoneModel PhoneModels" Corresponding SQL would be (assuming tables as follows: t_mobile_vendor for MobileVendor and t_phone_model for PhoneModel) SELECT * FROM t_mobile_vendor vendor LEFT OUTER JOIN t_phone_model model ON model.vendor_id=vendor.vendor_id (ii) Criteria query Criteria criteria = session.createCriteria(MobileVendor.class); criteria.setFetchMode("phoneModels", FetchMode.EAGER); In both cases, our query returns a list of MobileVendor objects with the phoneModels initialized. Only one query needs to be run to return all the PhoneModel and MobileVendor information required.

Inserting into Binary Search Tree - C# public class BinaryTreeNode { public BinaryTreeNode Left { get; set; } public BinaryTreeNode Right { get; set; } public int Data { get; set; } public BinaryTreeNode(int data) { this.Data = data; } } public void InsertIntoBST(BinaryTreeNode root, int data) { BinaryTreeNode _newNode = new BinaryTreeNode(data); BinaryTreeNode _current = root; BinaryTreeNode _previous = _current; while (_current != null) { if (data < _current.Data) { _previous = _current; _current = _current.Left; } else if (data > _current.Data) { _previous = _current; _current = _current.Right; } } if (data < _previous.Data) _previous.Left = _newNode; else _previous.Right = _newNode; }

This is interesting stuff; Jonas Bonér organized some general some latency data by Peter Norvig as a Gist, and others expanded on it. What's interesting is how, scaling time up by a billion, converts a CPU instruction cycle into approximately one heartbeat, and yields a disk seek time of "a semester in university". ### Latency numbers every programmer should know L1 cache reference ......................... 0.5 ns Branch mispredict ............................ 5 ns L2 cache reference ........................... 7 ns Mutex lock/unlock ........................... 25 ns Main memory reference ...................... 100 ns Compress 1K bytes with Zippy ............. 3,000 ns = 3 µs Send 2K bytes over 1 Gbps network ....... 20,000 ns = 20 µs SSD random read ........................ 150,000 ns = 150 µs Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs Round trip within same datacenter ...... 500,000 ns = 0.5 ms Read 1 MB sequentially from SSD* ..... 1,000,000 ns = 1 ms Disk seek ........................... 10,000,000 ns = 10 ms Read 1 MB sequentially from disk .... 20,000,000 ns = 20 ms Send packet CA->Netherlands->CA .... 150,000,000 ns = 150 ms Assuming ~1GB/sec SSD  Visual chart provided by [ayshen](https://gist.github.com/ayshen) Data by [Jeff Dean](http://research.google.com/people/jeff/) Originally by [Peter Norvig](http://norvig.com/21-days.html#answers) Lets multiply all these durations by a billion: Magnitudes: ### Minute: L1 cache reference 0.5 s One heart beat (0.5 s) Branch mispredict 5 s Yawn L2 cache reference 7 s Long yawn Mutex lock/unlock 25 s Making a coffee ### Hour: Main memory reference 100 s Brushing your teeth Compress 1K bytes with Zippy 50 min One episode of a TV show (including ad breaks) ### Day: Send 2K bytes over 1 Gbps network 5.5 hr From lunch to end of work day ### Week SSD random read 1.7 days A normal weekend Read 1 MB sequentially from memory 2.9 days A long weekend Round trip within same datacenter 5.8 days A medium vacation Read 1 MB sequentially from SSD 11.6 days Waiting for almost 2 weeks for a delivery ### Year Disk seek 16.5 weeks A semester in university Read 1 MB sequentially from disk 7.8 months Almost producing a new human being The above 2 together 1 year ### Decade Send packet CA->Netherlands->CA 4.8 years Average time it takes to complete a bachelor's degree

Today we’ll spend some time looking at three different ways to make Python submit a web form. In this case, we will be doing a web search with duckduckgo.com searching on the term “python” and saving the result as an HTML file. We will use Python’s included urllib modules and two 3rd party packages: requests and mechanize. We have three small scripts to cover, so let’s get cracking! Submitting a web form with urllib We will start with urllib and urllib2 since they are included in Python’s standard library. We’ll also import the webbrowser to open the search results for viewing. Here’s the code: import urllib import urllib2 import webbrowser url = "http://duckduckgo.com/html" data = urllib.urlencode({'q': 'Python'}) results = urllib2.urlopen(url, data) with open("results.html", "w") as f: f.write(results.read()) webbrowser.open("results.html") The first thing you have to do when you want to submit a web form is figure out what the form is called and what the url is that you will be posting to. If you go to duckduckgo’s website and view the source, you’ll notice that its action is pointing to a relative link, “/html”. So our url is “http://duckduckgo.com/html”. The input field is named “q”, so to pass duckduckgo a search term, we have to pass it to the “q” field. This is where the urllib.urlencode line comes in. It encodes our search term correctly and then we open the url and search. The results are read and written to disk. Finally, we open our saved results using the webbrowser module. Now let’s find out how this process differs when using the requests package. Submitting a web form with requests The requests package does form submissions a little bit more elegantly. Let’s take a look: import requests url = "http://duckduckgo.com/html" payload = {'q':'python'} r = requests.post(url, payload) with open("requests_results.html", "w") as f: f.write(r.content) With requests, you just need to create a dictionary with the field name as the key and the search term as the value. Then you use requests.post to do the search. Finally you use the resulting requests object, “r”, and access its content property which you save to disk. We skipped the webbrowser part in this example (and the next) for brevity. Now we should be ready to see how mechanize does its thing. Submitting a web form with mechanize The mechanize module has lots of fun features for browsing the internet with Python. Sadly it doesn’t support javascript. Anyway, let’s get on with the show! import mechanize url = "http://duckduckgo.com/html" br = mechanize.Browser() br.set_handle_robots(False) # ignore robots br.open(url) br.select_form(name="x") br["q"] = "python" res = br.submit() content = res.read() with open("mechanize_results.html", "w") as f: f.write(content) As you can see, mechanize is a little more verbose than the other two methods were. We also need to tell it to ignore the robots.txt directive or it will fail. Of course, if you want to be a good netizen, then you shouldn’t ignore it. Anyway, to start off, you need a Browser object. Then you open the url, select the form (in this case, “x”) and set up a dictionary with the search parameters as before. Note that in each method, the dict setup is a little different. Next you submit the query and read the result. Finally you save the result to disk and you’re done! Wrapping Up Of the three, requests was probably the simplest with urllib being a close second. Mechanize is made for doing a lot more then the other two though. It’s made for screen scraping and website testing, so it’s no surprise it’s a little more verbose. You can also do form submission with selenium, but you can read about that in this blog’s archives. I hope you found this article interesting and perhaps inspiring. See you next time!

Parking Lot Design Implementation in C#. public class ParkingLot { List _parkingSpots; public ParkingLot() { _parkingSpots = new List(); // 10 parking spots; 2 free; 6 Regular Paid; 2 Handicapped Free //2 free for (int i = 0; i p.IsFree == true && p.IsAvailable); } public ParkingSpot FindPaidSpot() { return this._parkingSpots.Find(p => p.IsFree == false && p.IsAvailable); } public int GetAvailableSpotsCount() { return this._parkingSpots.Count(p => p.IsAvailable); } public int GetTotalSpots() { return this._parkingSpots.Count; } } public class ParkingSpot { public bool IsAvailable { get; set; } public bool IsFree { get; set; } public IVehicle ParkedVehicle { get; set; } public ParkingMeter Meter { get; set; } public void Park(IVehicle vehicle) { if (this.ParkedVehicle == null) { this.ParkedVehicle = vehicle; this.IsAvailable = false; } else { throw new Exception("Parking Spot is Taken. Cannot Park here!"); } } public ParkingSpot() { this.IsAvailable = true; this.IsFree = true; } public ParkingSpot(bool isFree) { this.IsAvailable = true; this.IsFree = isFree; if (!this.IsFree) { this.Meter = new ParkingMeter(); } } } public class HandicappedParkingSpot : ParkingSpot { } public class ParkingMeter { public DateTime EndTime { get; set; } public int MinutesRemaining { get { if (DateTime.Now >= EndTime) return 0; else return (EndTime - DateTime.Now).Minutes; } } public int ParkingIntervalMins { get { return 1; } } public void Pay(int quarters) { EndTime = DateTime.Now.AddMinutes(quarters * ParkingIntervalMins); } } public interface IVehicle { string Make { get; set; } string Model { get; set; } void Drive(); } public class Car : IVehicle { public string Make { get; set; } public string Model { get; set; } public void Drive() { } public void Park() { } } public class Truck : IVehicle { public string Make { get; set; } public string Model { get; set; } public void Drive() { } public void Park() { } }

One of the advantages of code generation is the ability to see how a specific language feature or framework is used. As I discussed in the post NetBeans 7.2 beta: Faster and More Helpful, NetBeans 7.2 beta provides TestNG integration. I did not elaborate further in that post other than a single reference to that feature because I wanted to devote this post to the subject. I use this post to demonstrate how NetBeans 7.2 can be used to help a developer new to TestNG start using this alternative (to JUnit) test framework. NetBeans 7.2's New File wizard makes it easier to create an empty TestNG test case. This is demonstrated in the following screen snapshots that are kicked off by using New File | Unit Tests (note that "New File" is available under the "File" drop-down menu or by right-clicking in the Projects window). Running the TestNG test case creation as shown above leads to the following generated test code. TestNGDemo.java (Generated by NetBeans 7.2) package dustin.examples; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeClass; import org.testng.annotations.Test; import org.testng.Assert; /** * * @author Dustin */ public class TestNGDemo { public TestNGDemo() { } @BeforeClass public void setUpClass() { } @AfterClass public void tearDownClass() { } @BeforeMethod public void setUp() { } @AfterMethod public void tearDown() { } // TODO add test methods here. // The methods must be annotated with annotation @Test. For example: // // @Test // public void hello() {} } The test generated by NetBeans 7.2 includes comments indicate how test methods are added and annotated (similar to modern versions of JUnit). The generated code also shows some annotations for overall test case set up and tear down and for per-test set up and tear down (annotations are similar to JUnit's). NetBeans identifies import statements that are not yet used at this point (import org.testng.annotations.Test; and import org.testng.Assert;), but are likely to be used and so have been included in the generated code. I can add a test method easily to this generated test case. The following code snippet is a test method using TestNG. testIntegerArithmeticMultiplyIntegers() @Test public void testIntegerArithmeticMultiplyIntegers() { final IntegerArithmetic instance = new IntegerArithmetic(); final int[] integers = {4, 5, 6}; final int expectedProduct = 2 * 3 * 4 * 5 * 6; final int product = instance.multiplyIntegers(2, 3, integers); assertEquals(product, expectedProduct); } This, of course, looks very similar to the JUnit equivalent I used against the same IntegerArithmetic class that I used for testing illustrations in the posts Improving On assertEquals with JUnit and Hamcrest and JUnit's Built-in Hamcrest Core Matcher Support. The following screen snapshot shows the output in NetBeans 7.2 beta from right-clicking on the test case class and selecting "Run File" (Shift+F6). The text output of the TestNG run provided in the NetBeans 7.2 beta is reproduced next. [TestNG] Running: Command line suite [VerboseTestNG] RUNNING: Suite: "Command line test" containing "1" Tests (config: null) [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @BeforeClass dustin.examples.TestNGDemo.setUpClass() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @BeforeClass dustin.examples.TestNGDemo.setUpClass() finished in 33 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @BeforeMethod dustin.examples.TestNGDemo.setUp() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @BeforeMethod dustin.examples.TestNGDemo.setUp() finished in 2 ms [VerboseTestNG] INVOKING: "Command line test" - dustin.examples.TestNGDemo.testIntegerArithmeticMultiplyIntegers() [VerboseTestNG] PASSED: "Command line test" - dustin.examples.TestNGDemo.testIntegerArithmeticMultiplyIntegers() finished in 12 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @AfterMethod dustin.examples.TestNGDemo.tearDown() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @AfterMethod dustin.examples.TestNGDemo.tearDown() finished in 1 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @AfterClass dustin.examples.TestNGDemo.tearDownClass() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @AfterClass dustin.examples.TestNGDemo.tearDownClass() finished in 1 ms [VerboseTestNG] [VerboseTestNG] =============================================== [VerboseTestNG] Command line test [VerboseTestNG] Tests run: 1, Failures: 0, Skips: 0 [VerboseTestNG] =============================================== =============================================== Command line suite Total tests run: 1, Failures: 0, Skips: 0 =============================================== Deleting directory C:\Users\Dustin\AppData\Local\Temp\dustin.examples.TestNGDemo test: BUILD SUCCESSFUL (total time: 2 seconds) The above example shows how easy it is to start using TestNG, especially if one is moving to TestNG from JUnit and is using NetBeans 7.2 beta. Of course, there is much more to TestNG than this, but learning a new framework is typically most difficult at the very beginning and NetBeans 7.2 gets one off to a fast start.

This article is part 4 of our Thread Dump analysis series which will provide you with an overview of what is a JVM Thread Dump for the IBM VM and the different Threads and data points that you will find. As you will see and learn, the IBM VM Thread Dump format is different but provides even more out-of-the-box troubleshooting data. At this point, you should know how Threads interact with the Java EE container and what a Thread Dump is. Before we go any further in the deep dive analysis patterns, you also need to understand the IBM VM Thread Dump format since this is the typical Thread Dump data to expect when using IBM WAS on IBM VM. IBM VM Thread Dump breakdown overview In order for you to better understand, find below a diagram showing you a visual breakdown of an IBM 1.6 VM Thread Dump and its common data points found: As you can, there are extra runtime data that you will not find from a HotSpot VM Thread Dump. Please keep in mind that you may not need to review all these data points but you still need to understand what data is available depending of your problem case. The rest of the article will cover each Thread Dump portion in more detail. # Thread Dump generation event The first portion provides you with detail on how this Thread Dump was generated. IBM Thread Dump can be generated as a result of a “signal 3” or “user” e.g. kill -3 or automatically as a result of severe JVM conditions such as an OutOfMemoryError. 0SECTION TITLE subcomponent dump routine NULL =============================== 1TISIGINFO Dump Event "user" (00004000) received 1TIDATETIME Date: 2012/03/12 at 20:52:13 1TIFILENAME Javacore filename: /apps/wl11g/domains/app/javacore.20120312.205205.1949928.0004.txt 1TIREQFLAGS Request Flags: 0x81 (exclusive+preempt) 1TIPREPSTATE Prep State: 0x4 (exclusive_vm_access) 0SECTION TITLE subcomponent dump routine NULL =============================== 1TISIGINFO OUTOFMEMORY received 1TIDATETIME Date: 2012/06/01 at 09:52:12 1TIFILENAME Javacore filename: /usr/WebSphere/AppServer/javacore311328.1338524532.txt # HW and OS environment detail The next section provides you with some detail on the current hardware and OS that this IBM VM is running from: 0SECTION GPINFO subcomponent dump routine NULL ================================ 2XHOSLEVEL OS Level : AIX 5.3 2XHCPUS Processors - 3XHCPUARCH Architecture : ppc64 3XHNUMCPUS How Many : 6 3XHNUMASUP NUMA is either not supported or has been disabled by user # JRE detail and Java start-up arguments This section is very useful as it provides you with a full view on your JRE major version and patch level along with all JVM start-up arguments. 0SECTION ENVINFO subcomponent dump routine NULL ================================= 1CIJAVAVERSION JRE 1.6.0 IBM J9 2.4 AIX ppc64-64 build jvmap6460sr9-20101124_69295 1CIVMVERSION VM build 20101124_069295 1CIJITVERSION JIT enabled, AOT enabled - r9_20101028_17488ifx2 1CIGCVERSION GC - 20101027_AA 1CIRUNNINGAS Running as a standalone JVM ………………………………………………………………………………………… # User and environment variables This section provides you with a listing of current user and environment variables such as File Descriptor limit. 1CIUSERLIMITS User Limits (in bytes except for NOFILE and NPROC) NULL ------------------------------------------------------------------------ NULL type soft limit hard limit 2CIUSERLIMIT RLIMIT_AS unlimited unlimited 2CIUSERLIMIT RLIMIT_CORE 1073741312 unlimited 2CIUSERLIMIT RLIMIT_CPU unlimited unlimited 2CIUSERLIMIT RLIMIT_DATA unlimited unlimited 2CIUSERLIMIT RLIMIT_FSIZE unlimited unlimited 2CIUSERLIMIT RLIMIT_NOFILE 4096 4096 2CIUSERLIMIT RLIMIT_RSS 33554432 unlimited 2CIUSERLIMIT RLIMIT_STACK 33554432 4294967296 # Java Heap detail and GC history Similar to HotSpot VM 1.6+, IBM VM Thread Dump also contains information on the Java Heap capacity and utilization along with memory segments allocated for each memory space of the Java process. Please keep in mind that deeper Java Heap analysis will require you to analyze the Heap Dump binary snapshot as per below tutorial. http://javaeesupportpatterns.blogspot.com/2011/02/ibm-sdk-heap-dump-httpsession-footprint.htm Finally, a history of the garbage collection process is also present. 0SECTION MEMINFO subcomponent dump routine NULL ================================= 1STHEAPFREE Bytes of Heap Space Free: 51104BC8 1STHEAPALLOC Bytes of Heap Space Allocated: 80000000 1STSEGTYPE Internal Memory ………………………………………………………………………………………… 1STSEGTYPE Object Memory ………………………………………………………………………………………… 1STSEGTYPE Class Memory ………………………………………………………………………………………… 1STSEGTYPE JIT Code Cache ………………………………………………………………………………………… 1STSEGTYPE JIT Data Cache ………………………………………………………………………………………… STGCHTYPE GC History 3STHSTTYPE 00:52:07:523048405 GMT j9mm.51 - SystemGC end: newspace=466136480/483183616 oldspace=899251600/1610612736 loa=80530432/80530432 3STHSTTYPE 00:52:07:523046694 GMT j9mm.139 - Reference count end: weak=40149 soft=87504 phantom=33 threshold=17 maxThreshold=32 3STHSTTYPE 00:52:07:522164027 GMT j9mm.91 - GlobalGC end: workstackoverflow=0 overflowcount=0 weakrefs=40149 soft=87504 threshold=17 phantom=33 finalizers=4947 newspace=466136480/483183616 oldspace=899251600/1610612736 loa=80530432/80530432 3STHSTTYPE 00:52:07:522152764 GMT j9mm.90 - GlobalGC collect complete # Java and JVM object monitor lock and deadlock detail This Thread Dump portion is very important. Quite often Thread problems involve Threads waiting between each other due to locks on particular Object monitors e.g. Thread B waiting to acquire a lock on Object monitor held by Thread A. Deadlock conditions can also be triggered from time to time; especially for non-Thread safe implementations. The IBM VM Thread Dump provides a separate section where you can analyze lock(s) held by each Thread including waiting chain(s) e.g. Many Threads waiting to acquire the same Object monitor lock. 0SECTION LOCKS subcomponent dump routine NULL =============================== NULL 1LKPOOLINFO Monitor pool info: 2LKPOOLTOTAL Current total number of monitors: 1034 NULL 1LKMONPOOLDUMP Monitor Pool Dump (flat & inflated object-monitors): 2LKMONINUSE sys_mon_t:0x0000000115B53060 infl_mon_t: 0x0000000115B530A0: 3LKMONOBJECT java/util/Timer$TimerImpl@0x0700000000C92AA0/0x0700000000C92AB8: 3LKNOTIFYQ Waiting to be notified: 3LKWAITNOTIFY "Thread-7" (0x0000000114CAB400) ………………………………………………………………………… ## Threads waiting chain 2LKMONINUSE sys_mon_t:0x000000012462FE00 infl_mon_t: 0x000000012462FE40: 3LKMONOBJECT com/inc/server/app/Request@0x07000000142ADF30/0x07000000142ADF48: owner "Thread-30" (0x000000012537F300), entry count 1 3LKNOTIFYQ Waiting to be notified: 3LKWAITNOTIFY "Thread-26" (0x0000000125221F00) 3LKWAITNOTIFY "Thread-27" (0x0000000125252000) 3LKWAITNOTIFY "Thread-28" (0x000000012527B800) 3LKWAITNOTIFY "Thread-29" (0x00000001252DDA00) 3LKWAITNOTIFY "Thread-31" (0x0000000125386200) 3LKWAITNOTIFY "Thread-32" (0x0000000125423600) 3LKWAITNOTIFY "Thread-33" (0x000000012548C500) 3LKWAITNOTIFY "Thread-34" (0x00000001255D6000) 3LKWAITNOTIFY "Thread-35" (0x00000001255F7900) ………………………………………………………………………… # Java EE middleware, third party & custom application Threads Similar to the HotSpot VM Thread Dump format, this portion is the core of the Thread Dump and where you will typically spend most of your analysis time. The number of Threads found will depend on your middleware software that you use, third party libraries (that might have its own Threads) and your application (if creating any custom Thread, which is generally not a best practice). The following Thread in the example below is in BLOCK state which typically means it is waiting to acquire a lock on an Object monitor. You will need to search in the earlier section and determine which Thread is holding the lock so you can pinpoint the root cause. 3XMTHREADINFO "[STUCK] ExecuteThread: '162' for queue: 'weblogic.kernel.Default (self-tuning)'" J9VMThread:0x000000013ACF0800, j9thread_t:0x000000013AC88B20, java/lang/Thread:0x070000001F945798, state:B, prio=1 3XMTHREADINFO1 (native thread ID:0x1AD0F3, native priority:0x1, native policy:UNKNOWN) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at org/springframework/jms/connection/SingleConnectionFactory.createConnection(SingleConnectionFactory.java:207(Compiled Code)) 4XESTACKTRACE at org/springframework/jms/connection/SingleConnectionFactory.createQueueConnection(SingleConnectionFactory.java:222(Compiled Code)) 4XESTACKTRACE at org/springframework/jms/core/JmsTemplate102.createConnection(JmsTemplate102.java:169(Compiled Code)) 4XESTACKTRACE at org/springframework/jms/core/JmsTemplate.execute(JmsTemplate.java:418(Compiled Code)) 4XESTACKTRACE at org/springframework/jms/core/JmsTemplate.send(JmsTemplate.java:475(Compiled Code)) 4XESTACKTRACE at org/springframework/jms/core/JmsTemplate.send(JmsTemplate.java:467(Compiled Code)) ………………………………………………………………………………………………………… # JVM class loader summary Finally, the last section of the IBM VM Thread Dump provides you with a detailed class loader summary. This is very crucial data when dealing with Class Loader related issues and leaks. You will find the number and type of loaded Classes for each active Class loader in the running JVM. I suggest that you review the following case study for a complete tutorial on how to pinpoint root cause for this type of issues when using IBM VM. http://javaeesupportpatterns.blogspot.com/2011/04/class-loader-memory-leak-debugging.html 0SECTION CLASSES subcomponent dump routine NULL ================================= 1CLTEXTCLLOS Classloader summaries 1CLTEXTCLLSS 12345678: 1=primordial,2=extension,3=shareable,4=middleware,5=system,6=trusted,7=application,8=delegating 2CLTEXTCLLOADER p---st-- Loader *System*(0x0700000000878898) 3CLNMBRLOADEDLIB Number of loaded libraries 6 3CLNMBRLOADEDCL Number of loaded classes 3721 2CLTEXTCLLOADER -x--st-- Loader sun/misc/Launcher$ExtClassLoader(0x0700000000AE8F40), Parent *none*(0x0000000000000000) 3CLNMBRLOADEDLIB Number of loaded libraries 0 3CLNMBRLOADEDCL Number of loaded classes 91 2CLTEXTCLLOADER -----ta- Loader sun/misc/Launcher$AppClassLoader(0x07000000008786D0), Parent sun/misc/Launcher$ExtClassLoader(0x0700000000AE8F40) 3CLNMBRLOADEDLIB Number of loaded libraries 3 3CLNMBRLOADEDCL Number of loaded classes 15178 …………………………………………………………………………………………… I hope this article has helped to understand the basic view of an IBM VM Thread Dump. The next article (part 5) will provide you with a tutorial on how to analyze a JVM Thread Dump via a step by step tutorial and technique I have used over the last 10 years. Please feel free to post any comment and question.

Visual Studio 11 Consumer Preview enables creating Metro styled applications using C++ which is great news for any native developer. Let’s create a simple Metro application. Start up Visual Studio and create new project using the Visual C++>Windows Metro Style>Blank Application template. “Hello world” Open the BlankPage.xaml file and start poking around with XAML. For those who haven’t used it yet, it is similar to HTML development. Scroll the code down until you find the Grid element and insert a TextBlock inside it, the final result should be: The Margin property offsets the element from the default top left corner for 12 pixels from the left and 20 pixels from the top. I have used the style that comes native with all Metro applications and it is primarily used to increase the size of the text. If you want, you can set size via the FontSize attribute. If you run the application now, you should get a black screen with the familiar “Hello world” text in the upper left corner. You define how the UI will look like in the BlankPage.xaml file, but the logic is placed in the regular BlankPage.xaml.h and BlankPage.xaml.cpp which are placed as child items for the BlankPage.xaml file in Solution Explorer. This is a regular C++ class but something looks different. The code does not look native at all with all those strange hats around the code and ref new syntax. Although these are reminiscent of the C++/CLI syntax, these extensions are called C++/CX, which is short for Component extensions. Pure C++ is not used for developing Metro applications and using the pure WinRT would be cumbersome since everything in Metro world is actually a COM object. C++/CX extensions are supposed to cut the burden for C++ programmer with this nonstandard extension. It litters the code with AddRef and Release calls and essentially all hat pointers are nothing more than shared_ptr variant. C++ to XAML You can “name” the previously added TextBlock XAML element by adding the x:Name="txtHello" attribute and value. By doing this, you can refer to it from the code behind files using the following code (add it to the BlankPage::OnNavigatedTo method): txtHello->Text = "Hello world!!!"; Run application and you should see different text like on the image below. Although not very exciting and super-simple, think about how it is done in Win32, MFC or WTL, this is clearly easier. In the next post we will look at how to handle simple events.

A couple years ago I started a series of articles on XML parsing. I covered lxml’s etree and Python builtin minidom XML parsing library. For whatever reason I didn’t notice lxml’s objectify sub-package, but I saw it recently and decided to check it out. In my mind, the objectify module seems to be even more “Pythonic” than etree is. Let’s take some time and go over my old XML examples using objectify and see how it’s different! Let’s Get This Party Started! If you haven’t already, go out and download lxml or you won’t be able to follow along very well. Once you have it, we can continue. We’ll be using the following piece of XML for our parsing pleasure: 1181251680 040000008200E000 1181572063 1800 Bring pizza home 1234360800 1800 Check MS Office website for updates 604f4792-eb89-478b-a14f-dd34d3cc6c21-1234360800 dismissed Now we need to write some code that can parse and modify the XML. Let’s take a look at this little demo that shows a bunch of the neat abilities that objectify provides. from lxml import etree, objectify #---------------------------------------------------------------------- def parseXML(xmlFile): """""" with open(xmlFile) as f: xml = f.read() root = objectify.fromstring(xml) # returns attributes in element node as dict attrib = root.attrib # how to extract element data begin = root.appointment.begin uid = root.appointment.uid # loop over elements and print their tags and text for e in root.appointment.iterchildren(): print "%s => %s" % (e.tag, e.text) # how to change an element's text root.appointment.begin = "something else" print root.appointment.begin # how to add a new element root.appointment.new_element = "new data" # print the xml obj_xml = etree.tostring(root, pretty_print=True) print obj_xml # remove the py:pytype stuff #objectify.deannotate(root) etree.cleanup_namespaces(root) obj_xml = etree.tostring(root, pretty_print=True) print obj_xml # save your xml with open("new.xml", "w") as f: f.write(obj_xml) #---------------------------------------------------------------------- if __name__ == "__main__": f = r'path\to\sample.xml' parseXML(f) The code is pretty well commented, but we’ll spend a little time going over it anyway. First we pass it our sample XML file and objectify it. If you want to get access to a tag’s attributes, use the attrib property. It will return a dictionary of the attribute’s of the tag. To get to sub-tag elements, you just use dot notation. As you can see, to get to the begin tag’s value, we can just do something like this: begin = root.appointment.begin If you need to iterate over the children elements, you can use iterchildren. You may have to use a nested for loop structure to get everything. Changing an element’s value is as simple as just assigning it a new value. And if you need to create a new element, just add a period and the name of the new element (see below): root.appointment.new_element = "new data" When we add or change items using objectify, it will add some annotations to the XML, such as xmlns:py="http://codespeak.net/lxml/objectify/pytype" py:pytype="str". You may not want that included, so you'll have to call the following method to remove that stuff: [python] etree.cleanup_namespaces(root) You can also use “objectify.deannotate(root)” to do some deannotation chores, but I wasn’t able to get it to work for this example. To save the new XML, you actually seem to need lxml’s etree module to convert it to a string so you can save it. At this point, you should be able to parse most XML documents and edit them effectively with lxml’s objectify. I thought it was very intuitive and easy to pick up. Hopefully you will find it useful in your endeavors as well.

flexible programs focus on polymorphism and not inheritance . some languages focus on static type checking ( c++ , java , c# ) which links the concepts and reduces polymorphic opportunities . languages that separate the concepts can allow you to focus on polymorphism and create more robust code. javascript, python, ruby, and vb.net do not have typed variables and defer type checking to runtime. is the value of static type checking worth giving up the power of pure polymorphism at runtime? inheritance and polymorphism are independent but related entities – it is possible to have one without the other. if we use a language that requires variables to have a specific type ( c++ , c# , java ) then we might believe that these concepts are linked. if you only use languages that do not require variables to be declared with a specific type, i.e. var in javascript , def in python , def in ruby , dim in vb.net then you probably have no idea what i'm squawking about! :-) i believe that the benefits of pure polymorphism outweigh the value of static type checking. now that we have fast processors, sophisticated debuggers, and runtime exception constructs the value of type checking at compile time is minimal. some struggle with polymorphism , so let's define it: polymorphism is the ability to send a message to an object without knowing what its type is. polymorphism is the reason why we can drive each others cars and why we have no trouble using different light switches . a car is polymorphic because you can send commonly understood messages to any car ( start (), accelerate (), turnleft (), turnright (), etc) without knowing who built the car. a light switch is polymorphic because you can send the message turnon () and turnoff () to any light switch without knowing who manufactured it. polymorphism is literally what makes our economy work. it allows us to build functionally equivalent products that can have radically different implementations. this is the basis for price and quality differences in products, i.e. toasters, blenders, etc. polymorphism through inheritance the uml diagram above shows how polymorphism is stated in languages like c++ , java , and c# . the method (a.k.a operation) start () is declared to be abstract (in uml), which defers the implementation of the method to the subclasses in your target language. the method for start () is declared in class car and specifies only the method signature and not an implementation (technically polymorphism requires that no code exists for method start () in class car ). the code for method start () is then implemented separately in the volkswagenbeetle and sportscar subclasses. polymorphism implies that start () is implemented using different attributes in the subclasses, otherwise the start () method could simply been implemented in the super class car . even though most of us no longer code in c++ , it is instructive to see why a strong link between inheritance and polymorphism kills flexibility. // c++ polymorphism through inheritance class car { // declare signature as pure virtual function public virtual boolean start() = 0; } class volkswagenbeetle : car { public boolean start() { // implementation code } } class sportscar : car { public boolean start() { // implementation code } } // invocation of polymorphism car cars[] = { new volkswagenbeetle(), new sportscar() }; for( i = 0; i < 2; i++) cars[i].start(); the cars array is of type car and can only hold objects that derive from car ( volkswagenbeetle and sportscar ) and polymorphism works as expected. however, suppose i had the following additional class in my c++ program: // c++ lack of polymorphism with no inheritance class jalopy { public boolean start() { … } } // jalopy does not inherit from car, the following is illegal car cars[] = { new volkswagenbeetle(),new jalopy() }; for( i = 0; i < 2; i++) cars[i].start(); at compile time this will generate an error because the jalopy type is not derived from car . even though they both implement the start () method with an identical signature, the compiler will stop me because there is a static type error. strong type checking imposed at compile time means that all polymorphism has to come through inheritance. this leads to problems with deep inheritance hierarchies and multiple inheritance where there are all kinds of problems with unexpected side effects. even moderately complex programs become very hard to understand and maintain in c++ . historical note: c++ was dominant until the mid 1990s simply because it was an object oriented solution that was not interpreted. this meant that on the slow cpus of the time it had decent performance. we used c++ because we could not get comparable performance with any of the interpreted object-oriented languages of the time, i.e. smalltalk. weakening the link the negative effects of the tight link between inheritance and polymorphism lead both java and c# to introduce the concept of interface to pry apart the ideas of inheritance and polymorphism but keep strong type checking at compile time. first it is possible to implement the above c++ example exactly using inheritance as shown by the c# below: // c# polymorphism using inheritance class car { public virtual boolean start(); // declare signature } class volkswagenbeetle : car { public override boolean start() { // implementation code } } class sportscar : car { public override boolean start() { // implementation code } } // invocation of polymorphism car cars[] = { new volkswagenbeetle(), new sportscar() }; for( i = 0; i < 2; i++) cars[i].start(); in addition, through the use of the interface concept we can write the classes in java as follows: // java polymorphism using interface interface car { public boolean start(); } class volkswagenbeetle implements car { public boolean start() { // implementation code } } class sportscar implements car { public boolean start() { // implementation code } } by using an interface , the implementations of the volkswagenbeetle and sportscar can be completely independent as long as they continue to satisfy the car interface. in this manner, we can now get our jalopy class to be polymorphic with the other two classes simply by: class jalopy implements car { … } polymorphism without inheritance there are languages where you have polymorphism without using inheritance . some examples are javascript, python, ruby, vb.net, and small talk. in each of these languages it is possible to write car.start () without knowing anything about the object car and its method. # python polymorphism class volkswagenbeetle(car): def start(): # code to start volkswagen class sportscar(car): def start(): # code to start sportscar # invocation of polymorphism cars = [ volkswagenbeetle(), sportscar() ] for car in cars: car.start() the ability to get pure polymorphism stems from these languages only have a single variable type prior to runtime i.e. var in javascript, def in python, def in ruby, dim in vb.net. with only one variable type there can not be type error prior to runtime. historical note: it was only during the time frame when java and c# were introduced that cpu power was sufficient for interpreted languages to give sufficient performance at run time. the transition from having polymorphism and inheritance tightly coupled to being more loosely coupled depended on run time interpreters being able to execute practical applications with decent performance. there is no such thing as a free lunch you can end up with strange behaviors when you make method calls to objects that don’t implement a method when type checking is deferred to runtime, i.e. sending start () to an object with no start() method. when type checking is deferred to runtime you want an object to respond “ i have no idea how to start () ” if you send the start () method to it by accident. some purely polymorphic languages generally have a way of detecting missing methods: in visual basic you can get the notimplementedexception in ruby you either implement the method _ missing() method or catch a nomethoderror exception in smalltalk you get the # doesnotunderstand exception some languages don’t have exceptions but there are clunky work arounds: in python you have to use the getattr () call to see if an attribute exists for a name and then use callable() to figure out if it can be called. for the car example above it would look like: startcar = getattr(obj, "start", none) if callable(startcar): startcar () javascript ( ecmascript ) will only raise an exception for a missing method in firefox/spidermonkey. even if you have a well defined exception mechanism (i.e. try catch), when you defer type checking to runtime it becomes harder to prove that your programs work correctly. untyped variables at development time allow a developer to create collections of heterogeneous objects (i.e. sets, bags, vectors, maps, arrays). when you iterate over these heterogeneous collections there is always the possibility that a method will be called on an object that is not implemented. even if you get an exception when this happens, it can take a long time for subtle problems to be found. conclusion the concepts of polymorphism and inheritance are linked only if your language requires static type checking ( c++ , java , c# , etc). any language with only a generic type for variable declaration has full separation of polymorphism and inheritance ( javascript , python , ruby , vb.net ), whether they are compiled to byte code or are directly interpreted. the original compiled languages ( c++ , etc) performed static type checking because of performance issues. performing type checking at compile time created a strong link between inheritance and polymorphism. needing deep class inheritance structures and multiple inheritance lead to runtime side effects and code that was hard to understand. languages like c# and java used the notion of an interface to preserve type checking at compile time to weaken the link between inheritance and polymorphism. in addition, these languages compile to a byte code that is interpreted at runtime to give a balance been static type checking and runtime performance. languages like ruby , python , javascript , visual basic , and smalltalk take advantage of powerful cpus to use interpreters to defer type checking to run time (whether the source code is compiled to byte code or purely interpreted). by deferring type checking we break the link between inheritance and polymorphism, however, this power comes with the difficulty of proving that a subtle runtime problem won't emerge. the one caveat to pure polymorphism is that we may develop subtle bugs that can be difficult to track down and fix. pure polymorphism is only worth seeking if the language that you are using can reliably throw an exception when a method is not implemented. effective programmers are seeking polymorphism and not strong>inheritance. the benefits of pure polymorphism outweigh any advantage that compile time type checking provides, especially when we have access to very sophisticated debuggers and support for runtime exception handling. in general, i believe that the benefits of pure polymorphism outweigh the value of static type checking.