This article will describe the complete root cause analysis of a recent Java deadlock problem observed from a Weblogic 11g production system running on the IBM JVM 1.6.This case study will also demonstrate the importance of mastering Thread Dump analysis skills; including for the IBM JVM Thread Dump format. Environment specification Java EE server: Oracle Weblogic Server 11g & Spring 2.0 OS: AIX 5.3 Java VM: IBM JRE 1.6.0 Platform type: Portal & ordering application Monitoring and troubleshooting tools JVM Thread Dump (IBM JVM format) Compuware Server Vantage (Weblogic JMX monitoring & alerting) Problem overview A major stuck Threads problem was observed & reported from Compuware Server Vantage and affecting 2 of our Weblogic 11g production managed servers causing application impact and timeout conditions from our end users. Gathering and validation of facts As usual, a Java EE problem investigation requires gathering of technical and non-technical facts so we can either derived other facts and/or conclude on the root cause. Before applying a corrective measure, the facts below were verified in order to conclude on the root cause: · What is the client impact? MEDIUM (only 2 managed servers / JVM affected out of 16) · Recent change of the affected platform? Yes (new JMS related asynchronous component) · Any recent traffic increase to the affected platform? No · How does this problem manifest itself? A sudden increase of Threads was observed leading to rapid Thread depletion · Did a Weblogic managed server restart resolve the problem? Yes, but problem is returning after few hours (unpredictable & intermittent pattern) - Conclusion #1: The problem is related to an intermittent stuck Threads behaviour affecting only a few Weblogic managed servers at the time - Conclusion #2: Since problem is intermittent, a global root cause such as a non-responsive downstream system is not likely Thread Dump analysis – first pass The first thing to do when dealing with stuck Thread problems is to generate a JVM Thread Dump. This is a golden rule regardless of your environment specifications & problem context. A JVM Thread Dump snapshot provides you with crucial information about the active Threads and what type of processing / tasks they are performing at that time. Now back to our case study, an IBM JVM Thread Dump (javacore.xyz format) was generated which did reveal the following Java Thread deadlock condition below: 1LKDEADLOCK Deadlock detected !!! NULL --------------------- NULL 2LKDEADLOCKTHR Thread "[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'" (0x000000012CC08B00) 3LKDEADLOCKWTR is waiting for: 4LKDEADLOCKMON sys_mon_t:0x0000000126171DF8 infl_mon_t: 0x0000000126171E38: 4LKDEADLOCKOBJ weblogic/jms/frontend/FESession@0x07000000198048C0/0x07000000198048D8: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" (0x000000012E560500) 3LKDEADLOCKWTR which is waiting for: 4LKDEADLOCKMON sys_mon_t:0x000000012884CD60 infl_mon_t: 0x000000012884CDA0: 4LKDEADLOCKOBJ weblogic/jms/frontend/FEConnection@0x0700000019822F08/0x0700000019822F20: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'" (0x000000012CC08B00) This deadlock situation can be translated as per below: - Weblogic Thread #8 is waiting to acquire an Object monitor lock owned by Weblogic Thread #10 - Weblogic Thread #10 is waiting to acquire an Object monitor lock owned by Weblogic Thread #8 Conclusion: both Weblogic Threads #8 & #10 are waiting on each other; forever! Now before going any deeper in this root cause analysis, let me provide you a high level overview on Java Thread deadlocks. Java Thread deadlock overview Most of you are probably familiar with Java Thread deadlock principles but did you really experience a true deadlock problem? From my experience, true Java deadlocks are rare and I have only seen ~5 occurrences over the last 10 years. The reason is that most stuck Threads related problems are due to Thread hanging conditions (waiting on remote IO call etc.) but not involved in a true deadlock condition with other Thread(s). A Java Thread deadlock is a situation for example where Thread A is waiting to acquire an Object monitor lock held by Thread B which is itself waiting to acquire an Object monitor lock held by Thread A. Both these Threads will wait for each other forever. This situation can be visualized as per below diagram: Thread deadlock is confirmed…now what can you do? Once the deadlock is confirmed (most JVM Thread Dump implementations will highlight it for you), the next step is to perform a deeper dive analysis by reviewing each Thread involved in the deadlock situation along with their current task & wait condition.Find below the partial Thread Stack Trace from our problem case for each Thread involved in the deadlock condition: ** Please note that the real application Java package name was renamed for confidentiality purposes ** Weblogic Thread #8 "[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'" J9VMThread:0x000000012CC08B00, j9thread_t:0x00000001299E5100, java/lang/Thread:0x070000001D72EE00, state:B, prio=1 (native thread ID:0x111200F, native priority:0x1, native policy:UNKNOWN) Java callstack: at weblogic/jms/frontend/FEConnection.stop(FEConnection.java:671(Compiled Code)) at weblogic/jms/frontend/FEConnection.invoke(FEConnection.java:1685(Compiled Code)) at weblogic/messaging/dispatcher/Request.wrappedFiniteStateMachine(Request.java:961(Compiled Code)) at weblogic/messaging/dispatcher/DispatcherImpl.syncRequest(DispatcherImpl.java:184(Compiled Code)) at weblogic/messaging/dispatcher/DispatcherImpl.dispatchSync(DispatcherImpl.java:212(Compiled Code)) at weblogic/jms/dispatcher/DispatcherAdapter.dispatchSync(DispatcherAdapter.java:43(Compiled Code)) at weblogic/jms/client/JMSConnection.stop(JMSConnection.java:863(Compiled Code)) at weblogic/jms/client/WLConnectionImpl.stop(WLConnectionImpl.java:843) at org/springframework/jms/connection/SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:342) at org/springframework/jms/connection/SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:296) at org/app/JMSReceiver.receive() …………………………………………………………………… Weblogic Thread #10 "[STUCK] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'" J9VMThread:0x000000012E560500, j9thread_t:0x000000012E35BCE0, java/lang/Thread:0x070000001ECA9200, state:B, prio=1 (native thread ID:0x4FA027, native priority:0x1, native policy:UNKNOWN) Java callstack: at weblogic/jms/frontend/FEConnection.getPeerVersion(FEConnection.java:1381(Compiled Code)) at weblogic/jms/frontend/FESession.setUpBackEndSession(FESession.java:755(Compiled Code)) at weblogic/jms/frontend/FESession.consumerCreate(FESession.java:1025(Compiled Code)) at weblogic/jms/frontend/FESession.invoke(FESession.java:2995(Compiled Code)) at weblogic/messaging/dispatcher/Request.wrappedFiniteStateMachine(Request.java:961(Compiled Code)) at weblogic/messaging/dispatcher/DispatcherImpl.syncRequest(DispatcherImpl.java:184(Compiled Code)) at weblogic/messaging/dispatcher/DispatcherImpl.dispatchSync(DispatcherImpl.java:212(Compiled Code)) at weblogic/jms/dispatcher/DispatcherAdapter.dispatchSync(DispatcherAdapter.java:43(Compiled Code)) at weblogic/jms/client/JMSSession.consumerCreate(JMSSession.java:2982(Compiled Code)) at weblogic/jms/client/JMSSession.setupConsumer(JMSSession.java:2749(Compiled Code)) at weblogic/jms/client/JMSSession.createConsumer(JMSSession.java:2691(Compiled Code)) at weblogic/jms/client/JMSSession.createReceiver(JMSSession.java:2596(Compiled Code)) at weblogic/jms/client/WLSessionImpl.createReceiver(WLSessionImpl.java:991(Compiled Code)) at org/springframework/jms/core/JmsTemplate102.createConsumer(JmsTemplate102.java:204(Compiled Code)) at org/springframework/jms/core/JmsTemplate.doReceive(JmsTemplate.java:676(Compiled Code)) at org/springframework/jms/core/JmsTemplate$10.doInJms(JmsTemplate.java:652(Compiled Code)) at org/springframework/jms/core/JmsTemplate.execute(JmsTemplate.java:412(Compiled Code)) at org/springframework/jms/core/JmsTemplate.receiveSelected(JmsTemplate.java:650(Compiled Code)) at org/springframework/jms/core/JmsTemplate.receiveSelected(JmsTemplate.java:641(Compiled Code)) at org/app/JMSReceiver.receive() …………………………………………………………… As you can see in the above Thread Strack Traces, such deadlock did originate from our application code which is using the Spring framework API for the JMS consumer implementation (very useful when not using MDB’s). The Stack Traces are quite interesting and revealing that both Threads are in a race condition against the same Weblogic JMS consumer session / connection and leading to a deadlock situation: - Weblogic Thread #8 is attempting to reset and close the current JMS connection - Weblogic Thread #10 is attempting to use the same JMS Connection / Session in order to create a new JMS consumer - Thread deadlock is triggered! Root cause: non Thread safe Spring JMS SingleConnectionFactory implementation A code review and a quick research from Spring JIRA bug database did reveal the following Thread safe defect below with a perfect correlation with the above analysis: # SingleConnectionFactory's resetConnection is causing deadlocks with underlying OracleAQ's JMS connection https://jira.springsource.org/browse/SPR-5987 A patch for Spring SingleConnectionFactory was released back in 2009 which did involve adding proper synchronized{} block in order to prevent Thread deadlock in the event of a JMS Connection reset operation: synchronized (connectionMonitor) { //if condition added to avoid possible deadlocks when trying to reset the target connection if (!started) { this.target.start(); started = true; } } Solution Our team is currently planning to integrate this Spring patch in to our production environment shortly. The initial tests performed in our test environment are positive. Conclusion I hope this case study has helped understand a real-life Java Thread deadlock problem and how proper Thread Dump analysis skills can allow you to quickly pinpoint the root cause of stuck Thread related problems at the code level. Please don’t hesitate to post any comment or question.

Cross-site request forgery attacks (CSRF) are very common in web applications and can cause significant harm if allowed. If you have never heard of CSRF I recommend you check out OWASPs page about it. Luckily preventing CSRF attacks is quite simple, I’ll try to show you how they work and how we can defend from them in the least obtrusive way possible in Java based web apps. Imagine you are about to perform a money transfer in your bank’s secure web page, when you click on the transfer option a form page is loaded that allows you to choose the debit and credit accounts, and enter the amount of money to move. When you are satisfied with your options you press “submit” and send the form information to your bank’s web server, which in turns performs the transaction. Now add the following to the picture, a malicious website (which you think harmless of course) is open on another window/tab of your browser while you are innocently moving all your millions in your bank’s site. This evil site knows the bank’s web forms structure, and as you browse through it, it tries to post transactions withdrawing money from your accounts and depositing it on the evil overlord’s accounts, it can do it because you have an open and valid session with the banks site in the same browser! This is the basis for a CSRF attack. One simple and effective way to prevent it is to generate a random (i.e. unpredictable) string when the initial transfer form is loaded and send it to the browser. The browser then sends this piece of data along with the transfer options, and the server validates it before approving the transaction for processing. This way, malicious websites cannot post transactions even if they have access to a valid session in a browser. To implement this mechanism in Java I choose to use two filters, one to create the salt for each request, and another to validate it. Since the users request and subsequent POST or GETs that should be validated do not necessarily get executed in order, I decided to use a time based cache to store a list of valid salt strings. The first filter, used to generate a new salt for a request and store it in the cache can be coded as follows: package com.ricardozuasti.csrf; import com.google.common.cache.Cache; import com.google.common.cache.CacheBuilder; import com.google.common.cache.CacheLoader; import com.google.common.cache.LoadingCache; import java.io.IOException; import java.security.SecureRandom; import java.util.concurrent.ExecutionException; import java.util.concurrent.TimeUnit; import javax.servlet.*; import javax.servlet.http.HttpServletRequest; import org.apache.commons.lang.RandomStringUtils; public class LoadSalt implements Filter { @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { // Assume its HTTP HttpServletRequest httpReq = (HttpServletRequest) request; // Check the user session for the salt cache, if none is present we create one Cache csrfPreventionSaltCache = (Cache) httpReq.getSession().getAttribute("csrfPreventionSaltCache"); if (csrfPreventionSaltCache == null){ csrfPreventionSaltCache = CacheBuilder.newBuilder() .maximumSize(5000) .expireAfterWrite(20, TimeUnit.MINUTES) .build(); httpReq.getSession().setAttribute("csrfPreventionSaltCache", csrfPreventionSaltCache); } // Generate the salt and store it in the users cache String salt = RandomStringUtils.random(20, 0, 0, true, true, null, new SecureRandom()); csrfPreventionSaltCache.put(salt, Boolean.TRUE); // Add the salt to the current request so it can be used // by the page rendered in this request httpReq.setAttribute("csrfPreventionSalt", salt); chain.doFilter(request, response); } @Override public void init(FilterConfig filterConfig) throws ServletException { } @Override public void destroy() { } } I used Guava CacheBuilder to create the salt cache since it has both a size limit and an expiration timeout per entry. To generate the actual salt I used Apache Commons RandomStringUtils, powered by Java 6 SecureRandom to ensure a strong generation seed. This filter should be used in all requests ending in a page that will link, post or call via AJAX a secured transaction, so in most cases it’s a good idea to map it to every request (maybe with the exception of static content such as images, CSS, etc.). It’s mapping in your web.xml should look similar to: ... loadSalt com.ricardozuasti.csrf.LoadSalt ... loadSalt * ... As I said, to validate the salt before executing secure transactions we can write another filter: package com.ricardozuasti.csrf; import com.google.common.cache.Cache; import java.io.IOException; import javax.servlet.*; import javax.servlet.http.HttpServletRequest; public class ValidateSalt implements Filter { @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { // Assume its HTTP HttpServletRequest httpReq = (HttpServletRequest) request; // Get the salt sent with the request String salt = (String) httpReq.getParameter("csrfPreventionSalt"); // Validate that the salt is in the cache Cache csrfPreventionSaltCache = (Cache) httpReq.getSession().getAttribute("csrfPreventionSaltCache"); if (csrfPreventionSaltCache != null && salt != null && csrfPreventionSaltCache.getIfPresent(salt) != null){ // If the salt is in the cache, we move on chain.doFilter(request, response); } else { // Otherwise we throw an exception aborting the request flow throw new ServletException("Potential CSRF detected!! Inform a scary sysadmin ASAP."); } } @Override public void init(FilterConfig filterConfig) throws ServletException { } @Override public void destroy() { } } You should configure this filter for every request that needs to be secure (i.e. retrieves or modifies sensitive information, move money, etc.), for example: ... validateSalt com.ricardozuasti.csrf.ValidateSalt ... validateSalt /transferMoneyServlet ... After configuring both servlets all your secured requests should fail :). To fix it you have to add, to each link and form post that ends in a secure URL, the csrfPreventionSalt parameter containing the value of the request parameter with the same name. For example, in an HTML form within a JSP page: ... ... ... Of course you can write a custom tag, a nice Javascript code or whatever you prefer to inject the new parameter in every needed link/form.

This article will provide you with a tutorial on how you can quickly pinpoint the Java Thread contributors to a high CPU problem on the Windows OS. Windows, like other OS such as Linux, Solaris & AIX allow you to monitor the CPU utilization at the process level but also for individual Thread executing a task within a process. For this tutorial, we created a simple Java program that will allow you to learn this technique in a step by step manner. Troubleshooting tools The following tools will be used below for this tutorial: - Windows Process Explorer (to pinpoint high CPU Thread contributors) - JVM Thread Dump (for Thread correlation and root cause analysis at code level) High CPU simulator Java program The simple program below is simply looping and creating new String objects. It will allow us to perform this CPU per Thread analysis. I recommend that you import it in an IDE of your choice e.g. Eclipse and run it from there. You should observe an increase of CPU on your Windows machine as soon as you execute it. package org.ph.javaee.tool.cpu; /** * HighCPUSimulator * @author Pierre-Hugues Charbonneau * http://javaeesupportpatterns.blogspot.com * */ public class HighCPUSimulator { private final static int NB_ITERATIONS = 500000000; // ~1 KB data footprint private final static String DATA_PREFIX = "datadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadatadata"; /** * @param args */ public static void main(String[] args) { System.out.println("HIGH CPU Simulator 1.0"); System.out.println("Author: Pierre-Hugues Charbonneau"); System.out.println("http://javaeesupportpatterns.blogspot.com/"); try { for (int i = 0; i < NB_ITERATIONS; i++) { // Perform some String manipulations to slowdown and expose looping process... String data = DATA_PREFIX + i; } } catch (Throwable any) { System.out.println("Unexpected Exception! " + any.getMessage() + " [" + any + "]"); } System.out.println("HighCPUSimulator done!"); } } Step #1 – Launch Process Explorer The Process Explorer tool visually shows the CPU usage dynamically. It is good for live analysis. If you need historical data on CPU per Thread then you can also use Windows perfmon with % Processor Time & Thread Id data counters. You can download Process Explorer from the link below: http://technet.microsoft.com/en-us/sysinternals/bb896653 In our example, you can see that the Eclipse javaw.exe process is now using ~25% of total CPU utilization following the execution of our sample program. Step #2 – Launch Process Explorer Threads view The next step is to display the Threads view of the javaw.exe process. Simply right click on the javaw.exe process and select Properties. The Threads view will be opened as per below snapshot: - The first column is the Thread Id (decimal format) - The second column is the CPU utilization % used by each Thread - The third column is also another counter indicating if Thread is running on the CPU In our example, we can see our primary culprit is Thread Id #5996 using ~ 25% of CPU. Step #3 – Generate a JVM Thread Dump At this point, Process Explorer will no longer be useful. The goal was to pinpoint one or multiple Java Threads consuming most of the Java process CPU utilization which is what we achieved. In order to go the next level in your analysis you will need to capture a JVM Thread Dump. This will allow you to correlate the Thread Id with the Thread Stack Trace so you can pinpoint that type of processing is consuming such high CPU. JVM Thread Dump generation can be done in a few manners. If you are using JRockit VM you can simply use the jrcmd tool as per below example: Once you have the Thread Dump data, simply search for the Thread Id and locate the Thread Stack Trace that you are interested in. For our example, the Thread “Main Thread” which was fired from Eclipse got exposed as the primary culprit which is exactly what we wanted to demonstrate. "Main Thread" id=1 idx=0x4 tid=5996 prio=5 alive, native_blocked at org/ph/javaee/tool/cpu/HighCPUSimulator.main (HighCPUSimulator.java:31) at jrockit/vm/RNI.c2java(IIIII)V(Native Method) -- end of trace Step #4 – Analyze the culprit Thread(s) Stack Trace and determine root cause At this point you should have everything that you need to move forward with the root cause analysis. You will need to review each Thread Stack Trace and determine what type of problem you are dealing with. That final step is typically where you will spend most of your time and problem can be simple such as infinite looping or complex such as garbage collection related problems. In our example, the Thread Dump did reveal the high CPU originates from our sample Java program around line 31. As expected, it did reveal the looping condition that we engineered on purpose for this tutorial. for (int i = 0; i < NB_ITERATIONS; i++) { // Perform some String manipulations to slowdown and expose looping process... String data = DATA_PREFIX + i; } I hope this tutorial has helped you understand how you can analyze and help pinpoint root cause of Java CPU problems on Windows OS. Please stay tuned for more updates, the next article will provide you with a Java CPU troubleshooting guide including how to tackle that last analysis step along with common problem patterns.

Following on from the previous post on operator overloading I'm going to be looking at Implicit Conversions, and how we can combine them to with operator overloading to do some really neat things, including one way of creating a multi-parameter conversion. So what's an "Implicit Conversion" when it's at home? So lets start with some basic Scala syntax, if you've spent any time with Scala you've probably noticed it allows you to do things like: (1 to 4).foreach(println) // print out 1 2 3 4 Ever wondered how it does this? Lets make things more explicit, you could rewrite the above code as: val a : Int = 1 val b : Int = 4 val myRange : Range = a to b myRange.foreach(println) Scala is creating a Range object directly from two Ints, and a method called to. So what's going on here? Is this just a sprinkling of syntactic sugar to make writing loops easier? Is to just a keyword in like def or val? The answers to all this is no, there's nothing special going on here. to is simply a method defined in the RichInt class, which takes a parameter and returns a Range object (specifically a subclass of Range called Inclusive). You could rewrite it as the following if you really wanted to: val myRange : Range = a.to(b) Hang on though, RichInt may have a "to" method but Int certainly doesn't, in your example you're even explicitly casting your numbers to Ints Which brings me nicely on to the subject of this post, Implicit Conversions. This is how Scala does this. Implicit Conversions are a set of methods that Scala tries to apply when it encounters an object of the wrong type being used. In the case of the to example there's a method defined and included by default that will convert Ints into RichInts. So when Scala sees 1 to 4 it first runs the implicit conversion on the 1 converting it from an Int primitive into a RichInt. It can then call the to method on the new RichInt object, passing in the second Int (4) as the parameter. Hmm, think I understand, how's about another example? Certainly. Lets try to improve our Complex number class we created in the previous post. Using operator overloading we were able to support adding two complex numbers together using the + operator. eg. class Complex(val real : Double, val imag : Double) { def +(that: Complex) = new Complex(this.real + that.real, this.imag + that.imag) def -(that: Complex) = new Complex(this.real - that.real, this.imag - that.imag) override def toString = real + " + " + imag + "i" } object Complex { def main(args : Array[String]) : Unit = { var a = new Complex(4.0,5.0) var b = new Complex(2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 2.0i } } But what if we want to support adding a normal number to a complex number, how would we do that? We could certainly overload our "+" method to take a Double argument, ie something like... def +(n: Double) = new Complex(this.real + n, this.imag) Which would allow us to do... val sum = myComplexNumber + 8.5 ...but it'll break if we try... val sum = 8.5 + myComplexNumber To get around this we could use an Implicit Conversion. Here's how we create one. object ComplexImplicits { implicit def Double2Complex(value : Double) = new Complex(value,0.0) } Simple! Although you do need to be careful to import the ComplexImplicits methods before they can be used. You need to make sure you add the following to the top of your file (even if your Implicits object is in the same file)... import ComplexImplicits._ And that's the problem solved, you can now write val sum = 8.5 + myComplexNumber and it'll do what you expect! Nice. Is there anything else I can do with them? One other thing I've found them good for is creating easy ways of instantiating objects. Wouldn't it be nice if there were a simpler way of creating one of our complex numbers other than with new Complex(3.0,5.0). Sure you could get rid of the new by making it a case class, or implementing an apply method. But we can do better, how's about just (3.0,5.0) Awesome, but I'd need some sort of multi parameter implicit conversion, and I don't really see how that's possible!? The thing is, ordinarily (3.0,5.0) would create a Tuple. So we can just use that tuple as the parameter for our implicit conversion and convert it into a Complex. how we might go about doing this... implicit def Tuple2Complex(value : Tuple2[Double,Double]) = new Complex(value._1,value._2); And there we have it, a simple way to instantiate our Complex objects, for reference here's what the entire Complex code looks like now. import ComplexImplicits._ object ComplexImplicits { implicit def Double2Complex(value : Double) = new Complex(value,0.0) implicit def Tuple2Complex(value : Tuple2[Double,Double]) = new Complex(value._1,value._2); } class Complex(val real : Double, val imag : Double) { def +(that: Complex) : Complex = (this.real + that.real, this.imag + that.imag) def -(that: Complex) : Complex = (this.real - that.real, this.imag + that.imag) def unary_~ = Math.sqrt(real * real + imag * imag) override def toString = real + " + " + imag + "i" } object Complex { val i = new Complex(0,1); def main(args : Array[String]) : Unit = { var a : Complex = (4.0,5.0) var b : Complex = (2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 8.0i println(~b) // 3.60555 var c = 4 + b println(c) // 6.0 + 3.0i var d = (1.0,1.0) + c println(d) // 7.0 + 4.0i } }

So, I've been teaching myself Scala recently, and it's a very interesting language. One of the nice things I like about it, is it's support for creating DSLs, domain specific languages. A domain specific language - or at least my understanding of it - is a language that is written specifically for one problem domain. One example would be SQL, great for querying relational databases, useless for creating first person shooters. Of course Scala itself is not a DSL, it's a general purpose language. However it does offer several features that allow you to simulate a DSL, in particular operator overloading, and implicit conversions. In this post I'm going to focus on the first of these... Operator Overloading So what's operator overloading? Well operators are typically things such as +, -, and !. You know those things you use to do arithmetic on numbers, or occasionally for manipulating Strings. Well, operator overloading - just like method overloading - allows you to redefine their behaviour for a particular type, and give them meaning for your own custom classes. Hang on a minute! I'm sure someone once told me operator overloading was evil? Indeed, this is quite a controversial topic. It's considered far too open for abuse by some, and was so maligned in C++ that the creators of Java deliberately disallowed it (excepting "+" for String concatenation). I'm of a slightly different opinion, used responsibly it can be very useful. For example lots of different objects support a concept of addition, so why not just use an addition operator? Lets say you were developing a complex number class, and you want to support addition. Wouldn't it be nicer to write... Complex result = complex1 + complex2; ...rather than... Complex result = complex1.add(complex2); The first example is much more natural don't you think? So Scala allows you to overload operators then? Well, not really. In fact, technically not at all. So all this is just a tease? This is the most stupid blog post I've ever read. Scala's rubbish. I'm going back to Algol 68. Wait a second, I've not finished. You see Scala doesn't support operator overloading, because it doesn't have operators! Scala doesn't have operators? You've gone mad, I write stuff like "sum = 2 + 3" all the time, and what about all those funny list operations? "::", and ":/". They look like operators to me! Well they're not. The thing is, Scala has a rather relaxed attitude to what you can name a method. When you write... sum = 2 + 3, ...you're actually calling a method called + on a RichInt type with a value of 2. You could even rewrite it as... sum = 2.+(3) ...if you really really wanted to. Aha, I got it. So how do you go about overloading an operator then? Simple, it's exactly the same as writing a normal method. Here's an example. class Complex(val real : Double, val imag : Double) { def +(that: Complex) = new Complex(this.real + that.real, this.imag + that.imag) def -(that: Complex) = new Complex(this.real - that.real, this.imag - that.imag) override def toString = real + " + " + imag + "i" } object Complex { def main(args : Array[String]) : Unit = { var a = new Complex(4.0,5.0) var b = new Complex(2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 2.0i } } Ok that's nice, what if I wanted a "not" operator though, ie something like a "!" That's a unary prefix operator, and yes scala can support these, although in a more limited fashion than an infix operator like "+" Only four operators can be supported in this fashion, +, -, !, and ~. You simply need to call your methods unary_! or unary_~, etc. Here's how you might add a "~" to calculate the magnitude of a Complex number to our complex number class class Complex(val real : Double, val imag : Double) { // ... def unary_~ = Math.sqrt(real * real + imag * imag) } object Complex { def main(args : Array[String]) : Unit = { var b = new Complex(2.0,3.0) prinln(~b) // 3.60555 } } So that's all pretty simple, but please use responsibly. Don't create methods called "+" unless your class really does something that could be interpreted as addition. And never ever redefine the binary shift left operator "<<" as some sort of substitute for println. It's not clever and you'll make the Scala gods angry. Hope you found that useful. Next up I'll cover implicit conversions. Another nice feature of Scala that really allows you to write your code in a more natural way

On the surface these methods do the same thing in Java; Thread.yield(), Thread.sleep(0), Object.wait(0,1) and LockSupport.parkNanos(1) They all wait a sort period of time, but how much that is varies a surprising amount and between platforms. Timing a short delay The following code times how long it takes to repeatedly call those methods. import java.util.concurrent.locks.LockSupport; public class Pausing { public static void main(String... args) throws InterruptedException { int repeat = 10000; for (int i = 0; i < 3; i++) { long time0 = System.nanoTime(); for (int j = 0; j < repeat; j++) Thread.yield(); long time1 = System.nanoTime(); for (int j = 0; j < repeat; j++) Thread.sleep(0); long time2 = System.nanoTime(); synchronized (Thread.class) { for (int j = 0; j < repeat/10; j++) Thread.class.wait(0, 1); } long time3 = System.nanoTime(); for (int j = 0; j < repeat/10; j++) LockSupport.parkNanos(1); long time4 = System.nanoTime(); System.out.printf("The average time to yield %.1f μs, sleep(0) %.1f μs, " + "wait(0,1) %.1f μs and LockSupport.parkNanos(1) %.1f μs%n", (time1 - time0) / repeat / 1e3, (time2 - time1) / repeat / 1e3, (time3 - time2) / (repeat/10) / 1e3, (time4 - time3) / (repeat/10) / 1e3); } } } On Windows 7 The average time to yield 0.3 μs, sleep(0) 0.6 μs, wait(0,1) 999.9 μs and LockSupport.parkNanos(1) 1000.0 μs The average time to yield 0.3 μs, sleep(0) 0.6 μs, wait(0,1) 999.5 μs and LockSupport.parkNanos(1) 1000.1 μs The average time to yield 0.2 μs, sleep(0) 0.5 μs, wait(0,1) 1000.0 μs and LockSupport.parkNanos(1) 1000.1 μs On RHEL 5.x The average time to yield 1.1 μs, sleep(0) 1.1 μs, wait(0,1) 2003.8 μs and LockSupport.parkNanos(1) 3.8 μs The average time to yield 1.1 μs, sleep(0) 1.1 μs, wait(0,1) 2004.8 μs and LockSupport.parkNanos(1) 3.4 μs The average time to yield 1.1 μs, sleep(0) 1.1 μs, wait(0,1) 2005.6 μs and LockSupport.parkNanos(1) 3.1 μs In summary If you want to wait for a short period of time, you can't assume that all these methods do the same thing, nor will be the same between platforms.

You may be using JSON to transfer data (we were using it in our message queue). While this is good, it has the only benefit of being human-readable. If you don’t care about readability, you’d probably want to use a more efficient serialization mechanism. Multiple options exist: protobuf, MessagePack, protostuff, java serialization. The easiest of them to use is java serialization, but it is less efficient (with both memory and time) than the other solutions. There are some benchmarks that will help you choose the most efficient solution, but if you want it to be easy and almost drop-in replacement to your JSON solution, MessagePack might be the best option. I made a simple test to compare the JSON output to the MessagePack output in terms of size: 2300 vs 150 bytes for a simple message. Pretty good reduction, and if the messages are a lot, it’s a must to optimize. However, you need to register all classes in the message pack. There are two options: use @Message on all the objects in the serialized graph. This is a bit tedious, especially if you already have a lot of classes that are transferred. You have to go through the whole graph you can manually register all classes with the mesagpack. Again tedious, because you also have to register all classes that the message class contains as a field (recursively) That’s why I wrote the following code to loop all our message classes, and register them with the message pack on startup. It partly relies on spring classes, but if you are not using Spring, you can replace them: private MessagePack serializer = new MessagePack(); private ClassMapper classMapper = new DefaultClassMapper(); @PostConstruct public void init() { // we need to find all messages, and register their classes, and also all their fields' recursively ClassPathScanningCandidateComponentProvider provider = new ClassPathScanningCandidateComponentProvider(false); Set classes = provider.findCandidateComponents("com.foo.bar.messages"); // hacking MessagePack to allow Set handling Field fld = ReflectionUtils.findField(MessagePack.class, "registry"); ReflectionUtils.makeAccessible(fld); TemplateRegistry registry = (TemplateRegistry) ReflectionUtils.getField(fld, serializer); registry.register(Set.class, new SetTemplate(new AnyTemplate(registry))); registry.registerGeneric(Set.class, new GenericCollectionTemplate(registry, SetTemplate.class)); try { for (BeanDefinition def : classes) { Class clazz = Class.forName(def.getBeanClassName()); registerHierarcy(clazz, serializer, Sets.>newHashSet()); } } catch (ClassNotFoundException e) { throw new IllegalStateException(e); } } private void registerHierarcy(Class clazz, MessagePack serializer, Set> handledClasses) { if (!isEligibleForRegistration(clazz)) { return; } Class currentClass = clazz; while (currentClass != null && !currentClass.isEnum() && currentClass != Object.class) { for (Field field : currentClass.getDeclaredFields()) { registerHierarcy(field.getType(), serializer, handledClasses); // type parameters Type type = field.getGenericType(); if (type instanceof ParameterizedType) { for (Type typeParam : ((ParameterizedType) type).getActualTypeArguments()) { // avoid circular generics references, resulting in stackoverflow Class typeParamClass = (Class) typeParam; if (!handledClasses.contains(typeParamClass)) { handledClasses.add(typeParamClass); registerHierarcy(typeParamClass, serializer, handledClasses); } } } } currentClass = currentClass.getSuperclass(); } try { serializer.register(clazz); } catch (Exception ex) { logger.warn("Problem registering class " + clazz, ex.getMessage()); } } private boolean isEligibleForRegistration(Class clazz) { return !(clazz.isAnnotationPresent(Entity.class) || clazz == Class.class || Type.class.isAssignableFrom(clazz) || clazz.isInterface() || clazz.isArray() || ClassUtils.isPrimitiveOrWrapper(clazz) || clazz == String.class || clazz == Date.class || clazz == Object.class); }

What is Zebra? The Zebra is a JavaScript library that follows easy OOP concepts and implements a rich set of various self-made UI components. There are a huge number of attractive and powerful WEB UI frameworks on the market. Most of them are stuck to HTML, DOM and CSS. They build UIs by coloring DOM with CSS and manipulating the HTML DOM tree. Zebra UI is different. It is not based on HTML, DOM or CSS. Zebra UI components are implemented and rendered from scratch as a number of widgets organized in hierarchy. How does Zebra manage to build and render UIs on the web? The essential thing required by Zebra abstraction is the existence of a "Canvas-like " component. The "Canvas" component has to have set of graphical methods to paint lines, simple shapes, text, images and be able to catch input (keyboard, mouse, etc) events. For instance Java AWT/SWING, Eclipse SWT, .NET or other platforms will supply "Canvas-like" components. But the new HTML5 standard embeds "Canvas" element. This element is supported by most of the modern browsers and is what makes it possible and reasonable to "drop" Zebra UI into a web context. See Zebra demo http://zebra.gravitysoft.org The table below lists the most significant Zebra UI components, managers view, etc. Components marked by orange background are still in development: The Zebra Java to JavaScript converter is out of context in this article, nevertheless it plays key role in porting Java-based UI components onto the web. For demo purposes, a slightly outdated version of the Java to JavaScript converter is available: http://j2js.gravitysoft.org Zebra JavaScript Easy OOP concept An attractive, easy for use, and understandable programming model is very important in context of having supportable, extendable, elegant code. This is one of the painful problems in web development. Zebra introduces easy OOP concepts as a foundation. ~10.5kb of Zebra code helps to do the following: To get more information about Zebra OOP see the cheat sheet: Zebra easy OOP concept cheat sheet Zebra "Hello WEB" application The Zebra JavaScript demo indents to show an internal window with a "Hello web" title. The window is opened by pressing a button: // create canvas and store root panel in local variable var c = new zCanvas(html5Canvas), r = c.root; // create button var b = new Button("Hello WEB"); // set border layout manager r.setLayout(new BorderLayout()); // add button to center of root panel r.add(Layout.CENTER, b); // register the button event listener that opens external // window every time the button has been pressed b._(function() { var w = new Window("Hello WEB"); w.setSize(200,200); w.show(); }); Zebra UI features via JavaScript code snippets Zebra UI component design follows approaches similar to traditional Java AWT/SWING, .NET, and Eclipse SWT. UI components are organized as a hierarchy where some components are laid out on others. But Zebra does many things much more quickly and easily. Below are some Zebra UI features equpped with code snippets: Compound components. Zebra UI components often are built as a combination of several other components laid out on a panel. For instance the "Button" component is a panel that keeps a child component as its label. By default title is "Label", but developers can set other UI components as a button label. Take a look at the snippets below: // by default button uses label component as its content var button = new Button("Label"); // set image as the button label var button = new Button(new ImagePan("b.png")); // set image+label as the button content. The image+label is also // compound component that consists of image and label var bContent = new Panel(new FlowLayout()); bContent.add(new ImagePan()); bContent.add(new Label()); var button = new Button(bContent); The "BorderPan" UI component is one more example of a compound component. It consists of a label and content panel. Take a look at the snippets below: // border panel uses simple "Label" as its title by default var bp = new BorderPanel("Label", new Panel()); // border panel uses another border panel as its title var bp = new BorderPanel(new BorderPan("Label"), new Panel()); // border panel uses "Checkbox" as its title var bp = new BorderPanel(new Checkbox("Check me"), new Panel()); Full control over UI component rendering. Zebra UI components painting is fully in developers hands. Zebra calls "paint", "update", "paintOnTop" UI component methods with a graphical context as an input argument. These methods form a UI component face. Passed graphical context gives developers number of elementary methods to draw and fill primitive shapes, paint text, and images. The "paint" method defines the UI component "face": // create inner class instance with redefined component "face" var myEyesCandyComponent = new Panel([ function paint(g) { // paint what ever you want using passed graphical context g.drawLine(...); g.drawRect(...); g.fillArc(...); } ]); The "update" method forms a component background: // create inner class instance and override background // rendering with a custom implementation var myEyesCandyComponent = new Panel([ function update(g) { ... } ]); The "paintOnTop" method may be used, for instance, to render a focus indicator over the component "face": // render rectangle around component if the component holds focus var myEyesCandyComponent = new Panel([ function paintOnTop(g) { if (this.hasFocus()) g.drawRect(0,0, this.width, this.height); } ]); Zebra paint manager takes care of triggering the re-painting of invalidated-"dirty" areas. UI components are designed to make direct "repaint" method execution unnecessary. But custom UI components should call thr "repaint([x, y, w, h])" method every time a new dirty area has appeared. The "repaint" method execution triggers paint manager to recalculate current dirty area and schedule repainting when it is possible: var MyCustomComponent = Class(Panel, [ function setBorderColor(c) { this.color = c; this.repaint(); // inform paint manager the component // has to be completely re-painted } ]); Neat events handling. When it is possible, Zebra event handling follows a declarative pattern (see next feature bullet). It allows developers avoiding listeners registration and simplifies the event handling concept. To catch an event: Express an intention to get the desired event type by inheritance an appropriate interface. Interface is like a marker. Implement function(s) to treat desired event. For instance, imagine a custom component that needs to handle mouse events. Express the intention by implementing the "MouseListener" interface: // let Zebra know that the component wants getting mouse // events by implementing"MouseListener" interface var MyComponent = new Class(Panel, MouseListener, []); Suppose "mouse button has pressed" and "mouse cursor has entered the component" event types have to be handled. To do it, declare the following functions correspondently: var MyComponent = new Class(Panel, MouseListener, [ function mousePressed(e) { // handle mouse pressed event here }, function mouseEntered(e) { // handle mouse entered event here } ]); Global event handling. The special Zebra event manager keeps track of all events that have occurred for all instantiated UI components. The manager can be utilized to register global event listeners that get all events of the given type. For instance, listening to focus events globally can be done this way: // instantiate "FocusListener" interface to express that focus // events have to be catched zebra.ui.events.addListener(new FocusListener([ function focusLost(e) { ... }, function focusGained(e) { ... } ])); Catching children components events. The Parent UI component can listen to input events that have happened in its children by implementing the "ChildrenListener" interface and adding appropriate method(s) as follows: // implements "ChildrenListener" interface to express intention // to get children events var MyComponent = new Class(Panel, ChildrenListener, [ function childInputEvent(e){ // handle children events here } ]); Composite component (event transparent children). Often compound UI components have to prevent their children components from getting any events. Children components are becoming event transparent. This can be done by implementing "Composite" interface and adding the "catchInput" method. The method should return "true" if the passed as argument child has to be event transparent. For instance: var MyComponent = new Class(Panel, Composite, [ function catchInput(kid){ // return true if the given kid has to be event transparent return true; } ]); Declarative pattern. "First inherit an interface to express an intention to do something and then implement required method(s)". This approach allows Zebra to decouple various functional parts from each other. Extending in this context means adding new declarative patterns instead of overloading Zebra UI classes with new code and APIs. For instance, imagine we need to change the mouse cursor type every time the mouse pointers enter a UI component. Zebra UI components don’t know how to control mouse cursor type. It's the cursor manager ("zebra.ui.cursor") that does it: // inherits "CursorInfo" interface to let cursor manager know // the component controls mouse cursor var p = new Panel(CursorInfo, [ // declare method that returns mouse cursor type for the component function getCursorType(x, y){ return Cursor.WAIT; } ]); HTML5 Canvas transformation operations can be applied to UI. Rotation, zoom in, zoom out and other graphical transformation effects can be applied to Zebra UI: // zoom UI in 1.3 times vertically // and horizontally zCanvas.scale(1.3, 1.3); // rotate zCanvas.rotate(0.3); ... // set back to initial stat zCanvas.scale(null); zCanvas.rotate(null, null); Look and feel customization. A lot of Zebra UI components' visual characteristics are defined by a special properties file. If the properties file is not custom enough, a UI Wizard class can be implemented. The instance of the class is notified about all instantiated UI components by calling a "customize" method. It helps to customize UI components' look and feel "on the fly". For instance, let’s define own wizard class to color all label components with red background: // declare custom Wizard class var MyCustomWizard = Class(Wizard, [ // the method is called every time a new component has been // instantiated function customize(id, comp) { // customize just instantiated label component background if (id == Wizard.LABEL) comp.setBackground(Fill.red); } ]); Then setup your wizard with the properties file as follow: ... # specify wizard to be used for UI customization wizard = MyCustomWizard() Layered architecture. Zebra Canvas is the root panel that consists of a number of layers. Layers are a standard Zebra UI "Panel" that are stretched over the whole Canvas surface. Zebra holds layers as a stack. Every time an event occurs the Zebra event manager "asks" (starting from top to bottom layer) who wants to take control. The first met layer that grabs control is selected as the target. The picture below explains it: Let’s develop a layer that freezes (blocks any interaction) and un-freezes UI components by pressing the "CTRL + SHIFT + ALT" keys combination. Freezing is indicated by dimming UI components: // declare custom layer class that inherits "BaseLayer" class var Freezer = Class(BaseLayer, [ function () { this.$super("FREEZER"); // call super with unique layer ID this.isActive = false; // set background to be 100% transparent this.setBackground(null); }, function layerKeyPressed(code, mask) { var rm = KeyEvent.CTRL + KeyEvent.SHIFT + KeyEvent.ALT; if ((rm & mask) == rm) { // CTRL+SHIFT+ALT keys combination has been pressed if (this.isActive) this.setBackground(null); else this.setBackground(new Fill(255,255,255, 0.7)); this.isActive = ! this.isActive; } }, // methods below indicate if the layer is in active state (take control) function isLayerActive(){ return this.isActive;}, function isLayerActiveAt(x,y){return this.isActive; } ]); ... // add the layer to zebra canvas zCanvas.add(new Freezer()); The result of the layer work is demonstrated below: Un-frozen UI Frozen UI Layout management. Layout specifies a rule that says how to shape a number of child components on the given panel. Rule-based positioning is much better than absolute locations and fixed size usage. Gained advantages are the same as "vector vs pixel graphics". Zebra layout managers are independent from the Zebra UI package and can be re-used to layout other objects, for instance HTML elements. Let’s see how, for instance, diagonal layouts can be implemented. The custom layout manager lays out children components aligning its top left corners to diagonal: // declare diagonal layout manager var DiagonalLayout = Class(Layout, [ // "layout" method positions and shapes visible // children of target component function layout(target) { var x = 0, y = 0; for (var i=0; i

Here are some notes on adding Camel (from v2.3+) exception handling to a JavaDSL route. There are various approaches/options available. These notes cover the important distinctions between approaches... Default handling The default mode uses the DefaultErrorHandler strategy which simply propagates any exception back to the caller and ends the route immediately. This is rarely the desired behavior, at the very least, you should define a generic/global exception handler to log the errors and put them on a queue for further analysis (during development, testing, etc). onException(Exception) .to("log:GeneralError?level=ERROR") .to("activemq:queue:GeneralErrorQueue"); Try-catch-finally This approach mimics the Java for exception handling and is designed to be very readable and easy to implement. It inlines the try/catch/finally blocks directly in the route and is useful for route specific error handling. from("direct:start") .doTry() .process(new MyProcessor()) .doCatch(Exception.class) .to("mock:error"); .doFinally() .to("mock:end"); onException This approach defines the exception clause separately from the route. This makes the route and exception handling code more readable and reusable. Also, the exception handling will apply to any routes defined in its CamelContext. onException(Exception.class) .to("mock:error"); from("direct:start") .process(new MyProcessor()) .to("mock:end"); Handled/Continued These APIs provide valuable control over the flow. Adding handled(true) tells Camel to not propagate the error back to the caller (should almost always be used). The continued(true) tells Camel to resume the route where it left off (rarely used, but powerful). These can both be used to control the flow of the route in interesting ways, for example... from("direct:start") .process(new MyProcessor()) .to("mock:end"); //send the exception back to the client (rarely used, clients need a meaningful response) onException(ClientException.class) .handled(false) //default .log("error sent back to the client"); //send a readable error message back to the client and handle the error internally onException(HandledException.class) .handled(true) .setBody(constant("error")) .to("mock:error"); //ignore the exception and continue the route (can be dangerous, use wisely) onException(ContinuedException.class) .continued(true); Using a processor for more control If you need more control of the handler code, you can use an inline Processor to get a handle to the exception that was thrown and write your own handler code... onException(Exception.class) .handled(true) .process(new Processor() { public void process(Exchange exchange) throws Exception { Exception exception = (Exception) exchange.getProperty(Exchange.EXCEPTION_CAUGHT); //log, email, reroute, etc. } }); Summary Overall, the exception handling is very flexible and can meet almost any scenario you can come up with. For the sake of focusing on the basics, many advanced features haven't been covered here. For more details, see these pages on Camel's site... http://camel.apache.org/error-handling-in-camel.html http://camel.apache.org/try-catch-finally.html http://camel.apache.org/exception-clause.html

In a previous post I demonstrated how EclipseLink MOXy can be leveraged to produce both XML and JSON representations of your domain model. The same metadata is used for both representations and MOXy applies it to leverage the capabilities of the media type. In this post I'll focus on how null is handled in each of these representations. Domain Model By default a JAXB (JSR-222) implementation will not include a mapped field/property with a null value in the output. If you want the null value represented then you simply add the following annotation @XmlElement(nillable=true). package blog.json.nillable; import javax.xml.bind.annotation.*; @XmlRootElement public class Customer { private String firstName; private String middleName; private String lastName; public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getMiddleName() { return middleName; } public void setMiddleName(String middleName) { this.middleName = middleName; } @XmlElement(nillable=true) public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } } Demo In the demo code below we will set both the middleName and lastName properties to null. Since we have mapped these properties differently in the domain model we will examine the output to see the impact of using @XmlElement(nillable=true). package blog.json.nillable; import javax.xml.bind.*; public class Demo { public static void main(String[] args) throws Exception { Customer customer = new Customer(); customer.setFirstName("Jane"); customer.setMiddleName(null); customer.setLastName(null); JAXBContext jc = JAXBContext.newInstance(Customer.class); Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); // Output XML marshaller.marshal(customer, System.out); // Output JSON marshaller.setProperty("eclipselink.media-type", "application/json"); marshaller.marshal(customer, System.out); } } XML Output By default JAXB implementations do not include null values in the output, so there is no element corresponding to the middleName property. Since we annotated the lastName property with @XmlElement(nillable=true) and it had a null value, it is represented in the output. In XML an element with a null value is represented by adding the xsi:nil="true" attribute. Jane JSON Output Just like in the XML output, an entry for the middleName property does not appear in the JSON output. Since we annotated the lastName property with @XmlElement(nillable=true) and it had a null value, it is represented in the output. In JSON a null value is represented with null. { "customer" : { "firstName" : "Jane", "lastName" : null } }

Do you know what day of the week was the day you were born? Monday or maybe Saturday? Well, perhaps you know that. Everybody knows the day he’s born on, but do you know what day was the 31st of January in 1883? No? Well, there must be some method to determine any day in any century. We know that 2012 started at Sunday. After we know that, it’s easy to determine what day is the 2nd of January. It should be Monday. But things get a little more complex if we try to guess some date distant from January the 1st. Indeed 1st of Jan was on Sunday, but what day is 9th of May the same year. This is far more difficult to say. Of course we can go with a brute force approach and count from 1/Jan till 9/May, but that is quite slow and error prone. So what would we do if we had to code a program that answers this question? The easiest way is to use a library. Almost every major library has built-in functions that can answer what day is on a given date. Such are date() in PHP or getDate() in JavaScript. But the question remains: How these library functions know the answer and how can we code such library functions if our library doesn’t support such functionality? There must be some algorithm to help us. Overview Because months have different number of days, and most of them aren’t divisible by 7 without a remainder, months begin on different days of the week. Thus, if January begins on Sunday, the month of February the same year will begin on Wednesday. Of course, in common years February has 28 days, which fortunately is divisible by 7 and thus February and March both begin on the same day, which is great, but isn’t true for leap years. What Do We Know About the Calendar First thing to know is that each week has exactly 7 days. We also know that a common year has 365 days, while a leap year has one day more – 366. Most of the months have 30 or 31 days, but February has only 28 days in common years and 29 in leap years. Because 365 mod 7 = 1 in a common year each year begins exactly on the next day of the preceding year. Thus if 2011 started on Saturday, 2012 starts on Sunday. And yet again, that is because 2011 is not a leap year. What else do we know? Because a week has exactly seven days only February (with its 28 days in a common year) is divisible by 7 (28 mod 7 = 0) and has exactly four weeks in it. Thus in a common year February and March start on a same day. Unfortunately that is not true about the other months. All these things we know about the calendar are great, so we can make some conclusions. Although eleven of the months have either 30 or 31 days they don’t start on a same day, but some of the months do appear to start on a same day just because the number of days between them is divisible by 7 without a remainder. Let’s take a look on some examples. For instance September has 30 days, as does November, while October, which is in between them has 31 days. Thus 30+30+31 makes 91. Fortunately 91 mod 7 = 0. So for each year September and December start on the same day (as they are after February they don’t depend on leap years). The same thing occurs to April and July and the good news is that in leap years even January starts on the same day as April and July. Now we know that there are some relations between months. Thus, if we know somehow that the 13th of April is Monday, we’ll be sure that 13th of July is also Monday. Let’s see now a summary of these observations. We can also refer to the following diagram. For leap years there are other corresponding months. Let’s take a look at the following image. Another way to get the same information is the following table. We also know that leap years happen to occur once every four years. However, if there is a common year like the year 2001, which will be the next year that is common and starts and corresponds exactly on 2001? Because of leap years we can have a year starting on one of the seven days of the week and to be either leap or common. This means just 14 combinations. Following these observations we can refer to the following table. You can clearly see the pattern “6 4 2 0” Here’s the month table. Columns 2 and 3 differs only for January and February. Clearly the day table is as follows: Now let’s go back to the algorithm. Using these tables and applying a simple formula, we can calculate what day was on some given date. Here are the steps of this algorithm. Get the number for the corresponding century from the centuries table; Get the last two digits from the year; Divide the number from step 2 by 4 and get it without the remainder; Get the month number from the month table; Sum the numbers from steps 1 to 4; Divide it by 7 and take the remainder; Find the result of step 6 in the days table; Implementation First let’s take a look at a simple and practical example of the example above and then the code. Let’s answer the question from the first paragraph of this post. What day was on January 31st, 1883? Take a look at the centuries table: for 1800 – 1899 this is 2. Get the last two digits from the year: 83. Divide 83 by 4 without a remainder: 83/4 = 20 Get the month number from the month table: Jan = 0. Sum the numbers from steps 1 to 4: 2 + 83 + 20 + 0 = 105. Divide it by 7 and take the remainder: 105 mod 7 = 0 Find the result of step 6 in the days table: Sunday = 0. The following code in PHP implements the algorithm above. function get_century_code($century) { // XVIII if (1700 <= $century && $century <= 1799) return 4; // XIX if (1800 <= $century && $century <= 1899) return 2; // XX if (1900 <= $century && $century <= 1999) return 0; // XXI if (2000 <= $century && $century <= 2099) return 6; // XXII if (2100 <= $century && $century <= 2199) return 4; // XXIII if (2200 <= $century && $century <= 2299) return 2; // XXIV if (2300 <= $century && $century <= 2399) return 0; // XXV if (2400 <= $century && $century <= 2499) return 6; // XXVI if (2500 <= $century && $century <= 2599) return 4; // XXVII if (2600 <= $century && $century <= 2699) return 2; } /** * Get the day of a given date * * @param $date */ function get_day_from_date($date) { $months = array( 1 => 0,// January 2 => 3,// February 3 => 3,// March 4 => 6,// April 5 => 1,// May 6 => 4,// June 7 => 6,// July 8 => 2,// August 9 => 5,// September 10 => 0,// October 11 => 3,// November 12 => 5,// December ); $days = array( 0 => 'Sunday', 1 => 'Monday', 2 => 'Tuesday', 3 => 'Wednesday', 4 => 'Thursday', 5 => 'Friday', 6 => 'Saturday', ); // calculate the date $dateParts = explode('-', $date); $century = substr($dateParts[2], 0, 2); $year = substr($dateParts[2], 2); // 1. Get the number for the corresponding century from the centuries table $a = get_century_code($dateParts[2]); // 2. Get the last two digits from the year $b = $year; // 3. Divide the number from step 2 by 4 and get it without the remainder $c = floor($year / 4); // 4. Get the month number from the month table $d = $months[$dateParts[1]]; // 5. Sum the numbers from steps 1 to 4 $e = $a + $b + $c + $d; // 6. Divide it by 7 and take the remainder $f = $e % 7; // 7. Find the result of step 6 in the days table return $days[$f]; } // Sunday echo get_day_from_date('31-1-1883'); Application This algorithm can be applied in many different cases although most of the libraries have built-in functions that can do that. The only problem besides that is that there are much more efficient algorithms that don’t need additional space (tables) of data. However this algorithm isn’t difficult to implement and it gives a good outlook of some facts in the calendar.

How to create a real-time face detection system using HTML5, JavaScript, and OpenCV, leveraging WebRTC for webcam access and WebSockets for client-server communication.

This post shows code examples in Python (2.7) for sending data to Graphite. Once you have a Graphite server setup, with Carbon running/collecting, you need to send it data for graphing. Basically, you write a program to collect numeric values and send them to Graphite's backend aggregator (Carbon). To send data, you create a socket connection to the graphite/carbon server and send a message (string) in the format: "metric_path value timestamp\n" `metric_path`: arbitrary namespace containing substrings delimited by dots. The most general name is at the left and the most specific is at the right. `value`: numeric value to store. `timestamp`: epoch time. messages must end with a trailing newline. multiple messages maybe be batched and sent in a single socket operation. each message is delimited by a newline, with a trailing newline at the end of the message batch. Example message: "foo.bar.baz 42 74857843\n" Let's look at some (Python 2.7) code for sending data to graphite... Here is a simple client that sends a single message to graphite. Code: #!/usr/bin/env python import socket import time CARBON_SERVER = '0.0.0.0' CARBON_PORT = 2003 message = 'foo.bar.baz 42 %d\n' % int(time.time()) print 'sending message:\n%s' % message sock = socket.socket() sock.connect((CARBON_SERVER, CARBON_PORT)) sock.sendall(message) sock.close() Here is a command line client that sends a single message to graphite: Usage: $ python client-cli.py metric_path value Code: #!/usr/bin/env python import argparse import socket import time CARBON_SERVER = '0.0.0.0' CARBON_PORT = 2003 parser = argparse.ArgumentParser() parser.add_argument('metric_path') parser.add_argument('value') args = parser.parse_args() if __name__ == '__main__': timestamp = int(time.time()) message = '%s %s %d\n' % (args.metric_path, args.value, timestamp) print 'sending message:\n%s' % message sock = socket.socket() sock.connect((CARBON_SERVER, CARBON_PORT)) sock.sendall(message) sock.close() Here is a client that collects load average (Linux-only) and sends a batch of 3 messages (1min/5min/15min loadavg) to graphite. It will run continuously in a loop until killed. (adjust the delay for faster/slower collection interval): #!/usr/bin/env python import platform import socket import time CARBON_SERVER = '0.0.0.0' CARBON_PORT = 2003 DELAY = 15 # secs def get_loadavgs(): with open('/proc/loadavg') as f: return f.read().strip().split()[:3] def send_msg(message): print 'sending message:\n%s' % message sock = socket.socket() sock.connect((CARBON_SERVER, CARBON_PORT)) sock.sendall(message) sock.close() if __name__ == '__main__': node = platform.node().replace('.', '-') while True: timestamp = int(time.time()) loadavgs = get_loadavgs() lines = [ 'system.%s.loadavg_1min %s %d' % (node, loadavgs[0], timestamp), 'system.%s.loadavg_5min %s %d' % (node, loadavgs[1], timestamp), 'system.%s.loadavg_15min %s %d' % (node, loadavgs[2], timestamp) ] message = '\n'.join(lines) + '\n' send_msg(message) time.sleep(DELAY) Resources: Graphite Docs Graphite Docs - Getting Your Data Into Graphite Installing Graphite 0.9.9 on Ubuntu 12.04 LTS Installing and configuring Graphite END

In this article we'll look at how you can connect from Scala to RabbitMQ so you can support the AMQP protocol from your applications. In this example I'll use the Play Framework 2.0 as container (for more info on this see my other article on this subject) to run the application in, since Play makes developing with Scala a lot easier. This article will also use Akka actors to send and receive the messages from RabbitMQ. What is AMQP First, a quick introduction into AMQP. AMQP stands for "Advanced Message Queueing Protocol" and is an open standard for messaging. The AMQP homepage states their vision as this: "To become the standard protocol for interoperability between all messaging middleware". AMQP defines a transport level protocol for exchanging messages that can be used to integrate applications from a number of different platform, languages and technologies. There are a number of tools implementing this protocol, but one that is getting more and more attention is RabbitMQ. RabbitMQ is an open source, erlang based message broker that uses AMQP. All application that can speak AMQP can connect to and make use of RabbitMQ. So in this article we'll show how you can connect from your Play2/Scala/Akka based application to RabbitMQ. In this article we'll show you how to do implement the two most common scenarios: Send / recieve: We'll configure one sender to send a message every couple of seconds, and use two listeners that will read the messages, in a round robin fashion, from the queue. Publish / subscribe: For this example we'll create pretty much the same scenario, but this time, the listeners will both get the message at the same time. I assume you've got an installation of RabbitMQ. If not follow the instructions from their site. Setup basic Play 2 / Scala project For this example I created a new Play 2 project. Doing this is very easy: [email protected]:~/Dev/play-2.0-RC2$ ./play new Play2AndRabbitMQ _ _ _ __ | | __ _ _ _| | | '_ \| |/ _' | || |_| | __/|_|\____|\__ (_) |_| |__/ play! 2.0-RC2, http://www.playframework.org The new application will be created in /Users/jos/Dev/play-2.0/PlayAndRabbitMQ What is the application name? > PlayAndRabbitMQ Which template do you want to use for this new application? 1 - Create a simple Scala application 2 - Create a simple Java application 3 - Create an empty project > 1 OK, application PlayAndRabbitMQ is created. Have fun! I am used to work from Eclipse with the scala-ide pluging, so I execute play eclipsify and import the project in Eclipse. The next step we need to do is set up the correct dependencies. Play uses sbt for this and allows you to configure your dependencies from the build.scala file in your project directory. The only dependency we'll add is the java client library from RabbitMQ. Even though Lift provides a scala based AMQP library, I find using the RabbitMQ one directly just as easy. After adding the dependency my build.scala looks like this: import sbt._ import Keys._ import PlayProject._ object ApplicationBuild extends Build { val appName = "PlayAndRabbitMQ" val appVersion = "1.0-SNAPSHOT" val appDependencies = Seq( "com.rabbitmq" % "amqp-client" % "2.8.1" ) val main = PlayProject(appName, appVersion, appDependencies, mainLang = SCALA).settings( ) } Add rabbitMQ configuration to the config file For our examples we can configure a couple of things. The queue where to send the message to, the exchange to use, and the host where RabbitMQ is running. In a real world scenario we would have more configuration options to set, but for this case we'll just have these three. Add the following to your application.conf so that we can reference it from our application. #rabbit-mq configuration rabbitmq.host=localhost rabbitmq.queue=queue1 rabbitmq.exchange=exchange1 We can now access these configuration files using the ConfigFactory. To allow easy access create the following object: object Config { val RABBITMQ_HOST = ConfigFactory.load().getString("rabbitmq.host"); val RABBITMQ_QUEUE = ConfigFactory.load().getString("rabbitmq.queue"); val RABBITMQ_EXCHANGEE = ConfigFactory.load().getString("rabbitmq.exchange"); } Initialize the connection to RabbitMQ We've got one more object to define before we'll look at how we can use RabbitMQ to send and receive messages. to work with RabbitMQ we require a connection. We can get a connection to a server by using a ConnectionFactory. Look at the javadocs for more information on how to configure the connection. object RabbitMQConnection { private val connection: Connection = null; /** * Return a connection if one doesn't exist. Else create * a new one */ def getConnection(): Connection = { connection match { case null => { val factory = new ConnectionFactory(); factory.setHost(Config.RABBITMQ_HOST); factory.newConnection(); } case _ => connection } } } Start the listeners when the application starts We need to do one more thing before we can look at the RabbitMQ code. We need to make sure our message listeners are registered on application startup and our senders start sending. Play 2 provides a GlobalSettings object for this which you can extend to execute code when your application starts. For our example we'll use the following object (remember, this needs to be stored in the default namespace: import play.api.mvc._ import play.api._ import rabbitmq.Sender object Global extends GlobalSettings { override def onStart(app: Application) { Sender.startSending } } We'll look at this Sender.startSending operation, which initializes all the senders and receivers in the following sections. Setup send and receive scenario Let's look at the Sender.startSending code that will setup a sender that sends a msg to a specific queue. For this we use the following piece of code: object Sender { def startSending = { // create the connection val connection = RabbitMQConnection.getConnection(); // create the channel we use to send val sendingChannel = connection.createChannel(); // make sure the queue exists we want to send to sendingChannel.queueDeclare(Config.RABBITMQ_QUEUE, false, false, false, null); Akka.system.scheduler.schedule(2 seconds, 1 seconds , Akka.system.actorOf(Props( new SendingActor(channel = sendingChannel, queue = Config.RABBITMQ_QUEUE))) , "MSG to Queue"); } } class SendingActor(channel: Channel, queue: String) extends Actor { def receive = { case some: String => { val msg = (some + " : " + System.currentTimeMillis()); channel.basicPublish("", queue, null, msg.getBytes()); Logger.info(msg); } case _ => {} } } In this code we take the following steps: Use the factory to retrieve a connection to RabbitMQ Create a channel on this connection to use in communicating with RabbitMQ Use the channel to create the queue (if it doesn't exist yet) Schedule Akka to send a message to an actor every second. This all should be pretty straightforward. The only (somewhat) complex part is the scheduling part. What this schedule operation does is this. We tell Akka to schedule a message to be sent to an actor. We want a 2 seconds delay before it is fired, and we want to repeat this job every second. The actor that should be used for this is the SendingActor you can also see in this listing. This actor needs access to a channel to send a message and this actor also needs to know where to send the message it receives to. This is the queue. So every second this Actor will receive a message, append a timestamp, and use the provided channel to send this message to the queue: channel.basicPublish("", queue, null, msg.getBytes());. Now that we send a message each second it would be nice to have listeners on this queue that can receive messages. For receiving messages we've also created an Actor that listens indefinitely on a specific queue. class ListeningActor(channel: Channel, queue: String, f: (String) => Any) extends Actor { // called on the initial run def receive = { case _ => startReceving } def startReceving = { val consumer = new QueueingConsumer(channel); channel.basicConsume(queue, true, consumer); while (true) { // wait for the message val delivery = consumer.nextDelivery(); val msg = new String(delivery.getBody()); // send the message to the provided callback function // and execute this in a subactor context.actorOf(Props(new Actor { def receive = { case some: String => f(some); } })) ! msg } } } This actor is a little bit more complex than the one we used for sending. When this actor receives a message (kind of message doesn't matter) it starts listening on the queue it was created with. It does this by creating a consumer using the supplied channel and tells the consumers to start listening on the specified queue. The consumer.nextDelivery() method will block until a message is waiting in the configured queue. Once a message is received, a new Actor is created to which the message is sent. This new actor passes the message on to the supplied method, where you can put your business logic. To use this listener we need to supply the following arguments: Channel: Allows access to RabbitMQ Queue: The queue to listen to for messages f: The function that we'll execute when a message is received. The final step for this first example is glueing everything together. We do this by adding a couple of method calls to the Sender.startSending method. def startSending = { ... val callback1 = (x: String) => Logger.info("Recieved on queue callback 1: " + x); setupListener(connection.createChannel(),Config.RABBITMQ_QUEUE, callback1); // create an actor that starts listening on the specified queue and passes the // received message to the provided callback val callback2 = (x: String) => Logger.info("Recieved on queue callback 2: " + x); // setup the listener that sends to a specific queue using the SendingActor setupListener(connection.createChannel(),Config.RABBITMQ_QUEUE, callback2); ... } private def setupListener(receivingChannel: Channel, queue: String, f: (String) => Any) { Akka.system.scheduler.scheduleOnce(2 seconds, Akka.system.actorOf(Props(new ListeningActor(receivingChannel, queue, f))), ""); } In this code you can see that we define a callback function, and use this callback function, together with the queue and the channel to create the ListeningActor. We use the scheduleOnce method to start this listener in a separate thread. Now with this code in place we can run the application (play run) open up localhost:9000 to start the application and we should see something like the following output. [info] play - Starting application default Akka system. [info] play - Application started (Dev) [info] application - MSG to Exchange : 1334324531424 [info] application - MSG to Queue : 1334324531424 [info] application - Recieved on queue callback 2: MSG to Queue : 1334324531424 [info] application - MSG to Exchange : 1334324532522 [info] application - MSG to Queue : 1334324532522 [info] application - Recieved on queue callback 1: MSG to Queue : 1334324532522 [info] application - MSG to Exchange : 1334324533622 [info] application - MSG to Queue : 1334324533622 [info] application - Recieved on queue callback 2: MSG to Queue : 1334324533622 [info] application - MSG to Exchange : 1334324534722 [info] application - MSG to Queue : 1334324534722 [info] application - Recieved on queue callback 1: MSG to Queue : 1334324534722 [info] application - MSG to Exchange : 1334324535822 [info] application - MSG to Queue : 1334324535822 [info] application - Recieved on queue callback 2: MSG to Queue : 1334324535822 Here you can clearly see the round-robin way messages are processed. Setup publish and subscribe scenario Once we've got the above code running, adding publish / subscribe functionality is very trivial. Instead of the SendingActor we now use a PublishingActor: class PublishingActor(channel: Channel, exchange: String) extends Actor { /** * When we receive a message we sent it using the configured channel */ def receive = { case some: String => { val msg = (some + " : " + System.currentTimeMillis()); channel.basicPublish(exchange, "", null, msg.getBytes()); Logger.info(msg); } case _ => {} } } An exchange is used by RabbitMQ to allow multiple recipients to receive the same message (and a whole lot of other advanced functionality). The only change in the code from the other actor is that this time we send the message to an exchange instead of to a queue. The listener code is exactly the same, the only thing we need to do is connect a queue to a specific exchange. So that listeners on that queue receive the messages sent to to the exchange. We do this, once again, from the setup method we used earlier. ... // create a new sending channel on which we declare the exchange val sendingChannel2 = connection.createChannel(); sendingChannel2.exchangeDeclare(Config.RABBITMQ_EXCHANGEE, "fanout"); // define the two callbacks for our listeners val callback3 = (x: String) => Logger.info("Recieved on exchange callback 3: " + x); val callback4 = (x: String) => Logger.info("Recieved on exchange callback 4: " + x); // create a channel for the listener and setup the first listener val listenChannel1 = connection.createChannel(); setupListener(listenChannel1,listenChannel1.queueDeclare().getQueue(), Config.RABBITMQ_EXCHANGEE, callback3); // create another channel for a listener and setup the second listener val listenChannel2 = connection.createChannel(); setupListener(listenChannel2,listenChannel2.queueDeclare().getQueue(), Config.RABBITMQ_EXCHANGEE, callback4); // create an actor that is invoked every two seconds after a delay of // two seconds with the message "msg" Akka.system.scheduler.schedule(2 seconds, 1 seconds, Akka.system.actorOf(Props( new PublishingActor(channel = sendingChannel2 , exchange = Config.RABBITMQ_EXCHANGEE))), "MSG to Exchange"); ... We also created an overloaded method for setupListener, which, as an extra parameter, also accepts the name of the exchange to use. private def setupListener(channel: Channel, queueName : String, exchange: String, f: (String) => Any) { channel.queueBind(queueName, exchange, ""); Akka.system.scheduler.scheduleOnce(2 seconds, Akka.system.actorOf(Props(new ListeningActor(channel, queueName, f))), ""); } In this small piece of code you can see that we bind the supplied queue (which is a random name in our example) to the specified exchange. After that we create a new listener as we've seen before. Running this code now will result in the following output: [info] play - Application started (Dev) [info] application - MSG to Exchange : 1334325448907 [info] application - MSG to Queue : 1334325448907 [info] application - Recieved on exchange callback 3: MSG to Exchange : 1334325448907 [info] application - Recieved on exchange callback 4: MSG to Exchange : 1334325448907 [info] application - MSG to Exchange : 1334325450006 [info] application - MSG to Queue : 1334325450006 [info] application - Recieved on exchange callback 4: MSG to Exchange : 1334325450006 [info] application - Recieved on exchange callback 3: MSG to Exchange : 1334325450006 As you can see, in this scenario both listeners receive the same message. That pretty much wraps it up for this article. As you've seen using the Java based client api for RabbitMQ is more than sufficient, and easy to use from Scala. Note though that this example is not production ready, you should take care to close connections, nicely shutdown listeners and actors. All this shutdown code isn't shown here.

our new tutorial tells us how to create an animated 3d sphere (through direct access to pixels on the canvas). the sphere itself moves around the canvas continuously. this example should work in most modern browsers (like firefox, chrome, safari and even in ie). in the end, you should to get something like this: here are our demo and downloadable package: live demo download in package ok, download the source files and let's start coding ! step 1. html this is the markup of our page. index.html i prepared 2 canvas objects here: the first for the source image, and the second one for our sphere. step 2. css css/main.css .container { height: 631px; margin: 50px auto; position: relative; width: 1024px; z-index: 1; } #obj { position: absolute; z-index: 2; } we should put our sphere object above our main canvas. step 3. js js/script.js var canvas, ctx; var canvasobj, ctxobj; var idstw = 256; var idsth = 256; var ixspeed = 4; var iyspeed = 3; var ilastx = idstw / 2; var ilasty = idsth / 2; var oimage; var amap = []; var abitmap; var mathsphere = function(px, py) { var x = px - idstw / 2; var y = py - idsth / 2; var r = math.sqrt(x * x + y * y); var maxr = idstw / 2; if (r > maxr) return {'x':px, 'y':py}; var a = math.atan2(y, x); var k = (r / maxr) * (r / maxr) * 0.5 + 0.5; var dx = math.cos(a) * r * k; var dy = math.sin(a) * r * k; return {'x': dx + idstw / 2, 'y': dy + idsth / 2}; } window.onload = function(){ // load background oimage = new image(); oimage.src="images/bg.jpg"; oimage.onload = function () { // creating canvas and context objects canvas = document.getelementbyid('slideshow'); ctx = canvas.getcontext('2d'); canvasobj = document.getelementbyid('obj'); ctxobj = canvasobj.getcontext('2d'); // clear context ctx.clearrect(0, 0, ctx.canvas.width, ctx.canvas.height); // and draw source image ctx.drawimage(oimage, 0, 0); abitmap = ctx.getimagedata(0, 0, idstw, idsth); for (var y = 0; y < idsth; y++) { for (var x = 0; x < idstw; x++) { var t = mathsphere(x, y); amap[(x + y * idsth) * 2 + 0] = math.max(math.min(t.x, idstw - 1), 0); amap[(x + y * idsth) * 2 + 1] = math.max(math.min(t.y, idsth - 1), 0); } } // begin updating scene updatescene(); }; function updatescene() { // update last coordinates ilastx = ilastx + ixspeed; ilasty = ilasty + iyspeed; // reverse speed if (ilastx > ctx.canvas.width - idstw/2) { ixspeed = -3; } if (ilastx < idstw/2) { ixspeed = 3; } if (ilasty > ctx.canvas.height - idsth/2) { iyspeed = -3; } if (ilasty < idsth/2) { iyspeed = 3; } // shifting of the second object canvasobj.style.left = ilastx - math.floor(idstw / 2) + 'px'; canvasobj.style.top = ilasty - (math.floor(idsth / 2)) + 'px'; // draw result sphere var adata = ctx.getimagedata(ilastx - math.ceil(idstw / 2), ilasty - math.ceil(idsth / 2), idstw, idsth + 1); for (var j = 0; j < idsth; j++) { for (var i = 0; i < idstw; i++) { var u = amap[(i + j * idsth) * 2]; var v = amap[(i + j * idsth) * 2 + 1]; var x = math.floor(u); var y = math.floor(v); var kx = u - x; var ky = v - y; for (var c = 0; c < 4; c++) { abitmap.data[(i + j * idsth) * 4 + c] = (adata.data[(x + y * idsth) * 4 + c] * (1 - kx) + adata.data[((x + 1) + y * idsth) * 4 + c] * kx) * (1-ky) + (adata.data[(x + (y + 1) * idsth) * 4 + c] * (1 - kx) + adata.data[((x + 1) + (y + 1) * idsth) * 4 + c] * kx) * (ky); } } } ctxobj.putimagedata(abitmap,0,0); // update timer settimeout(updatescene, 16); } }; during initialization, the script prepares two canvas objects and two contexts. then, it loads our main background image, and draws it as our first context. then it prepares a hash table of sphere transformations: amap. and, in the end – it starts the timer, which updates the main scene. in this function (updatescene) we update the coordinates of our sphere object, and draw the updated sphere at our second location. live demo download in package conclusion i hope that today’s 3d html5 sphere lesson has been interesting for you. we have done another nice html5 example. i will be glad to see your thanks and comments. good luck!

The foldLeft() method, available for all collections in Scala, allows a given 2-argument function to run against consecutive elements of that collection, where the result of that function is passed as the first argument in the next invocation. Second argument is always the current item in the collection. Doesn't sound very encouraging but as we will see soon there are great some use-cases waiting to be discovered. Before we dive into foldLeft, let us have a look at reduce - simplified version of foldLeft. I always believed that a working code is worth a thousand words: val input = List(3, 5, 7, 11) input.reduce((total, cur) => total + cur) or more readable: def op(total: Int, cur: Int) = total + cur input reduce op The result is 26 (sum). The code is more-or-less readable: to reduce method we are passing 2-argument function op (operation). Both parameters of that function (and its return value) need to have the same type as the collection. reduce() will invoke that operation on two first items of the collection: op(3, 5) //8 The result (8) is passed as a first argument to a subsequent invocation of op where the second argument is the next collection element: op(8, 7) //15 and finally: op(15, 11) //26 From the logical standpoint the following composed operation has been invoked: op(op(op(3, 5), 7), 11) When we realize that op() is basically an addition: (((3 + 5) + 7) + 11) So far so good - reduce() reduces a collection of a given type to a single value of the same type. Example use-cases include adding up numbers, concatenating a sequence of strings, etc.: List("Foo", "Bar", "Buzz").reduce(_ + _) Note the shorthand notation for code block without naming the parameters: _ + _. Obviously we are not limited to addition operator: def factorial(x: Int) = (2 to x).reduce(_ * _) It is worth to mention two special cases: when the collection has only one element, reduce() returns this very element. When it is empty, reduce() will throw an exception. Let's face it, typically we implement factorial for the first (and last) time somewhere at the beginning of the university and to add up numbers we have a convenience method: input.sum Besides the problem with empty collections is a bit painful - after all the sum of empty set of numbers is intuitively equal to 0 and the concatenation of an empty set of strings is... an empty string. Here is where foldLeft() enters with the ability to specify initial value: input.foldLeft(0)(op) In this case the op() function is first called with initial value 0 as the first argument and with the first collection element: op(0, 3) The subsequent iterations remain the same. If the collection is empty, foldLeft() returns the initial value. It is sad how many tutorial stop right here. After all we can simply prepend initial value to the input list and happily use reduce(): (0 :: input).reduce(op) (0 :: Nil).reduce(op) //empty list is prepended by 0 Even worse, many suggest “simplified" foldLeft() syntax, I doubt it simplifies anything: (0 /: input)(op) This is equivalent to input.foldLeft(0)(op) but intended for people who love Perl. So, closing this way too long introduction, let us see the true power behind foldLeft(). Let us assume that we have an object of type [T] on which we would like to perform a set of transformations. Transformation is nothing more than a function that accepts and returns an object of type [T]. We can return the same instance (no-op transformation), wrap the original object (the Decorator pattern) or mutate it. It is not hard to imagine that the order of transformations is important. For example let us use an ordinary string and set of transformations represented by functions of String => String: val reverse = (s: String) => s.reverse val toUpper = (s: String) => s.toUpperCase val appendBar = (s: String) => s + "bar" Remembering that a result of a first transformation is an argument of the second one we can say: appendBar(toUpper(reverse("foo"))) //OOFbar toUpper(reverse(appendBar("foo"))) //RABOOF I think that's obvious. Unfortunately we need a method taking an arbitrary (possibly empty or created dynamically) list of transformations to apply: def applyTransformations(initial: String, transformations: Seq[String => String]) = //??? applyTransformations("foo", List(reverse, toUpper, appendBar)) applyTransformations("foo", List(appendBar, reverse, toUpper)) applyTransformations("foo", List.fill(7)(appendBar)) The last line performs appendBar transformation 7 times on an initial value "foo". How to implement applyTransformations method? The programmer with highly imperative background would probably come up with something like this: def applyTransformations(initial: String, transformations: Seq[String => String]) = { var cur = initial for(transformation <- transformations) { cur = transformation(cur) } cur } Boring loop over all transformations, the intermediate result is stored in a variable. This implementation has several drawbacks. First - it's imperative (!) Scala tries to embrace the functional programming paradigm and this code seems very low-level. Our second take is much more idiomatic as far as Scala is concerned - we use recursion and pattern matching: @tailrec def applyTransformations(initial: String, transformations: Seq[String => String]): String = transformations match { case head :: tail => applyTransformations(head(initial), tail) case Nil => initial } A little bit harder to comprehend compared to imperative solution. If the list of transformations is empty - return current value. If it's not, apply the first transformation (head(initial)) and recursively call myself with the rest of the transformations (tail). Turns out this problem can be implemented in much, much more concise way, without explicit loops and recursion. Have you noticed how the problem with nested transformations (appendBar(toUpper(reverse("foo")))) is similar to how the foldLeft() works (op(op(op(3, 5), 7), 11))? def applyTransformations(initial: String, transformations: Seq[String => String]) = transformations.foldLeft(initial) { (cur, transformation) => transformation(cur) } Understanding how the code above works requires a little bit of time - but it is really rewarding afterwards. Also it allows you to fully grasp the power of foldLeft(). Before you go further try to figure this out yourself. Few tips: The type of foldLeft() result [B] doesn't necessarily have to be the same as the collection type [A]. It is the type of the initial value. In our example the input collection contains functions but the initial value is String. Function passed as an argument to foldLeft() does not need to accept both arguments of [A] type and return that type as well - as it was with reduce(). In fact, the signature of foldLeft() is as follows: def foldLeft[B](initial: B)(op: (B, A) => B): B The value returned by op function should be of the same type as its first argument. Also the whole foldLeft() invocation will have the same type. Let's think about it: the type of the first argument of op() is compatible with the initial value (initial: B) because in the first iteration it is the initial value that is passed as the first argument of op. A second argument is the first element of the input collection of type [A]. In the second iteration the result of op() invocation (of type [B]) is passed as the first argument of subsequent invocation of op. This time the second element of the input collection is used as the second argument. And it goes on until it reaches the end of the collection. I think the pseudo-code would be much easier to comprehend. First some example invocation: List(reverse, toUpper, appendBar).foldLeft("foo") { (cur, transformation) => transformation(cur) } Subsequent iterations (pseudo-code): val initial = "foo" val temp1 = (initial, reverse) => reverse(initial) val temp2 = (temp1, toUpper) => toUpper(temp1) val temp3 = (temp2, appendBar) => appendBar(temp2) And after inlining temporary variables: val initial = "foo" appendBar(toUpper(reverse(initial))) Isn't this the result we've been waiting for? As it turns out, foldLeft() is not only useful when we need to reduce (aggregate) collection to a single value, like adding up numbers - in fact, reduce() or sum() are better suited in this case. foldLeft() seems to be a great fit when we need to iterate over an arbitrary collection but every iteration requires some sort of result from previous one. By the way this is the reason why fold and reduce operations can't be executed in parallel. In comments to the original article Cezary Bartoszuk suggested an alternative way of using foldLeft() in this problem: def composeAll[A](ts: Seq[A => A]): A => A = ts.foldLeft(identity[A] _)(_ compose _) def applyTransformations(init: String, ts: Seq[String => String]): String = composeAll(ts.reverse)(init) If this solution isn't clear to your, once again few tips. First of all identity[A] _ is an identity function - always returning an argument untouched. Secondly val composed = appendBar compose toUpper is equivalent to: val composed = (s: String) => appendBar(toUpper(s)) So another mathematical term: function composition. This was a translation of my article "Ukryta potęga foldLeft()" originally published on scala.net.pl.

Below is an implementation of a Regular Expression HashMap. It works with key-value pairs which the key is a regular expression. It compiles the key (regular expression) while adding (i.e. putting), so there is no compile time while getting. Once getting an element, you don't give regular expression; you give any possible value of a regular expression. As a result, this behaviour provides to map numerous values of a regular expression into the same value. The class does not depend to any external libraries, uses only default java.util. So, it will be used simply when a behaviour like that is required. import java.util.ArrayList; import java.util.HashMap; import java.util.regex.Pattern; /** * This class is an extended version of Java HashMap * and includes pattern-value lists which are used to * evaluate regular expression values. If given item * is a regular expression, it is saved in regexp lists. * If requested item matches with a regular expression, * its value is get from regexp lists. * * @author cb * * @param : Key of the map item. * @param : Value of the map item. */ public class RegExHashMap extends HashMap { // list of regular expression patterns private ArrayList regExPatterns = new ArrayList(); // list of regular expression values which match patterns private ArrayList regExValues = new ArrayList(); /** * Compile regular expression and add it to the regexp list as key. */ @Override public V put(K key, V value) { regExPatterns.add(Pattern.compile(key.toString())); regExValues.add(value); return value; } /** * If requested value matches with a regular expression, * returns it from regexp lists. */ @Override public V get(Object key) { CharSequence cs = new String(key.toString()); for (int i = 0; i < regExPatterns.size(); i++) { if (regExPatterns.get(i).matcher(cs).matches()) { return regExValues.get(i); } } return super.get(key); } }

Recently I was working with a project where I was in need to format a number in such a way which can apply leading zero for particular format. So after doing such R and D I have found a great way to apply this leading zero format. I was having need that I need to pad number in 5 digit format. So following is a table in which format I need my leading zero format. 1-> 00001 20->00020 300->00300 4000->04000 50000->5000 So in the above example you can see that 1 will become 00001 and 20 will become 00200 format so on. So to display an integer value in decimal format I have applied interger.Tostring(String) method where I have passed “Dn” as the value of the format parameter, where n represents the minimum length of the string. So if we pass 5 it will have padding up to 5 digits. So let’s create a simple console application and see how its works. Following is a code for that. using System; namespace LeadingZero { class Program { static void Main(string[] args) { int a = 1; int b = 20; int c = 300; int d = 4000; int e = 50000; Console.WriteLine(string.Format("{0}------>{1}",a,a.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", b, b.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", c, c.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", d, d.ToString("D5"))); Console.WriteLine(string.Format("{0}------>{1}", e, e.ToString("D5"))); Console.ReadKey(); } } } As you can see in the above code I have use string.Format function to display value of integer and after using integer value’s ToString method. Now Let’s run the console application and following is the output as expected. Here you can see the integer number are converted into the exact output that we requires. That’s it you can see it’s very easy. We have written code in nice clean way and without writing any extra code or loop. Hope you liked it. Stay tuned for more.. Till than happy programming.

Here is an A - Z list of some Javascript idioms and patterns. The idea is to convey in simple terms some features of the actual Javascript language (rather than how it can interact with DOM). Enjoy... Array Literals An array literal can be defined using a comma separated list in square brackets. var months = ['jan', 'feb', 'mar', 'apr', 'may', 'jun', 'jul', 'aug', 'sep', 'oct', 'nov', 'dec']; console.log(months[0]); // outputs jan console.log(months.length) // outputs 12 Arrays in javascript have a wide selection methods including push() and pop(). Suppose the world got taken over by a dictator who wanted to get rid of the last month of the year? The dictator would just do... months.pop(); And of course, the dictator will eventually want to add a month after himself when everyone will have to worship him: months.push("me"); Callbacks Since functions are objects, they can be passed as arguments to other functions. function peakOil(callback) { //... code callback(); // the parentheses mean the function is executed! } function changeCivilisationCallback(){ //... } // Now pass the changeCivilisationCallback to peakOil. // Note: no changeCivilisationCallback parentheses because it is not // executed at this point. // It will be excuted later inside peak oil. peakOil(changeCivilisationCallback); In the example above, the chanceCivilisationCallback callback function is invoked by peakOil. Logic could be added to check if the energy returns from solar panels and wind farms were sufficient in which case another callback, other than changeCivilisationCallback could be added. Configuration Object Instead of passing around a bunch of related properties... function addCar(colour, wheelsize, regplate) {...} Use a configuration object function addCar(carConf) {...} var myCarConf = { colour: "blue", wheelsize: "32", regplate: "00D98788" }; addCar(myCarConf); The use of a configuration object makes it makes it easier to write clean APIs that don't need to take a huge long list of parameters. They also means you are less likely to get silly errors if parameters are in the wrong order. Closures There are three ways to creats objects in Javascript: using literals, using the constuctor function and by using a closure. What closures offer that the other two approaches do not is encapsulation. Closures make it possible to hide away functions and variables. var counter = function(count) { console.log(">> setting count to " + this.count); return { getCount: function(){ return ++count; } } } mycounter = counter(0); console.log(mycounter.getCount()); // outputs 1 console.log(mycounter.getCount()); // outputs 2 console.log(mycounter.getCount()); // outputs 3 console.log(mycounter.getCount()); // outputs 4 // Same again with offset this time. mycounterWithOffset = counter(10); console.log(mycounterWithOffset .getCount()); // outputs 11 console.log(mycounterWithOffset .getCount()); // outputs 12 console.log(mycounterWithOffset .getCount()); // outputs 13 console.log(mycounterWithOffset .getCount()); // outputs 14 Note: The closure is the object literal returned from annoymous function. It "closes" over the count variable. No-one can access it except for the closure. It is encapsulated. The closure also has a sense of state. Note also how the it maintains the value of the counter. Constructor Functions (Built in) There are no classes in Javascript but there are construtor functions which use the new keyword syntax similar to the class based object creation in Java or other languages. Javascript has some built-in constructor functions. These include Object(), Date(), String() etc. var person = new Object(); // person variable is an Object person.name = "alex"; // properties can then be dynamically added Constructor Functions (Custom) When a function is invoked with the keyword new, it is referred to as a Constructor function. The new means that the new object will have a hidden link to value of the function's prototype member and the this keyword will be bound to the new object. function MyConstrutorFunction() { this.goodblog = "dublintech.blogspot.com"; } var newObject = new MyConstrutorFunction(); console.log(typeof newObject); // "object" console.log(newObject.goodblog); // "dublintech.blogspot.com" var noNewObject = MyConstrutorFunction(); console.log(typeof noNewObject); // "undefined" console.log(window.tastes); // "yummy" The convention is that constructor functions should begin with a capital letter. Note: if the new keyword is not used, then the 'this' variable inside the function will refer to the global object. Can you smell a potential mess? Hence why the capital letter convention for constructor functions is used. The capital letter means: "I am a constructor function, please use the new keyword". Currying Currying is the process of reducing the number of arguments passed to a function by setting some argument(s) to predefined values. Consider this function. function outputNumbers(begin, end) { var i; for (i = begin; i <= end; i++) { print(i); } } outputNumbers(0, 5); // outputs 0, 1, 2, 3, 4, 5 outputNumbers(1, 5); // outputs 1, 2, 3, 4, 5 Suppose, we want a similar function with a fixed "begin" value. Let's say the "begin" value was always 1. We could do: function outputNumbersFixedStart(start) { return function(end) { return outputNumbers(start, end); } } And then define a variable to be this new function... var outputFromOne = outputNumbersFixedStart(1); outputFromOne(3); 1, 2, 3 outputFromOne(5); 1, 2, 3, 4, 5 Delete Operator The delete operator can be used to remove properties from objects and arrays. var person = {name: 'Alex', age: 56}; // damn I don't want them to know my age remove it delete person.age; console.log("name" in person); // outputs true because it is still there console.log("age" in person); // outputs false var colours = ['red', 'green', 'blue'] // is red really in the array? console.log(colours.indexOf('red') > -1); // outputs true. // remove red, it's going out of fashion! delete colours[colours.indexOf('red')]; console.log(colours.indexOf('red') > -1); // outputs false console.log(colours.length) // length is still three, remember it's javascript! You cannot delete global variables or prototype attributes. console.log(delete Object.prototype) // can't be deleted, outputs false function MyFunction() { // ... } console.log(delete MyFunction.prototype) // can't be deleted, outputs false var myglobalVar = 1; console.log(delete this.myglobalVar) // can't be delete, outputs false Dynamic Arguments Arguments for a function do not have to be specifed in the function definition function myFunction(){ // ... Note myfunction has no arguments in signature for(var i=0; i < arguments.length; i++){ alert(arguments[i].value); } } myFunction("tony", "Magoo"); // any argument can be specified The arguments parameter is an array available to functions and gives access to all arguments that were specified in the invocation. for-in iterations for-in loops (also called enumeration) should be used to iterate over nonarray objects. var counties = { dublin: "good", kildare: "not bad", cork: "avoid" } for (var i in counties) { if (counties.hasOwnProperty(i)) { // filter out prototype properties console.log(i, ":", counties[i]); } } Function declaration In a function declaration, the function stands on its own and does not need to be assigned to anything. function multiple(a, b) { return a * b; } // Note, no semi colon is needed Function expressions When function is defined as part of something else's definition, it is considered a function expression. multiply = function multiplyFunction(a, b) { return a * b; }; // Note the semi colan must be placed after the function definition console.log(multiply(5, 10)); // outputs 50 In the above example, the function is named. It can also be anonymous, in which case the name property will be a blank string. multiply = function (a, b) { return a * b; } // Note the semi colan must be placed after the function definition console.log(multiply(5, 10)); // outputs 50 Functional Inheritance Functional inheritance is mechanism of inheritance that provides encapsulation by using closures. Before trying to understand the syntax, take an example first. Suppose we want to represent planets in the solar system. We decided to have a planet base object and then several planet child objects which inherit from the base object. Here is the base planet object: var planet = function(spec) { var that = {}; that.getName = function() { return spec.radius; }; that.getNumberOfMoons()= function() { return spec.numberOfMoons; }; return that; } Now for some planets. Let's start with Earth and Jupiter and to amuse ourselves let's add a function for Earth for people to leave and a function to Jupiter for people arriving. Sarah Palin has taken over and things have got pretty bad!!! var earth = function(spec) { var that = planet{spec}; // No need for new keyword! that.peopleLeave = function() { // ... people leave } return that; } var jupiter = function(spec) { var that = planet(spec); that.peopleArrive = function() { // .. people arrive } return that; } Now put the earth and jupiter in motion... var myEarth = earth({name:"earth",numberofmoons:1}); var myjupiter=jupiter({name:"jupiter",numberofmoons:66}); The three key points here: There is code reuse. There is encapsulation. The name and numberOfMoons properties are encapsulated. The child objects can add in their own specific functionality. Now an explanation of the syntax: The base object planet accepts some data in the spec object. The base object planet creates a closures called that which is returned. The that object has access to everything in the spec object. But, nothing else does. This provides a layer of encapsulation. The child objects, earth and jupiter, set up their own data and pass it to base planet object. The planet object returns a closure which contains base functionality. The child classes receive this closure and add further methods and variables to it. Hoisting No matter where var's are declared in a function, javascript will "hoist" them meaning that they behave as if they were declared at the top of the function. mylocation = "dublin"; // global variable function outputPosition() { console.log(mylocation); // outputs "undefined" not "dublin" var mylocation = "fingal" ; console.log(mylocation); // outputs "fingal" } outputPosition(); In the function above, the var declaration in the function means that the first log will "see" the mylocation in the function scope and not the one declared in the global scope. After declaration, the local mylocation var will have the value "undefined", hence why this is outputted first. Functions that are assigned to variables can also be hoisted. The only difference being that when functions are hoisted, their definitions also are - not just their declarations. Immediate Function Expressions Immediate function expression are executed as soon as they are defined. (function() { console.log("I ain't waiting around"); }()); There are two aspects of the syntax to note here. Firsty, there is a () immediately after the function definiton, this makes it execute. Secondly, the function can only execute if it is a function expression as opposed to a function declaration. The outer () make the function an expression. Another way to define a an immediate function expression is: var anotherWay = function() { console.log("I ain't waiting around"); }() JSON JavaScript Object Notation (JSON) is a notation used to represent objects. It is very similar to the format used for Javascript Object literals except the property names must be wrapped in quotes. The JSON format is not exclusive to javascript; it can be used by any language (Python, Ruby etc). JSON makes it very easy to see what's an array and what's an object. In XML this would be much harder. An external document - such as XSD - would have to be consulted. In this example, Mitt Romney has an array describing who might vore for him and an object which is his son. {"name": "Mitt Romney", "party": "republicans", "scary": "of course", "romneysMostLikelyVoters": ["oilguzzlers", "conservatives"], son : {name:'George Romney} Loose typing Javascript is loosely typed. This means that variables do not need to be typed. It also means there is no complex class hierarchies and there is no casting. var number1 = 50; var number2 = "51"; function output(varToOutput) { // function does not care about what type the parameter passed is. console.log(varToOutput); } output(number1); // outputs 50 output(number2); // outputs 51 Memoization Memoization is a mechanism whereby functions can cache data from previous executions. function myFunc(param){ if (!myFunc.cache) { myFunc.cache = {}; // If the cache doesn't exist, create it. } if (!myFunc.cache[param]) { //... Imagine the code to work out result below // is computationally intensive. var result = { //... }; myFunc.cache[param] = result; // now result is cached. } return myFunc.cache[param]; } Method When a function is stored as a property of an object, it is referred to as a method. var myObject { myProperty: function () { //... // the this keyword in here will refer to the myObject instance. // This means the "method" can read and change variables in the // object. } } Modules The goal of modules is to enable javascript code bases to more modular. Functions and variables are collated into a module and then the module can decide what functions and what variables the outside world can see - in the same way as encapsulations works in the object orientated paradigms. In javascript we create modules by combining characteristics of closures and immediate function expressions. var bankAccountModule = (function moduleScope() { var balance = 0; //private function doSomethingPrivate(){ // private method //... } return { //exposed to public addMoney: function(money) { //... }, withDrawMoney: function(money) { //... }, getBalance: function() { return balance; } }()); In the example above, we have a bank account module: The function expression moduleScope has its own scope. The private variable balance and the private function doSomethingPrivate, exist only within this scope and are only visible to functions within this scope. The moduleScope function returns an object literal. This is a closure which has access to the private variables and functions of moduleScope. The returned object's properties are "public" and accesible to the outside world. The returned object is automatically assigned to bankAccountModule The immediate function ()) syntax is used. This means that the module is initialised immediately. Because the returned object (the closure) is assigned to bankAccountModule, it means we can access the bankAccountModule as: bankAccountModule.addMoney(20); bankAccoumtModule.withdrawMoney(15); By convention, the filename of a module should match its namespace. So in this example, the filename should be bankAccountModule.js. Namespace Pattern Javascript doesn't have namespaces built into the language, meaning it is easy for variables to clash. Unless variables are defined in a function, they are considered global. However, it is possible to use "." in variables names. Meaning you can pretend you have name spaces. DUBLINTECH.myName = "Alex" DUBLINTECH.myAddress = "Dublin Object Literal Notation In javascript you can define an object as collection of name value pairs. The values can be property values or functions. var ireland = { capital: "Dublin", getCapital: function () { return this.capital; } }; Prototype properties (inheritance) Every object has a prototype object. It is useful when you want to add a property to all instances of a particular object. Suppose you have a constructor function, which representent Irish people who bought in the boom. function IrishPersonBoughtInTheBoom(){ } var mary = new IrishPersonBoughtInTheBoom (); var tony = new IrishPersonBoughtInTheBoom (); var peter = new IrishPersonBoughtInTheBoom (); ... Now, the Irish economy goes belly up, the property bubble explodes and you want to add a debt property to all instances of this function. To do this you would do: IrishPersonBoughtInTheBoom.prototype.debt = "ouch"; Then... console.log(mary.debt); // outputs "ouch" console.log(tony.debt); // outputs "ouch" console.log(peter.debt); // outputs "ouch" Now, when this approach is used, all instances of IrishPersonBoughtInTheBoom share the save copy of the debt property. This means, that they all have the same value as illustrated in this example. Returning functions A function always returns a value. If return is not specified for a function, the undefined value type will be returned. Javascript functions can also return some data or another function. var counter = function() { //... var count = 0; return function () { return count = count + 1; } } var nextValue = counter(); nextValue(); // outputs 1 nextValue(); // outputs 2 Note, in this case the inner function which is returned "closes" over the count variable - making it a closure - since it encapsulates its own count variable. This means it gets its own copy which is different to the variable return by nextValue.count. this keyword The this keyword in Java has different meanings, depending on the context it is used. In summary: In a method context, this refers to the object that contains the method. In a function context, this refers to the global object. Unless the function is a property of another object. In which case the this refers to that object. If this is used in a constructor, the this in the constructor function refers to the object which uses the constructor function. When the apply or call methods are used the value of this refers to what was explictly specified in the apply or call invocation. typeof typeof is a unary operator with one operand. It is used to determine the types of things (a bit like getClass() in Java). The values outputted by typeof are "number", "string", "boolean", "undefined", "function", "object". console.log(typeof "tony"); // outputs string console.log(typeof 6); // outputs number console.log(false); // outputs boolean console.log(this.doesNotExist); // outputs undefined if the global scope has no such var console.log(typeof function(){}); // outputs function console.log(typeof {name:"I am an object"}); //outputs object console.log(typeof ["I am an array"]) // typedef outputs object for arrays console.log(typeof null) // typedef outputs object for nulls Some implementations return "object" for typeof for regular expressions; others return "function". But the biggest problem with typeof is that it returns object for null. To test for null, use strict equality... if (myobject === null) { ... } Self-redefining functions This is a good performance technique. Suppose you have a function and the first time it is called you want it to perform some set up code that you never want to perfom again. You can execute the set up code and then make the function redefine itself after that so that the setup code is never re-excuted. var myFunction = function () { //set up code only to this once alert("set up, only called once"); // set up code now complete. // redefine function so that set up code is not re-executed myFunction = function() { alert("no set up code"); } } myFunction(); // outputs - Set up, only called once myFunction(); // outputs - no set up code this time myFunction(); // outputs - no set up code this time Note, any properties added to the set up part of this function will be lost when the function redefines itself. In addition, if this function is used with a different name (i.e. it is assigned to a variable), the re-definition will not happen and the set up code will re-execute. Scope In javascript there is a global scope and a function scope available for variables. The var keyword does not need to be used to define variable in the global scope but it must be used to define variable in the local function scope. When a variable is scoped to a local function shares the name with a global variable, the local scope takes precedence - unless var was not used to declare the local variable in which case any local references are pointing to the global reference. There is no block scope in javascript. By block we mean the code between {}, aka curly braces. var myFunction = function () { var noBlockScope = function ( ) { if (true) { // you'd think that d would only be visible to this if statement var d = 24; } if (true) { // this if statement can see the variable defined in the other if statement console.log(d); } } noBlockScope(); Single var pattern You can define all variables used by a function in one place. It is ensures tidy code and is considered best practise. function scrum() { var numberOfProps = 2, numberOfHookers = 1, numberOfSecondRows = 2, numberOfBackRow = 3 // function body... } If a variable is declared but not initialized with a value it will have the value undefined. Strict Equality In javascript it is possible to compare two objects using ==. However, in some cases this will perform type conversion which can yield unexpected equality matches. To ensure there is strict comparison (i.e. no type conversions) use the === syntax. console.log(1 == true) // outputs true console.log(1 === true) // outputs false console.log(45 == "45") // outputs true console.log(45 === "45") // outputs false Truthy and Falsey When javascript expects a boolean, you may specify a value of any type. Values that convert to true are said to be truthy and values that convert to false are said to be falsey. Example of truthy values are objects, arrays, functions, strings and numbers: // This will output 'Wow, they were all true' if ({} && {sillyproperty:"sillyvalue"} && [] && ['element'] && function() {} && "string" && 89) { console.log("wow, they were all true"); } Examples of falsey values are empty strings, undefined, null and the value 0. // This will out put: 'none of them were true' if (!("" || undefined || null || 0)) { console.log("none of them were true"); } Undefined and null In javascript, the undefined value means not initialised or unknown where null means an absence of a value. References JavaScript patterns Stoyan Stefanov JavaScript, The Definitive Guide David Flanagan JavaScript, The Good Parts Doug Crockford.

In my recent projects I've had to do a lot with certificates, java and HTTPS with client-side authentication. In most of these projects, either during testing, or setting up a new environment, I've run into various SSL configuration errors that often resulted in a rather uncomprehensive error such as: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352) at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:397) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148) at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150) at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732) In most of the cases it was misconfiguration where keystores didn't containt the correct certificates, the certificate chain was incomplete or the client didn't supply a valid certificate. So in the last project I decided to document what was happening and what caused specific errors during the SSL handshake. In this article I'll show you why specific SSL errors occur, how you can detect them by analyzing the handshake information, and how to solve them. For this I use the following scenario: Server uses a certificate issued by a CA and requires client authentication. The server uses a simple truststore that lists this CA as trusted. Client connects using a certificate issued by this single trusted CA and has it's own trustore that also contains this certificate from the server. Not a very complicated situation, but one you often see. Note that the following information can also be used to identify problems when you don't work with client certificates or use self-signed certificates. The way to determine the problem in those cases, is pretty much the same. Happy Flow First we'll look at the happy flow, what happens in the handshake when we use client certificates. We won't look at the complete negotiation phase, but only until both the client and the server have exchanged their certificates and have validated the received certificate. If everything goes well until that point, the rest should work. The following is what you see when you run the client and the server using the java VM parameter: -Djavax.net.debug=ssl:handshake. The first thing that happens is that the client sends a ClientHello message using the TLS protocol version he supports, a random number and a list of suggested cipher suites and compression methods. From our client this looks like this: Client sends: *** ClientHello, TLSv1 RandomCookie: GMT: 1331663143 bytes = { 141, 219, 18, 140, 148, 60, 33, 241, 10, 21, 31, 90, 88, 145, 34, 153, 238, 105, 148, 72, 163, 210, 233, 49, 99, 224, 226, 64 } Session ID: {} Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV] Compression Methods: { 0 } *** The server responds, very originally, with a ServerHello message, that contains the choices made based on the information provided by the client another random number and (optionally) a session id. Server sends: *** ServerHello, TLSv1 RandomCookie: GMT: 1331663143 bytes = { 172, 233, 79, 197, 14, 21, 187, 161, 114, 206, 7, 38, 188, 228, 120, 102, 115, 214, 155, 86, 211, 41, 156, 179, 138, 2, 230, 81 } Session ID: {79, 96, 145, 39, 203, 136, 206, 69, 170, 46, 194, 17, 154, 175, 13, 138, 143, 199, 162, 193, 110, 86, 113, 109, 248, 187, 220, 169, 47, 180, 44, 68} Cipher Suite: SSL_RSA_WITH_RC4_128_MD5 Compression Method: 0 Extension renegotiation_info, renegotiated_connection: *** So in this case we're going to use SSL_RSA_WITH_RC4_128_MD5 as Cipher Suite. The next step is also done by the server. The server next sends a Certificate message that contains its complete certificate chain: Server sends: *** Certificate chain chain [0] = [ [ Version: V1 Subject: CN=server, C=NL Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5 Key: Sun RSA public key, 1024 bits modulus: 143864428144045085986129639694300995179398936575198896494655652087658861594939489453166811774109137006267822033915476680673848164790815913192075840268069822357600376998775923266017630332239546722181180383155088413406178660120548292599278819762883993031950564327152510982887716901499177102158407884939613382007 public exponent: 65537 Validity: [From: Wed Mar 14 13:32:04 CET 2012, To: Thu Mar 14 13:32:04 CET 2013] Issuer: CN=Application CA, OU=GKD, O=Smartjava, L=Maasland, ST=ZH, C=NL SerialNumber: [ a881d144 5e631f21] ] Algorithm: [SHA1withRSA] Signature: 0000: C3 56 81 7F 33 91 8A FF 84 5E 0B BA 7A 01 D8 41 .V..3....^..z..A 0010: 6B 47 B2 F7 8F FB B5 77 23 D8 FB B2 35 19 6E C4 kG.....w#...5.n. 0020: A4 6A BC 23 BB 69 92 F6 85 5A 1E CB FE 23 C6 98 .j.#.i...Z...#.. 0030: A0 57 F8 FB E9 DB B0 40 BD 8E F8 35 F8 77 E1 09 [email protected].. 0040: 5A 2E 45 71 80 F6 89 E7 0B 93 E2 48 EB 40 92 13 Z.Eq.......H.@.. 0050: 14 AA 1F 59 AA 98 67 46 9B 52 33 49 9A 3C 91 9B ...Y..gF.R3I.<.. 0060: F1 CB 8A BD 7D D4 DD 76 C4 15 00 36 A3 B2 87 A7 .......v...6.... 0070: D5 FF 52 E3 68 D4 F0 E0 32 86 74 02 DD 92 EC 1D ..R.h...2.t..... ] chain [1] = [ [ Version: V3 Subject: CN=Application CA, OU=SL, O=SmartJava, L=Waalwijk, ST=ZH, C=NL Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5 Key: Sun RSA public key, 1024 bits modulus: 159927271510058538658170959055540487654246676457579822126433656091883150307639380685203152841988861440546492270915750324654620063428634486478674507234742748515614639629692189315918046446256610037776978028900716455223387878926383828815082154427031884246429239077082613371662803582187768145965112751392402313823 public exponent: 65537 Validity: [From: Mon Mar 12 13:35:16 CET 2012, To: Wed Apr 11 14:35:16 CEST 2012] Issuer: CN=Application CA, OU=CA, O=Blaat, L=Waalwijk, ST=ZH, C=NL SerialNumber: [ fe7636c5 6804e69c] Certificate Extensions: 3 [1]: ObjectId: 2.5.29.14 Criticality=false SubjectKeyIdentifier [ KeyIdentifier [ 0000: 6C CC 48 03 E4 BE 07 D6 9E F6 4C 78 53 54 A2 B8 l.H.......LxST.. 0010: 7B DA 40 09 ..@. ] ] [2]: ObjectId: 2.5.29.35 Criticality=false AuthorityKeyIdentifier [ KeyIdentifier [ 0000: 6C CC 48 03 E4 BE 07 D6 9E F6 4C 78 53 54 A2 B8 l.H.......LxST.. 0010: 7B DA 40 09 ..@. ] ] [3]: ObjectId: 2.5.29.19 Criticality=false BasicConstraints:[ CA:true PathLen:2147483647 ] ] Algorithm: [SHA1withRSA] Signature: 0000: 1A 30 08 15 01 8E A6 36 5F 38 22 C6 81 5E 69 B1 .0.....6_8"..^i. 0010: 42 9A 1E FF 0F C4 D7 40 5F 85 0E 42 35 E0 CC 00 B......@_..B5... 0020: 6E A5 2E 70 6B 79 64 C5 99 AE A4 29 CB 26 DE 60 n..pkyd....).&.` 0030: 0B A6 AB 19 06 6F 19 54 6C 1A 88 9E 3A 6A D4 BB .....o.Tl...:j.. 0040: CB 28 85 2F 72 4D DE 35 C0 9B F4 2F EF 8E 6D E8 .(./rM.5.../..m. 0050: 30 AC 12 7D B4 0D A3 08 DA D4 60 46 94 BD 12 AF 0.........`F.... 0060: 44 F7 C3 B8 9D 69 2D 6A 32 C8 4D AE 12 60 05 09 D....i-j2.M..`.. 0070: FE AE D0 1A 72 6D 91 CE DA 7C 8E D5 31 14 31 4C ....rm......1.1L ] In this message you can see that the issuer of this certificate is our example CA. Our client checks to see if this certificate is trusted, which it is in this case. Since we require the client to authenticate itself the server requests a certificate from the client and after that sends a helloDone. Server sends: *** CertificateRequest Cert Types: RSA, DSS Cert Authorities: *** ServerHelloDone In this message you can see that the server provides a list of Cert Authorities it trusts. The client will use this information to determine if it has a keypair that matches this CA. In our happy flow, it has one and responds with a Certificate message. Client sends: *** Certificate chain chain [0] = [ [ Version: V1 Subject: CN=Application 3, OU=Smartjava, O=Smartjava, L=NL, ST=ZH, C=NL Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5 Key: Sun RSA public key, 1024 bits modulus: 90655907749318585147523875906892969031300830816947226352221659107570169820452561428696751943383590982109524990627182456571533992582229229163232831159652561902456847954385746762477844009336466314872376131553489447601649924116778337873632641536164462534398137791450495316700015095054427027256393580022887087767 public exponent: 65537 Validity: [From: Mon Mar 12 15:13:24 CET 2012, To: Tue Mar 12 15:13:24 CET 2013] Issuer: CN=Application CA, OU=Smartjava, O=Smartjava, L=Maasland, ST=ZH, C=NL SerialNumber: [ b247ffb2 ce060768] ] Algorithm: [SHA1withRSA] Signature: 0000: 97 58 36 C5 28 87 B3 16 9B DD 31 0C E0 C6 23 76 .X6.(.....1...#v 0010: 72 82 5B 13 4D 23 B6 0E A9 2F 9F 0C 3F 97 15 6E r.[.M#.../..?..n 0020: 7B 38 EC DE E2 57 D7 AA 07 12 E3 98 B7 86 A7 CE .8...W.......... 0030: 57 8E A1 29 96 C9 F0 30 57 67 C7 F1 F2 98 90 64 W..)...0Wg.....d 0040: 6C B9 6C 05 24 8B 56 3F B1 FF 03 62 3D 81 DB 45 l.l.$.V?...b=..E 0050: D3 1F C1 B2 DD 77 CF 74 54 EB 9D 82 23 89 1A 70 .....w.tT...#..p 0060: F8 C4 68 6A B7 41 C7 DE 7B B6 3A 0C 17 E7 FA 98 ..hj.A....:..... 0070: 19 0C D8 91 FB 5E FE D2 B3 92 FD 2D 2A 6B 51 10 .....^.....-*kQ. ] chain [1] = [ [ Version: V3 Subject: CN=Application CA, OU=Smartjava, O=Smartjava, L=Maasland, ST=ZH, C=NL Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5 Key: Sun RSA public key, 1024 bits modulus: 159927271510058538658170959055540487654246676457579822126433656091883150307639380685203152841988861440546492270915750324654620063428634486478674507234742748515614639629692189315918046446256610037776978028900716455223387878926383828815082154427031884246429239077082613371662803582187768145965112751392402313823 public exponent: 65537 Validity: [From: Mon Mar 12 13:35:16 CET 2012, To: Wed Apr 11 14:35:16 CEST 2012] Issuer: CN=Application CA, OU=Smartjava, O=Smartjava, L=Maasland, ST=ZH, C=NL SerialNumber: [ fe7636c5 6804e69c] Certificate Extensions: 3 [1]: ObjectId: 2.5.29.14 Criticality=false SubjectKeyIdentifier [ KeyIdentifier [ 0000: 6C CC 48 03 E4 BE 07 D6 9E F6 4C 78 53 54 A2 B8 l.H.......LxST.. 0010: 7B DA 40 09 ..@. ] ] [2]: ObjectId: 2.5.29.35 Criticality=false AuthorityKeyIdentifier [ KeyIdentifier [ 0000: 6C CC 48 03 E4 BE 07 D6 9E F6 4C 78 53 54 A2 B8 l.H.......LxST.. 0010: 7B DA 40 09 ..@. ] ] [3]: ObjectId: 2.5.29.19 Criticality=false BasicConstraints:[ CA:true PathLen:2147483647 ] ] Algorithm: [SHA1withRSA] Signature: 0000: 1A 30 08 15 01 8E A6 36 5F 38 22 C6 81 5E 69 B1 .0.....6_8"..^i. 0010: 42 9A 1E FF 0F C4 D7 40 5F 85 0E 42 35 E0 CC 00 B......@_..B5... 0020: 6E A5 2E 70 6B 79 64 C5 99 AE A4 29 CB 26 DE 60 n..pkyd....).&.` 0030: 0B A6 AB 19 06 6F 19 54 6C 1A 88 9E 3A 6A D4 BB .....o.Tl...:j.. 0040: CB 28 85 2F 72 4D DE 35 C0 9B F4 2F EF 8E 6D E8 .(./rM.5.../..m. 0050: 30 AC 12 7D B4 0D A3 08 DA D4 60 46 94 BD 12 AF 0.........`F.... 0060: 44 F7 C3 B8 9D 69 2D 6A 32 C8 4D AE 12 60 05 09 D....i-j2.M..`.. 0070: FE AE D0 1A 72 6D 91 CE DA 7C 8E D5 31 14 31 4C ....rm......1.1L ] This certificate is checked on the server side and if all is well, the final steps in the handshake are executed to setup the secured connection. Note that there is a CertificateVerify step. In this step the client signs a message with its private key. This is done so the server can verify the client has access to its private key. This might seem a step where things can go wrong in an incorrectly configured environment. In the default java implementation this won't happen. In the phase where the client has to determine which certificate to present to the server, the java implementation already checks if the privatekey is available. What could possibly go wrong So what could possibly go wrong in this handshake? In the next couple of sections we'll look at some scenarios, and how to detect them. Passwords Now that we've seen what happens when things go right, lets look at a couple of scenarios where things go wrong. We'll start simple with the following exception, that we get at the moment we start up the client application: Exception in thread "main" java.security.UnrecoverableKeyException: Cannot recover key at sun.security.provider.KeyProtector.recover(KeyProtector.java:311) at sun.security.provider.JavaKeyStore.engineGetKey(JavaKeyStore.java:121) at sun.security.provider.JavaKeyStore$JKS.engineGetKey(JavaKeyStore.java:38) at java.security.KeyStore.getKey(KeyStore.java:763) at com.sun.net.ssl.internal.ssl.SunX509KeyManagerImpl.(SunX509KeyManagerImpl.java:113) at com.sun.net.ssl.internal.ssl.KeyManagerFactoryImpl$SunX509.engineInit(KeyManagerFactoryImpl.java:48) at javax.net.ssl.KeyManagerFactory.init(KeyManagerFactory.java:239) at org.apache.http.conn.ssl.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:186) at org.apache.http.conn.ssl.SSLSocketFactory.(SSLSocketFactory.java:260) This very helpful message is thrown when (from the javadoc) " .. a key in the keystore cannot be recovered". There are a couple of reasons this can happen, but normally this occurs when the key in the keystore is accessed with the wrong password. Usually when you use the keytool to create and manage your keys, the keystore password is usually the same as the key password. However, if you import keys from a PKCS#12 type keystore, the password of the keystore can be easily set to a different value. Not all the SSL client allow you to specify a different password for the key and the keystore. If that is the case you can use the following command, to change the password of the key: keytool -keypasswd -alias -keystore It is also possible to set an incorrect password for the keystore. Luckily in that case the error message that is thrown is much more helpful: Exception in thread "main" java.io.IOException: Keystore was tampered with, or password was incorrect at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:771) at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:38) at java.security.KeyStore.load(KeyStore.java:1185) ... Caused by: java.security.UnrecoverableKeyException: Password verification failed at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:769) ... 3 more If this occurs at the server side, we can see the same message when the SSL listener is being set up. Incomplete CA Chains Now lets look at the first of the "peer not authenticated" exceptions. In the logging we see this exception at the client side: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352) at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:397) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148) at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150) So enable SSL logging, run again, and we'll start with analyzing the handshake. We'll start by looking from the client side. If we look through the logging we find the following CertificateRequest message from the server and the ServerHelloDone. *** CertificateRequest Cert Types: RSA, DSS Cert Authorities: *** ServerHelloDone So thus far, everything went ok. The server has already sent its certificate, and since our client doesn't throw an error on that part, we can assume it is trusted by the client. So something seems to be wrong with the steps that come after this message from the server. If you look closer at this message, you can see that the server doesn't specify a set of Cert Authorities it trusts. This could be a misconfiguration at the server side, or it could just be that the server expects one of the trusted Root CAs. In any case, the client is free to send any certificate he wants. So the client sends the following certificate: *** Certificate chain chain [0] = [ [ Version: V1 Subject: CN=Application4, OU=Smartjava, O=Smartjava, L=NL, ST=NB, C=NL Signature Algorithm: SHA1withDSA, OID = 1.2.840.10040.4.3 ... ] chain [1] = [ [ Version: V3 Subject: [email protected], CN=CA2, OU=Smartjava, O=Smartjava, L=Waalwijk, ST=NB, C=NL Signature Algorithm: SHA1withDSA, OID = 1.2.840.10040.4.3 ... ] According to the specification the client now continues with the key exchange and generates secrets to exchange. Somewhere along the lines we can see the following: pool-1-thread-1, WRITE: TLSv1 Handshake, length = 32 pool-1-thread-1, READ: TLSv1 Alert, length = 2 pool-1-thread-1, RECV TLSv1 ALERT: fatal, internal_error pool-1-thread-1, called closeSocket() This means we've received an internal error. So something at the server side went wrong. Looking at the server we see the following in the SSL dump: *** Certificate chain chain [0] = [ [ Version: V1 Subject: CN=Application4, OU=Smartjava, O=Smartjava, L=NL, ST=NB, C=NL Signature Algorithm: SHA1withDSA, OID = 1.2.840.10040.4.3 ... ] chain [1] = [ [ Version: V3 Subject: [email protected], CN=CA2, OU=Smartjava, O=Smartjava, L=Waalwijk, ST=NB, C=NL Signature Algorithm: SHA1withDSA, OID = 1.2.840.10040.4.3 ... ] *** qtp1735121130-17, handling exception: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty qtp1735121130-17, SEND TLSv1 ALERT: fatal, description = internal_error qtp1735121130-17, WRITE: TLSv1 Alert, length = 2 You can see that we received the certificate from the client, and directly after that we get this error. This error however doesn't really tell us anything. We do however have enough information to at least limit the possible errors. We know that the server didn't sent a list of CAs, we can see that the client sent a valid certificate, and that server somehow isn't able to process it. It looks like a problem with the server truststore. In this case the best approach is to look at the certificates the server trusts. Either in the cacerts file or in it's own truststore. Validate whether the CA certificate our client sends is in the server's truststore, and the server actually loads the stores we expect. It's of course also possible that the client has an incomplete chain of trust for the certificate received from the server. In that case we once again get the "peer not authenticated" error at the client side. If we look at the SSL debug logging, we see the following exception occuring at the client side: pool-1-thread-1, handling exception: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty pool-1-thread-1, SEND TLSv1 ALERT: fatal, description = internal_error pool-1-thread-1, WRITE: TLSv1 Alert, length = 2 This exception occured directly after the server has sent its certificate using a "Certificate message": *** Certificate chain chain [0] = [ [ Version: V1 Subject: CN=server, C=NL Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5 Following the same reasoning as for the server we can conclude that there is something wrong with the client side truststore. For completeness sake, the server receives this error message when this situation occurs at the client: qtp1500389297-17, READ: TLSv1 Alert, length = 2 qtp1500389297-17, RECV TLSv1 ALERT: fatal, internal_error qtp1500389297-17, called closeSocket() qtp1500389297-17, handling exception: javax.net.ssl.SSLException: Received fatal alert: internal_error qtp1500389297-17, called close() qtp1500389297-17, called closeInternal(true) Invalid keys For the next exercise lets look at the following error that occurs during this handshake. In the logging at the client side we see the following error message in the SSL output: ool-1-thread-1, WRITE: TLSv1 Handshake, length = 32 pool-1-thread-1, READ: TLSv1 Alert, length = 2 pool-1-thread-1, RECV TLSv1 ALERT: fatal, internal_error pool-1-thread-1, called closeSocket() pool-1-thread-1, handling exception: javax.net.ssl.SSLException: Received fatal alert: internal_error pool-1-thread-1, IOException in getSession(): javax.net.ssl.SSLException: Received fatal alert: internal_error Which results in the very unhelpful: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352) at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:397) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148) at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150) at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732) When you receive an internal error, there is usually something wrong at the server side. So looking at the serverside, lets see what caused this error. *** qtp2044601711-16, handling exception: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty qtp2044601711-16, SEND TLSv1 ALERT: fatal, description = internal_error Hmm.. somewhat more useful. It seems that there is something wrong with the algorithm we used, the client seems to have provided an incorrect certificate. But what is wrong? If you look back at the happy flow, you can send that at a certain time the server asks the client for a certificate using a "Certificate" message. Lets look a bit closer at this message and the response: *** CertificateRequest Cert Types: RSA, DSS Cert Authorities: *** ServerHelloDone matching alias: application4 *** Certificate chain chain [0] = [ [ Version: V1 Subject: CN=Application4, OU=Smartjava, O=Smartjava, L=NL, ST=NB, C=NL Signature Algorithm: SHA1withDSA, OID = 1.2.840.10040.4.3 Key: Sun DSA Public Key ... What you can see here is that the server specifies the cert types it accepts, and the authorities it accepts. The client responses in this case however with a DSA public key. Depending on the server implementation this can cause this strange message. Another possible scenario I've seen (especially with self-signed certificates) is that with a "CertificateRequest" message like this: *** CertificateRequest Cert Types: RSA, DSS Cert Authorities: *** ServerHelloDone This client won't respond with a certificate at all, if you only have DSA based keys in your keystore. It won't throw an error on the client side, but will cause a "null certificate chain" message as the server side. I haven't seen this scenario, though, when you don't use self-signed certificates. Certificate expiration So far we've seen how you can analyze the SSL handshake to determine where to look for configuration errors. In this last example we'll look at what happens when a certificate expires. In this case we once again see the very cryptic message at the client side: pool-1-thread-1, READ: TLSv1 Alert, length = 2 pool-1-thread-1, RECV TLSv1 ALERT: fatal, certificate_unknown pool-1-thread-1, called closeSocket() pool-1-thread-1, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown pool-1-thread-1, IOException in getSession(): javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown pool-1-thread-1, called close() pool-1-thread-1, called closeInternal(true) pool-1-thread-1, called close() pool-1-thread-1, called closeInternal(true) javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:352) at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:128) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:397) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148) at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150) If we look at the phase of the SSL handshake we're in, we can see that we've already sent our client certificate and finishing up the handshake when we receive this error. The error on the serverside is actually pretty helpful. After receiving the invalid certificate, in the debug logging, it shows us the following: *** qtp1735121130-17, SEND TLSv1 ALERT: fatal, description = certificate_unknown qtp1735121130-17, WRITE: TLSv1 Alert, length = 2 [Raw write]: length = 7 0000: 15 03 01 00 02 02 2E ....... qtp1735121130-17, called closeSocket() qtp1735121130-17, handling exception: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed qtp1735121130-17, called close() qtp1735121130-17, called closeInternal(true) It tells us that during the validation of the certificate, a timestamp check failed. This tells us that we should look at the validity of the certificates in our certificate chain to see what is happening. Summary In this article you've seen a couple of common causes for SSL exceptions and ways to identify the exception. Their can be many causes for these kind of exceptions, the most common though are the following: Incorrect certificate chains in the client truststore Incorrect certificate chains in the server truststore Invalid key algorithm used for private keys Expired certificate or expired CA certificate Incorrect passwords used to access the keys Multiple private keys to choose from If you're presented with a such an exception a good general approach is this. You first check the keystores that are involved. Use the java keytool for this: keytool -list -v -keystore This will print out all the certificates and keys in the keystore. Check whether the keys are of a supported type, the required CA certificates are stored and that your application is using the correct one (spent hours figuring out an issue because I was looking into a truststore for my private key). If everything seems to be OK at first glance it's time to enable ssl debugging (-Djavax.net.debug=ssl:handshake) and check the handshake messages that are sent. Wikipedia has a nice overview of which message is sent at a specific time. For more information on the content of the messages look at the RFC 5246 (or the one of the SSL/TLS version you're using, but the handshake changes are minimal between versions). Using the messages and the handshake, determine at what place in the handshake things go wrong, taking into account that the client will continue with the handshake, while the server is processing it's certificate.