Following on from the previous post on operator overloading I'm going to be looking at Implicit Conversions, and how we can combine them to with operator overloading to do some really neat things, including one way of creating a multi-parameter conversion. So what's an "Implicit Conversion" when it's at home? So lets start with some basic Scala syntax, if you've spent any time with Scala you've probably noticed it allows you to do things like: (1 to 4).foreach(println) // print out 1 2 3 4 Ever wondered how it does this? Lets make things more explicit, you could rewrite the above code as: val a : Int = 1 val b : Int = 4 val myRange : Range = a to b myRange.foreach(println) Scala is creating a Range object directly from two Ints, and a method called to. So what's going on here? Is this just a sprinkling of syntactic sugar to make writing loops easier? Is to just a keyword in like def or val? The answers to all this is no, there's nothing special going on here. to is simply a method defined in the RichInt class, which takes a parameter and returns a Range object (specifically a subclass of Range called Inclusive). You could rewrite it as the following if you really wanted to: val myRange : Range = a.to(b) Hang on though, RichInt may have a "to" method but Int certainly doesn't, in your example you're even explicitly casting your numbers to Ints Which brings me nicely on to the subject of this post, Implicit Conversions. This is how Scala does this. Implicit Conversions are a set of methods that Scala tries to apply when it encounters an object of the wrong type being used. In the case of the to example there's a method defined and included by default that will convert Ints into RichInts. So when Scala sees 1 to 4 it first runs the implicit conversion on the 1 converting it from an Int primitive into a RichInt. It can then call the to method on the new RichInt object, passing in the second Int (4) as the parameter. Hmm, think I understand, how's about another example? Certainly. Lets try to improve our Complex number class we created in the previous post. Using operator overloading we were able to support adding two complex numbers together using the + operator. eg. class Complex(val real : Double, val imag : Double) { def +(that: Complex) = new Complex(this.real + that.real, this.imag + that.imag) def -(that: Complex) = new Complex(this.real - that.real, this.imag - that.imag) override def toString = real + " + " + imag + "i" } object Complex { def main(args : Array[String]) : Unit = { var a = new Complex(4.0,5.0) var b = new Complex(2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 2.0i } } But what if we want to support adding a normal number to a complex number, how would we do that? We could certainly overload our "+" method to take a Double argument, ie something like... def +(n: Double) = new Complex(this.real + n, this.imag) Which would allow us to do... val sum = myComplexNumber + 8.5 ...but it'll break if we try... val sum = 8.5 + myComplexNumber To get around this we could use an Implicit Conversion. Here's how we create one. object ComplexImplicits { implicit def Double2Complex(value : Double) = new Complex(value,0.0) } Simple! Although you do need to be careful to import the ComplexImplicits methods before they can be used. You need to make sure you add the following to the top of your file (even if your Implicits object is in the same file)... import ComplexImplicits._ And that's the problem solved, you can now write val sum = 8.5 + myComplexNumber and it'll do what you expect! Nice. Is there anything else I can do with them? One other thing I've found them good for is creating easy ways of instantiating objects. Wouldn't it be nice if there were a simpler way of creating one of our complex numbers other than with new Complex(3.0,5.0). Sure you could get rid of the new by making it a case class, or implementing an apply method. But we can do better, how's about just (3.0,5.0) Awesome, but I'd need some sort of multi parameter implicit conversion, and I don't really see how that's possible!? The thing is, ordinarily (3.0,5.0) would create a Tuple. So we can just use that tuple as the parameter for our implicit conversion and convert it into a Complex. how we might go about doing this... implicit def Tuple2Complex(value : Tuple2[Double,Double]) = new Complex(value._1,value._2); And there we have it, a simple way to instantiate our Complex objects, for reference here's what the entire Complex code looks like now. import ComplexImplicits._ object ComplexImplicits { implicit def Double2Complex(value : Double) = new Complex(value,0.0) implicit def Tuple2Complex(value : Tuple2[Double,Double]) = new Complex(value._1,value._2); } class Complex(val real : Double, val imag : Double) { def +(that: Complex) : Complex = (this.real + that.real, this.imag + that.imag) def -(that: Complex) : Complex = (this.real - that.real, this.imag + that.imag) def unary_~ = Math.sqrt(real * real + imag * imag) override def toString = real + " + " + imag + "i" } object Complex { val i = new Complex(0,1); def main(args : Array[String]) : Unit = { var a : Complex = (4.0,5.0) var b : Complex = (2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 8.0i println(~b) // 3.60555 var c = 4 + b println(c) // 6.0 + 3.0i var d = (1.0,1.0) + c println(d) // 7.0 + 4.0i } }

So, I've been teaching myself Scala recently, and it's a very interesting language. One of the nice things I like about it, is it's support for creating DSLs, domain specific languages. A domain specific language - or at least my understanding of it - is a language that is written specifically for one problem domain. One example would be SQL, great for querying relational databases, useless for creating first person shooters. Of course Scala itself is not a DSL, it's a general purpose language. However it does offer several features that allow you to simulate a DSL, in particular operator overloading, and implicit conversions. In this post I'm going to focus on the first of these... Operator Overloading So what's operator overloading? Well operators are typically things such as +, -, and !. You know those things you use to do arithmetic on numbers, or occasionally for manipulating Strings. Well, operator overloading - just like method overloading - allows you to redefine their behaviour for a particular type, and give them meaning for your own custom classes. Hang on a minute! I'm sure someone once told me operator overloading was evil? Indeed, this is quite a controversial topic. It's considered far too open for abuse by some, and was so maligned in C++ that the creators of Java deliberately disallowed it (excepting "+" for String concatenation). I'm of a slightly different opinion, used responsibly it can be very useful. For example lots of different objects support a concept of addition, so why not just use an addition operator? Lets say you were developing a complex number class, and you want to support addition. Wouldn't it be nicer to write... Complex result = complex1 + complex2; ...rather than... Complex result = complex1.add(complex2); The first example is much more natural don't you think? So Scala allows you to overload operators then? Well, not really. In fact, technically not at all. So all this is just a tease? This is the most stupid blog post I've ever read. Scala's rubbish. I'm going back to Algol 68. Wait a second, I've not finished. You see Scala doesn't support operator overloading, because it doesn't have operators! Scala doesn't have operators? You've gone mad, I write stuff like "sum = 2 + 3" all the time, and what about all those funny list operations? "::", and ":/". They look like operators to me! Well they're not. The thing is, Scala has a rather relaxed attitude to what you can name a method. When you write... sum = 2 + 3, ...you're actually calling a method called + on a RichInt type with a value of 2. You could even rewrite it as... sum = 2.+(3) ...if you really really wanted to. Aha, I got it. So how do you go about overloading an operator then? Simple, it's exactly the same as writing a normal method. Here's an example. class Complex(val real : Double, val imag : Double) { def +(that: Complex) = new Complex(this.real + that.real, this.imag + that.imag) def -(that: Complex) = new Complex(this.real - that.real, this.imag - that.imag) override def toString = real + " + " + imag + "i" } object Complex { def main(args : Array[String]) : Unit = { var a = new Complex(4.0,5.0) var b = new Complex(2.0,3.0) println(a) // 4.0 + 5.0i println(a + b) // 6.0 + 8.0i println(a - b) // 2.0 + 2.0i } } Ok that's nice, what if I wanted a "not" operator though, ie something like a "!" That's a unary prefix operator, and yes scala can support these, although in a more limited fashion than an infix operator like "+" Only four operators can be supported in this fashion, +, -, !, and ~. You simply need to call your methods unary_! or unary_~, etc. Here's how you might add a "~" to calculate the magnitude of a Complex number to our complex number class class Complex(val real : Double, val imag : Double) { // ... def unary_~ = Math.sqrt(real * real + imag * imag) } object Complex { def main(args : Array[String]) : Unit = { var b = new Complex(2.0,3.0) prinln(~b) // 3.60555 } } So that's all pretty simple, but please use responsibly. Don't create methods called "+" unless your class really does something that could be interpreted as addition. And never ever redefine the binary shift left operator "<<" as some sort of substitute for println. It's not clever and you'll make the Scala gods angry. Hope you found that useful. Next up I'll cover implicit conversions. Another nice feature of Scala that really allows you to write your code in a more natural way

On the surface these methods do the same thing in Java; Thread.yield(), Thread.sleep(0), Object.wait(0,1) and LockSupport.parkNanos(1) They all wait a sort period of time, but how much that is varies a surprising amount and between platforms. Timing a short delay The following code times how long it takes to repeatedly call those methods. import java.util.concurrent.locks.LockSupport; public class Pausing { public static void main(String... args) throws InterruptedException { int repeat = 10000; for (int i = 0; i < 3; i++) { long time0 = System.nanoTime(); for (int j = 0; j < repeat; j++) Thread.yield(); long time1 = System.nanoTime(); for (int j = 0; j < repeat; j++) Thread.sleep(0); long time2 = System.nanoTime(); synchronized (Thread.class) { for (int j = 0; j < repeat/10; j++) Thread.class.wait(0, 1); } long time3 = System.nanoTime(); for (int j = 0; j < repeat/10; j++) LockSupport.parkNanos(1); long time4 = System.nanoTime(); System.out.printf("The average time to yield %.1f μs, sleep(0) %.1f μs, " + "wait(0,1) %.1f μs and LockSupport.parkNanos(1) %.1f μs%n", (time1 - time0) / repeat / 1e3, (time2 - time1) / repeat / 1e3, (time3 - time2) / (repeat/10) / 1e3, (time4 - time3) / (repeat/10) / 1e3); } } } On Windows 7 The average time to yield 0.3 μs, sleep(0) 0.6 μs, wait(0,1) 999.9 μs and LockSupport.parkNanos(1) 1000.0 μs The average time to yield 0.3 μs, sleep(0) 0.6 μs, wait(0,1) 999.5 μs and LockSupport.parkNanos(1) 1000.1 μs The average time to yield 0.2 μs, sleep(0) 0.5 μs, wait(0,1) 1000.0 μs and LockSupport.parkNanos(1) 1000.1 μs On RHEL 5.x The average time to yield 1.1 μs, sleep(0) 1.1 μs, wait(0,1) 2003.8 μs and LockSupport.parkNanos(1) 3.8 μs The average time to yield 1.1 μs, sleep(0) 1.1 μs, wait(0,1) 2004.8 μs and LockSupport.parkNanos(1) 3.4 μs The average time to yield 1.1 μs, sleep(0) 1.1 μs, wait(0,1) 2005.6 μs and LockSupport.parkNanos(1) 3.1 μs In summary If you want to wait for a short period of time, you can't assume that all these methods do the same thing, nor will be the same between platforms.

You may be using JSON to transfer data (we were using it in our message queue). While this is good, it has the only benefit of being human-readable. If you don’t care about readability, you’d probably want to use a more efficient serialization mechanism. Multiple options exist: protobuf, MessagePack, protostuff, java serialization. The easiest of them to use is java serialization, but it is less efficient (with both memory and time) than the other solutions. There are some benchmarks that will help you choose the most efficient solution, but if you want it to be easy and almost drop-in replacement to your JSON solution, MessagePack might be the best option. I made a simple test to compare the JSON output to the MessagePack output in terms of size: 2300 vs 150 bytes for a simple message. Pretty good reduction, and if the messages are a lot, it’s a must to optimize. However, you need to register all classes in the message pack. There are two options: use @Message on all the objects in the serialized graph. This is a bit tedious, especially if you already have a lot of classes that are transferred. You have to go through the whole graph you can manually register all classes with the mesagpack. Again tedious, because you also have to register all classes that the message class contains as a field (recursively) That’s why I wrote the following code to loop all our message classes, and register them with the message pack on startup. It partly relies on spring classes, but if you are not using Spring, you can replace them: private MessagePack serializer = new MessagePack(); private ClassMapper classMapper = new DefaultClassMapper(); @PostConstruct public void init() { // we need to find all messages, and register their classes, and also all their fields' recursively ClassPathScanningCandidateComponentProvider provider = new ClassPathScanningCandidateComponentProvider(false); Set classes = provider.findCandidateComponents("com.foo.bar.messages"); // hacking MessagePack to allow Set handling Field fld = ReflectionUtils.findField(MessagePack.class, "registry"); ReflectionUtils.makeAccessible(fld); TemplateRegistry registry = (TemplateRegistry) ReflectionUtils.getField(fld, serializer); registry.register(Set.class, new SetTemplate(new AnyTemplate(registry))); registry.registerGeneric(Set.class, new GenericCollectionTemplate(registry, SetTemplate.class)); try { for (BeanDefinition def : classes) { Class clazz = Class.forName(def.getBeanClassName()); registerHierarcy(clazz, serializer, Sets.>newHashSet()); } } catch (ClassNotFoundException e) { throw new IllegalStateException(e); } } private void registerHierarcy(Class clazz, MessagePack serializer, Set> handledClasses) { if (!isEligibleForRegistration(clazz)) { return; } Class currentClass = clazz; while (currentClass != null && !currentClass.isEnum() && currentClass != Object.class) { for (Field field : currentClass.getDeclaredFields()) { registerHierarcy(field.getType(), serializer, handledClasses); // type parameters Type type = field.getGenericType(); if (type instanceof ParameterizedType) { for (Type typeParam : ((ParameterizedType) type).getActualTypeArguments()) { // avoid circular generics references, resulting in stackoverflow Class typeParamClass = (Class) typeParam; if (!handledClasses.contains(typeParamClass)) { handledClasses.add(typeParamClass); registerHierarcy(typeParamClass, serializer, handledClasses); } } } } currentClass = currentClass.getSuperclass(); } try { serializer.register(clazz); } catch (Exception ex) { logger.warn("Problem registering class " + clazz, ex.getMessage()); } } private boolean isEligibleForRegistration(Class clazz) { return !(clazz.isAnnotationPresent(Entity.class) || clazz == Class.class || Type.class.isAssignableFrom(clazz) || clazz.isInterface() || clazz.isArray() || ClassUtils.isPrimitiveOrWrapper(clazz) || clazz == String.class || clazz == Date.class || clazz == Object.class); }

A few weeks ago, I co-authored an article (with my colleague Rama Ramani) about how the Screen Actors Guild Awards website migrated its Drupal deployment from LAMP to Windows Azure: Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. Since then, Rama and another colleague, Jason Roth, have been working on writing up how the SAG Awards website was managed and monitored in Windows Azure. The article below is the fruit of their work…a very interesting/educational read. Overview Drupal is an open source content management system that runs on PHP. Windows Azure offers a flexible platform for hosting, managing, and scaling Drupal deployments. This paper focuses on an approach to host Drupal sites on Windows Azure, based on learning from a BPD Customer Programs Design Win engagement with the Screen Actors Guild Awards Drupal website. This paper covers guidelines and best practices for managing an existing Drupal web site in Windows Azure. For more information on how to migrate Drupal applications to Windows Azure, see Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. The target audience for this paper is Drupal administrators who have some exposure to Windows Azure. More detailed pointers to Windows Azure content is provided throughout the paper as links. Drupal Application Architecture on Windows Azure Before reviewing the management and monitoring guidelines, it is important to understand the architecture of a typical Drupal deployment on Windows Azure. First, the following diagram displays the basic architecture of Drupal running on Windows and IIS7. In the Windows Server scenario, you could have one or more machines hosting the web site in a farm. Those machines would either persist the site content to the file system or point to other network shares. For Windows Azure, the basic architecture is the same, but there are some differences. In Windows Azure the site is hosted on a web role. A web role instance is hosted on a Windows Server 2008 virtual machine within the Windows Azure datacenter. Like the web farm, you can have multiple instances running the site. But there is no persistence guarantee for the data on the file system. Because of this, much of the shared site content should be stored in Windows Azure Blob storage. This allows them to be highly available and durable. Usually, a large portion of the site caters to static content which lends well to caching. And caching can be applied in a set of places – browser level caching, CDN to cache content in the edge closer to the browser clients, caching in Azure to reduce the load on backend, etc. Finally, the database can be located in SQL Azure. The following diagram shows these differences. For monitoring and management, we will look at Drupal on Windows Azure from three perspectives: Availability: Ensure the web site does not go down and that all tiers are setup correctly. Apply best practices to ensure that the site is deployed across data centers and perform backup operations regularly. Scalability: Correctly handle changes in user load. Understand the performance characteristics of the site. Manageability: Correctly handle updates. Make code and site changes with no downtime when possible. Although some management tasks span one or more of these categories, it is still helpful to discuss Drupal management on Windows Azure within these focus areas. Availability One main goal is that the Drupal site remains running and accessible to all end-users. This involves monitoring both the site and the SQL Azure database that the site depends on. In this section, we will briefly look at monitoring and backup tasks. Other crossover areas that affect availability will be discussed in the next section on scalability. Monitoring With any application, monitoring plays an important role with managing availability. Monitoring data can reveal whether users are successfully using the site or whether computing resources are meeting the demand. Other data reveals error counts and possibly points to issues in a specific tier of the deployment. There are several monitoring tools that can be used. The Windows Azure Management Portal. Windows Azure diagnostic data. Custom monitoring scripts. System Center Operations Manager. Third party tools such as Azure Diagnostics Manager and Azure Storage Explorer. The Windows Azure Management Portal can be used to ensure that your deployments are successful and running. You can also use the portal to manage features such as Remote Desktop so that you can directly connect to machines that are running the Drupal site. Windows Azure diagnostics allows you to collect performance counters and logs off of the web role instances that are running the Drupal site. Although there are many options for configuring diagnostics in Azure, the best solution with Drupal is to use a diagnostics configuration file. The following configuration file demonstrates some basic performance counters that can monitor resources such as memory, processor utilization, and network bandwidth. For more information about setting up diagnostic configuration files, see How to Use the Windows Azure Diagnostics Configuration File. This information is stored locally on each role instance and then transferred to Windows Azure storage per a defined schedule or on-demand. See Getting Started with Storing and Viewing Diagnostic Data in Windows Azure Storage. Various monitoring tools, such as Azure Diagnostics Manager, help you to more easily analyze diagnostic data. Monitoring the performance of the machines hosting the Drupal site is only part of the story. In order to plan properly for both availability and scalability, you should also monitor site traffic, including user load patterns and trends. Standard and custom diagnostic data could contribute to this, but there are also third-party tools that monitor web traffic. For example, if you know that spikes occur in your application during certain days of the week, you could make changes to the application to handle the additional load and increase the availability of the Drupal solution. Backup Tasks To remain highly available, it is important to backup your data as a defense-in-depth strategy for disaster recovery. This is true even though SQL Azure and Windows Azure Storage both implement redundancy to prevent data loss. One obvious reason is that these services cannot prevent administrator error if data is accidentally deleted or incorrectly changed. SQL Azure does not currently have a formal backup technology, although there are many third-party tools and solutions that provide this capability. Usually the database size for a Drupal site is relatively small. In the case of SAG Awards, it was only ~100-150 MB. So performing an entire backup using any strategy was relatively fast. If your database is much larger, you might have to test various backup strategies to find the one that works best. Apart from third-party SQL Azure backup solutions, there are several strategies for obtaining a backup of your data: · Use the Drush tool and the portabledb-export command. · Periodically copy the database using the CREATE DATABASE Transact-SQL command. · Use Data-tier applications (DAC) to assist with backup and restore of the database. SQL Azure backup and data security techniques are described in more detail in the topic, Business Continuity in SQL Azure. Note that bandwidth costs accrue with any backup operation that transfers information outside of the Windows Azure datacenter. To reduce costs, you can copy the database to a database within the same datacenter. Or you can export the data-tier applications to blob storage in the same datacenter. Another potential backup task involves the files in Blob storage. If you keep a master copy of all media files uploaded to Blob storage, then you already have an on-premises backup of those files. However, if multiple administrators are loading files into Blob storage for use on the Drupal site, it is a good idea to enumerate the storage account and to download any new files to a central location. The following PHP script demonstrates how this can be done by backing up all files in Blob storage after a specified modification date. setProxy(true, 'YOUR_PROXY_IF_NEEDED', 80); $blobs = (array)$blobObj->listBlobs(AZURE_STORAGE_CONTAINER, '', '', 35000); backupBlobs($blobs, $blobObj); function backupBlobs($blobs, $blobObj) { foreach ($blobs as $blob) { if (strtotime($blob->lastmodified) >= DEFAULT_BACKUP_FROM_DATE && strtotime($blob->lastmodified) <= DEFAULT_BACKUP_TO_DATE) { $path = pathinfo($blob->name); if ($path['basename'] != '$$$.$$$') { $dir = $path['dirname']; $oldDir = getcwd(); if (handleDirectory($dir)) { chdir($dir); $blobObj->getBlob( AZURE_STORAGE_CONTAINER, $blob->name, $path['basename'] ); chdir($oldDir); } } } } } function handleDirectory($dir) { if (!checkDirExists($dir)) { return mkdir($dir, 0755, true); } return true; } function checkDirExists($dir) { if(file_exists($dir) && is_dir($dir)) { return true; } return false; } ?> This script has a dependency on the Windows Azure SDK for PHP. Also note there are several parameters that you must modify such as the storage account, secret, and backup location. As with SQL Azure, bandwidth and transaction charges apply to a backup script like this. Scalability Drupal sites on Windows Azure can scale as load increased through typical strategies of scale-up, scale-out, and caching. The following sections describe the specifics of how these strategies are implemented in Windows Azure. Typically you make scalability decisions based on monitoring and capacity planning. Monitoring can be done in staging during testing or in production with real-time load. Capacity planning factors in projections for changes in user demand. Scale Up When you configure your web role prior to deployment, you have the option of specifying the Virtual Machine (VM) size, such as Small or ExtraLarge. Each size tier adds additional memory, processing power, and network bandwidth to each instance of your web role. For cost efficiency and smaller units of scale, you can test your application under expected load to find the smallest virtual machine size that meets your requirements. The workload usually in most popular Drupal websites can be separated out into a limited set of Drupal admins making content changes and a large user base who perform mostly read-only workload. End users can be allowed to make ‘writes’, such as uploading blogs or posting in forums, but those changes are not ‘content changes’. Drupal admins are setup to operate without caching so that the writes are made directly to SQL Azure or the corresponding backend database. This workload performs well with Large or ExtraLarge VM sizes. Also, note that the VM size is closely tied to all hardware resources, so if there are many content-rich pages that are streaming content, then the VM size requirements are higher. To make changes to the Virtual Machine size setting, you must change the vmsize attribute of the WebRole element in the service definition file, ServiceDefinition.csdef. A virtual machine size change requires existing applications to be redeployed. Scale Out In addition to the size of each web role instance, you can increase or decrease the number of instances that are running the Drupal site. This spreads the web requests across more servers, enabling the site to handle more users. To change the number of running instances of your web role, see How to Scale Applications by Increasing or Decreasing the Number of Role Instances. Note that some configuration changes can cause your existing web role instances to recycle. You can choose to handle this situation by applying the configuration change and continue running. This is done by handling the RoleEnvironment.Changing event. For more information see, How to Use the RoleEnvironment.Changing Event. A common question for any Windows Azure solution is whether there is some type of built-in automatic scaling. Windows Azure does not provide a service that provides auto-scaling. However, it is possible to create a custom solution that scales Azure services using the Service Management API. For an example of this approach, see An Auto-Scaling Module for PHP Applications in Windows Azure. Caching Caching is an important strategy for scaling Drupal applications on Windows Azure. One reason for this is that SQL Azure implements throttling mechanisms to regulate the load on any one database in the cloud. Code that uses SQL Azure should have robust error handling and retry logic to account for this. For more information, see Error Messages (SQL Azure Database). Because of the potential for load-related throttling as well as for general performance improvement, it is strongly recommended to use caching. Although Windows Azure provides a Caching service, this service does not currently have interoperability with PHP. Because of this, the best solution for caching in Drupal is to use a module that uses an open-source caching technology, such as Memcached. Outside of a specific Drupal module, you can also configure Memcached to work in PHP for Windows Azure. For more information, see Running Memcached on Windows Azure for PHP. Here is also an example of how to get Memcached working in Windows Azure using a plugin: Windows Azure Memcached plugin. In a future paper, we hope to cover this architecture in more detail. For now, here are several design and management considerations related to caching. Area Consideration Design and Implementation For a technology like Memcached, will the cache be collocated (spread across all web role instances)? Or will you attempt to setup a dedicated cache ring with worker roles that only run Memcached? Configuration What memory is required and how will items in the cache be invalidated? Performance and Monitoring What mechanisms will be used to detect the performance and overall health of the cache? For ease of use and cost savings, collocation of the cache across the web role instances of the Drupal site works best. However, this assumes that there is available reserve memory on each instance to apply toward caching. It is possible to increase the virtual machine size setting to increase the amount of available memory on each machine. It is also possible to add additional web role instances to add to the overall memory of the cache while at the same time improving the ability of the web site to respond to load. It is possible to create a dedicated cache cluster in the cloud, but the steps for this are beyond the scope of this paper[RR1] . For Windows Azure Blob storage, there is also a caching feature built into the service called the Content Delivery Network (CDN). CDN provides high-bandwidth access to files in Blob storage by caching copies of the files in edge nodes around the world. Even within a single geographic region, you could see performance improvements as there are many more edge nodes than Windows Azure datacenters. For more information, see Delivering High-Bandwidth Content with the Windows Azure CDN. Manageability It is important to note that each hosted service has a Staging environment and a Production environment. This can be used to manage deployments, because you can load and test and application in staging before performing a VIP swap with production. From a manageability standpoint, Drupal has an advantage on Windows Azure in the way that site content is stored. Because the data necessary to serve pages is stored in the database and blob storage, there is no need to redeploy the application to change the content of the site. Another best practice is to use a separate storage account for diagnostic data than the one that is used for the application itself. This can improve performance and also helps to separate the cost of diagnostic monitoring from the cost of the running application. As mentioned previously, there are several tools that can assist with managing Windows Azure applications. The following table summarizes a few of these choices. Tool Description Windows Azure Management Portal The web interface of the Windows Azure management portal shows deployments, instance counts and properties, and supports many different common management and monitoring tasks. Azure Diagnostics Managerq[RR2] [JR3] A Red Gate Software product that provides advanced monitoring and management of diagnostic data. This tool can be very useful for easily analyzing the performance of the Drupal site to determine appropriate scaling decisions. Azure Storage Explorer A tool created by Neudesic for viewing Windows Azure storage account. This can be useful for viewing both diagnostic data and the files in Blob storage.

What is Zebra? The Zebra is a JavaScript library that follows easy OOP concepts and implements a rich set of various self-made UI components. There are a huge number of attractive and powerful WEB UI frameworks on the market. Most of them are stuck to HTML, DOM and CSS. They build UIs by coloring DOM with CSS and manipulating the HTML DOM tree. Zebra UI is different. It is not based on HTML, DOM or CSS. Zebra UI components are implemented and rendered from scratch as a number of widgets organized in hierarchy. How does Zebra manage to build and render UIs on the web? The essential thing required by Zebra abstraction is the existence of a "Canvas-like " component. The "Canvas" component has to have set of graphical methods to paint lines, simple shapes, text, images and be able to catch input (keyboard, mouse, etc) events. For instance Java AWT/SWING, Eclipse SWT, .NET or other platforms will supply "Canvas-like" components. But the new HTML5 standard embeds "Canvas" element. This element is supported by most of the modern browsers and is what makes it possible and reasonable to "drop" Zebra UI into a web context. See Zebra demo http://zebra.gravitysoft.org The table below lists the most significant Zebra UI components, managers view, etc. Components marked by orange background are still in development: The Zebra Java to JavaScript converter is out of context in this article, nevertheless it plays key role in porting Java-based UI components onto the web. For demo purposes, a slightly outdated version of the Java to JavaScript converter is available: http://j2js.gravitysoft.org Zebra JavaScript Easy OOP concept An attractive, easy for use, and understandable programming model is very important in context of having supportable, extendable, elegant code. This is one of the painful problems in web development. Zebra introduces easy OOP concepts as a foundation. ~10.5kb of Zebra code helps to do the following: To get more information about Zebra OOP see the cheat sheet: Zebra easy OOP concept cheat sheet Zebra "Hello WEB" application The Zebra JavaScript demo indents to show an internal window with a "Hello web" title. The window is opened by pressing a button: // create canvas and store root panel in local variable var c = new zCanvas(html5Canvas), r = c.root; // create button var b = new Button("Hello WEB"); // set border layout manager r.setLayout(new BorderLayout()); // add button to center of root panel r.add(Layout.CENTER, b); // register the button event listener that opens external // window every time the button has been pressed b._(function() { var w = new Window("Hello WEB"); w.setSize(200,200); w.show(); }); Zebra UI features via JavaScript code snippets Zebra UI component design follows approaches similar to traditional Java AWT/SWING, .NET, and Eclipse SWT. UI components are organized as a hierarchy where some components are laid out on others. But Zebra does many things much more quickly and easily. Below are some Zebra UI features equpped with code snippets: Compound components. Zebra UI components often are built as a combination of several other components laid out on a panel. For instance the "Button" component is a panel that keeps a child component as its label. By default title is "Label", but developers can set other UI components as a button label. Take a look at the snippets below: // by default button uses label component as its content var button = new Button("Label"); // set image as the button label var button = new Button(new ImagePan("b.png")); // set image+label as the button content. The image+label is also // compound component that consists of image and label var bContent = new Panel(new FlowLayout()); bContent.add(new ImagePan()); bContent.add(new Label()); var button = new Button(bContent); The "BorderPan" UI component is one more example of a compound component. It consists of a label and content panel. Take a look at the snippets below: // border panel uses simple "Label" as its title by default var bp = new BorderPanel("Label", new Panel()); // border panel uses another border panel as its title var bp = new BorderPanel(new BorderPan("Label"), new Panel()); // border panel uses "Checkbox" as its title var bp = new BorderPanel(new Checkbox("Check me"), new Panel()); Full control over UI component rendering. Zebra UI components painting is fully in developers hands. Zebra calls "paint", "update", "paintOnTop" UI component methods with a graphical context as an input argument. These methods form a UI component face. Passed graphical context gives developers number of elementary methods to draw and fill primitive shapes, paint text, and images. The "paint" method defines the UI component "face": // create inner class instance with redefined component "face" var myEyesCandyComponent = new Panel([ function paint(g) { // paint what ever you want using passed graphical context g.drawLine(...); g.drawRect(...); g.fillArc(...); } ]); The "update" method forms a component background: // create inner class instance and override background // rendering with a custom implementation var myEyesCandyComponent = new Panel([ function update(g) { ... } ]); The "paintOnTop" method may be used, for instance, to render a focus indicator over the component "face": // render rectangle around component if the component holds focus var myEyesCandyComponent = new Panel([ function paintOnTop(g) { if (this.hasFocus()) g.drawRect(0,0, this.width, this.height); } ]); Zebra paint manager takes care of triggering the re-painting of invalidated-"dirty" areas. UI components are designed to make direct "repaint" method execution unnecessary. But custom UI components should call thr "repaint([x, y, w, h])" method every time a new dirty area has appeared. The "repaint" method execution triggers paint manager to recalculate current dirty area and schedule repainting when it is possible: var MyCustomComponent = Class(Panel, [ function setBorderColor(c) { this.color = c; this.repaint(); // inform paint manager the component // has to be completely re-painted } ]); Neat events handling. When it is possible, Zebra event handling follows a declarative pattern (see next feature bullet). It allows developers avoiding listeners registration and simplifies the event handling concept. To catch an event: Express an intention to get the desired event type by inheritance an appropriate interface. Interface is like a marker. Implement function(s) to treat desired event. For instance, imagine a custom component that needs to handle mouse events. Express the intention by implementing the "MouseListener" interface: // let Zebra know that the component wants getting mouse // events by implementing"MouseListener" interface var MyComponent = new Class(Panel, MouseListener, []); Suppose "mouse button has pressed" and "mouse cursor has entered the component" event types have to be handled. To do it, declare the following functions correspondently: var MyComponent = new Class(Panel, MouseListener, [ function mousePressed(e) { // handle mouse pressed event here }, function mouseEntered(e) { // handle mouse entered event here } ]); Global event handling. The special Zebra event manager keeps track of all events that have occurred for all instantiated UI components. The manager can be utilized to register global event listeners that get all events of the given type. For instance, listening to focus events globally can be done this way: // instantiate "FocusListener" interface to express that focus // events have to be catched zebra.ui.events.addListener(new FocusListener([ function focusLost(e) { ... }, function focusGained(e) { ... } ])); Catching children components events. The Parent UI component can listen to input events that have happened in its children by implementing the "ChildrenListener" interface and adding appropriate method(s) as follows: // implements "ChildrenListener" interface to express intention // to get children events var MyComponent = new Class(Panel, ChildrenListener, [ function childInputEvent(e){ // handle children events here } ]); Composite component (event transparent children). Often compound UI components have to prevent their children components from getting any events. Children components are becoming event transparent. This can be done by implementing "Composite" interface and adding the "catchInput" method. The method should return "true" if the passed as argument child has to be event transparent. For instance: var MyComponent = new Class(Panel, Composite, [ function catchInput(kid){ // return true if the given kid has to be event transparent return true; } ]); Declarative pattern. "First inherit an interface to express an intention to do something and then implement required method(s)". This approach allows Zebra to decouple various functional parts from each other. Extending in this context means adding new declarative patterns instead of overloading Zebra UI classes with new code and APIs. For instance, imagine we need to change the mouse cursor type every time the mouse pointers enter a UI component. Zebra UI components don’t know how to control mouse cursor type. It's the cursor manager ("zebra.ui.cursor") that does it: // inherits "CursorInfo" interface to let cursor manager know // the component controls mouse cursor var p = new Panel(CursorInfo, [ // declare method that returns mouse cursor type for the component function getCursorType(x, y){ return Cursor.WAIT; } ]); HTML5 Canvas transformation operations can be applied to UI. Rotation, zoom in, zoom out and other graphical transformation effects can be applied to Zebra UI: // zoom UI in 1.3 times vertically // and horizontally zCanvas.scale(1.3, 1.3); // rotate zCanvas.rotate(0.3); ... // set back to initial stat zCanvas.scale(null); zCanvas.rotate(null, null); Look and feel customization. A lot of Zebra UI components' visual characteristics are defined by a special properties file. If the properties file is not custom enough, a UI Wizard class can be implemented. The instance of the class is notified about all instantiated UI components by calling a "customize" method. It helps to customize UI components' look and feel "on the fly". For instance, let’s define own wizard class to color all label components with red background: // declare custom Wizard class var MyCustomWizard = Class(Wizard, [ // the method is called every time a new component has been // instantiated function customize(id, comp) { // customize just instantiated label component background if (id == Wizard.LABEL) comp.setBackground(Fill.red); } ]); Then setup your wizard with the properties file as follow: ... # specify wizard to be used for UI customization wizard = MyCustomWizard() Layered architecture. Zebra Canvas is the root panel that consists of a number of layers. Layers are a standard Zebra UI "Panel" that are stretched over the whole Canvas surface. Zebra holds layers as a stack. Every time an event occurs the Zebra event manager "asks" (starting from top to bottom layer) who wants to take control. The first met layer that grabs control is selected as the target. The picture below explains it: Let’s develop a layer that freezes (blocks any interaction) and un-freezes UI components by pressing the "CTRL + SHIFT + ALT" keys combination. Freezing is indicated by dimming UI components: // declare custom layer class that inherits "BaseLayer" class var Freezer = Class(BaseLayer, [ function () { this.$super("FREEZER"); // call super with unique layer ID this.isActive = false; // set background to be 100% transparent this.setBackground(null); }, function layerKeyPressed(code, mask) { var rm = KeyEvent.CTRL + KeyEvent.SHIFT + KeyEvent.ALT; if ((rm & mask) == rm) { // CTRL+SHIFT+ALT keys combination has been pressed if (this.isActive) this.setBackground(null); else this.setBackground(new Fill(255,255,255, 0.7)); this.isActive = ! this.isActive; } }, // methods below indicate if the layer is in active state (take control) function isLayerActive(){ return this.isActive;}, function isLayerActiveAt(x,y){return this.isActive; } ]); ... // add the layer to zebra canvas zCanvas.add(new Freezer()); The result of the layer work is demonstrated below: Un-frozen UI Frozen UI Layout management. Layout specifies a rule that says how to shape a number of child components on the given panel. Rule-based positioning is much better than absolute locations and fixed size usage. Gained advantages are the same as "vector vs pixel graphics". Zebra layout managers are independent from the Zebra UI package and can be re-used to layout other objects, for instance HTML elements. Let’s see how, for instance, diagonal layouts can be implemented. The custom layout manager lays out children components aligning its top left corners to diagonal: // declare diagonal layout manager var DiagonalLayout = Class(Layout, [ // "layout" method positions and shapes visible // children of target component function layout(target) { var x = 0, y = 0; for (var i=0; i

Here are some notes on adding Camel (from v2.3+) exception handling to a JavaDSL route. There are various approaches/options available. These notes cover the important distinctions between approaches... Default handling The default mode uses the DefaultErrorHandler strategy which simply propagates any exception back to the caller and ends the route immediately. This is rarely the desired behavior, at the very least, you should define a generic/global exception handler to log the errors and put them on a queue for further analysis (during development, testing, etc). onException(Exception) .to("log:GeneralError?level=ERROR") .to("activemq:queue:GeneralErrorQueue"); Try-catch-finally This approach mimics the Java for exception handling and is designed to be very readable and easy to implement. It inlines the try/catch/finally blocks directly in the route and is useful for route specific error handling. from("direct:start") .doTry() .process(new MyProcessor()) .doCatch(Exception.class) .to("mock:error"); .doFinally() .to("mock:end"); onException This approach defines the exception clause separately from the route. This makes the route and exception handling code more readable and reusable. Also, the exception handling will apply to any routes defined in its CamelContext. onException(Exception.class) .to("mock:error"); from("direct:start") .process(new MyProcessor()) .to("mock:end"); Handled/Continued These APIs provide valuable control over the flow. Adding handled(true) tells Camel to not propagate the error back to the caller (should almost always be used). The continued(true) tells Camel to resume the route where it left off (rarely used, but powerful). These can both be used to control the flow of the route in interesting ways, for example... from("direct:start") .process(new MyProcessor()) .to("mock:end"); //send the exception back to the client (rarely used, clients need a meaningful response) onException(ClientException.class) .handled(false) //default .log("error sent back to the client"); //send a readable error message back to the client and handle the error internally onException(HandledException.class) .handled(true) .setBody(constant("error")) .to("mock:error"); //ignore the exception and continue the route (can be dangerous, use wisely) onException(ContinuedException.class) .continued(true); Using a processor for more control If you need more control of the handler code, you can use an inline Processor to get a handle to the exception that was thrown and write your own handler code... onException(Exception.class) .handled(true) .process(new Processor() { public void process(Exchange exchange) throws Exception { Exception exception = (Exception) exchange.getProperty(Exchange.EXCEPTION_CAUGHT); //log, email, reroute, etc. } }); Summary Overall, the exception handling is very flexible and can meet almost any scenario you can come up with. For the sake of focusing on the basics, many advanced features haven't been covered here. For more details, see these pages on Camel's site... http://camel.apache.org/error-handling-in-camel.html http://camel.apache.org/try-catch-finally.html http://camel.apache.org/exception-clause.html

So I recently wrote a post about good old reliable Apache Commons StringUtils, which provoked a couple of comments, one of which was that Google Guava provides better mechanisms for joining and splitting Strings. I have to admit, this is a corner of Guava I've yet to explore. So thought I ought to take a closer look, and compare with StringUtils, and I have to admit I was surprised at what I found. Splitting strings eh? There can't be many different ways of doing this surely? Well Guava and StringUtils do take a sylisticly different approach. Lets start with the basic usage. // Apache StringUtils... String[] tokens1 = StringUtils.split("one,two,three",','); // Guava splitter... Iterable tokens2 = Splitter.on(',').split("one,two,three"); So, my first observation is that Splitter is more object orientated. You have to create a splitter object, which you then use to do the splitting. Whereas the StringUtils splitter methods uses a more functional style, with static methods. Here I much prefer Splitter. Need a reusable splitter that splits comma separated lists? A splitter that also trims leading and trailing white space, and ignores empty elements? Not a problem: Splitter niceCommaSplitter = Splitter.on(',') .omitEmptyString() .trimResults(); niceCommaSplitter.split("one,, two, three"); //"one","two","three" niceCommaSplitter.split(" four , five "); //"four","five" That looks really useful, any other differences? The other thing to notice is that Splitter returns an Iterable, whereas StringUtils.split returns a String array. Don't really see that making much of a difference, most of the time I just want to loop through the tokens in order anyway! I also didn't think it was a big deal, until I examined the performance of the two approaches. To do this I tried running the following code: final String numberList = "One,Two,Three,Four,Five,Six,Seven,Eight,Nine,Ten"; long start = System.currentTimeMillis(); for(int i=0; i<1000000; i++) { StringUtils.split(numberList , ','); } System.out.println(System.currentTimeMillis() - start); start = System.currentTimeMillis(); for(int i=0; i<1000000; i++) { Splitter.on(',').split(numberList ); } System.out.println(System.currentTimeMillis() - start); On my machine this output the following times: 594 31 Guava's Splitter is almost 10 times faster! Now this is a much bigger difference than I was expecting, Splitter is over 10 times faster than StringUtils. How can this be? Well, I suspect it's something to do with the return type. Splitter returns an Iterable, whereas StringUtils.split gives you an array of Strings! So Splitter doesn't actually need to create new String objects. It's also worth noting you can cache your Splitter object, which results in an even faster runtime. Blimey, end of argument? Guava's Splitter wins every time? Hold on a second. This isn't quite the full story. Notice we're not actually doing anything with the result of the Strings? Like I mentioned, it looks like the Splitter isn't actually creating any new Strings. I suspect it's actually deferring this to the Iterator object it returns. So can we test this? Sure thing. Here's some code to repeatedly check the lengths of the generated substrings: final String numberList = "One,Two,Three,Four,Five,Six,Seven,Eight,Nine,Ten"; long start = System.currentTimeMillis(); for(int i=0; i<1000000; i++) { final String[] numbers = StringUtils.split(numberList, ','); for(String number : numbers) { number.length(); } } System.out.println(System.currentTimeMillis() - start); Splitter splitter = Splitter.on(','); start = System.currentTimeMillis(); for(int i=0; i<1000000; i++) { Iterable numbers = splitter.split(numberList); for(String number : numbers) { number.length(); } } System.out.println(System.currentTimeMillis() - start); On my machine this outputs: 609 2048 Guava's Splitter is almost 4 times slower! Indeed, I was expecting them to be about the same, or maybe Guava slightly faster, so this is another surprising result. Looks like by returning an Iterable, Splitter is trading immediate gains, for longer term pain. There's also a moral here about making sure performance tests are actually testing something useful. In conclusion I think I'll still use Splitter most of the time. On small lists the difference in performance is going to be negligible, and Splitter just feels much nicer to use. Still I was surprised by the result, and if you're splitting lots of Strings and performance is an issue, it might be worth considering switching back to Commons StringUtils.

In a previous post I demonstrated how EclipseLink MOXy can be leveraged to produce both XML and JSON representations of your domain model. The same metadata is used for both representations and MOXy applies it to leverage the capabilities of the media type. In this post I'll focus on how null is handled in each of these representations. Domain Model By default a JAXB (JSR-222) implementation will not include a mapped field/property with a null value in the output. If you want the null value represented then you simply add the following annotation @XmlElement(nillable=true). package blog.json.nillable; import javax.xml.bind.annotation.*; @XmlRootElement public class Customer { private String firstName; private String middleName; private String lastName; public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getMiddleName() { return middleName; } public void setMiddleName(String middleName) { this.middleName = middleName; } @XmlElement(nillable=true) public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } } Demo In the demo code below we will set both the middleName and lastName properties to null. Since we have mapped these properties differently in the domain model we will examine the output to see the impact of using @XmlElement(nillable=true). package blog.json.nillable; import javax.xml.bind.*; public class Demo { public static void main(String[] args) throws Exception { Customer customer = new Customer(); customer.setFirstName("Jane"); customer.setMiddleName(null); customer.setLastName(null); JAXBContext jc = JAXBContext.newInstance(Customer.class); Marshaller marshaller = jc.createMarshaller(); marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); // Output XML marshaller.marshal(customer, System.out); // Output JSON marshaller.setProperty("eclipselink.media-type", "application/json"); marshaller.marshal(customer, System.out); } } XML Output By default JAXB implementations do not include null values in the output, so there is no element corresponding to the middleName property. Since we annotated the lastName property with @XmlElement(nillable=true) and it had a null value, it is represented in the output. In XML an element with a null value is represented by adding the xsi:nil="true" attribute. Jane JSON Output Just like in the XML output, an entry for the middleName property does not appear in the JSON output. Since we annotated the lastName property with @XmlElement(nillable=true) and it had a null value, it is represented in the output. In JSON a null value is represented with null. { "customer" : { "firstName" : "Jane", "lastName" : null } }

Do you know what day of the week was the day you were born? Monday or maybe Saturday? Well, perhaps you know that. Everybody knows the day he’s born on, but do you know what day was the 31st of January in 1883? No? Well, there must be some method to determine any day in any century. We know that 2012 started at Sunday. After we know that, it’s easy to determine what day is the 2nd of January. It should be Monday. But things get a little more complex if we try to guess some date distant from January the 1st. Indeed 1st of Jan was on Sunday, but what day is 9th of May the same year. This is far more difficult to say. Of course we can go with a brute force approach and count from 1/Jan till 9/May, but that is quite slow and error prone. So what would we do if we had to code a program that answers this question? The easiest way is to use a library. Almost every major library has built-in functions that can answer what day is on a given date. Such are date() in PHP or getDate() in JavaScript. But the question remains: How these library functions know the answer and how can we code such library functions if our library doesn’t support such functionality? There must be some algorithm to help us. Overview Because months have different number of days, and most of them aren’t divisible by 7 without a remainder, months begin on different days of the week. Thus, if January begins on Sunday, the month of February the same year will begin on Wednesday. Of course, in common years February has 28 days, which fortunately is divisible by 7 and thus February and March both begin on the same day, which is great, but isn’t true for leap years. What Do We Know About the Calendar First thing to know is that each week has exactly 7 days. We also know that a common year has 365 days, while a leap year has one day more – 366. Most of the months have 30 or 31 days, but February has only 28 days in common years and 29 in leap years. Because 365 mod 7 = 1 in a common year each year begins exactly on the next day of the preceding year. Thus if 2011 started on Saturday, 2012 starts on Sunday. And yet again, that is because 2011 is not a leap year. What else do we know? Because a week has exactly seven days only February (with its 28 days in a common year) is divisible by 7 (28 mod 7 = 0) and has exactly four weeks in it. Thus in a common year February and March start on a same day. Unfortunately that is not true about the other months. All these things we know about the calendar are great, so we can make some conclusions. Although eleven of the months have either 30 or 31 days they don’t start on a same day, but some of the months do appear to start on a same day just because the number of days between them is divisible by 7 without a remainder. Let’s take a look on some examples. For instance September has 30 days, as does November, while October, which is in between them has 31 days. Thus 30+30+31 makes 91. Fortunately 91 mod 7 = 0. So for each year September and December start on the same day (as they are after February they don’t depend on leap years). The same thing occurs to April and July and the good news is that in leap years even January starts on the same day as April and July. Now we know that there are some relations between months. Thus, if we know somehow that the 13th of April is Monday, we’ll be sure that 13th of July is also Monday. Let’s see now a summary of these observations. We can also refer to the following diagram. For leap years there are other corresponding months. Let’s take a look at the following image. Another way to get the same information is the following table. We also know that leap years happen to occur once every four years. However, if there is a common year like the year 2001, which will be the next year that is common and starts and corresponds exactly on 2001? Because of leap years we can have a year starting on one of the seven days of the week and to be either leap or common. This means just 14 combinations. Following these observations we can refer to the following table. You can clearly see the pattern “6 4 2 0” Here’s the month table. Columns 2 and 3 differs only for January and February. Clearly the day table is as follows: Now let’s go back to the algorithm. Using these tables and applying a simple formula, we can calculate what day was on some given date. Here are the steps of this algorithm. Get the number for the corresponding century from the centuries table; Get the last two digits from the year; Divide the number from step 2 by 4 and get it without the remainder; Get the month number from the month table; Sum the numbers from steps 1 to 4; Divide it by 7 and take the remainder; Find the result of step 6 in the days table; Implementation First let’s take a look at a simple and practical example of the example above and then the code. Let’s answer the question from the first paragraph of this post. What day was on January 31st, 1883? Take a look at the centuries table: for 1800 – 1899 this is 2. Get the last two digits from the year: 83. Divide 83 by 4 without a remainder: 83/4 = 20 Get the month number from the month table: Jan = 0. Sum the numbers from steps 1 to 4: 2 + 83 + 20 + 0 = 105. Divide it by 7 and take the remainder: 105 mod 7 = 0 Find the result of step 6 in the days table: Sunday = 0. The following code in PHP implements the algorithm above. function get_century_code($century) { // XVIII if (1700 <= $century && $century <= 1799) return 4; // XIX if (1800 <= $century && $century <= 1899) return 2; // XX if (1900 <= $century && $century <= 1999) return 0; // XXI if (2000 <= $century && $century <= 2099) return 6; // XXII if (2100 <= $century && $century <= 2199) return 4; // XXIII if (2200 <= $century && $century <= 2299) return 2; // XXIV if (2300 <= $century && $century <= 2399) return 0; // XXV if (2400 <= $century && $century <= 2499) return 6; // XXVI if (2500 <= $century && $century <= 2599) return 4; // XXVII if (2600 <= $century && $century <= 2699) return 2; } /** * Get the day of a given date * * @param $date */ function get_day_from_date($date) { $months = array( 1 => 0,// January 2 => 3,// February 3 => 3,// March 4 => 6,// April 5 => 1,// May 6 => 4,// June 7 => 6,// July 8 => 2,// August 9 => 5,// September 10 => 0,// October 11 => 3,// November 12 => 5,// December ); $days = array( 0 => 'Sunday', 1 => 'Monday', 2 => 'Tuesday', 3 => 'Wednesday', 4 => 'Thursday', 5 => 'Friday', 6 => 'Saturday', ); // calculate the date $dateParts = explode('-', $date); $century = substr($dateParts[2], 0, 2); $year = substr($dateParts[2], 2); // 1. Get the number for the corresponding century from the centuries table $a = get_century_code($dateParts[2]); // 2. Get the last two digits from the year $b = $year; // 3. Divide the number from step 2 by 4 and get it without the remainder $c = floor($year / 4); // 4. Get the month number from the month table $d = $months[$dateParts[1]]; // 5. Sum the numbers from steps 1 to 4 $e = $a + $b + $c + $d; // 6. Divide it by 7 and take the remainder $f = $e % 7; // 7. Find the result of step 6 in the days table return $days[$f]; } // Sunday echo get_day_from_date('31-1-1883'); Application This algorithm can be applied in many different cases although most of the libraries have built-in functions that can do that. The only problem besides that is that there are much more efficient algorithms that don’t need additional space (tables) of data. However this algorithm isn’t difficult to implement and it gives a good outlook of some facts in the calendar.

An old customer recently asked me if I had a solution for how to integrate between their existing JMS infrastructure on Websphere MQ with RabbitMQ. Although I know that RabbitMQ has the shovel plugin which can bridge between Rabbit instances I've yet not found a good plugin for JMS <-> AMQP forwarding. The first thing that came to my mind was to utilize a Spring Integration mediation as SI has excellent support for both JMS and Rabbit. Curious as I am I started a PoC and this is the result. It takes messages of a JMS queue and forwards to an AMQP exchange that is bound to a queue the consumer application is supposed to listen to. I used an external HornetQ instance in JBoss 6.1 as the JMS Provider, but I am 100% secure that the same setup would work for Websphere MQ as they both implement JMS. Be aware that I've done no performance tweaking or QoS setup yet as this is just a proof-of-concept. For a real setup you'd probably have to think about delivery guarantees versus performance and etc... The code will be available at a GitHub repository near you soon.. SpringContext in XML: org.jnp.interfaces.NamingContextFactory jnp://localhost:1099 org.jnp.interfaces:org.jboss.naming ConnectionFactory Maven POM: 4.0.0 org.rl si.jmstorabbit 0.0.1-SNAPSHOT jar si.jmstorabbit http://maven.apache.org UTF-8 2.2.5.Final 2.1.0.RELEASE springsource-release http://repository.springsource.com/maven/bundles/release false springsource-external http://repository.springsource.com/maven/bundles/external false org.springframework.integration spring-integration-core ${spring.integration.version} org.springframework.integration spring-integration-file ${spring.integration.version} org.springframework.integration spring-integration-amqp ${spring.integration.version} org.springframework.integration spring-integration-jms ${spring.integration.version} junit junit 3.8.1 test org.springframework spring-context 3.0.7.RELEASE jboss jnp-client 4.2.2.GA org.hornetq hornetq-core-client ${hornet.version} org.hornetq hornetq-jms-client ${hornet.version} org.hornetq hornetq-jms ${hornet.version} jboss jboss-common-client 3.2.3 org.jboss.netty netty 3.2.7.Final javax.jms jms 1.1

See original post at https://muhammadkhojaye.blogspot.com/2012/04/how-to-run-amazon-elastic-mapreduce-job.html Introduction Amazon EMR is a web service which can be used to easily and efficiently process enormous amounts of data. It uses a hosted Hadoop framework running on the web-scale infrastructure of Amazon EC2 and Amazon S3. Amazon EMR removes most of the cumbersome details of Hadoop while taking care of provisioning of Hadoop, running the job flow, terminating the job flow, moving the data between Amazon EC2 and Amazon S3, and optimizing Hadoop. In this tutorial, we will use a developed WordCount Java example using Hadoop and thereafter, we execute our program on Amazon Elastic MapReduce. Prerequisites You must have valid AWS account credentials. You should also have a general familiarity with using the Eclipse IDE before you begin. The reader can also use any other IDE of their choice. Step 1 – Develop MapReduce WordCount Java Program In this section, we are first going to develop a WordCount application. A WordCount program will determine how many times different words appear in a set of files. In Eclipse (or whatever the IDE you are using), Create simple Java Project with the name "WordCount". Create a java class name Map and override the map method as follow, public class Map extends Mapper { private final static IntWritable one = new IntWritable(1); private Text word = new Text(); @Override public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { String line = value.toString(); StringTokenizer tokenizer = new StringTokenizer(line); while (tokenizer.hasMoreTokens()) { word.set(tokenizer.nextToken()); context.write(word, one); } } } Create a java class named Reduce and override the reduce method as shown below, public class Reduce extends Reducer { @Override protected void reduce(Text key, java.lang.Iterable values, org.apache.hadoop.mapreduce.Reducer.Context context) throws IOException, InterruptedException { int sum = 0; for (IntWritable value : values) { sum += value.get(); } context.write(key, new IntWritable(sum)); } } Create a java class named WordCount and defined the main method as below, public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); Job job = new Job(conf, "wordcount"); job.setJarByClass(WordCount.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); job.setMapperClass(Map.class); job.setReducerClass(Reduce.class); job.setInputFormatClass(TextInputFormat.class); job.setOutputFormatClass(TextOutputFormat.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); } Export the WordCount program in a jar using eclipse and save it to some location on disk. Make sure that you have provided the Main Class (WordCount.jar) during extraction ofu8u the jar file as shown below. Our jar is ready!!! Step 2 – Upload the WordCount JAR and Input Files to Amazon S3 Now we are going to upload the WordCount jar to Amazon S3. First, go to the following URL: https://console.aws.amazon.com/s3/home Next, click “Create Bucket”, give your bucket a name, and click the “Create” button. Select your new S3 bucket in the left-hand pane. Upload the WordCount JAR and sample input file for counting the words. Step 3 – Running an Elastic MapReduce job Now that the JAR is uploaded into S3, all we need to do is to create a new Job flow. let's execute the steps below. (I encourage readers to check out the following link for details regarding each step, How to Create a Job Flow Using a Custom JAR ) Sign in to the AWS Management Console and open the Amazon Elastic MapReduce console at https://console.aws.amazon.com/elasticmapreduce/ Click Create New Job Flow. In the DEFINE JOB FLOW page, enter the following details, a) Job Flow Name = WordCountJob b) Select Run your own applications) Select Custom JAR in the drop-down list) Click Continue In the SPECIFY PARAMETERS page, enter values in the boxes using the following table as a guide, and then click Continue.JAR Location = bucketName/jarFileLocationJAR Arguments =s3n://bucketName/inputFileLocations3n://bucketName/outputpath Please note that the output path must be unique each time we execute the job. The Hadoop always create a folder with the same name specified here. After executing the job, just wait and monitor your job that runs through the Hadoop flow. You can also look for errors by using the Debug button. The job should be complete within 10 to 15 minutes (can also depend on the size of the input). After completing the job, You can view results in the S3 Browser panel. You can also download the files from S3 and can analyze the outcome of the job. Amazon Elastic MapReduce Resources Amazon Elastic MapReduce Documentation,http://aws.amazon.com/documentation/elasticmapreduce/ Amazon Elastic MapReduce Getting Started Guide,http://docs.amazonwebservices.com/ElasticMapReduce/latest/GettingStartedGuide/ Amazon Elastic MapReduce Developer Guide,http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/ Apache Hadoop,http://hadoop.apache.org/ See more at https://muhammadkhojaye.blogspot.com/2012/04/how-to-run-amazon-elastic-mapreduce-job.html

How to create a real-time face detection system using HTML5, JavaScript, and OpenCV, leveraging WebRTC for webcam access and WebSockets for client-server communication.

is a really easy way to provide property replacements in Spring configurations with values from a standard Java Properties file. But what if you don’t want a property hard coded into a file – a clear text password for instance? Spring provides all the bits and pieces to write your own property replacement. Let me introduce my CustomPropertyConfigurer. I’ll demonstrate using a variation on the theme of the Spring JDBC Template. MyQuery is a simple extension of org.springframework.jdbc.core.JDBCTemplate that gets the current timestamp from a MySQL database. Here’s the, hopefully familiar, configuration: Except the jdbc.properties file does not contain the password: jdbc.driverClassName=com.mysql.jdbc.Driver jdbc.url=jdbc:mysql://rob-7 jdbc.username=rob I will set the jdbc.password property myself from what is entered on the command line. public static void main(String... args) { char[] password = System.console().readPassword("Password: "); Properties properties = new Properties(); properties.setProperty("jdbc.password", new String(password)); ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( new String[] { "rob/MyQuery.xml"}, false); context.addBeanFactoryPostProcessor( new CustomPropertyConfigurer(properties)); context.refresh(); MyQuery myQuery = context.getBean(MyQuery.class); myQuery.run(); context.close(); } Where the CustomPropertyConfigurer is: import java.util.Properties; import org.springframework.beans.BeansException; import org.springframework.beans.factory.BeanDefinitionStoreException; import org.springframework.beans.factory.config.BeanDefinition; import org.springframework.beans.factory.config.BeanDefinitionVisitor; import org.springframework.beans.factory.config.BeanFactoryPostProcessor; import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; import org.springframework.util.PropertyPlaceholderHelper; import org.springframework.util.StringValueResolver; public class CustomPropertyConfigurer implements BeanFactoryPostProcessor { private final Properties properties; public CustomPropertyConfigurer(Properties properties) { this.properties = properties; } public void postProcessBeanFactory( ConfigurableListableBeanFactory beanFactoryToProcess) throws BeansException { BeanDefinitionVisitor visitor = new BeanDefinitionVisitor( new BeanDirectoryResolver()); String[] beanNames = beanFactoryToProcess.getBeanDefinitionNames(); for (int i = 0; i < beanNames.length; i++) { BeanDefinition bd = beanFactoryToProcess.getBeanDefinition( beanNames[i]); try { visitor.visitBeanDefinition(bd); } catch (BeanDefinitionStoreException ex) { throw new BeanDefinitionStoreException( bd.getResourceDescription(), beanNames[i], ex.getMessage()); } } } class BeanDirectoryResolver implements StringValueResolver { private final PropertyPlaceholderHelper helper; public BeanDirectoryResolver() { helper = new PropertyPlaceholderHelper("${", "}"); } public String resolveStringValue(String strVal) { return helper.replacePlaceholders(strVal, properties); } } } The CustomPropertyConfigurer gets applied first. It leaves any properties it can’t resolve (all but the password) for the standard to resolve. Unit tests running against a different jdbc.propeties file can continue to provide the password as before. Here it is running: There are many other examples of configuration values that might only be discovered at runtime – file names, schedule dates, form values etc. So long as the value can be a String, a CustomPropertyConfigurer provides a simple way of passing these values to Spring.

In my previous post I showed you how to install and use Redis with Node.js. Today I’m going to take that a step further and walk through some of the things you can do with node_redis using Redis’s TTL and EXPIRE commands. Note: If you haven’t gone through my previous article make sure to do that now as I’ll assume you have Node.js and Redis up and running. Create a new folder and put a new text file in it called: app.js Inside the app.js file we will add some simple code to set a value that doesn’t have a time to live (or expiration on it): var redis = require("redis") , client = redis.createClient(); client.on("error", function (err) { console.log("Error " + err); }); client.on("connect", runSample); function runSample() { // Set a value client.set("string key", "Hello World", function (err, reply) { console.log(reply.toString()); }); // Get a value client.get("string key", function (err, reply) { console.log(reply.toString()); }); } When we connect to Redis and everything is ready the runSample function is called which in turn sets a value and then reads it back. Expected output: OK Hello World Lets set a timeout on a value using the EXPIRE command and see what happens. Replace the original code with this: var redis = require('redis') , client = redis.createClient(); client.on('error', function (err) { console.log('Error ' + err); }); client.on('connect', runSample); function runSample() { // Set a value with an expiration client.set('string key', 'Hello World', redis.print); // Expire in 3 seconds client.expire('string key', 3); // This timer is only to demo the TTL // Runs every second until the timeout // occurs on the value var myTimer = setInterval(function() { client.get('string key', function (err, reply) { if(reply) { console.log('I live: ' + reply.toString()); } else { clearTimeout(myTimer); console.log('I expired'); client.quit(); } }); }, 1000); } Note: Be aware that the timer I use is just to demo the EXPIRE, you should be very careful about using timers in production Nodejs projects. Run the program. Expected results: Reply: OK I live: Hello World I live: Hello World I live: Hello World I expired Now we will check to see how much time a value has left before it expires: var redis = require('redis') , client = redis.createClient(); client.on('error', function (err) { console.log('Error ' + err); }); client.on('connect', runSample); function runSample() { // Set a value client.set('string key', 'Hello World', redis.print); // Expire in 3 seconds client.expire('string key', 3); // This timer is only to demo the TTL // Runs every second until the timeout // occurs on the value var myTimer = setInterval(function() { client.get('string key', function (err, reply) { if(reply) { console.log('I live: ' + reply.toString()); client.ttl('string key', writeTTL); } else { clearTimeout(myTimer); console.log('I expired'); client.quit(); } }); }, 1000); } function writeTTL(err, data) { console.log('I live for this long yet: ' + data); } Run the program. Expected results: Reply: OK I live: Hello World I live for this long yet: 2 I live: Hello World I live for this long yet: 1 I live: Hello World I live for this long yet: 0 I expired

This post shows code examples in Python (2.7) for sending data to Graphite. Once you have a Graphite server setup, with Carbon running/collecting, you need to send it data for graphing. Basically, you write a program to collect numeric values and send them to Graphite's backend aggregator (Carbon). To send data, you create a socket connection to the graphite/carbon server and send a message (string) in the format: "metric_path value timestamp\n" `metric_path`: arbitrary namespace containing substrings delimited by dots. The most general name is at the left and the most specific is at the right. `value`: numeric value to store. `timestamp`: epoch time. messages must end with a trailing newline. multiple messages maybe be batched and sent in a single socket operation. each message is delimited by a newline, with a trailing newline at the end of the message batch. Example message: "foo.bar.baz 42 74857843\n" Let's look at some (Python 2.7) code for sending data to graphite... Here is a simple client that sends a single message to graphite. Code: #!/usr/bin/env python import socket import time CARBON_SERVER = '0.0.0.0' CARBON_PORT = 2003 message = 'foo.bar.baz 42 %d\n' % int(time.time()) print 'sending message:\n%s' % message sock = socket.socket() sock.connect((CARBON_SERVER, CARBON_PORT)) sock.sendall(message) sock.close() Here is a command line client that sends a single message to graphite: Usage: $ python client-cli.py metric_path value Code: #!/usr/bin/env python import argparse import socket import time CARBON_SERVER = '0.0.0.0' CARBON_PORT = 2003 parser = argparse.ArgumentParser() parser.add_argument('metric_path') parser.add_argument('value') args = parser.parse_args() if __name__ == '__main__': timestamp = int(time.time()) message = '%s %s %d\n' % (args.metric_path, args.value, timestamp) print 'sending message:\n%s' % message sock = socket.socket() sock.connect((CARBON_SERVER, CARBON_PORT)) sock.sendall(message) sock.close() Here is a client that collects load average (Linux-only) and sends a batch of 3 messages (1min/5min/15min loadavg) to graphite. It will run continuously in a loop until killed. (adjust the delay for faster/slower collection interval): #!/usr/bin/env python import platform import socket import time CARBON_SERVER = '0.0.0.0' CARBON_PORT = 2003 DELAY = 15 # secs def get_loadavgs(): with open('/proc/loadavg') as f: return f.read().strip().split()[:3] def send_msg(message): print 'sending message:\n%s' % message sock = socket.socket() sock.connect((CARBON_SERVER, CARBON_PORT)) sock.sendall(message) sock.close() if __name__ == '__main__': node = platform.node().replace('.', '-') while True: timestamp = int(time.time()) loadavgs = get_loadavgs() lines = [ 'system.%s.loadavg_1min %s %d' % (node, loadavgs[0], timestamp), 'system.%s.loadavg_5min %s %d' % (node, loadavgs[1], timestamp), 'system.%s.loadavg_15min %s %d' % (node, loadavgs[2], timestamp) ] message = '\n'.join(lines) + '\n' send_msg(message) time.sleep(DELAY) Resources: Graphite Docs Graphite Docs - Getting Your Data Into Graphite Installing Graphite 0.9.9 on Ubuntu 12.04 LTS Installing and configuring Graphite END

In this article we'll look at how you can connect from Scala to RabbitMQ so you can support the AMQP protocol from your applications. In this example I'll use the Play Framework 2.0 as container (for more info on this see my other article on this subject) to run the application in, since Play makes developing with Scala a lot easier. This article will also use Akka actors to send and receive the messages from RabbitMQ. What is AMQP First, a quick introduction into AMQP. AMQP stands for "Advanced Message Queueing Protocol" and is an open standard for messaging. The AMQP homepage states their vision as this: "To become the standard protocol for interoperability between all messaging middleware". AMQP defines a transport level protocol for exchanging messages that can be used to integrate applications from a number of different platform, languages and technologies. There are a number of tools implementing this protocol, but one that is getting more and more attention is RabbitMQ. RabbitMQ is an open source, erlang based message broker that uses AMQP. All application that can speak AMQP can connect to and make use of RabbitMQ. So in this article we'll show how you can connect from your Play2/Scala/Akka based application to RabbitMQ. In this article we'll show you how to do implement the two most common scenarios: Send / recieve: We'll configure one sender to send a message every couple of seconds, and use two listeners that will read the messages, in a round robin fashion, from the queue. Publish / subscribe: For this example we'll create pretty much the same scenario, but this time, the listeners will both get the message at the same time. I assume you've got an installation of RabbitMQ. If not follow the instructions from their site. Setup basic Play 2 / Scala project For this example I created a new Play 2 project. Doing this is very easy: [email protected]:~/Dev/play-2.0-RC2$ ./play new Play2AndRabbitMQ _ _ _ __ | | __ _ _ _| | | '_ \| |/ _' | || |_| | __/|_|\____|\__ (_) |_| |__/ play! 2.0-RC2, http://www.playframework.org The new application will be created in /Users/jos/Dev/play-2.0/PlayAndRabbitMQ What is the application name? > PlayAndRabbitMQ Which template do you want to use for this new application? 1 - Create a simple Scala application 2 - Create a simple Java application 3 - Create an empty project > 1 OK, application PlayAndRabbitMQ is created. Have fun! I am used to work from Eclipse with the scala-ide pluging, so I execute play eclipsify and import the project in Eclipse. The next step we need to do is set up the correct dependencies. Play uses sbt for this and allows you to configure your dependencies from the build.scala file in your project directory. The only dependency we'll add is the java client library from RabbitMQ. Even though Lift provides a scala based AMQP library, I find using the RabbitMQ one directly just as easy. After adding the dependency my build.scala looks like this: import sbt._ import Keys._ import PlayProject._ object ApplicationBuild extends Build { val appName = "PlayAndRabbitMQ" val appVersion = "1.0-SNAPSHOT" val appDependencies = Seq( "com.rabbitmq" % "amqp-client" % "2.8.1" ) val main = PlayProject(appName, appVersion, appDependencies, mainLang = SCALA).settings( ) } Add rabbitMQ configuration to the config file For our examples we can configure a couple of things. The queue where to send the message to, the exchange to use, and the host where RabbitMQ is running. In a real world scenario we would have more configuration options to set, but for this case we'll just have these three. Add the following to your application.conf so that we can reference it from our application. #rabbit-mq configuration rabbitmq.host=localhost rabbitmq.queue=queue1 rabbitmq.exchange=exchange1 We can now access these configuration files using the ConfigFactory. To allow easy access create the following object: object Config { val RABBITMQ_HOST = ConfigFactory.load().getString("rabbitmq.host"); val RABBITMQ_QUEUE = ConfigFactory.load().getString("rabbitmq.queue"); val RABBITMQ_EXCHANGEE = ConfigFactory.load().getString("rabbitmq.exchange"); } Initialize the connection to RabbitMQ We've got one more object to define before we'll look at how we can use RabbitMQ to send and receive messages. to work with RabbitMQ we require a connection. We can get a connection to a server by using a ConnectionFactory. Look at the javadocs for more information on how to configure the connection. object RabbitMQConnection { private val connection: Connection = null; /** * Return a connection if one doesn't exist. Else create * a new one */ def getConnection(): Connection = { connection match { case null => { val factory = new ConnectionFactory(); factory.setHost(Config.RABBITMQ_HOST); factory.newConnection(); } case _ => connection } } } Start the listeners when the application starts We need to do one more thing before we can look at the RabbitMQ code. We need to make sure our message listeners are registered on application startup and our senders start sending. Play 2 provides a GlobalSettings object for this which you can extend to execute code when your application starts. For our example we'll use the following object (remember, this needs to be stored in the default namespace: import play.api.mvc._ import play.api._ import rabbitmq.Sender object Global extends GlobalSettings { override def onStart(app: Application) { Sender.startSending } } We'll look at this Sender.startSending operation, which initializes all the senders and receivers in the following sections. Setup send and receive scenario Let's look at the Sender.startSending code that will setup a sender that sends a msg to a specific queue. For this we use the following piece of code: object Sender { def startSending = { // create the connection val connection = RabbitMQConnection.getConnection(); // create the channel we use to send val sendingChannel = connection.createChannel(); // make sure the queue exists we want to send to sendingChannel.queueDeclare(Config.RABBITMQ_QUEUE, false, false, false, null); Akka.system.scheduler.schedule(2 seconds, 1 seconds , Akka.system.actorOf(Props( new SendingActor(channel = sendingChannel, queue = Config.RABBITMQ_QUEUE))) , "MSG to Queue"); } } class SendingActor(channel: Channel, queue: String) extends Actor { def receive = { case some: String => { val msg = (some + " : " + System.currentTimeMillis()); channel.basicPublish("", queue, null, msg.getBytes()); Logger.info(msg); } case _ => {} } } In this code we take the following steps: Use the factory to retrieve a connection to RabbitMQ Create a channel on this connection to use in communicating with RabbitMQ Use the channel to create the queue (if it doesn't exist yet) Schedule Akka to send a message to an actor every second. This all should be pretty straightforward. The only (somewhat) complex part is the scheduling part. What this schedule operation does is this. We tell Akka to schedule a message to be sent to an actor. We want a 2 seconds delay before it is fired, and we want to repeat this job every second. The actor that should be used for this is the SendingActor you can also see in this listing. This actor needs access to a channel to send a message and this actor also needs to know where to send the message it receives to. This is the queue. So every second this Actor will receive a message, append a timestamp, and use the provided channel to send this message to the queue: channel.basicPublish("", queue, null, msg.getBytes());. Now that we send a message each second it would be nice to have listeners on this queue that can receive messages. For receiving messages we've also created an Actor that listens indefinitely on a specific queue. class ListeningActor(channel: Channel, queue: String, f: (String) => Any) extends Actor { // called on the initial run def receive = { case _ => startReceving } def startReceving = { val consumer = new QueueingConsumer(channel); channel.basicConsume(queue, true, consumer); while (true) { // wait for the message val delivery = consumer.nextDelivery(); val msg = new String(delivery.getBody()); // send the message to the provided callback function // and execute this in a subactor context.actorOf(Props(new Actor { def receive = { case some: String => f(some); } })) ! msg } } } This actor is a little bit more complex than the one we used for sending. When this actor receives a message (kind of message doesn't matter) it starts listening on the queue it was created with. It does this by creating a consumer using the supplied channel and tells the consumers to start listening on the specified queue. The consumer.nextDelivery() method will block until a message is waiting in the configured queue. Once a message is received, a new Actor is created to which the message is sent. This new actor passes the message on to the supplied method, where you can put your business logic. To use this listener we need to supply the following arguments: Channel: Allows access to RabbitMQ Queue: The queue to listen to for messages f: The function that we'll execute when a message is received. The final step for this first example is glueing everything together. We do this by adding a couple of method calls to the Sender.startSending method. def startSending = { ... val callback1 = (x: String) => Logger.info("Recieved on queue callback 1: " + x); setupListener(connection.createChannel(),Config.RABBITMQ_QUEUE, callback1); // create an actor that starts listening on the specified queue and passes the // received message to the provided callback val callback2 = (x: String) => Logger.info("Recieved on queue callback 2: " + x); // setup the listener that sends to a specific queue using the SendingActor setupListener(connection.createChannel(),Config.RABBITMQ_QUEUE, callback2); ... } private def setupListener(receivingChannel: Channel, queue: String, f: (String) => Any) { Akka.system.scheduler.scheduleOnce(2 seconds, Akka.system.actorOf(Props(new ListeningActor(receivingChannel, queue, f))), ""); } In this code you can see that we define a callback function, and use this callback function, together with the queue and the channel to create the ListeningActor. We use the scheduleOnce method to start this listener in a separate thread. Now with this code in place we can run the application (play run) open up localhost:9000 to start the application and we should see something like the following output. [info] play - Starting application default Akka system. [info] play - Application started (Dev) [info] application - MSG to Exchange : 1334324531424 [info] application - MSG to Queue : 1334324531424 [info] application - Recieved on queue callback 2: MSG to Queue : 1334324531424 [info] application - MSG to Exchange : 1334324532522 [info] application - MSG to Queue : 1334324532522 [info] application - Recieved on queue callback 1: MSG to Queue : 1334324532522 [info] application - MSG to Exchange : 1334324533622 [info] application - MSG to Queue : 1334324533622 [info] application - Recieved on queue callback 2: MSG to Queue : 1334324533622 [info] application - MSG to Exchange : 1334324534722 [info] application - MSG to Queue : 1334324534722 [info] application - Recieved on queue callback 1: MSG to Queue : 1334324534722 [info] application - MSG to Exchange : 1334324535822 [info] application - MSG to Queue : 1334324535822 [info] application - Recieved on queue callback 2: MSG to Queue : 1334324535822 Here you can clearly see the round-robin way messages are processed. Setup publish and subscribe scenario Once we've got the above code running, adding publish / subscribe functionality is very trivial. Instead of the SendingActor we now use a PublishingActor: class PublishingActor(channel: Channel, exchange: String) extends Actor { /** * When we receive a message we sent it using the configured channel */ def receive = { case some: String => { val msg = (some + " : " + System.currentTimeMillis()); channel.basicPublish(exchange, "", null, msg.getBytes()); Logger.info(msg); } case _ => {} } } An exchange is used by RabbitMQ to allow multiple recipients to receive the same message (and a whole lot of other advanced functionality). The only change in the code from the other actor is that this time we send the message to an exchange instead of to a queue. The listener code is exactly the same, the only thing we need to do is connect a queue to a specific exchange. So that listeners on that queue receive the messages sent to to the exchange. We do this, once again, from the setup method we used earlier. ... // create a new sending channel on which we declare the exchange val sendingChannel2 = connection.createChannel(); sendingChannel2.exchangeDeclare(Config.RABBITMQ_EXCHANGEE, "fanout"); // define the two callbacks for our listeners val callback3 = (x: String) => Logger.info("Recieved on exchange callback 3: " + x); val callback4 = (x: String) => Logger.info("Recieved on exchange callback 4: " + x); // create a channel for the listener and setup the first listener val listenChannel1 = connection.createChannel(); setupListener(listenChannel1,listenChannel1.queueDeclare().getQueue(), Config.RABBITMQ_EXCHANGEE, callback3); // create another channel for a listener and setup the second listener val listenChannel2 = connection.createChannel(); setupListener(listenChannel2,listenChannel2.queueDeclare().getQueue(), Config.RABBITMQ_EXCHANGEE, callback4); // create an actor that is invoked every two seconds after a delay of // two seconds with the message "msg" Akka.system.scheduler.schedule(2 seconds, 1 seconds, Akka.system.actorOf(Props( new PublishingActor(channel = sendingChannel2 , exchange = Config.RABBITMQ_EXCHANGEE))), "MSG to Exchange"); ... We also created an overloaded method for setupListener, which, as an extra parameter, also accepts the name of the exchange to use. private def setupListener(channel: Channel, queueName : String, exchange: String, f: (String) => Any) { channel.queueBind(queueName, exchange, ""); Akka.system.scheduler.scheduleOnce(2 seconds, Akka.system.actorOf(Props(new ListeningActor(channel, queueName, f))), ""); } In this small piece of code you can see that we bind the supplied queue (which is a random name in our example) to the specified exchange. After that we create a new listener as we've seen before. Running this code now will result in the following output: [info] play - Application started (Dev) [info] application - MSG to Exchange : 1334325448907 [info] application - MSG to Queue : 1334325448907 [info] application - Recieved on exchange callback 3: MSG to Exchange : 1334325448907 [info] application - Recieved on exchange callback 4: MSG to Exchange : 1334325448907 [info] application - MSG to Exchange : 1334325450006 [info] application - MSG to Queue : 1334325450006 [info] application - Recieved on exchange callback 4: MSG to Exchange : 1334325450006 [info] application - Recieved on exchange callback 3: MSG to Exchange : 1334325450006 As you can see, in this scenario both listeners receive the same message. That pretty much wraps it up for this article. As you've seen using the Java based client api for RabbitMQ is more than sufficient, and easy to use from Scala. Note though that this example is not production ready, you should take care to close connections, nicely shutdown listeners and actors. All this shutdown code isn't shown here.

our new tutorial tells us how to create an animated 3d sphere (through direct access to pixels on the canvas). the sphere itself moves around the canvas continuously. this example should work in most modern browsers (like firefox, chrome, safari and even in ie). in the end, you should to get something like this: here are our demo and downloadable package: live demo download in package ok, download the source files and let's start coding ! step 1. html this is the markup of our page. index.html i prepared 2 canvas objects here: the first for the source image, and the second one for our sphere. step 2. css css/main.css .container { height: 631px; margin: 50px auto; position: relative; width: 1024px; z-index: 1; } #obj { position: absolute; z-index: 2; } we should put our sphere object above our main canvas. step 3. js js/script.js var canvas, ctx; var canvasobj, ctxobj; var idstw = 256; var idsth = 256; var ixspeed = 4; var iyspeed = 3; var ilastx = idstw / 2; var ilasty = idsth / 2; var oimage; var amap = []; var abitmap; var mathsphere = function(px, py) { var x = px - idstw / 2; var y = py - idsth / 2; var r = math.sqrt(x * x + y * y); var maxr = idstw / 2; if (r > maxr) return {'x':px, 'y':py}; var a = math.atan2(y, x); var k = (r / maxr) * (r / maxr) * 0.5 + 0.5; var dx = math.cos(a) * r * k; var dy = math.sin(a) * r * k; return {'x': dx + idstw / 2, 'y': dy + idsth / 2}; } window.onload = function(){ // load background oimage = new image(); oimage.src="images/bg.jpg"; oimage.onload = function () { // creating canvas and context objects canvas = document.getelementbyid('slideshow'); ctx = canvas.getcontext('2d'); canvasobj = document.getelementbyid('obj'); ctxobj = canvasobj.getcontext('2d'); // clear context ctx.clearrect(0, 0, ctx.canvas.width, ctx.canvas.height); // and draw source image ctx.drawimage(oimage, 0, 0); abitmap = ctx.getimagedata(0, 0, idstw, idsth); for (var y = 0; y < idsth; y++) { for (var x = 0; x < idstw; x++) { var t = mathsphere(x, y); amap[(x + y * idsth) * 2 + 0] = math.max(math.min(t.x, idstw - 1), 0); amap[(x + y * idsth) * 2 + 1] = math.max(math.min(t.y, idsth - 1), 0); } } // begin updating scene updatescene(); }; function updatescene() { // update last coordinates ilastx = ilastx + ixspeed; ilasty = ilasty + iyspeed; // reverse speed if (ilastx > ctx.canvas.width - idstw/2) { ixspeed = -3; } if (ilastx < idstw/2) { ixspeed = 3; } if (ilasty > ctx.canvas.height - idsth/2) { iyspeed = -3; } if (ilasty < idsth/2) { iyspeed = 3; } // shifting of the second object canvasobj.style.left = ilastx - math.floor(idstw / 2) + 'px'; canvasobj.style.top = ilasty - (math.floor(idsth / 2)) + 'px'; // draw result sphere var adata = ctx.getimagedata(ilastx - math.ceil(idstw / 2), ilasty - math.ceil(idsth / 2), idstw, idsth + 1); for (var j = 0; j < idsth; j++) { for (var i = 0; i < idstw; i++) { var u = amap[(i + j * idsth) * 2]; var v = amap[(i + j * idsth) * 2 + 1]; var x = math.floor(u); var y = math.floor(v); var kx = u - x; var ky = v - y; for (var c = 0; c < 4; c++) { abitmap.data[(i + j * idsth) * 4 + c] = (adata.data[(x + y * idsth) * 4 + c] * (1 - kx) + adata.data[((x + 1) + y * idsth) * 4 + c] * kx) * (1-ky) + (adata.data[(x + (y + 1) * idsth) * 4 + c] * (1 - kx) + adata.data[((x + 1) + (y + 1) * idsth) * 4 + c] * kx) * (ky); } } } ctxobj.putimagedata(abitmap,0,0); // update timer settimeout(updatescene, 16); } }; during initialization, the script prepares two canvas objects and two contexts. then, it loads our main background image, and draws it as our first context. then it prepares a hash table of sphere transformations: amap. and, in the end – it starts the timer, which updates the main scene. in this function (updatescene) we update the coordinates of our sphere object, and draw the updated sphere at our second location. live demo download in package conclusion i hope that today’s 3d html5 sphere lesson has been interesting for you. we have done another nice html5 example. i will be glad to see your thanks and comments. good luck!

As you know I'm open-sourcing and completely overhauling my PhD system. One of my goals was to replace internal JMS queues with AMQP. Today I'll show you how I did it and why I was forced to change RabbitMQ to Apache Qpid. AMQP In short. AMQP is an open standard application layer protocol for message-oriented middleware. The most important feature is that AMQP is a wire-level protocol and is interoperable by design. JMS is just an API. Altough JMS brokers can be used in .NET applications (see my post: ActiveMQ and .NET combined!), the whole JMS specification does not guarantee interoperability. Also, the AMQP standard is by design more flexible and powerful (e.g., supports two-way communication by design) - they simply learnt from JMS mistakes :). Oh, forgot to mention. The AMQP was originally developed by banks :) so I don't have to say that AMQP is secure, fault-tolerant, and so on. RabbitMQ RabbitMQ is the most mature AMQP broker. RabbitMQ is written in Erlang so you have to download that first (RabbitMQ Windows installer does it for you). Download it from here: http://www.rabbitmq.com/. I also recommend installing the web management console. From Rabbit's sbin directory execute: rabbitmq-plugins enable rabbitmq_management If you're on Windows and you installed a Rabbit service you have to restart it. That's it. Spring Well, it turned out that VMware bought RabbitMQ and SpringSource developers are now developing it. Given this fact, you shouldn't be surprised that Spring - RabbitMQ integration is childishly simple. Add spring-rabbit dependency to your Maven project, and then in Spring configuration paste the following: The default configuration assumes that RabbitMQ is running on a local server using the default port and default credentials (guest/guest). Of course all these settings are configurable. To sent a message to "myqueue" queue, just inject an instance of AmqpTemplate into your service and send the message. An example would be: @Service public class HomeController { @Autowired private AmqpTemplate amqpTemplate; public void sendMessage(Bundle bundle) throws IOException { byte[] body = IOUtils.toByteArray(bundle.getInputStream()); MessageProperties messageProperties = new MessageProperties(); messageProperties.setContentType(bundle.getContentType()); messageProperties.setContentLength(bundle.getSize()); messageProperties.setTimestamp(new Date()); messageProperties.setDeliveryMode(MessageDeliveryMode.PERSISTENT); Message message = new Message(body, messageProperties); amqpTemplate.send(message); } } You can open the web console http://localhost:55672/mgmt/ and see 1 message in "myqueue" queue. Apache Camel To read a message from Apache Camel you first have to add camel-amqp dependency to your POM. Then just copy and paste the following route definition: Run the route by executing mvn:camel-run and... you'll see an error. Making a long story short, Apache Camel 2.9.0 doesn't work with RabbitMQ. This is because the camel-amqp component is using the Apache Qpid client under the hood. The current Qpid version is 0.14, but Qpid guys forgot to upload new jars to the Maven public repo. Thus camel-amqp is still using Qpid 0.12 whose client doesn't seem to negotiate protocols. Even if you exclude qpid-commons and qpid-client dependencies and explicitly add Qpid 0.14 ones (download them and install in your local repo) there will be an exception thrown from the camel-amqp component as there is no longer a default ConnectionFactory constructor. Thus I was forced to install Qpid. Qpid I downloaded the Java server and simply ran it. There is no web management console, but that's OK. You can use JConsole for JMX. Spring AMQP and Qpid In order to make Spring AMQP work with Qpid copy and paste the following configuration: As you can see in the above snippet I explicitly created AMPQComponent with connectionFactory set to Apache Qpid AMQConnectionFactory object. Source code and working example This solution is a part of the Qualitas project. I use Spring MVC to handle uploads of business processes bundles (e.g., zipped archive of a WS-BPEL process) and send it to an AMQP queue. Then Apache Camel consumes the message, does additional processing of the bundle, and installs it on a remote business process execution engine. The projects you are most interested in are: qualitas-webapp (Spring MVC sending messages to AMQP) qualitas-internall-installation (Apache Camel route consuming messages from AMQP) To check out 0.0.2-SNAPSHOT tag from here: http://code.google.com/p/qualitas/source/browse/. Qualitas Read more about Qualitas project here: http://code.google.com/p/qualitas/. Happy to welcome new developers on board! cheers, Łukasz

This is the first part of a two part article about caching in WCF services. In this part I will explain the in-process memory cache available in .NET 4.0. In the second part I will describe the Windows AppFabric distributed memory cache. The .NET framework has provided a cache for ASP.NET applications since version 1.0. For other types of applications like WPF applications or console application, caching was never possible out of the box. Only WCF services were able to use the ASP.NET cache if they were configured to run in ASP.NET compatibility mode. But this mode has some performance drawbacks and only works when the WCF service is hosted inside IIS and uses an HTTP-based binding. With the release of the .NET 4.0 framework this has luckily changed. Microsoft has now developed an in-process memory cache that does not rely on the ASP.NET framework. This cache can be found in the “System.Runtime.Caching.dll” assembly. In order to explain the working of the cache, I have a created a simple sample application. It consists of a very slow repository called “SlowRepository”. public class SlowRepository { public IEnumerable GetPizzas() { Thread.Sleep(10000); return new List() { "Hawaii", "Pepperoni", "Bolognaise" }; } } This repository is used by my sample WCF service to gets its data. public class PizzaService : IPizzaService { private const string CacheKey = "availablePizzas"; private SlowRepository repository; public PizzaService() { this.repository = new SlowRepository(); } public IEnumerable GetAvailablePizzas() { ObjectCache cache = MemoryCache.Default; if(cache.Contains(CacheKey)) return (IEnumerable)cache.Get(CacheKey); else { IEnumerable availablePizzas = repository.GetPizzas(); // Store data in the cache CacheItemPolicy cacheItemPolicy = new CacheItemPolicy(); cacheItemPolicy.AbsoluteExpiration = DateTime.Now.AddHours(1.0); cache.Add(CacheKey, availablePizzas, cacheItemPolicy); return availablePizzas; } } } When the WCF service method GetAvailablePizzas is called, the service first retrieves the default memory cache instance ObjectCache cache = MemoryCache.Default; Next, it checks if the data is already available in the cache. If so, the cached data is used. If not, the repository is called to get the data and afterwards the data is stored in the cache. For my sample service, I also choose to restrict the maximum memory to 20% of the total physical memory. This can be done in the web.config.