Dynamic and static properties are really interesting. Learn about managing dynamic property in Spring.

A Robust Splitter Aggregator Design Strategy - Messaging Gateway Adapter Pattern What do we mean by robust? In the context of this article, robustness refers to an ability to manage exception conditions within a flow without immediately returning to the caller. In some processing scenarios n of m responses is good enough to proceed to conclusion. Example processing scenarios that typically have these tendencies are: Quotations for finance, insurance and booking systems. Fan-out publishing systems. Why do we need Robust Splitter Aggregator Designs? First and foremost an introduction to a typical Splitter Aggregator pattern maybe necessary. The Splitter is an EIP pattern that describes a mechanism for breaking composite messages into parts in order that they can be processed individually. A Router is an EIP pattern that describes routing messages into channels - aiming them at specific messaging endpoints. The Aggregator is an EIP pattern that collates and stores a set of messages that belong to a group, and releases them when that group is complete. Together, those three EIP constructs form a powerful mechanism for dividing processing into distinct units of work. Spring Integration (SI) uses the same pattern terminology as EIP and so readers of that methodology will be quite comfortable with Spring Integration Framework constructs. The SI Framework allows significant customisations of all three of those constructs and furthermore, by simply using asynchronous channels as you would in any other multi-threaded configuration, allows those units of work to be executed in parallel. An interesting challenge working with SI Splitter Aggregator designs is building appropriately robust flows that operate predictably in a number of invocation scenarios. A simple splitter aggregator design can be used in many circumstances and operate without heavy customisation of the SI constructs. However, some service requirements demand a more robust processing strategy and therefore more complex configuration. The following sections describe and show what a Simple Splitter Aggregator design actually looks like, the type of processing your design must be able to deal with and then goes on to suggest candidate solutions for more robust processing. A Simple Splitter Aggregator Design The following Splitter Aggregator design shows a simple flow that receives document request messages into messaging gateway, splits the message into two processing routes and then aggregates the response. Note that the diagram has been built from EIP constructs in OmniGraffle rather than being an Integration Graph view from within STS; the channels are missing from the diagram for the sake of brevity. SI Constructs in detail: Messaging Gateways - there are three messaging gateways. A number of configurations are available for gateway specifications but significantly can return business objects, exceptions and nulls (following a timeout). The gateway to the far left is the service gateway for which we are defining the flow. The other two gateways, between the Router and Aggregator, are external systems that will be providing responses to business questions that our flow generates. The Splitter - a single splitter exists and is responsible for consuming the document message and producing a collection of messages for onward processing. The Java signature for the, most often, custom Splitter specifies a single object argument and a collection for return. The Recipient List Router - a single router exists, any appropriate router can be used, chose the one that closely matches your requirements - you can easily route by expression or payload type. The primary purpose of the router is route a collection of messages supplied by the splitter. This is a pretty typical Splitter Aggregator configuration. Aggregator - a single construct that is responsible for collecting messages together in a group in order that further processing can take place on the gateway responses. Although the Aggregator can be configured with attributes and bean definitions to provide alternative grouping and release strategies, most often the default aggregation strategy suffices. Interesting Aspects of Splitter Aggregator Operation Gateway - the inbound gateway, the one on the far left, may or may not have an error handling bean reference defined on it. If it does then that bean will have an opportunity to handle an exceptions thrown within the flow to the right of that gateway. If not, any exception will be thrown straight out of the gateway. Gateway - an optional default-reply-timeout can be set on each of the gateways, there are significant implications for setting this value, ensure that they're well understood. An expired timeout will result in a null being returned from the gateway. This is the very same condition that can lead to a thread getting parked if an upstream gateway also has no default-reply-timeout set. Splitter Input Channel - this can be a simple direct channel or a direct channel with a dispatcher defined on it. If the channel has a dispatcher specified the flow downstream of this point will be asynchronous, multi-threaded. This also changes the upstream gateway semantics as it usually means that an otherwise impotent default-reply-timeout becomes active. Splitter - the splitter must return a single object. The single object returned by the splitter is a collection, a java.util.List. The SI framework will take each member of that list and feed it into the output-channel of the Splitter - as with this example, usually straight into a router. The contract for Splitter List returns is as its use in Java - it may contain zero, one or more elements. If the splitter returns an empty list it's unlikely that the router will have any work to do and so the flow invocation will be complete. However, if the List contains one item, the SI framework will extract that item from the list and push it into the router, if this gets routed successfully, the flow will continue. Router - the router will simply route messages into one of two gateways in this example. Gateways - the two gateways that are used between the Splitter and Aggregator are interesting. In this example I have used the generic gateway EIP pattern to represent a message sub-system but not defined it explicitly - we could use an HTTP outbound gateway, another SI flow or any other external system. Of course, for each of those sub-systems, a number of responses is possible. Depending on the protocol and external system, the message request may fail to send, the response fail to arrive, a long running process invoked, a network error or timeout or a general processing exception. Aggregator - the single aggregator will wait for a number of responses depending on what's been created by the Splitter. In the case where the splitter return list is empty the Aggregator will not get invoked. In the case where the Splitter return list has one entry, the aggregator will be waiting for one gateway response to complete the group. In the case where the Splitter list has n entries the Aggregator will be waiting for n entries to complete the group. Custom correlation strategies, release strategies and message stores can be injected amongst a set of rich configuration aspects. Interesting Aspects of Simple Splitter Aggregator Operation The primary deciding factor for establishing whether this type of simple gateway is adequate for requirements is to understand what happens in the event of failure. If any exception occurring in your SI flow results in the flow invocation being abandoned and that suits your requirements, there's no need to read any further. If, however, you need to continue processing following failure in one of the gateways the remainder of this article may be of more interest. Exceptions, from any source, generated between the splitter and aggregator, will result in an empty or partial group being discarded by the Aggregator. The exception will propagate back to the closest upstream gateway for either handling by a custom bean or re-throwing by the gateway. Note that a custom release strategy on the Aggregator is difficult to use and especially so alongside timeouts but would not help in this case as the exception will propagate back to the leftmost gateway before the aggregator is invoked. It's also possible to configure exception handlers on the innermost gateways, the exception message could be caught but how do you route messages from a custom exception handler into the aggregator to complete the group, inject the aggregator channel definition into the custom exception handler? This is a poor approach and would involve unpacking an exception message payload, copying the original message headers into a new SI message and then adding the original payload - only four or five lines of code, but dirty it is. Following exception generation, exception messages (without modification) cannot be routed into an Aggregator to complete the group. The original message, the one that contains the correlation and sequence ids for the group and group position are buried inside the SI messages exception payload. If processing needs to continue following exception generation, it should be clear that in order to continue processing, the following must take place: the aggregation group needs to be completed, any exceptions must be caught and handled before getting back to the closet upstream gateway, the correlation and sequence identifiers that allow group completion in the aggregator are buried within the exception message payload and will require extraction and setting on the message that's bound for the aggregator A More Robust Solution - Messaging Gateway Adapter Pattern Dealing with exceptions and null returns from gateways naturally leads to a design that implements a wrapper around the messaging gateway. This affords a level of control that would otherwise be very difficult to establish. This adapter technique allows all returns from messaging gateways to be caught and processed as the messaging gateway is injected into the Service Activator and called directly from that. The messaging gateway no longer responds to the aggregator directly, it responds to a custom Java code Spring bean configured in the Service Activator namespace definition. As expected, processing that does not undergo exception will continue as normal. Those flows that experience exception conditions or unexpected or missing responses from messaging gateways need to process messages in such as way that message groups bound for aggregation can be completed. If the Service Activator were to allow the exception to be propagated outside of it's backing bean, the group would not complete. The same applies not just for exceptions but any return object that does not carry the prerequisite group correlation id and sequence headers - this is where the adaptation is applied. Exception messages or null responses from messaging gateways are caught and handled as shown in the following example code: import com.l8mdv.sample.*; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.integration.Message; import org.springframework.integration.MessageHeaders; import org.springframework.integration.support.MessageBuilder; import org.springframework.util.Assert; public class AvsServiceImpl implements AvsService { private static final Logger logger = LoggerFactory.getLogger(AvsServiceImpl.class); public static final String MISSING_MANDATORY_ARG = "Mandatory argument is missing."; private AvsGateway avsGateway; public AvsServiceImpl(final AvsGateway avsGateway) { this.avsGateway = avsGateway; } public Message service(Message message) { Assert.notNull(message, MISSING_MANDATORY_ARG); Assert.notNull(message.getPayload(), MISSING_MANDATORY_ARG); MessageHeaders requestMessageHeaders = message.getHeaders(); Message responseMessage = null; try { logger.debug("Entering AVS Gateway"); responseMessage = avsGateway.send(message); if (responseMessage == null) responseMessage = buildNewResponse(requestMessageHeaders, AvsResponseType.NULL_RESULT); logger.debug("Exited AVS Gateway"); return responseMessage; } catch (Exception e) { return buildNewResponse(responseMessage, requestMessageHeaders, AvsResponseType.EXCEPTION_RESULT, e); } } private Message buildNewResponse(MessageHeaders requestMessageHeaders, AvsResponseType avsResponseType) { Assert.notNull(requestMessageHeaders, MISSING_MANDATORY_ARG); Assert.notNull(avsResponseType, MISSING_MANDATORY_ARG); AvsResponse avsResponse = new AvsResponse(); avsResponse.setError(avsResponseType); return MessageBuilder.withPayload(avsResponse) .copyHeadersIfAbsent(requestMessageHeaders).build(); } private Message buildNewResponse(Message responseMessage, MessageHeaders requestMessageHeaders, AvsResponseType avsResponseType, Exception e) { Assert.notNull(responseMessage, MISSING_MANDATORY_ARG); Assert.notNull(responseMessage.getPayload(), MISSING_MANDATORY_ARG); Assert.notNull(requestMessageHeaders, MISSING_MANDATORY_ARG); Assert.notNull(avsResponseType, MISSING_MANDATORY_ARG); Assert.notNull(e, MISSING_MANDATORY_ARG); AvsResponse avsResponse = new AvsResponse(); avsResponse.setError(avsResponseType, responseMessage.getPayload(), e); return MessageBuilder.withPayload(avsResponse) .copyHeadersIfAbsent(requestMessageHeaders).build(); } } Notice the last line of the catch clause of the exception handling block. This line of code copies the correlation and sequence headers into the response message, this is mandatory if the aggregation group is going to be allowed to complete and will always be necessary following an exception as shown here. Consequences of using this technique There's no doubt that introducing a Messaging Gateway Adapter into SI config makes the configuration more complex to read and follow. The key factor here is that there is no longer a linear progression through the configuration file. This because the Service Activator must forward reference a Gateway or a Gateway defined before it's adapting Service Activator - in both cases the result is the same. Resources Note:- The design for the software that drove creation of this meta-pattern was based on a requirement that a number of external risk assessment services would be accessed by a single, central Risk Assessment Service. In order to satisfy clients of the service, invocation had to take place in parallel and continue despite failure in any one of those external services. This requirement lead to the design of the Messaging Gateway Adapter Pattern for the project. Spring Integration Reference Manual The solution approach for this problem was discussed directly with Mark Fisher (SpringSource) in the context of building Risk Assessment flows for a large US financial institution. Although the configuration and code is protected by NDA and copyright, it's acceptable to express the design intention and similar code in this article.

eclipse comes with a large set of different views: they allow the developer to represent the information in various forms and with different angles. most of these views are navigation oriented: a perfect example for this is the projects view or the outline view . but over time i add more projects, more resources to my project, and at a certain time things get overwhelming. i have a lot of projects, and i do not want to switch between workspace too often. yes, i can open and close projects, but this gets cumbersome too. thankfully, there is a solution in eclipse: working sets . working sets allow me to group elements for display in views. with that, i can do operations on a set of elements in that working set. especially as i’m using many projects the same time, working sets are a big help to focus on the right set of things at a time. i can define a set of things i want to look at, work with, or whatever: it allows me to get be productive in the universe of my environment. building/compiling a working set a nice feature of using working sets is to build a set of projects. instead of selecting a set of projects and then to compile them together, i use a working set. i use the menu project > build working set > select working set… to create or change working sets: menu to select working set if i have no working set defined, then this will show the following dialog where i can press new… to create a new one: creating new working set to create a working set of c/c++ projects, i select c/c++ and press next : new c c++ working set next i give a name and select the project(s) which shall be in my working set, and press finish : defining working set to build my set of projects, i can select the working set and press ok : selected working set search in a working set it is possible to limit the search to a working set. for this i can choose a working set as scope in the search dialogs: search in a working set managing working sets to manage working sets, i press ctrl+3 (see quick access ) and choose manage working sets… : ctrl+3 with manage working sets… note: i can add extra tool-bars and menus for working sets too (this is explained later). then i can manage my working sets: manage working set configuration project view filtering with working sets i can filter the projects shown in the project view based on working sets. for this i select the small triangle and select/define a working set or choose one from the most recently used sets: working sets for project view with this i can easily filter and focus on a subset of projects: project view with applied working set really cool tip: i’m using working sets as well to avoid too many workspaces. instead of having projects spread over different workspaces, i can keep them in one workspace and use working sets instead. there is an added benefit of using working sets: having too many projects open at the same time in eclipse can slow down the ide: using working sets allows me just to switch quickly between the set of projects i’m working on. but it does not stop at filtering by projects: you can filter even things inside the project structure. i simply deselect things i don’t want to see and can focus on what is important for me: filtering project files export and import of working sets note: import and export of working sets is not part of the standard codewarrior eclipse distribution. you get the import/export feature installed with the mqx plugins (www.freescale.com/mqx) or with the anyedit plugins (http://andrei.gmxhome.de/anyedit/index.html). to export a working set, i use the menu file > export > other > export working set : export working sets with mqx plugins note: the anyedit plugins come as well with an import/expert working set wizard. the file format is different, and the mqx plugin allows drag&drop of the file into eclipse. this gives the following dialog where i can specify the file name and the root of projects: export working set dialog this will store the settings in an xml file. importing the working set is done with file > import > other > import working set . tip: i’m using *.wsd extension for working sets. that way i can simply drag&drop the file into eclipse to import it. other kinds of working sets working sets do not stop at projects and files: select working set type i can create working sets of breakpoints or analysis/trace points. or i can create working sets of any resource files or tasks. the possibilities are nearly endless and depend as well on the extra plugins installed. window working sets and now back to the really cool part. one question remains: what are window working sets? window working sets the thing is that every view and dialog has its own working set setting. in my example below i use a working set ‘coldfire’ for the projects view, but my search dialog has a ‘kinetis’ working set configured: two different working sets sometimes i want this, but not always. what i need is a ‘global’ working set. and here the window working sets comes to rescue me. for this i’m going to add some menus and toolbars to make it really easy… for this i choose the menu window > customize perspecti ve. in the command groups i enable ‘window working set’. additionally it is a good idea to enable ‘working set manipulation’ as well: window working set commands the same way i can enable the toolbar and menu visibility. this gives me added tool-bars to switch between working sets and to add/remove things from a working set quickly: working set toolbars in a similar way, it gives me menu access as well: working sets menu and here is the trick: using ‘ window working set ‘ really means ‘ using the global workbench working set ‘. to select the global workbench working set, i use the toolbar icon to switch between window (or workbench) working sets: selecting global window or workbench working set in the individual views i choose to use the window working sets instead a selection of working sets: selected global window or workbench working set now my working set settings are shared and common for all views: if i switch the working set, it will switch for all views where i have set it to ‘window working set’: window working set applied to multiple views that way my working set is the same across views, and switching between different project settings is done with a simple mouse click. summary working sets are an extremely powerful feature to get focus on a subset of things inside eclipse, based on my workflow. as with many great eclipse features, i need to know about it until you really appreciate the power of it. who knows how many other hidden treasures are buried in eclipse i hope this article helps to save you a few mouse clicks. happy work-setting but over time i add more projects, more resources to my project, and at a certain time things get overwhelming. i have a lot of projects, and i do not want to switch between workspace too often.

Spring Integration (SI) Gateways Spring Integration Gateways () provide a semantically rich interface to message sub-systems. Gateways are specified using namespace constructs, these reference a specific Java interface () that is backed by an object dynamically implemented at run-time by the Spring Integration framework. Furthermore, these Java interfaces can, if you so wish, be defined entirely independent of any Spring artefacts - that's both code and configuration. One of the primary advantages of using the SI gateway as an interface to message sub-systems is that it's possible to automatically adopt the benefit of rich, default and customisable, gateway configuration. One such configuration attribute deserves further scrutiny and discussion primarily because it's easy to misunderstand and misconfigure around - default-reply-timeout. Primary Motivator for Gateway Analysis During recent consulting engagements, I've encountered a number of deployments that use Spring Integration Gateway specifications that may, in some circumstances, lead to production operational instability. This has often been in high-pressure environments or those where technology support is not backed by adequate training, testing, review or technology mentoring. How do gateways behave in Spring Integration (R2.0.5) One of the key sections, regarding gateways, in the Spring Integration manual clearly explains gateway semantics. Below is a 2-dimensional table of possible non-standard gateway returns for each of the scenarios that the SI Manual (r2.0.5) refers to. Gateway Non-standard Responses Runtime Events default-reply-timeout=x Single-threaded default-reply-timeout=x Multi-threaded default-reply-timeout=null Single-threaded default-reply-timeout=null Multi-threaded 1. Long Running Process Thread Parked null returned Thread Parked Thread Parked 2. Null Returned Downstream null returned null returned Thread Parked Thread Parked 3. void method Downstream null returned null returned Thread Parked Thread Parked 4. Runtime Exception Error handler invoked or exception thrown. Error handler invoked or exception thrown. Error handler invoked or exception thrown. Error handler invoked or exception thrown. The key parts of this table are the conditions that lead to invoking threads being parked (noted in red), nulls returned (noted in orange) and exceptions (noted in green). Each contributor consists of configuration that is under the developers control, deployed code that is under developers control and conditions that are usually not under developers control. Clearly, the column headings in the table above are divided into two sections; two gateway configuration attributes. The default-reply-timeout is set by the SI configured and is the amount of time that a client call is wiling to wait for a response from the gateway. Secondly, synchronous flows are represented by Single-threaded flows, asynchronous by Multi-threaded flows. A synchronous, or single-threaded flow, is one such as the following: The implicit input channel (gateway-request-channel) has no associated dispatcher configured. An asynchronous, or multi-threaded flow, is one such as the following: The explicit input channel has a dispatcher configured ("taskExecutor"). This task executor specifies a thread pool that supplies threads for execution and whose configuration as above marks a thread boundary. Note: This is not the only way of making channels asynchronous The other configuration attribute referenced is default-reply-timeout, this is set on the gateway namespace configuration such as the example above. Note that both of these runtime aspects are set by the configurer during SI flow design and implementation. They are entirely under developer control. The 'Runtime Events' column indicates gateway relevant runtime events that have to be considered during gateway configuration - these are obviously not under developer control. Trigger conditions for these events are not as unusual as one may hope. 1. Long Running Processes It's not uncommon for thread pools to become exhausted because all pooled threads are waiting for an external resource accessed through a socket, this may be a long running database query, a firewall keeping a connection open despite the server terminating etc. There is significant potential for these types of trigger. Some long-running processes terminate naturally, sometimes they never completed - an application restart is required. 2. Null returned downstream A null may be returned from a downstream SI construct such as a Transformer, Service Activator or Gateway. A Gateway may return null in some circumstances such as following a gateway timeout event. 3. Void method downstream Any custom code invoked during an SI flow may use a void method signature. This can also be caused by configuration in circumstances where flows are determined dynamically at runtime. 4. Runtime Exception RuntimeException's can be triggered during normal operation and are generally handled by catching them at the gateway or allowing them to propagate through. The reason that they are coloured green in the table above is that they are generally much easier to handle than timeouts. Gateway Timeout Handling Strategies There are four possible outcomes from invoking a gateway with a request message, all of these as a result of specific runtime events: a) an ordinary message response, b) an exception message, c) a null or d) no-response. Ordinary business responses and exceptions are straight forward to understand and will not be covered further in this article. The two significant outcomes that will be explored further are strategies for dealing with nulls and no-response. Generally speaking, long running processes either terminate or not. Long running processes that terminate may eventually return a message through the invoked gateway or timeout depending on timeout configuration, in which case a null may be returned. The severity of this as a problem depends on throughput volume, length of long running process and system resources (thread-pool size). Configuration exists for default-reply-timeout In the case where a long running process event is underway and a default-reply-timeout has been set, as long as the long running process completes before the default-reply-timeout expires, there is no problem to deal with. However, if the long running process does not complete before that timeout expires one of three outcomes will apply. Firstly, if the long running process terminates subsequent to the reply timeout expiry, the gateway will have already returned null to the invoker so the null response needs handling by the invoker. The thread handling the long-running process will be returned to the pool. Secondly, if the long running process does not terminate and a reply timeout has been set, the gateway will return null to the gateway invoker but the thread executing the long-running process will not get returned to the pool. Thirdly, and most significantly, if a default-reply-timeout has been configured but the long running process is running on the same thread as the invoker, i.e. synchronous channels supply messages to that process, the thread will not return, the default-reply-timeout has no affect. Assuming the most common processing scenario, a long running process completes either before or after the reply timeout expiry. When a null is returned by the gateway, the invoker is forced to deal with a null response. It's often unacceptable to force gateway consumers to deal with null responses and is not necessary as with a little additional configuration, this can be avoided. Absent Configuration for default-reply-timeout The most significant danger exists around gateways that have no default-reply-timeout configuration set. A long running process or a null returned from downstream will mean that the invoking thread is parked. This is true for both synchronous and asynchronous flows and may ultimately force an application to be restarted because the invoker thread pool is likely to start on a depletion course if this continues to occur. Spring Integration Timeout Handling Design Strategies For those Spring Integration configuration designers that are comfortable with gateway invokers dealing with null responses, exceptions and set default-reply-timeouts on gateways, there's no need to read further. However, if you wish to provide clients of your gateway a more predictable response, a couple of strategies exist for handling null responses from gateways in order that invokers are protected from having to deal with them. Firstly, the simpliest solution is to wrap the gateway with a service activator. The gateway must have the default-reply-timeout attribute value set in order to avoid unnecessary parking of threads. In order to avoid the consequence of long-running threads it's also very prudent to use a dispatcher soon after entry to the gateway - this breaks the thread boundary. Whilst this is a valid technical approach, the impact is that we have forced a different entry point to our message sub-system. Entry is now via a Service Activator rather than a Gateway. A side affect of this change is that the testing entry point changes. Integration tests that would normally reference a gateway to send a message now have to locate the backing implementation for the Service Activator, not ideal. An alternative approach toward solving this problem would be to configure two gateways with a Service Activator between them. Only one of the gateways would be exposed to invokers, the outer one. Both Gateways would reference the same service interface. The outer gateway specification would not specify the default-reply-timeout but would specify the input and output channels in the same way that a single gateway would. The Service Activator between the Gateways would handle null gateway responses and possibly any exceptions if preferred to the gateway error handler approach. An example is as follows: The Service Activator bean (enrollmentServiceGatewayHandler) deals with both null and exception responses from the adapted gateway (enrollmentServiceAdaptedGateway), in the situation where these are generated a business response detailing the error is generated. Spring Integration R2.1 Changes async-executor on gateway spec

Within Spring Integration, one form of EIP scatter-gather is provided by the splitter and aggregator constructs.

Introduction In recent times, many programming languages that run on JVM have emerged. Many of these languages support the concept of writing code in a functional style. Programmers have started realizing the benefits of functional programming and are beginning to rediscover the powerful style of this programming paradigm. The emergence of multiple languages on JVM have only helped to reignite the strong interest in this paradigm. Java at its core is an imperative programming language. However in recent past many new languages like Scala, Clojure, Groovy etc. have become popular which supports functional programming style and yet run on JVM. However none of these languages can be considered as pure functional language since all of them allow Java code to be called from within them and Java on its own is not a functional language. Still they have different degree of support for writing code in functional style and have their own benefits. Functional programming requires different kind of thinking and has its own advantages as compared to imperative programming. It seems that Java has also realized functional programming advantages and is slowly inching towards it. First sign of this can be seen in the form of Lambda Expressions that will be supported in Java 8. Although it's too early to comment on this as the draft for Java 8 is still under review and is expected to be released next year, but it does show that Java has plans of supporting functional programming style going forward. In this article we will first discuss what functional programming is and how it is different from imperative programming. Later we will see where does each of the above mentioned Java based programming languages i.e. Scala, Clojure and Groovy fits in the world of functional programming and what each of them has to offer. And at the last we will sneak-peak into Java 8's lambda expressions. Why Functional Programming? Computers of current era are shipped with multicore processors. Going forward the number of processors in a machine is only going to increase. The code we write today and tomorrow will probably never run on a single processor system. In order to get best out of this, software must be designed to make more and more use of concurrency and hence keep all available processors busy. Java does provide concurrency concepts like threads, synchronization, locks etc. to execute code in parallel. But shared memory multi-threading approach in Java causes more trouble than solving the problem. Java based functional programming languages like Scala, Clojure, Groovy etc. looks into these problems with a different angle and provides less complex and less error-prone solutions as compared to imperative programming. They provide immutability concepts out of the box and hence eliminate need of synchronization and associated risk of deadlocks or livelocks. Concepts like Actors, Agents and DataFlow variables provide high level concurrency abstraction and makes very easy to write concurrent programs. What is Functional Programming? Functional Programming is a concept which treats functions as first class citizens. At the core of functional programming is immutability. It emphasizes on application of functions in contrast to imperative programming style which emphasizes on change in state. Functional programming has no side effects whereas programming in imperative style can result in side-effects. Let's elaborate more on each of these characteristics to understand the concept behind functional programming. Immutable state - The state of an object doesn't change and hence need not be protected or synchronized. That might sound a bit awkward at first, since if nothing changes, one might think that we are not writing a useful program. However that's not what immutable state means. In functional programming, change in state occurs via series of transformations which keeps the object immutable and yet achieves change in state. Functions as first class citizens - There was a major shift in the way programs were written when Object oriented concepts came into picture. Everything was conceptualized as object and any action to be performed was treated as method call on objects. Hence there is a series of method calls executed on objects to get the desired work done. In functional programming world, it's more about thinking in terms of communication chain between functions than method calls on objects. This makes functions as first class citizens of functional programming since everything is modelled around functions. Higher-order functions - Functions in functional programming are higher order functions since following actions can be performed with them. 1. Functions can be passed within functions as arguments. 2. Functions can be created within functions just as objects can be created in functions 3. Functions can be returned from functions Functions with no side-effects - In functional programming, function execution has no side-effects. In other words a function code will always return same result for same argument when called multiple times. It doesn't change anything outside its boundaries and is also not affected by any external change outside it's boundary. It doesn't change input value and can only produce new output. However once the output has been produced and returned by function, it also becomes immutable and cannot be modified by any other function. In other words, they support referential transparency i.e. if a function takes an input and returns some output, multiple invocation of that function at different point of time will always return same output as long as input remains same. This is one of the main motivations behind using functional language as it makes easy to understand and predict behaviour of program. Characteristics like immutability and no side-effects are extremely helpful while writing multi-threaded code and developers need not to worry for synchronizing the state. Hence functional code is very easy to distribute across multiple cores as they don't have any side effects. JVM based Functional Programming Languages There are many JVM based languages which supports functional programming paradigm. However I intend to limit discussion around following. Scala Clojure Groovy Lambda Expressions in Java 8 Lambda Expressions is not a programming language but a feature that will be supported in Java8. The reason for including it in this article is to emphasize on the fact that going forward Java will also support writing code in functional style. Scala Scala is a statically typed multi-paradigm programming language designed to integrate features of object oriented programming and functional programming. Since it is static, one cannot change class definition at run time i.e. one cannot add new methods or variables at run-time. However Scala does provide functional programming concepts i.e. immutability, higher-order functions, nested functions etc. Apart from supporting Java's concurrency model, it also provides concept of Actor model out of the box for event based asynchronous message passing between objects. The code written in Scala gets compiled into very efficient bytecode which can then be executed on JVM. Creating immutable list in Scala is very simple and doesn't require any extra effort. "val" keyword does the trick. val numbers = List(1,2,3,4) Functions can be passed as arguments. Let's see this with an example. Suppose we have a list of 10 numbers and we want to calculate sum of all the numbers in list. val numbers = List(1,2,3,4,5,6,7,8,9,10) val total = numbers.foldLeft(0){(a,b) => a+b } As can be seen in above example, we are passing a function to add two variables "a" and "b" to another function "foldLeft" which is provided by Scala library on collections. We have also not used any iteration logic and temporary variable to calculate the sum. "foldLeft" method eliminates the need to maintain state in temporary variable which would have otherwise be required if we were to write this code in pure Java way (as mentioned below). int total = 0; for(int number in numbers){ total+=number; } Scala function can easily be executed in parallel without any need for synchronization since it does not mutate state. This was just a small example to showcase the power of Scala as functional programming language. There are whole lot of features available in Scala to write code in functional style. Clojure Clojure is a dynamic language with an excellent support for writing code in functional style. It is a dialect of "lisp" programming language with an efficient and robust infrastructure for multithreaded programming. Clojure is predominantly a functional programming language, and features a rich set of immutable, persistent data structures. When mutable state is needed, Clojure offers a software transactional memory system and reactive Agent system that ensure clean, correct multithreaded designs. Apart from this since Clojure is a dynamic language, it allows to modify class definition at run time by adding new methods or modifying existing one at run time. This makes it different from Scala which is a statically typed language. Immutability is in the root of Clojure. To create immutable list just following needs to be done. By default list in Clojure is immutable, so does not require any extra effort. (def numbers (list 1 2 3 4 5 6 7 8 9 10)) To add numbers without maintaining state, reduce function can be used as mentioned below (reduce + 0 '(1 2 3 4 5 6 7 8 9 10)) As can be seen, adding list of numbers just requires one line of code without mutating any state. This is the beauty about functional programming languages and plays an important role for parallel execution. Groovy Groovy is again a dynamic language with some support for functional programming. Amongst the 3 languages, Groovy can be considered weakest in terms of functional programming features. However because of it's dynamic nature and close resemblance to Java, it has been widely accepted and considered good alternative to Java. Groovy does not provide immutable objects out of the box but has excellent support for higher order functions. Immutable objects can be created with annotation @Immutation, but it's far less flexible than immutablity support in Scala and Clojure. In Groovy functions can be passed around just as any other variable in the form of Closures. The same example in Groovy can be written as follows def numbers = [1,2,3,4,5,6,7,8,9,10] def total = numbers.inject(0){a,b -> a+b } However the point to be noted is that variables "numbers" and "total" are not immutable and can be modified at any point of time. Hence writing multithreaded code can be a bit challenging. But Groovy does provide the concept of Actors, Agents and DataFlow variables via library called GPars(Groovy Parallel System) which reduces the challenges associated with multithreaded code to a greater extent. Java8 Lambda Expression Java has finally realized the power of writing code in functional style and is going to support the concept of closures starting from Java8. JSR 335 - Lambda Expressions for the JavaTM Programming Language aims to support programming in a multicore environment by adding closures and related features to the Java language. So it will finally be possible to pass around functions similar to variables in pure Java code. Currently if someone wants to try out and play around lambda expressions, Project Lambda of OpenJDK provides prototype implementation of JSR-335. Following code snippet should run fine with OpenJDK Project Lambda compiler. ExecutorService executor = Executors.newCachedThreadPool(); executor.submit(() -> {System.out.println("I am running")}) As can be seen above, a closure(function) has been passed to executor's submit method. It does not take any argument and hence empty brackets () have been placed. This function just prints "I am running" when executed. Just as we can pass functions to function, it will also be possible to create closure within functions and return closure from function. I would recommend to try out OpenJDK to get a feel of lambda expressions which is going to be part of Java8 Conclusion So this was all about functional programming, it's concepts, benefits and options available on JVM to write function code. Functional programming requires a different mind-set and can be very useful if used correctly. Functional Programming along with Object Oriented Programming can be a jewel in crown. As discussed there are various options available to write code in functional style that can be executed on JVM. Choice depends on various factors and there is no one language that can be considered best in all aspects. However one thing is for sure, going forward we are going to see more and more usage of functional programming.

rate this eclipse is good, but like any other tool: it gets better after i have it customized for my special needs. eclipse stores a lot of settings in the workspace, see my article about copy my workspace settings . but is there a way to apply some settings to every workspace? at least to the new ones? because importing/exporting the settings can get really tedious as i have many workspace. and indeed, there are global settings in eclipse. and i want to have them changed… warning: changing eclipse global preferences might break an eclipse installation. so better have a backup of the changed files at hand! i’m using here the eclipse based codewarrior for mcu10.2 , but things are pretty much the same for any eclipse based product (see the documentation in defining your own global preferences ). question: where are the global preferences stored? the first thing to check is the eclipse\configuration\.settings folder: here some plugins store their global preferences. for example: the recent workspace settings are in org.eclipse.ui.ide.prefs. #fri apr 06 16:46:14 cest 2012 recent_workspaces_protocol=3 max_recent_workspaces=10 show_workspace_selection_dialog=true eclipse.preferences.version=1 recent_workspaces=c\:\\tmp\\wsp_test\nc\:\\tmp\\wsp_10.2 but what about all the other settings? looking at the codewarrior installation, inside the eclipse folder, i find the cwide.ini file. cwide.ini file this file defines the eclipse startup options for launching the ide (cwide.exe for codewarrior). the interesting part is this line: -declipse.plugincustomization=cwide.properties this tells eclipse to use the cwide.properties as a default configuration file. if i inspect that file, it has the following content: org.eclipse.debug.ui/org.eclipse.debug.ui.switch_perspective_on_suspend=always org.eclipse.debug.ui/org.eclipse.debug.ui.switch_to_perspective=always org.eclipse.ui.editors/spellingengine=org.eclipse.cdt.internal.ui.text.spelling.cspellingengine ok, that gives me an idea how settings could look like. but the question is: how to know the settings and syntax? what works (most of the time) is following approach: launch eclipse with a new workspace export the settings using file > export > general > preferences to a file change the setting in window > preferences export the settings using file > export > general > preferences to a different file compare/inspect the exported information and find the settings apply the settings to the cwide.properties file, without the /instance/ part restart the ide and check if it works with a new workspace the last check is necessary as not all settings might work that way, see this forum post . this is maybe best illustrated with an example. i have configured my workspace to use 2 for tab width and to insert spaces for tabs: changed preferences for tabs if i compare the two exported .epf files, this gives me: diffing eclipse preference files that means the two following lines are configuring what i have changed: /instance/org.eclipse.ui.editors/tabwidth=2 /instance/org.eclipse.ui.editors/spacesfortabs=true for the cwide.properties file i need to cut off the /instance/ part, so i have this added to the cwide.properties : # set tab width to 2 org.eclipse.ui.editors/tabwidth=2 # using spaces for tabs org.eclipse.ui.editors/spacesfortabs=true note: preferences are applied in following order: global preferences, then local (workspace) preferences this does not overwrite an existing setting of my workspace. as i can see from above diff, my initial workspace settings do not have any settings for tabwidth and spacesfortabs. creating a new workspace use and apply my new settings. but once i have the them, they will not be overwritten with new global ones. which makes sense: the local settings are winning. note: post a comment if you know an elegant way how to enforce/overwrite workspace settings with global ones.

Learn how Apache Camel implements the EIPs and offers a standardized, internal domain-specific language (DSL) to integrate applications.

Continuing on the theme of temporary storage for transient messages used within Spring Integration flows, the claim-check model offers configurable storage for message payloads. The advantage in using this Enterprise Integration pattern, compared against header enrichment, is that objects don't have to be packed into the header using a Header Enrichment technique. They can be stored in a local Java Map, an IMDB, cache or anything else that be used to hold data. Several advantages using this approach are evident. Firstly, performance and efficiency. When using header enrichment, if message payloads need to be managed outside of the JVM that generates the enriched message header, the object will not be available unless it's serialised and transported around the distributed application. This could be costly in terms of performance and transport efficiency. The key factor here is the frequency of remote dispatch and the size of the header object. In specific circumstances the claim-check pattern may offer an advantage here, objects can be serialised and/or transformed into a storage specific format and stored internally in memory or externally in a data store. Secondly, accessibility. It's conceivable that message payloads undergoing claim-check processing may need to be accessed by third party applications that are unable to receive Spring Integration messages. The claim-check pattern allows this type of processing to take place. Thirdly, resiliency is offered. A data store can be chosen that guarantees persistence for messages in order that they can be recovered following failure. The following code details how the claim-check pattern can be used: The gateway used is specified as the following Java class: package com.l8mdv.sample; import org.springframework.integration.Message; import org.springframework.integration.annotation.Gateway; public interface ClaimCheckGateway { public static final String CLAIM_CHECK_ID = "CLAIM_CHECK_ID"; @Gateway (requestChannel = "claim-check-in-channel") public Message send(Message message); } Lastly, this can all be tested by using the following JUnit test case: package com.l8mdv.sample; import org.junit.Assert; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.integration.Message; import org.springframework.integration.support.MessageBuilder; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import static com.l8mdv.sample.ClaimCheckGateway.CLAIM_CHECK_ID; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration( locations = {"classpath:META-INF/spring/claim-check.xml"} ) public class ClaimCheckIntegrationTest { @Autowired ClaimCheckGateway claimCheckGateway; @Test public void locatePayloadInHeader() { String payload = "Sample test message."; Message message = MessageBuilder.withPayload(payload).build(); Message response = claimCheckGateway.send(message); Assert.assertTrue(response.getPayload().equals(payload)); Assert.assertTrue(response.getHeaders().get(CLAIM_CHECK_ID) != null); } }

Looking for the best Eclipse shortcuts? Here are the top 10.

A few weeks ago, I co-authored an article (with my colleague Rama Ramani) about how the Screen Actors Guild Awards website migrated its Drupal deployment from LAMP to Windows Azure: Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. Since then, Rama and another colleague, Jason Roth, have been working on writing up how the SAG Awards website was managed and monitored in Windows Azure. The article below is the fruit of their work…a very interesting/educational read. Overview Drupal is an open source content management system that runs on PHP. Windows Azure offers a flexible platform for hosting, managing, and scaling Drupal deployments. This paper focuses on an approach to host Drupal sites on Windows Azure, based on learning from a BPD Customer Programs Design Win engagement with the Screen Actors Guild Awards Drupal website. This paper covers guidelines and best practices for managing an existing Drupal web site in Windows Azure. For more information on how to migrate Drupal applications to Windows Azure, see Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. The target audience for this paper is Drupal administrators who have some exposure to Windows Azure. More detailed pointers to Windows Azure content is provided throughout the paper as links. Drupal Application Architecture on Windows Azure Before reviewing the management and monitoring guidelines, it is important to understand the architecture of a typical Drupal deployment on Windows Azure. First, the following diagram displays the basic architecture of Drupal running on Windows and IIS7. In the Windows Server scenario, you could have one or more machines hosting the web site in a farm. Those machines would either persist the site content to the file system or point to other network shares. For Windows Azure, the basic architecture is the same, but there are some differences. In Windows Azure the site is hosted on a web role. A web role instance is hosted on a Windows Server 2008 virtual machine within the Windows Azure datacenter. Like the web farm, you can have multiple instances running the site. But there is no persistence guarantee for the data on the file system. Because of this, much of the shared site content should be stored in Windows Azure Blob storage. This allows them to be highly available and durable. Usually, a large portion of the site caters to static content which lends well to caching. And caching can be applied in a set of places – browser level caching, CDN to cache content in the edge closer to the browser clients, caching in Azure to reduce the load on backend, etc. Finally, the database can be located in SQL Azure. The following diagram shows these differences. For monitoring and management, we will look at Drupal on Windows Azure from three perspectives: Availability: Ensure the web site does not go down and that all tiers are setup correctly. Apply best practices to ensure that the site is deployed across data centers and perform backup operations regularly. Scalability: Correctly handle changes in user load. Understand the performance characteristics of the site. Manageability: Correctly handle updates. Make code and site changes with no downtime when possible. Although some management tasks span one or more of these categories, it is still helpful to discuss Drupal management on Windows Azure within these focus areas. Availability One main goal is that the Drupal site remains running and accessible to all end-users. This involves monitoring both the site and the SQL Azure database that the site depends on. In this section, we will briefly look at monitoring and backup tasks. Other crossover areas that affect availability will be discussed in the next section on scalability. Monitoring With any application, monitoring plays an important role with managing availability. Monitoring data can reveal whether users are successfully using the site or whether computing resources are meeting the demand. Other data reveals error counts and possibly points to issues in a specific tier of the deployment. There are several monitoring tools that can be used. The Windows Azure Management Portal. Windows Azure diagnostic data. Custom monitoring scripts. System Center Operations Manager. Third party tools such as Azure Diagnostics Manager and Azure Storage Explorer. The Windows Azure Management Portal can be used to ensure that your deployments are successful and running. You can also use the portal to manage features such as Remote Desktop so that you can directly connect to machines that are running the Drupal site. Windows Azure diagnostics allows you to collect performance counters and logs off of the web role instances that are running the Drupal site. Although there are many options for configuring diagnostics in Azure, the best solution with Drupal is to use a diagnostics configuration file. The following configuration file demonstrates some basic performance counters that can monitor resources such as memory, processor utilization, and network bandwidth. For more information about setting up diagnostic configuration files, see How to Use the Windows Azure Diagnostics Configuration File. This information is stored locally on each role instance and then transferred to Windows Azure storage per a defined schedule or on-demand. See Getting Started with Storing and Viewing Diagnostic Data in Windows Azure Storage. Various monitoring tools, such as Azure Diagnostics Manager, help you to more easily analyze diagnostic data. Monitoring the performance of the machines hosting the Drupal site is only part of the story. In order to plan properly for both availability and scalability, you should also monitor site traffic, including user load patterns and trends. Standard and custom diagnostic data could contribute to this, but there are also third-party tools that monitor web traffic. For example, if you know that spikes occur in your application during certain days of the week, you could make changes to the application to handle the additional load and increase the availability of the Drupal solution. Backup Tasks To remain highly available, it is important to backup your data as a defense-in-depth strategy for disaster recovery. This is true even though SQL Azure and Windows Azure Storage both implement redundancy to prevent data loss. One obvious reason is that these services cannot prevent administrator error if data is accidentally deleted or incorrectly changed. SQL Azure does not currently have a formal backup technology, although there are many third-party tools and solutions that provide this capability. Usually the database size for a Drupal site is relatively small. In the case of SAG Awards, it was only ~100-150 MB. So performing an entire backup using any strategy was relatively fast. If your database is much larger, you might have to test various backup strategies to find the one that works best. Apart from third-party SQL Azure backup solutions, there are several strategies for obtaining a backup of your data: · Use the Drush tool and the portabledb-export command. · Periodically copy the database using the CREATE DATABASE Transact-SQL command. · Use Data-tier applications (DAC) to assist with backup and restore of the database. SQL Azure backup and data security techniques are described in more detail in the topic, Business Continuity in SQL Azure. Note that bandwidth costs accrue with any backup operation that transfers information outside of the Windows Azure datacenter. To reduce costs, you can copy the database to a database within the same datacenter. Or you can export the data-tier applications to blob storage in the same datacenter. Another potential backup task involves the files in Blob storage. If you keep a master copy of all media files uploaded to Blob storage, then you already have an on-premises backup of those files. However, if multiple administrators are loading files into Blob storage for use on the Drupal site, it is a good idea to enumerate the storage account and to download any new files to a central location. The following PHP script demonstrates how this can be done by backing up all files in Blob storage after a specified modification date. setProxy(true, 'YOUR_PROXY_IF_NEEDED', 80); $blobs = (array)$blobObj->listBlobs(AZURE_STORAGE_CONTAINER, '', '', 35000); backupBlobs($blobs, $blobObj); function backupBlobs($blobs, $blobObj) { foreach ($blobs as $blob) { if (strtotime($blob->lastmodified) >= DEFAULT_BACKUP_FROM_DATE && strtotime($blob->lastmodified) <= DEFAULT_BACKUP_TO_DATE) { $path = pathinfo($blob->name); if ($path['basename'] != '$$$.$$$') { $dir = $path['dirname']; $oldDir = getcwd(); if (handleDirectory($dir)) { chdir($dir); $blobObj->getBlob( AZURE_STORAGE_CONTAINER, $blob->name, $path['basename'] ); chdir($oldDir); } } } } } function handleDirectory($dir) { if (!checkDirExists($dir)) { return mkdir($dir, 0755, true); } return true; } function checkDirExists($dir) { if(file_exists($dir) && is_dir($dir)) { return true; } return false; } ?> This script has a dependency on the Windows Azure SDK for PHP. Also note there are several parameters that you must modify such as the storage account, secret, and backup location. As with SQL Azure, bandwidth and transaction charges apply to a backup script like this. Scalability Drupal sites on Windows Azure can scale as load increased through typical strategies of scale-up, scale-out, and caching. The following sections describe the specifics of how these strategies are implemented in Windows Azure. Typically you make scalability decisions based on monitoring and capacity planning. Monitoring can be done in staging during testing or in production with real-time load. Capacity planning factors in projections for changes in user demand. Scale Up When you configure your web role prior to deployment, you have the option of specifying the Virtual Machine (VM) size, such as Small or ExtraLarge. Each size tier adds additional memory, processing power, and network bandwidth to each instance of your web role. For cost efficiency and smaller units of scale, you can test your application under expected load to find the smallest virtual machine size that meets your requirements. The workload usually in most popular Drupal websites can be separated out into a limited set of Drupal admins making content changes and a large user base who perform mostly read-only workload. End users can be allowed to make ‘writes’, such as uploading blogs or posting in forums, but those changes are not ‘content changes’. Drupal admins are setup to operate without caching so that the writes are made directly to SQL Azure or the corresponding backend database. This workload performs well with Large or ExtraLarge VM sizes. Also, note that the VM size is closely tied to all hardware resources, so if there are many content-rich pages that are streaming content, then the VM size requirements are higher. To make changes to the Virtual Machine size setting, you must change the vmsize attribute of the WebRole element in the service definition file, ServiceDefinition.csdef. A virtual machine size change requires existing applications to be redeployed. Scale Out In addition to the size of each web role instance, you can increase or decrease the number of instances that are running the Drupal site. This spreads the web requests across more servers, enabling the site to handle more users. To change the number of running instances of your web role, see How to Scale Applications by Increasing or Decreasing the Number of Role Instances. Note that some configuration changes can cause your existing web role instances to recycle. You can choose to handle this situation by applying the configuration change and continue running. This is done by handling the RoleEnvironment.Changing event. For more information see, How to Use the RoleEnvironment.Changing Event. A common question for any Windows Azure solution is whether there is some type of built-in automatic scaling. Windows Azure does not provide a service that provides auto-scaling. However, it is possible to create a custom solution that scales Azure services using the Service Management API. For an example of this approach, see An Auto-Scaling Module for PHP Applications in Windows Azure. Caching Caching is an important strategy for scaling Drupal applications on Windows Azure. One reason for this is that SQL Azure implements throttling mechanisms to regulate the load on any one database in the cloud. Code that uses SQL Azure should have robust error handling and retry logic to account for this. For more information, see Error Messages (SQL Azure Database). Because of the potential for load-related throttling as well as for general performance improvement, it is strongly recommended to use caching. Although Windows Azure provides a Caching service, this service does not currently have interoperability with PHP. Because of this, the best solution for caching in Drupal is to use a module that uses an open-source caching technology, such as Memcached. Outside of a specific Drupal module, you can also configure Memcached to work in PHP for Windows Azure. For more information, see Running Memcached on Windows Azure for PHP. Here is also an example of how to get Memcached working in Windows Azure using a plugin: Windows Azure Memcached plugin. In a future paper, we hope to cover this architecture in more detail. For now, here are several design and management considerations related to caching. Area Consideration Design and Implementation For a technology like Memcached, will the cache be collocated (spread across all web role instances)? Or will you attempt to setup a dedicated cache ring with worker roles that only run Memcached? Configuration What memory is required and how will items in the cache be invalidated? Performance and Monitoring What mechanisms will be used to detect the performance and overall health of the cache? For ease of use and cost savings, collocation of the cache across the web role instances of the Drupal site works best. However, this assumes that there is available reserve memory on each instance to apply toward caching. It is possible to increase the virtual machine size setting to increase the amount of available memory on each machine. It is also possible to add additional web role instances to add to the overall memory of the cache while at the same time improving the ability of the web site to respond to load. It is possible to create a dedicated cache cluster in the cloud, but the steps for this are beyond the scope of this paper[RR1] . For Windows Azure Blob storage, there is also a caching feature built into the service called the Content Delivery Network (CDN). CDN provides high-bandwidth access to files in Blob storage by caching copies of the files in edge nodes around the world. Even within a single geographic region, you could see performance improvements as there are many more edge nodes than Windows Azure datacenters. For more information, see Delivering High-Bandwidth Content with the Windows Azure CDN. Manageability It is important to note that each hosted service has a Staging environment and a Production environment. This can be used to manage deployments, because you can load and test and application in staging before performing a VIP swap with production. From a manageability standpoint, Drupal has an advantage on Windows Azure in the way that site content is stored. Because the data necessary to serve pages is stored in the database and blob storage, there is no need to redeploy the application to change the content of the site. Another best practice is to use a separate storage account for diagnostic data than the one that is used for the application itself. This can improve performance and also helps to separate the cost of diagnostic monitoring from the cost of the running application. As mentioned previously, there are several tools that can assist with managing Windows Azure applications. The following table summarizes a few of these choices. Tool Description Windows Azure Management Portal The web interface of the Windows Azure management portal shows deployments, instance counts and properties, and supports many different common management and monitoring tasks. Azure Diagnostics Managerq[RR2] [JR3] A Red Gate Software product that provides advanced monitoring and management of diagnostic data. This tool can be very useful for easily analyzing the performance of the Drupal site to determine appropriate scaling decisions. Azure Storage Explorer A tool created by Neudesic for viewing Windows Azure storage account. This can be useful for viewing both diagnostic data and the files in Blob storage.

An old customer recently asked me if I had a solution for how to integrate between their existing JMS infrastructure on Websphere MQ with RabbitMQ. Although I know that RabbitMQ has the shovel plugin which can bridge between Rabbit instances I've yet not found a good plugin for JMS <-> AMQP forwarding. The first thing that came to my mind was to utilize a Spring Integration mediation as SI has excellent support for both JMS and Rabbit. Curious as I am I started a PoC and this is the result. It takes messages of a JMS queue and forwards to an AMQP exchange that is bound to a queue the consumer application is supposed to listen to. I used an external HornetQ instance in JBoss 6.1 as the JMS Provider, but I am 100% secure that the same setup would work for Websphere MQ as they both implement JMS. Be aware that I've done no performance tweaking or QoS setup yet as this is just a proof-of-concept. For a real setup you'd probably have to think about delivery guarantees versus performance and etc... The code will be available at a GitHub repository near you soon.. SpringContext in XML: org.jnp.interfaces.NamingContextFactory jnp://localhost:1099 org.jnp.interfaces:org.jboss.naming ConnectionFactory Maven POM: 4.0.0 org.rl si.jmstorabbit 0.0.1-SNAPSHOT jar si.jmstorabbit http://maven.apache.org UTF-8 2.2.5.Final 2.1.0.RELEASE springsource-release http://repository.springsource.com/maven/bundles/release false springsource-external http://repository.springsource.com/maven/bundles/external false org.springframework.integration spring-integration-core ${spring.integration.version} org.springframework.integration spring-integration-file ${spring.integration.version} org.springframework.integration spring-integration-amqp ${spring.integration.version} org.springframework.integration spring-integration-jms ${spring.integration.version} junit junit 3.8.1 test org.springframework spring-context 3.0.7.RELEASE jboss jnp-client 4.2.2.GA org.hornetq hornetq-core-client ${hornet.version} org.hornetq hornetq-jms-client ${hornet.version} org.hornetq hornetq-jms ${hornet.version} jboss jboss-common-client 3.2.3 org.jboss.netty netty 3.2.7.Final javax.jms jms 1.1

is a really easy way to provide property replacements in Spring configurations with values from a standard Java Properties file. But what if you don’t want a property hard coded into a file – a clear text password for instance? Spring provides all the bits and pieces to write your own property replacement. Let me introduce my CustomPropertyConfigurer. I’ll demonstrate using a variation on the theme of the Spring JDBC Template. MyQuery is a simple extension of org.springframework.jdbc.core.JDBCTemplate that gets the current timestamp from a MySQL database. Here’s the, hopefully familiar, configuration: Except the jdbc.properties file does not contain the password: jdbc.driverClassName=com.mysql.jdbc.Driver jdbc.url=jdbc:mysql://rob-7 jdbc.username=rob I will set the jdbc.password property myself from what is entered on the command line. public static void main(String... args) { char[] password = System.console().readPassword("Password: "); Properties properties = new Properties(); properties.setProperty("jdbc.password", new String(password)); ConfigurableApplicationContext context = new ClassPathXmlApplicationContext( new String[] { "rob/MyQuery.xml"}, false); context.addBeanFactoryPostProcessor( new CustomPropertyConfigurer(properties)); context.refresh(); MyQuery myQuery = context.getBean(MyQuery.class); myQuery.run(); context.close(); } Where the CustomPropertyConfigurer is: import java.util.Properties; import org.springframework.beans.BeansException; import org.springframework.beans.factory.BeanDefinitionStoreException; import org.springframework.beans.factory.config.BeanDefinition; import org.springframework.beans.factory.config.BeanDefinitionVisitor; import org.springframework.beans.factory.config.BeanFactoryPostProcessor; import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; import org.springframework.util.PropertyPlaceholderHelper; import org.springframework.util.StringValueResolver; public class CustomPropertyConfigurer implements BeanFactoryPostProcessor { private final Properties properties; public CustomPropertyConfigurer(Properties properties) { this.properties = properties; } public void postProcessBeanFactory( ConfigurableListableBeanFactory beanFactoryToProcess) throws BeansException { BeanDefinitionVisitor visitor = new BeanDefinitionVisitor( new BeanDirectoryResolver()); String[] beanNames = beanFactoryToProcess.getBeanDefinitionNames(); for (int i = 0; i < beanNames.length; i++) { BeanDefinition bd = beanFactoryToProcess.getBeanDefinition( beanNames[i]); try { visitor.visitBeanDefinition(bd); } catch (BeanDefinitionStoreException ex) { throw new BeanDefinitionStoreException( bd.getResourceDescription(), beanNames[i], ex.getMessage()); } } } class BeanDirectoryResolver implements StringValueResolver { private final PropertyPlaceholderHelper helper; public BeanDirectoryResolver() { helper = new PropertyPlaceholderHelper("${", "}"); } public String resolveStringValue(String strVal) { return helper.replacePlaceholders(strVal, properties); } } } The CustomPropertyConfigurer gets applied first. It leaves any properties it can’t resolve (all but the password) for the standard to resolve. Unit tests running against a different jdbc.propeties file can continue to provide the password as before. Here it is running: There are many other examples of configuration values that might only be discovered at runtime – file names, schedule dates, form values etc. So long as the value can be a String, a CustomPropertyConfigurer provides a simple way of passing these values to Spring.

As you know I'm open-sourcing and completely overhauling my PhD system. One of my goals was to replace internal JMS queues with AMQP. Today I'll show you how I did it and why I was forced to change RabbitMQ to Apache Qpid. AMQP In short. AMQP is an open standard application layer protocol for message-oriented middleware. The most important feature is that AMQP is a wire-level protocol and is interoperable by design. JMS is just an API. Altough JMS brokers can be used in .NET applications (see my post: ActiveMQ and .NET combined!), the whole JMS specification does not guarantee interoperability. Also, the AMQP standard is by design more flexible and powerful (e.g., supports two-way communication by design) - they simply learnt from JMS mistakes :). Oh, forgot to mention. The AMQP was originally developed by banks :) so I don't have to say that AMQP is secure, fault-tolerant, and so on. RabbitMQ RabbitMQ is the most mature AMQP broker. RabbitMQ is written in Erlang so you have to download that first (RabbitMQ Windows installer does it for you). Download it from here: http://www.rabbitmq.com/. I also recommend installing the web management console. From Rabbit's sbin directory execute: rabbitmq-plugins enable rabbitmq_management If you're on Windows and you installed a Rabbit service you have to restart it. That's it. Spring Well, it turned out that VMware bought RabbitMQ and SpringSource developers are now developing it. Given this fact, you shouldn't be surprised that Spring - RabbitMQ integration is childishly simple. Add spring-rabbit dependency to your Maven project, and then in Spring configuration paste the following: The default configuration assumes that RabbitMQ is running on a local server using the default port and default credentials (guest/guest). Of course all these settings are configurable. To sent a message to "myqueue" queue, just inject an instance of AmqpTemplate into your service and send the message. An example would be: @Service public class HomeController { @Autowired private AmqpTemplate amqpTemplate; public void sendMessage(Bundle bundle) throws IOException { byte[] body = IOUtils.toByteArray(bundle.getInputStream()); MessageProperties messageProperties = new MessageProperties(); messageProperties.setContentType(bundle.getContentType()); messageProperties.setContentLength(bundle.getSize()); messageProperties.setTimestamp(new Date()); messageProperties.setDeliveryMode(MessageDeliveryMode.PERSISTENT); Message message = new Message(body, messageProperties); amqpTemplate.send(message); } } You can open the web console http://localhost:55672/mgmt/ and see 1 message in "myqueue" queue. Apache Camel To read a message from Apache Camel you first have to add camel-amqp dependency to your POM. Then just copy and paste the following route definition: Run the route by executing mvn:camel-run and... you'll see an error. Making a long story short, Apache Camel 2.9.0 doesn't work with RabbitMQ. This is because the camel-amqp component is using the Apache Qpid client under the hood. The current Qpid version is 0.14, but Qpid guys forgot to upload new jars to the Maven public repo. Thus camel-amqp is still using Qpid 0.12 whose client doesn't seem to negotiate protocols. Even if you exclude qpid-commons and qpid-client dependencies and explicitly add Qpid 0.14 ones (download them and install in your local repo) there will be an exception thrown from the camel-amqp component as there is no longer a default ConnectionFactory constructor. Thus I was forced to install Qpid. Qpid I downloaded the Java server and simply ran it. There is no web management console, but that's OK. You can use JConsole for JMX. Spring AMQP and Qpid In order to make Spring AMQP work with Qpid copy and paste the following configuration: As you can see in the above snippet I explicitly created AMPQComponent with connectionFactory set to Apache Qpid AMQConnectionFactory object. Source code and working example This solution is a part of the Qualitas project. I use Spring MVC to handle uploads of business processes bundles (e.g., zipped archive of a WS-BPEL process) and send it to an AMQP queue. Then Apache Camel consumes the message, does additional processing of the bundle, and installs it on a remote business process execution engine. The projects you are most interested in are: qualitas-webapp (Spring MVC sending messages to AMQP) qualitas-internall-installation (Apache Camel route consuming messages from AMQP) To check out 0.0.2-SNAPSHOT tag from here: http://code.google.com/p/qualitas/source/browse/. Qualitas Read more about Qualitas project here: http://code.google.com/p/qualitas/. Happy to welcome new developers on board! cheers, Łukasz

This is the first part of a two part article about caching in WCF services. In this part I will explain the in-process memory cache available in .NET 4.0. In the second part I will describe the Windows AppFabric distributed memory cache. The .NET framework has provided a cache for ASP.NET applications since version 1.0. For other types of applications like WPF applications or console application, caching was never possible out of the box. Only WCF services were able to use the ASP.NET cache if they were configured to run in ASP.NET compatibility mode. But this mode has some performance drawbacks and only works when the WCF service is hosted inside IIS and uses an HTTP-based binding. With the release of the .NET 4.0 framework this has luckily changed. Microsoft has now developed an in-process memory cache that does not rely on the ASP.NET framework. This cache can be found in the “System.Runtime.Caching.dll” assembly. In order to explain the working of the cache, I have a created a simple sample application. It consists of a very slow repository called “SlowRepository”. public class SlowRepository { public IEnumerable GetPizzas() { Thread.Sleep(10000); return new List() { "Hawaii", "Pepperoni", "Bolognaise" }; } } This repository is used by my sample WCF service to gets its data. public class PizzaService : IPizzaService { private const string CacheKey = "availablePizzas"; private SlowRepository repository; public PizzaService() { this.repository = new SlowRepository(); } public IEnumerable GetAvailablePizzas() { ObjectCache cache = MemoryCache.Default; if(cache.Contains(CacheKey)) return (IEnumerable)cache.Get(CacheKey); else { IEnumerable availablePizzas = repository.GetPizzas(); // Store data in the cache CacheItemPolicy cacheItemPolicy = new CacheItemPolicy(); cacheItemPolicy.AbsoluteExpiration = DateTime.Now.AddHours(1.0); cache.Add(CacheKey, availablePizzas, cacheItemPolicy); return availablePizzas; } } } When the WCF service method GetAvailablePizzas is called, the service first retrieves the default memory cache instance ObjectCache cache = MemoryCache.Default; Next, it checks if the data is already available in the cache. If so, the cached data is used. If not, the repository is called to get the data and afterwards the data is stored in the cache. For my sample service, I also choose to restrict the maximum memory to 20% of the total physical memory. This can be done in the web.config.

this is a live timing map application for f1 championship races made using javascript and google maps markers. the live timing data is supplied by formula1.com. it’s interactive, you can press over a driver to track him or press into an empty map zone to untrack and have a general view. it has also been made with a responsive design to adapt it to mobile browsers using jquerymobile framework. how it works: the client side: until the race start date a countdown and a demo race is showed. when the countdown finishes it will connect to server (using ajax) to get the live timing data from server (every five seconds) and the interface will be updated using this data. the server side: it uses a django app for the web page and the static race data (circuit, laps, drivers) is put into the html using the django template system. for the dynamic data (live timing) i have modified the source of a c program for the linux terminal called live-f1 to generate a json with the data that the client requires instead of printing it on terminal screen. enjoy the race!

Message groups in JMS provide a way to identify a set of related messages. The messages could be related by anything - a customer order number, for example. Basically a JMS broker provides a guarantee that any messages that belong to a specific group will always be consumed by a common consumer. For instance, imagine that we’ve used the splitter pattern to split out line items from an order but want to aggregate those line items together later in a route. In order to perform that aggregation you need to guarantee that all of the messages being aggregated together are consumed by the same consumer. Below is an example of using message groups with ActiveMQ within Apache Camel. package com.brinksys.camel; import org.apache.activemq.ActiveMQConnectionFactory; import org.apache.activemq.broker.BrokerService; import org.apache.activemq.camel.component.ActiveMQComponent; import org.apache.activemq.pool.PooledConnectionFactory; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; import org.apache.camel.Processor; import org.apache.camel.ProducerTemplate; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.impl.DefaultCamelContext; import java.util.concurrent.TimeUnit; public class App { private static BrokerService broker; public static void main(String[] args) throws Exception { try { startBroker(); CamelContext ctx = createCamelContext(); ctx.start(); ctx.addRoutes(new RouteBuilder() { @Override public void configure() throws Exception { /* Our direct route will take a message, and set the message to group 1 if the body is an integer, * otherwise set the group to 2. * * This demonstrates the following concepts: * 1) Header Manipulation * 2) Checking the payload type of the body and using it in a choice. * 3) JMS Message groups */ from("direct:begin") .choice() .when(body().isInstanceOf(Integer.class)).setHeader("JMSXGroupID",constant("1")) .otherwise().setHeader("JMSXGroupID",constant("2")) .end() .to("amq:queue:Message.Group.Test"); /* These two are competing consumers */ from("amq:queue:Message.Group.Test").routeId("Route A").log("Received: ${body}"); from("amq:queue:Message.Group.Test").routeId("Route B").log("Received: ${body}"); } }); sendMessages(ctx.createProducerTemplate()); Thread.sleep(TimeUnit.SECONDS.toMillis(10)); stopBroker(); } catch (Exception e) { e.printStackTrace(); } } private static CamelContext createCamelContext() throws Exception { CamelContext camelContext = new DefaultCamelContext(); ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory("vm://localhost/"); PooledConnectionFactory pooledConnectionFactory = new PooledConnectionFactory(activeMQConnectionFactory); pooledConnectionFactory.setMaxConnections(8); pooledConnectionFactory.setMaximumActive(500); ActiveMQComponent activeMQComponent = ActiveMQComponent.activeMQComponent(); activeMQComponent.setUsePooledConnection(true); activeMQComponent.setConnectionFactory(pooledConnectionFactory); camelContext.addComponent("amq", activeMQComponent); return camelContext; } private static void sendMessages(ProducerTemplate pt) throws Exception { for (int i = 0; i < 10; i++) { pt.sendBody("direct:begin", Integer.valueOf(i)); } for (int i = 0; i < 10; i++) { pt.sendBody("direct:begin", "next group"); } pt.sendBody("direct:begin", Integer.valueOf(1)); pt.sendBody("direct:begin", "foo"); pt.sendBody("direct:begin", Integer.valueOf(2)); } private static void startBroker() throws Exception { broker = new BrokerService(); broker.addConnector("vm://localhost"); broker.start(); } private static void stopBroker() throws Exception { broker.stop(); } } The result of running this main method is as follows: 2445 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 0 2447 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 1 2460 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 2 2466 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 3 2472 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 4 2479 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 5 2482 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 6 2485 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 7 2488 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 8 2490 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 9 2493 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2496 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2499 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2501 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2504 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2505 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2508 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2510 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2513 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2515 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: next group 2517 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 1 2535 [Camel (camel-1) thread #1 - JmsConsumer[Message.Group.Test]] INFO Route B - Received: foo 2538 [Camel (camel-1) thread #0 - JmsConsumer[Message.Group.Test]] INFO Route A - Received: 2 You’ll notice that all messages with a groupId of 1 are consumed by one route and the messages with a groupId of 2 are consumed by the other consumer. You’ll also see how relatively simple it is to inspect the body of our original message to check it’s type and set the header in the route that begins our orchestration. If you wish to run this source code, I’ve set up a little Git repository on github for hosting some camel examples. As of the time I write this, only the message group example is available, but others should appear soon.

I created a new MVC 3 web application called breadcrumb and I added a reference to the site map provider via the NuGet Package Manager.

I am starting a little series about Quartz scheduler internals, tips and tricks, this is chapter 0 - how to configure persistent job store.

The latest incarnation of the Spring Framework (3.1) brought an interesting feature called Flash Attributes. It is a remedy for the problem mentioned a long time ago, in one of my posts: Spring MVC - Session Attributes handling. This problem can be described in few words: if we want to pass the attributes via redirect between two controllers, we cannot use request attributes (they will not survive the redirect), and we cannot use Spring's @SessionAttributes (because of the way Spring handles it), only an ordinary HttpSession can be used, which is not very convenient. Below you will find an example of Flash Attributes usage, before you start reviewing it, read Using flash attributes section of Spring documentation. Suppose that we have two controllers: AController and BController, first one will prepare some data and pass to the second using Flash Attributes after the form submission. On the AController we will have something like this: @RequestMapping(method = RequestMethod.POST) public String handleFormSubmission(..., final RedirectAttributes redirectAttrs) { ... redirectAttrs.addFlashAttribute("AttributeName", value); return "redirect:to_some_url_handled_by_BController"; } When the form will be submitted, attribute value will be stored as Flash Attribute named "AttributeName", and thanks to the Spring, will be passed to BController, where it can be used for example in following way: @Controller ... @SessionAttributes("AttributeName") public class SearchCriteriaHandler { ... @RequestMapping(method = RequestMethod.GET) public void handleGetRequest(@ModelAttribute("AttributeName") final SomeType value) { ... } ... } Before your handler method will be called, Spring Framework will populate the Model with the available Flash Attributes - at this point value passed from AController will become a model attribute for the BController. Note, that because we also defined this attribute as the Session Attribute, it will be automatically stored for future usage within this controller, after the GET request handling. Let me say that I was waiting for this feature for the long time, ;)