Spring Integration (SI) is a framework enabling a collection of individual applications to integrate together to deliver a business enterprise system. The framework is essentially a lightweight messaging system that enables spring based applications to communicate with one another and supports integration with external systems via declarative adaptors. It is based on the 'filters and pipes' design architecture. A key feature of it is that it achieves this integration in a minimally intrusive way. The framework is built on 3 main components: Messages Encapsulate the data to be transferred from one place to another. They comprise of a header (holds meta data such as message-id, timestamp, etc) and a payload (your data typically in the form of a POJO). Channels Provide a mechanism to transport messages from one endpoint to another. Represents the pipes in the pipes & filters architecture. SI offers two types of channels, namely Pollable and Subscribable Channels. The former rely on consumers to periodically check for messages whereas the latter is directly responsible for notifying registered consumers when messages become available. Endpoints Consumer/Producer of messages. Performs some action based on the payload. Endpoints come in various flavours, each performing a different function. These include Transformers (transform data), Routers (route data), Filters (filter data), Splitter (splits messages), Aggregator (aggregates group of messages into single message), Service Activator (connecting messages to Services) and Channel Adapters (connect channels to external applications). The basic idea behind the SI framework is that applications communicate with each other by sending/receiving messages. These messages would typically contain the information (payload) required by the next application in the process pipeline. The transport of messages from one application to another is performed by Channel components. The Endpoints perform some action based on the payload. This could be routing the messages to another endpoint or processing the payload itself. The objective of this post is to provide an introduction to Spring Integration. To help achieve this, I developed a sample application which will be discussed below. The source for this sample application is available at here. The project was built and run using spring-integration-4.0.0, maven 3.2.1 and jdk1.6. The main dependency is for the relevant spring-integration jar as declared in the pom.xml: org.springframework.integration spring-integration-stream 4.0.0.RC1 I ran the application using the maven exec plugin. This allows me to clean, package and run the application by invoking mvn clean package exec:java -P OnlineShop from the command line. Developing a sample application: Tabernus My goal as usual was to build something very simple which would help me to become familiar with key concepts of this framework and to this end I've knocked up a simple app which does not connect up individual systems but rather invokes methods on a POJO. Extending this to actual working applications shouldn't be too difficult. The scenario I'm going to model revolves around purchasing items from an online store (Tabernus). This store only sells 3 types of items: Books, Music CDs, and software. During a Sale, the owners have decided to apply different discounts based on the item type. In this instance books, music and software benefit from discounts of 5%, 10%, and 15% respectively. The following diagram shows our domain entities. The class diagram shows that a Customer can place an Order comprising of a number of OrderItems which are of type Book, MusicCD or Software. The problem I need to solve is to design a system which can interrogate each Order and apply the correct discount based on the item type. Subsequently it should be able to compute the total cost of the order once the discounts have been applied. To model this using Spring Integration we need the following pipeline The above diagram shows various components most of which can be divided into 2 categories, channels (blue cylinder shapes) and endpoints (rectangular boxes). The exception to this is the Poller component whose purpose is to enable the various endpoints to function correctly and discussion of it will be given later. We'll start off by briefly covering the various stages in this pipeline as indicated by the numbers in red. Following this we will delve deeper into how we build this pipeline using the SI framework. The pipeline is comprised of 6 major stages as reflected by the numbers in the diagram, The Gateway component represents the entry point to the messaging system. All new Orders will be submitted to this component which will in turn wrap them as messages and place them into the channel appropriately named ordersChannel. Using the Splitter component - each Order is decomposed into a collection of it's constituent OrderItem instances. Each of these is wrapped in a Message and placed in the orderItemsChannel. The Router component considers each OrderItem in turn and places it in the relevant channel, e.g. Book items will be placed in the bookItemsChannel etc. This allows us to consider the different item types separately. The ServiceActivator needs to consider messages within each of the 3 channels and calculate the correct discount based on the channel. After completing the calculation for each OrderItem, it will place the OrderItem in the processedItemsChannel. The Aggregator component will collect all OrderItem instances placed in the processedItemsChannel and reconstruct the original Order. This will subsequently be placed in the deliveriesChannel, which represents the end of the pipeline. The Poller Component is required to configure how often the various endpoints will interrogate their respective input channels for messages. To implement the pipeline shown above using the SI framework, we need to implement the various end points. configure the pipeline in an xml file (Shop.xml) - identifying the various channels and endpoints and how they wire up together. At this point I should mention that SI offers 2 approach to configuring your process pipeline, annotations based and xml. In this article I'll be using the latter. Let's start to look at some code. We'll consider each stage described above and show the java implementation of the endpoint and xml configuration required to wire up the components. Step 1 - Gateway To begin with, we need to implement the Client that will invoke the Gateway component to place the Order. The client (OnlineShop.java) is shown below, public class OnlineShop { public static void main(String[] args) { AbstractApplicationContext context = new ClassPathXmlApplicationContext("/META-INF/com/prodcod/shop.xml", OnlineShop.class); Shop shop = (Shop) context.getBean("shop"); final Order order = createOrder(); shop.placeOrder(order); context.close(); } The logic here is quite simple. The client creates a dummy Order and passes this as an argument when it invokes the placeOrder() method on the gateway component. The gateway component referred here as Shop is injected by Spring. The Gateway component looks like: // Gateway component public interface Shop { @Gateway(requestChannel="ordersChannel") void placeOrder(Order order); } As you can see, this is simply an interface, whose implementation will be provided by Spring when it is injected into the client application. This is achieved by the use of the @Gateway annotation which informs Spring that this is a Gateway component and it needs to provide the implementation. Additionally the annotation accepts an attribute, requestChannel which defines the channel on which the Order instance will be placed. The framework does this by simply wrapping our instance of Order within a Message instance and placing it in the channel, 'ordersChannel'. The Gateway component and the 'ordersChannel' are declared as follows in the file shop.xml Step 2 - Splitter The next end point is the Splitter component. Appropriately named, it's role is to take a single message containing a payload of a collection of items and splitting it into a number of messages, each of which contains a single element from the collection. In our case, we want to decompose the Order into it's constituent OrderItem instances. It does this by taking a Message containing the payload of Order from 'ordersChannel' and then processing it before sending messages (each containing an OrderItem instance) to the 'orderItemsChannel'. Our implementation of the splitter is called OrderSplitter and is defined as below, public class OrderSplitter extends AbstractMessageSplitter{ @Override protected Object splitMessage(Message message) { return ((Order)message.getPayload()).getOrderItems(); } } Implementing a splitter is quite easy and involves extending the AbstractMessageSplitter class and overriding the splitMessage() method. This simply takes a message containing the payload of Order and returns it's collection of OrderItems. Step 3 - Router Having decomposed the Order into it's constituent OrderItems, we now need to separate them into groups of Books, MusicCD, and Software. This is achieved using a router. Our implementation of the Router looks like, public class OrderItemRouter { public String routeOrder(OrderItem orderItem) { String channel = ""; if(isBook(orderItem)) { channel = "bookItemsChannel"; } else if(isMusic(orderItem)) { channel = "musicItemsChannel"; } else if(isSoftware(orderItem)) { channel = "softwareItemsChannel"; } return channel; } ..................... ..................... } Nothing too complicated here. For each OrderItem, the method routeOrder() will determine it's item type and return the name of the channel that this message should be sent to. The channel name is returned by the method. Spring will then ensure that the message containing the OrderItem is relayed to the named channel. The configuration for OrderItemRouter looks like, The config identifies that the class OrderItemRouter is a Router component which will consume messages from the orderItemsChannel. Further Spring needs to invoke the method routeOrder() which contains the logic to perform the routing. The channels for each item type are declared as follows Step 4 - ServiceActivator The next step is to calculate the discounted price for each item type and this is performed by a ServiceActivator component. This is implemented as follows public class Shopkeeper { private static final BigDecimal BOOK_DISCOUNT = new BigDecimal(0.05); private static final BigDecimal MUSIC_DISCOUNT = new BigDecimal(0.10); private static final BigDecimal SOFTWARE_DISCOUNT = new BigDecimal(0.15); /** * Performs discount on books * @param bookOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processBooks(OrderItem bookOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(bookOrderItem, BOOK_DISCOUNT); bookOrderItem.setDiscountedPrice(finalPrice); return bookOrderItem; } /** * Performs discount on music * @param musicOrderItem OrderItem comprising of a music item * @return OrderItem with discount price newly calculated */ public OrderItem processMusic(OrderItem musicOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(musicOrderItem, MUSIC_DISCOUNT); musicOrderItem.setDiscountedPrice(finalPrice); return musicOrderItem; } /** * Performs discount on software * @param softwareOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processSoftware(OrderItem softwareOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(softwareOrderItem, SOFTWARE_DISCOUNT); softwareOrderItem.setDiscountedPrice(finalPrice); return softwareOrderItem; } } This class exposes 3 methods to compute the new discounted price for each item type. Each method returns the OrderItem instance with the new price. The ServiceActivator is configured as follows: This tells Spring that the Shopkeeper class is a ServiceActivator and will consume messages from any of the 3 channels defined in the input-channel attribute. When a message appears in one of these channels, Spring will invoke the appropriate method on the ServiceActivator class as specfied by the attribute method. Anything returned from all three methods will be placed in the processedItems channel, ready for the next step of the processing pipeline. Step 5 - Aggregator The final stage is to take the individual OrderItems with their newly computed discounted prices and reconstruct the Order. This is achieved using an aggregator. Our implementation of an aggregator is listed below public class OrderCompleter { public Order prepareDelivery(List orderItems) { final Order order = new Order(); order.setOrderItems(orderItems); return order; } } The aggregator exposes a method that takes a collection of OrderItem objects. These will come from the processedItems channel declared as Recall this is the output channel for the service activator class as discussed above. The aggregator is configured in the xml file as The configuration tells Spring that the aggregator component will consume messages from the processedItems channel. These will be processed by the method prepareDelivery on the class OrderCompleter. Any output from this class will be relayed to the channel-adaptor deliveries, which is declared as The stdout-channel-adapter component writes to the systems STDOUT output stream. Step 6 - Poller To complete the setup we have to configure a poller component. This is required to enable the channels to work correctly. All our channels are of a queue type and so their respective consumers need to know when to query them. This is achieved using a poller mechanism. It is configured in the following way In this case, we have declared a global poller (as indicated by the default attribute). This will be used by the various end points to determine when they should interrogate their respective input-channels for messages. The second attribute fixed-delay is used to configure the polling interval. Running the Application Building and running the app shows the following output: The logging shows that the Customer submitted an Order for 3 items, one of each type. All items cost £100 each. The Order was then split into 3 OrderItems each of which was routed to the correct processing channel based on the item type. The ServiceActivator (Shopkeeper) then calculated the discount for each item and this was set on the OrderItem instance. The OrderItems were then aggregated using the OrderCompleter class which displays the final discounted price of £270 to be paid by the Customer. Note that the messages are logged to be in different stages of the processing pipeline despite starting off in the same order. This completes the tutorial on the Spring Integration Framework. Any comments relating to corrections, omissions, etc are welcome.

You should by now have worked your way through Part 1: Getting Started with Ansible. If you haven't, go and do that now. In this article, I'll be demonstrating a very simple application deployment workflow, deploying an insanely simple node.js application from a github repository, and configuring it to start with supervisord, and be reverse-proxied with Nginx. As with last time, we'll be using Parallax as the starting point for this. I've actually gone through and put the config in there already (if you don't feel like doing it yourself ;) - name: Install all the packages and stuff required for a demobox hosts: demoboxes user: user sudo: yes roles: - redis - nginx - nodejs - zeromq # - deploy_thingy In the 9c818d0b8f version, you'll be able to see that I've created a new role, inventively called "deploy_thingy". **Updated** I've been recommended that my __template__ role be based on the output of ansible-galaxy init $rolename So I've recreated the __template__ role to be based on an ansible-galaxy role template. There's not that many changes, but it does include a new directory 'default/' containing the Galaxy metadata required if you wish to push back to the public galaxy role index. In an attempt to make creating new roles easier, I put a __template__ role into the file tree when I first created Parallax, so that all you do to create a new role is execute: cp -R __template__ new_role_name in the roles/ directory. . ├── files │ ├── .empty │ ├── thingy.nginx.conf │ └── thingy.super.conf ├── handlers │ ├── .empty │ └── main.yml ├── meta │ ├── .empty │ └── main.yml ├── tasks │ └── main.yml └── templates └── .empty In this role, we define some dependencies in meta/main.yml, there's two files in the files/ directory, and there's a set of tasks defined in tasks/main.yml. There's also some handlers defined in handlers/main.yml. Let's have a quick glance at the meta/main.yml file. --- dependencies: - { role: nodejs } - { role: nginx } This basically sets the requirement that this role, deploy_thingy depends on services installed by the roles: nginx and nodejs. Although these roles are explicitly stated to be installed in site.yml, this gives us a level of belt-and-braces configuration, in case the deploy_thingy role were ever included without the other two roles being explicitly stated, or if it were configured to run before its dependencies had explicitly been set to run. tasks/main.yml is simple. --- - name: Create directory under /srv for thingy file: path=/srv/thingy state=directory mode=755 - name: Git checkout from github git: repo=https://github.com/tomoconnor/shiny-octo-computing-machine.git dest=/srv/thingy - name: Drop Config for supervisord into the conf.d directory copy: src=thingy.super.conf dest=/etc/supervisor/conf.d/thingy.conf notify: reread supervisord - name: Drop Reverse Proxy Config for Nginx copy: src=thingy.nginx.conf dest=/etc/nginx/sites-enabled/thingy.conf notify: restart nginx We'll create somewhere for it to live, check the code out of my git repository [1], Then drop two config files in place, one to configure supervisor(d), and one to configure Nginx. Because the command to configure supervisor(d) and nginx change the configuration of those services, there are notify: handlers to reload the configuration, or restart the service. Let's have a quick peek at those handlers now: --- - name: reread supervisord shell: /usr/bin/supervisorctl reread && /usr/bin/supervisorctl update - name: restart nginx service: name=nginx state=restarted When the supervisor config changes (and we add something to /etc/supervisor/conf.d), we need to tell supervisord to re-read it's configuration files, at which point, it will see the new services, and then run supervisorctl update, which will set the state of the newly added items from 'available' to 'started'. When we change the nginx configuration, we'll hit nginx with a restart. It's possible to do softer actions, like reload here, but I've chosen service restart for simplicity. I've also changed the basic Ansible config, and configuration of roles/common/files/insecure_sudoers so that it will still ask you for a sudo password in light of some minor criticism. I've found that if you're developing Ansible playbooks on an isolated system, then there's no great harm in disabling SSH Host Key Checking (in ansible.cfg), similarly how there's no great problems in disabling sudo authentication, so it's effectively like NOPASSWD use. However, Micheil made a very good point that in live environments it's a bit dodgy to say the least. So I've commented those lines out of the playbook in Parallax, so that it should give users a reasonable level of basic security. At the end of the day, it's up to you how you use Parallax, and if you find that disabling security works for you, then fine. It's not like you haven't been warned. But I digress. The next thing to do is to edit site.yml, and ensure that the new role we've created gets mapped to a hostgroup in the play configuration. In the latest version of Parallax this is already done for you, but as long as the role name in the list matches the directory in roles/, it should be ready to go. Now if we run: ansible-playbook -k -K -i playbooks/example/hosts playbooks/example/site.yml It should go through the playbook, installing stuff, then finally do the git clone from github, deploy the configuration files, and trigger a reread of supervisord, and a restart of nginx. If I now test that it's working, with: curl -i http://192.168.20.151/ HTTP/1.1 200 OK Server: nginx/1.4.1 (Ubuntu) Date: Mon, 27 Jan 2014 14:51:29 GMT Content-Type: text/html; charset=utf-8 Content-Length: 170 Connection: keep-alive X-Powered-By: Express ETag: "1827834703" That X-Powered-By: Express line shows that Nginx is indeed working, and that the node.js application is running too. You can get more information about stuff that supervisord is controlling by running: sudo supervisorctl status on the target host. $ sudo supervisorctl status thingy RUNNING pid 19756, uptime 0:00:06 If the Nginx side is configured, but the node.js application isn't running, you'd get a HTTP 502 error, as follows: curl -i http://192.168.20.151/ HTTP/1.1 502 Bad Gateway Server: nginx/1.4.1 (Ubuntu) Date: Mon, 27 Jan 2014 14:59:34 GMT Content-Type: text/html Content-Length: 181 Connection: keep-alive So, that's it. A very simple guide to deploying a very simple application with Ansible. Of course, it should be obvious that you can deploy *anything* from a git repository, it really boils down to the configuration of supervisord. For that matter, it doesn't have to be supervisord. I consider configuring supervisord for process controlling to be outside of the scope of this article, but I might touch on it in future in more detail. Next up, Part 3: Ansible and Amazon Web Services. 1: It's really simple, and I'm not very node-savvy, so I'm sorry if it sucks.

Data-driven testing is a powerful way of testing a given scenario with different combinations of values. In this article, we look at several ways to do data-driven unit testing in JUnit. Suppose, for example, you are implementing a Frequent Flyer application that awards status levels (Bronze, Silver, Gold, Platinum) based on the number of status points you earn. The number of points needed for each level is shown here: level minimum status points result level Bronze 0 Bronze Bronze 300 Silver Bronze 700 Gold Bronze 1500 Platinum Our unit tests need to check that we can correctly calculate the status level achieved when a frequent flyer earns a certain number of points. This is a classic problem where data-driven tests would provide an elegant, efficient solution. Data-driven testing is well-supported in modern JVM unit testing libraries such as Spock and Spec2. However, some teams don’t have the option of using a language other than Java, or are limited to using JUnit. In this article, we look at a few options for data-driven testing in plain old JUnit. Parameterized Tests in JUnit JUnit provides some support for data-driven tests, via the Parameterized test runner. A simple data-driven test in JUnit using this approach might look like this: @RunWith(Parameterized.class) public class WhenEarningStatus { @Parameters(name = "{index}: {0} initially had {1} points, earns {2} points, should become {3} ") public static Iterable data() { return Arrays.asList(new Object[][]{ {Bronze, 0, 100, Bronze}, {Bronze, 0, 300, Silver}, {Bronze, 100, 200, Silver}, {Bronze, 0, 700, Gold}, {Bronze, 0, 1500, Platinum}, }); } private Status initialStatus; private int initialPoints; private int earnedPoints; private Status finalStatus; public WhenEarningStatus(Status initialStatus, int initialPoints, int earnedPoints, Status finalStatus) { this.initialStatus = initialStatus; this.initialPoints = initialPoints; this.earnedPoints = earnedPoints; this.finalStatus = finalStatus; } @Test public void shouldUpgradeStatusBasedOnPointsEarned() { FrequentFlyer member = FrequentFlyer.withFrequentFlyerNumber("12345678") .named("Joe", "Jones") .withStatusPoints(initialPoints) .withStatus(initialStatus); member.earns(earnedPoints).statusPoints(); assertThat(member.getStatus()).isEqualTo(finalStatus); } } You provide the test data in the form of a list of Object arrays, identified by the _@Parameterized@ annotation. These object arrays contain the rows of test data that you use for your data-driven test. Each row is used to instantiate member variables of the class, via the constructor. When you run the test, JUnit will instantiate and run a test for each row of data. You can use the name attribute of the @Parameterized annotation to provide a more meaningful title for each test. There are a few limitations to the JUnit parameterized tests. The most important is that, since the test data is defined at a class level and not at a test level, you can only have one set of test data per test class. Not to mention that the code is somewhat cluttered - you need to define member variables, a constructor, and so forth. Fortunatly, there is a better option. Using JUnitParams A more elegant way to do data-driven testing in JUnit is to use [https://code.google.com/p/junitparams/|JUnitParams]. JUnitParams (see [http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22JUnitParams%22|Maven Central] to find the latest version) is an open source library that makes data-driven testing in JUnit easier and more explicit. A simple data-driven test using JUnitParam looks like this: @RunWith(JUnitParamsRunner.class) public class WhenEarningStatusWithJUnitParams { @Test @Parameters({ "Bronze, 0, 100, Bronze", "Bronze, 0, 300, Silver", "Bronze, 100, 200, Silver", "Bronze, 0, 700, Gold", "Bronze, 0, 1500, Platinum" }) public void shouldUpgradeStatusBasedOnPointsEarned(Status initialStatus, int initialPoints, int earnedPoints, Status finalStatus) { FrequentFlyer member = FrequentFlyer.withFrequentFlyerNumber("12345678") .named("Joe", "Jones") .withStatusPoints(initialPoints) .withStatus(initialStatus); member.earns(earnedPoints).statusPoints(); assertThat(member.getStatus()).isEqualTo(finalStatus); } } Test data is defined in the @Parameters annotation, which is associated with the test itself, not the class, and passed to the test via method parameters. This makes it possible to have different sets of test data for different tests in the same class, or mixing data-driven tests with normal tests in the same class, which is a much more logical way of organizing your classes. JUnitParam also lets you get test data from other methods, as illustrated here: @Test @Parameters(method = "sampleData") public void shouldUpgradeStatusFromEarnedPoints(Status initialStatus, int initialPoints, int earnedPoints, Status finalStatus) { FrequentFlyer member = FrequentFlyer.withFrequentFlyerNumber("12345678") .named("Joe", "Jones") .withStatusPoints(initialPoints) .withStatus(initialStatus); member.earns(earnedPoints).statusPoints(); assertThat(member.getStatus()).isEqualTo(finalStatus); } private Object[] sampleData() { return $( $(Bronze, 0, 100, Bronze), $(Bronze, 0, 300, Silver), $(Bronze, 100, 200, Silver) ); } The $ method provides a convenient short-hand to convert test data to the Object arrays that need to be returned. You can also externalize @Test @Parameters(source=StatusTestData.class) public void shouldUpgradeStatusFromEarnedPoints(Status initialStatus,int initialPoints, int earnedPoints,Status finalStatus){ ... } The test data here comes from a method in the StatusTestData class: public class StatusTestData{ public static Object[] provideEarnedPointsTable(){ return $( $(Bronze,0, 100,Bronze), $(Bronze,0, 300,Silver), $(Bronze,100,200,Silver) ); } } This method needs to be static, return an object array, and start with the word "provide". Getting test data from external methods or classes in this way opens the way to retrieving test data from external sources such as CSV or Excel files. JUnitParam provides a simple and clean way to implement data-driven tests in JUnit, without the overhead and limitations of the traditional JUnit parameterized tests. Testing with non-Java languages If you are not constrained to Java and/or JUnit, more modern tools such as Spock (https://code.google.com/p/spock/) and Spec2 provide great ways of writing clean, expressive unit tests in Groovy and Scala respectively. In Groovy, for example, you could write a test like the following: class WhenEarningStatus extends Specification{ def"should earn status based on the number of points earned"(){ given: def member =FrequentFlyer.withFrequentFlyerNumber("12345678") .named("Joe","Jones") .withStatusPoints(initialPoints) .withStatus(initialStatus); when: member.earns(earnedPoints).statusPoints() then: member.status == finalStatus where: initialStatus | initialPoints | earnedPoints | finalStatus Bronze |0 |100 |Bronze Bronze |0 |300 |Silver Bronze |100 |200 |Silver Silver |0 |700 |Gold Gold |0 |1500 |Platinum } } John Ferguson Smart is a specialist in BDD, automated testing, and software life cycle development optimization, and author of BDD in Action and other books. John runsregular courses in Australia, London and Europe on related topics such as Agile Requirements Gathering, Behaviour Driven Development, Test Driven Development, andAutomated Acceptance Testing. Blog Links >>

In this blog, we will discuss one particular data grid platform from Redhat namely JBoss Data Grid (JDG). We will firstly cover how to access and install this data grid platform and then we will demonstrate how to develop and deploy a simple remote client/server data grid application which utilises the HotRod protocol. We will be using the latest release JDG 6.2 from Redhat in this article. Installation Overview To start using JDG, firstly log on to the redhat site https://access.redhat.com/home and download the software from the Downloads section of the site. We wish to download JDG 6.2 server by clicking on the appropriate links in the Downloads section. For future reference, it is also useful to download the quickstart and maven repository zip files. To install JDG, we simply unzip the JDG server package into an appropriate directory in your environment. JDG Overview In this section, we will provide a brief overview of the contents of the JDG installation package and the most notable configuration options available to users. Out of the box, users are provided with two runtime options either to run JDG in standalone or clustered mode. We can start JDG in either mode by invoking the stanadalone or clustered start up scripts in the / bin directory. To configure the JDG in either mode we need to configure the files standalone.xml and clustered.xml. In our case we will creating a distributed cache which will run on 3 node JDG cluster so we will be utilizing the clustered startup script. In order to set up and add new cache instances to JDG, we modify the infinispan subsystems in the appropriate xml configuration file above. We should also note the principal difference between the standalone and clustered configuration file is that in the clustered configuration file there is a JGroups subsystem configured element which allows for communication and messaging between configured cache instances running in a JDG cluster. Development Environment Setup and Configuration In this section, we will detail how to develop and configure a simple datagrid application which will be deployed to a 3 node JDG cluster. We will demonstrate how to configure and deploy a distributed cache in JDG and also show how to develop a HotRod Java client application which will be used to insert, update and display entries in the distributed cache. We will firstly discuss setting a new distributed cache on a 3 node JDG cluster. In this example, we will run our JDG cluster on a single machine by running each JDG instance on different ports. Firstly, we will create 3 instances of JDG by creating 3 directories (server1, server2, server3) on our host machine and unzipping each JDG installation into each directory. We will now configure each node in our cluster by copying and renaming the clustered.xml configuration file in the \server1\jboss-datagrid-6.2.0-server\standalone\configuration directory. We will name each of the cluster configuration files as "clustered1.xml", "clustered2.xml" and "clustered3.xml" for the JDG instances denoted by "server1", "server2" and "server3" respectively. We will now set up a new distributed cache on our JDG cluster by modifying the infinispan subsystem element in each clustered.xml file. We will demonstrate this for the node denoted "server1" here by modifying the file "clustered1.xml". The cache configuration shown here will be the same across all 3 nodes. To setup a new distributed cache named "directory-dist-cache", we configure the following elements in the file named "clustered1.xml" ......... ...... .............. ...... ...... /socket-binding-group> We will discuss the key elements and attributes relating to the configuration above. In the infinispan endpoint subsystem, we will configure hotrod clients to connect to the JDG server instance on socket 11222. The name of the cache container to host each of the cache instances will be held in the container named "clusteredcache". We have configured the infinispan core subsystem to the default cache container named "clusteredcacahe" whereby we will allow for jmx statistics to be collected relating the configured cache entries i.e statistics="true" We have created a new distributed cache named "directory-dist-cache" whereby there will be two copies of each cache entry held on two of the 3 cluster nodes. We have also set up an eviction policy whereby should there be more than 20 entries in our cache then cache entries will be removed using the LRU algorithm We should have configured nodes "server2" and "server3" to start up with a port offset of 100 and 200 respectively by configuring the socketing binding group element appropriately. Please view the socket bindings noted below. To set the socket binding element with a port offset of 100 on "server2", we configure "clustered2.xml" with the following entry: ...... ...... /socket-binding-group> To set the socket binding element with a port offset of 200 on "server3", we configure "clustered3.xml" with the following entry: ...... ...... /socket-binding-group> Before discussing the setup and configuration of our Hotrod client which will be used to interact with our JDG clustered HotRod server, we will start up each server instance to ensure our newly configured JDG distributed cache starts up correctly. Open up 3 Windows or Linux consoles and execute the following start up commands: Console 1: 1) Navigate to \server1\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the first instance of our JDG cluster denoted "server1": clustered -c=clustered1.xml -Djboss.node.name=server1 Console 2: 1) Navigate to \server2\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the second instance of our JDG cluster denoted "server2": clustered -c=clustered2.xml -Djboss.node.name=server2 Console 3: 1) Navigate to \server3\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the third instance of our JDG cluster denoted "server3": clustered -c=clustered3.xml -Djboss.node.name=server3 Providing all 3 JDG instances have started up correctly, you should see output in the console window whereby we can see there are 3 JDG instances in the JGroups view: HotRod Client Development Setup Now that the Hotrod server is up and running, we need to develop a Hotrod Java client which will interact with the clustered server application. The development environment consists of the following tools. 1) JDK Hotspot 1.7.0_45 2) IDE - Eclipse Kepler Build id: 20130919-0819 The HotRod client application is a simple application consisting of two Java classes. The application allows users to retrieve a reference to the distributed cache from the JDG server and then perform these actions: a) add new cinema objects. b) add and remove shows to each cinema object. c) print the list of all cinemas and shows stored in our distributed cache. The source code can be downloaded from github @ https://github.com/davewinters/JDG. We could use maven here to build and execute our application by configuring the maven settings.xml to point to the maven repository files we downloaded earlier and set up a maven project file (pom.xml) to build and execute the client application. In this article we will build our application using the Eclipse IDE and run the client application on the command line. To create a HotRod client application and execute the sample application, one should complete the following steps: 1) Create a new Java Project in Eclipse 2) Create a new package named uk.co.c2b2.jdg.hotrod and import the source code that has been downloaded from Github mentioned previously. 3) Now we need to configure the build path in Eclipse to contain the appropriate JDG client jar files which are required to compile the application. You should include all the client jar files in the project build path. These jar files are contained in the JDG installation zip file. For example on my machine these jar files are located in the directory: \server1\jboss-datagrid-6.2.0-server\client\hotrod\java 4. Providing the Eclipse build path has been configured appropriately, the application source should compile without issue. 5. We will need to execute the Hotrod application by opening the console window and executing the following command. Note the path specified here will differ depending on where the JDG client jar files and application class files are located in your environment: java -classpath ".;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\commons-pool-1.6-redhat-4.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-client-hotrod-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-commons-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-query-dsl-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-remote-query-client-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-logging-3.1.2.GA-redhat-1.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-marshalling-1.4.2.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-marshalling-river-1.4.2.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\protobuf-java-2.5.0.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\protostream-1.0.0.CR1-redhat-1.jar" uk/co/c2b2/jdg/hotrod/CinemaDirectory 6. The Hotrod client at runtime provides the end user with a number of different options to interact with the distributed cache as we can view from the console window below. Client Application Principal API Details We will not provide a detailed overview of the Hotrod application code however we will describe the principal API and code details briefly. In order to interact with the distributed cache on the JDG cluster using the Hotrod protocol, we will use the RemoteCacheManager Object which will allow us to retrieve a remote reference to the distributed cache. We have initialised a Properties object with the list of JDG instances and the associated with HotRod server port on each instance. We can add Cinema objects into the distributed cache using the RemoteCache.put() method. private RemoteCacheManager cacheManager; private RemoteCache cache; ..... Properties properties = new Properties(); properties.setProperty(ConfigurationProperties.SERVER_LIST, "127.0.0.1:11222;127.0.0.1:11322;127.0.0.1:11422"); cacheManager = new RemoteCacheManager(properties); cache = cacheManager.getCache("directory-dist-cache"); ..... cache.put(cinemaKey, cinemalist); In the webinar below, I describe in further detail how to set up a JDG cluster and how to develop and run the JDG application discussed above. For further details on JDG please visit: http://www.redhat.com/products/jbossenterprisemiddleware/data-grid/ Webinar: Introduction to JBoss Data Grid -- Installation, Configuration and Development In this webinar we will look at the basics of setting up JBoss Data Grid covering installation, configuration and development. We will look at practical examples of storing data, viewing the data in the cache and removing it. We will also take a look at the different clustered modes and what effect these have on the storage of your data:

a powerful multi-format xml schema (xsd) documentation generator and a tool for rapid development of custom xsd documentation generators according to user needs. about docflex/xml "xsddoc" template set template processor template designer integrations generation of xsd diagrams apache ant & maven links about docflex/xml docflex/xml is a java-based software system for development and execution of high performance template-driven documentation generators from any data stored in xml files. the actual doc/report generators are programmed in the form of special templates using a graphic template designer , which represents the templates visually in a form resembling the output they generate. further, the templates are interpreted by a template processor , which takes on input the xml files and produces by them the result documentation. this article describes an application of docflex/xml for the task of generation of high-quality xml schema documentation. that includes the following features of docflex/xml system: " xsddoc " template set that implements the ready-to-use xml schema documentation generator itself. template processor makes the templates works. currently, it provides three interchangeable output generators for html, rtf, txt (plain text) formats. template designer provides a high quality gui to design/modify templates. if you need a special xml schema doc generator, the simplest way to create it is to modify the standard xsddoc templates. the template designer enables you to do that. integrations with altova xmlspy and oxygen xml editor . if you are a user of one of those popular xml editors, you can turn it also into a dynamically linked diagramming engine for docflex, so that to include automatically the xsd diagrams generated by xmlspy/oxygenxml into the xml schema documentation generated by docflex (with the full support of hyperlinks). "xsddoc" template set it is the implementation of xml schema documentation itself, which provides the following functionality: generation of single documentation by any number of xml schema (xsd) files together, in particular: highly navigable framed (javadoc-like) html documentation single-file html documentation rtf documentation (further convertible to pdf) processing of any referenced xml schemas, in particular: correct processing of all , , elements found across all involved xsd files. automatic loading and processing (i.e. inclusion in the documentation scope) all directly/indirectly referenced xsd files. sophisticated documenting of xsd components , including: component diagrams (with hyperlinks to everything depicted on them; see also integrations ) xml representation summary (a textual alternative to diagrams) lists of related components. for elements this includes also the list of possible containing elements . (such a list is never present in the output generated by xslt-based doc generators). list of usage locations support of any xml schema design patterns . this comes down mainly to the following: special treatment of local elements (see below) support and documenting of substitution groups support of importing, inclusion and redefinition of schema files special documenting of local elements . local elements are those components that are declared locally within other xsd components. w3c xml schema spec allows you to declare any number of local elements that may share the same name but have different content. that's because their meaning is local and there will be no collisions with other declarations. that, however, creates a problem for documenting, because in a documentation both global and local elements may appear simultaneously in various lists according to their common properties. if each element component is identified only by its name, you will get the lists with multiple repeating names but little clue what they mean. moreover, some xml schemas may contain lots of identical local element declarations (that is, they have the same both name and content). so, you'll get in those lists a mess of repeating names, some of which referencing to effectively the same entities, whereas others to complete different ones. in xsddoc , those problems are solved in two ways: adding extensions to local element names. the extension provides more information about the element (e.g. where it can be inserted or its global type or where it is defined). that makes the whole string identifying the element unique. here is how it looks. the grey text is the name extension: unifying local elements by type. on the left you can see a documentation generated with such unification. on the right, all local elements are documented straight as they are. click on each screenshot to view the docs: we believe the first documentation (on the left) is easier to understand and use. processing of xhtml markup . you can format your xml schema annotations with xhtml tags, which will be recognized and rendered with the appropriate formatting in both html and rtf output, as shown on the following screenshots (click to see more details): here, on the left you can see the xml source of an xml schema, whose annotations are heavily laden with xhtml markup (including insertion of images). the next is the html documentation generated by that schema. on the right is a page of rtf documentation also generated by that schema. possibility of unlimited customization : xsddoc is controlled by more than 400 parameters, which allow you to adjust the generated documentation within huge range of included details. template parameters serve the same role as options in traditional doc generators. the difference is that docflex template architecture makes the support/implementation of template parameters very cheap (typically, the most of efforts takes writing their descriptions). so, there may be hundreds of parameters controlling a large template application. if parameters are not enough, you can modify the templates themselves using the template designer . in case of html output, you can also apply your own css styles to change how the generated documentation looks. template processor the template processor (also called simply "generator") makes everything work. it consists of two logical parts: 1. template interpreter 2. output generator the output generator actually has three different implementations for each currently supported output format: html, rtf, txt (plain text). the plain-text output can be used to generate documentation in formats not supported directly by docflex. the template processor is started directly from java command line with the following arguments: ● main template ● template parameters ● initial xsd files to be processed (documented) ● xml catalogs (to redirect physical location of input files) ● destination directory/file ● output format (this selects which output generator will be used) ● output format options (specify settings to control the selected output generator) actually, the number of settings may be so large that the template processor provides a special gui to specify everything interactively (click to enlarge): template designer although docflex templates are stored as plain-text files (with an xml-like format), they are not supposed for editing manually. rather, a special graphic template designer must be used, which visualizes the templates in the form of template components they are made of. those components are the actual constructs of the template language (not some textual statements, operators, blocks etc.) the following screenshots show templates open in the template designer (click to see a lot more): that approach has a number of advantages, among them: the processing structures represented by template components may be displayed in a way that visually expresses what a component does (for instance, it may resemble the output it generates). that representation may be both expressive and compact (after all, it is not just a text), which allows you easily to navigate a template, understand what it does and modify anything you need. as template components are visual and interactive, they may have very complex internal structure, for instance, contain lots of properties and nested components. at that, you don't need to scroll and navigate some kind of enormous text, which encodes all of this (as it would be in case of a script). rather, you just need to invoke some property dialogs and expand/collapse some component sections. a template component may be easily copied, pasted and deleted as a whole. at that, you don't need to bother that the template syntax is restored after that. the template designer will also ensure that each component is created, copied or moved only in the allowed place. the highly structured nature of templates eliminates the need for most of various named identifiers. many connections between different template components are also maintained by the template designer (i.e. modified automatically when necessary). as template files are stored and read only programmatically, there is no need to know and understand their syntax. there will be no syntax errors either. the actual syntax of template files may be optimized not for human programmers, but for faster loading and processing of templates by the template processor . there is no need in a compilation phase. the separation of template semantics from the particular structure of template files helps for faster and easier evolution of the template language. the obsolete constructs of older template versions can be automatically converted into new structures. both old and new templates will look and work up-to-date. integrations generation of xsd diagrams docflex/xml is able to work with any kind of diagrams (i.e. inserting them automatically in the generated output). that is supported on the level of templates, along with the generation of hypertext imagemaps, as shown on the following screenshot (click to see a lot more): docflex/xml provides no diagramming engine of its own. instead, it includes integrations with two most popular xml editors that do generate xsd diagrams: ● altova xmlspy ● oxygen xml editor effectively, the third-party software is used as dynamically linked diagramming engine. the advantage of such integrations is that when you are the user of one of those xml editors, you will get in the documentation generated by docflex the same diagrams as you see in your xml editor. here is how such a documentation with diagrams looks (click on a screenshot to view the real html): apache ant & maven as a pure java application, docflex/xml can be run in any environment that runs java itself. the template processor can be easily integrated with ant (that can be specified just in the ant build file). in case of maven, docflex/xml includes a simple maven plugin. it is possible also to use all diagraming integrations with both ant and maven. links docflex/xml (home page): http://www.filigris.com/docflex-xml/ docflex/xml xsddoc: http://www.filigris.com/docflex-xml/xsddoc/ xsddoc examples: http://www.filigris.com/docflex-xml/xsddoc/examples/ xmlspy integration: http://www.filigris.com/docflex-xml/xmlspy/ oxygenxml integration: http://www.filigris.com/docflex-xml/oxygenxml/ free downloads: http://www.filigris.com/downloads/ this original article: http://www.filigris.com/ann/docflex-xsd/

This is a sample that I have wanted to try for sometime - A Websocket application to tail the contents of a file. The following is the final view of the web-application: There are a few parts to this application: Generating a File to tail: I chose to use a set of 100 random quotes as a source of the file content, every few seconds the application generates a quote and writes this quote to the temporary file. Spring Integration is used for wiring this flow for writing the contents to the file: Just a quick note, Spring Integration flows can now also be written using a Java Based DSL, and this flow using Java is available here Tailing the file and sending the content to a broker The actual tailing of the file itself can be accomplished by OS specific tail command or by using a library like Apache Commons IO. Again in my case I decided to use Spring Integration which provides Inbound channel adapters to tail a file purely using configuration, this flow looks like this: and its working Java equivalent There is a reference to a "fileContentRecordingService" above, this is the component which will direct the lines of the file to a place where the Websocket client will subscribe to. Websocket server configuration Spring Websocket support makes it super simple to write a Websocket based application, in this instance the entire working configuration is the following: @Configuration @EnableWebSocketMessageBroker public class WebSocketDefaultConfig extends AbstractWebSocketMessageBrokerConfigurer { @Override public void configureMessageBroker(MessageBrokerRegistry config) { //config.enableStompBrokerRelay("/topic/", "/queue/"); config.enableSimpleBroker("/topic/", "/queue/"); config.setApplicationDestinationPrefixes("/app"); } @Override public void registerStompEndpoints(StompEndpointRegistry registry) { registry.addEndpoint("/tailfilesep").withSockJS(); } } This may seem a little over the top, but what these few lines of configuration does is very powerful and the configuration can be better understood by going through the reference here. In brief, it sets up a websocket endpoint at '/tailfileep' uri, this endpoint is enhanced with SockJS support, Stomp is used as a sub-protocol, endpoints `/topic` and `/queue` is configured to a real broker like RabbitMQ or ActiveMQ but in this specific to an in-memory one. Going back to the "fileContentRecordingService" once more, this component essentially takes the line of the file and sends it this in-memory broker, SimpMessagingTemplate facilitates this wiring: public class FileContentRecordingService { @Autowired private SimpMessagingTemplate simpMessagingTemplate; public void sendLinesToTopic(String line) { this.simpMessagingTemplate.convertAndSend("/topic/tailfiles", line); } } Websocket UI configuration The UI is angularjs based, the client controller is set up this way and internally uses the javascript libraries for sockjs and stomp support: var tailFilesApp = angular.module("tailFilesApp",[]); tailFilesApp.controller("TailFilesCtrl", function ($scope) { function init() { $scope.buffer = new CircularBuffer(20); } $scope.initSockets = function() { $scope.socket={}; $scope.socket.client = new SockJS("/tailfilesep); $scope.socket.stomp = Stomp.over($scope.socket.client); $scope.socket.stomp.connect({}, function() { $scope.socket.stomp.subscribe("/topic/tailfiles", $scope.notify); }); $scope.socket.client.onclose = $scope.reconnect; }; $scope.notify = function(message) { $scope.$apply(function() { $scope.buffer.add(angular.fromJson(message.body)); }); }; $scope.reconnect = function() { setTimeout($scope.initSockets, 10000); }; init(); $scope.initSockets(); }); The meat of this code is the "notify" function which the callback acting on the messages from the server, in this instance the new lines coming into the file and showing it in a textarea. This wraps up the entire application to tail a file. A complete working sample without any external dependencies is available at this github location, instructions to start it up is also available at that location. Conclusion Spring Websockets provides a concise way to create Websocket based applications, this sample provides a good demonstration of this support. I had presented on this topic recently at my local JUG (IndyJUG) and a deck with the presentation is available here

In this post, I will show how to integrate Apache Tiles 3 with Spring MVC.

This is the fourth in a series of posts on the new JBoss Red Deer automated test framework. This post describes features in Red Deer that make it easier to debug test failures. Writing any automated UI tests can be a challenging task. Your tests have to be able to manipulate the UI, deal with platform-dependent display characteristics, and timing issues. But, what about when things go wrong and you have to debug a failing test? Are you looking at a bug in the program that you’re testing? Or, perish the thought, what if the bug is not in the program you’re testing, but it’s in your test program? The primary goal of Red Deer is to make this task easier by providing a open source solution that is reliable and extensible. Red Deer also makes your life easier when things go wrong by providing your with multiple debugging features. This post describes these features and illustrates how you can enable/disable Red Deer’s debugging features, and how you can use them to debug test failures. Maven and Eclipse Debugging is Still Usable Before discussing Red Deer’s debugging features, it’s important to remember that in no way does Red Deer prevent you from performing the types of debugging that you can use with Maven or Eclipse. You can use the Eclipse debugger to set breakpoints in Red Deer test programs in the same manner as any other programs. Likewise, to debug a Red Deer test program with maven, all you have to do is to run maven with -DdebugPort=, and in Eclipse, select the test class that you want to execute, select "Debug As," and create a new debug configuration. In the debug configuration, specify that you want to run the test class as a Remote Java Application, and the port number. Starting the debugger makes the waiting test execution run and then stop on the first breakpoint: Debugging Features Added by Red Deer The Red Deer debugging features that we’ll discuss in this post are: Red Deer Debug Logging Automatic Screenshots for Failed Tests Pausing Failing Tests Recording Screencasts Let’s start by looking at the simplest of Red Deer’s debugging tools; logging. Red Deer Debug Logging By default, Red Deer creates a DEBUG level log whenever a test is run. The log is written to stdout when you run tests from a shell and to the console view when you run tests in Eclipse. Some example output of a Red Deer log looks like this: INFO [thread][class] Log message-N INFO [thread][class] Log message-N+1 ERROR [thread][class] Hey! Something failed here DEBUG [thread][class] And, here’s some additional debug information INFO [thread][class] Log message-N+2 To disable debug logging, you set the “logDebug” JVM argument to false. For example: -DlogDebug=false In addition, you can also filter the log contents with the “logMessageFilter” JVM argument. The supported filter values are: debug error warn info trace none all You can set multiple filters by specifying filter values in a single string using vertical pipe separators. For example: -DlogMessageFilter=error|warn Note that the filter values are case insensitive. Automatic Screenshots for Failed Tests As is the case with debug logging, screenshot saving for failed tests is enabled by default by Red Deer. What happens is that when a Red Deer test fails, at the point of failure a screenshot is taken and stored for you. The screenshot illustrates the state of the UI when the test failure occured. By default, Red Deer saves the screenshot files in the “target/screenshot” directory. You can select the directory into which Red Deer will save the screenshot files by setting this JVM argument: relativeScreenshotDirectory For example: -DrelativeScreenshotDirectory=/home/jsmith/screenshots Let’s take a look at this in action. The following screenshot was generated when a test failed. The server address in question is valid. The error was due to a temporary network connectivity issue. What happens under the hood here is that when a test fails, the Red Deer watchdog process takes over, and invokes an extension of the org.eclipse.swt.graphics.ImageLoader class to create a screenshot, before it terminates the test. Having a screenshot is helpful, as it shows the state of the UI Red Deer determined that something went very wrong, but it's limited in that after the failure, Red Deer immediately stops the test and exits. In some cases having Red Deer take a screenshot and then exit may be adequate as the source of the failure is obvious. In other cases, however, you might prefer that Red Deer enable you to investigate the cause of the failure before the test is stopped and Red Deer exits. Pausing Failing Tests In some ways, an automated test failure is like an automobile crash. Something unexpected happens, and before you can react, the test goes off the road, rolls over and finishes in a ditch. Wouldn't life be easier if you could press a pause button to avoid bad things like this from happening? Likewise, it would be helpful if Red Deer, instead of terminating the test immediately could enable you to press a "pause" button and freeze the test so that you could examine the state of the accident so that you could understand its causes. Luckily Red Deer is a built-in feature to enable you to pause failing tests. Unlike the screenshot feature, having Red Deer pause when a test fails is not enabled by default. To enable the feature, all you have to do is to set the "pauseFailedTest" JVM argument to “true.” For example: -DpauseFailedTest=true With this argument set to “true,” when a test fails, it connects to the Red Deer watchdog process and test execution is paused. You can cause the test execution to continue by pressing the Enter key. Note that in the current version of Red Deer, pauseFailedTest only works when your test extends the RedDeerTest class. In a future release of Red Deer, it will work on all test types. Recording Screencasts Up to now, the Red Deer debugging features that we’ve discussed have all had one characteristic in common; they all require that you run the test and manually observe the results. This means that for an error that is subtle and difficult to track, you have may have to rerun the test several times in order to be able to see the error. It would be more helpful if you had an easy way to pause the execution of the test at the point where it fails and “rewind” the execution to a point before the failure, without having to take the time, and occupy system resources, to rerun the test from its beginning. Red Deer solves this problem by enabling you to save a screencast of all failed tests. Red Deer performs the screencast captures through an extension of the org.monte.screenrecorder.ScreenRecorder (http://www.randelshofer.ch/monte/) class As is the case with pausing tests, Red Deer’s screencast feature is disabled by default. To enable the the recording of screenshots, you set the recordScreenCast JVM argument to “true.” For example: -DrecordScreenCast=true The screencast files are stored in a subdirectory named, appropriately enough, “screencast.” References https://github.com/jboss-reddeer/reddeer/wiki/Debugging-RedDeer Acknowledgements Many thanks to Jirka Peterka and Vlado Pakan for their input to this post!

1 Start with What You Know Now Traditionally, a product strategy is the result of months of market research and business analysis work. It is intended to be factual, reliable, and ready to be implemented. But in an agile, dynamic environment a product strategy is best created differently: Start with your idea, state the vision behind it, and capture your initial strategy. Then identify the biggest risk or the crucial leap-of-faith assumption, address it, and change and improve your strategy. Repeat this process until you are confident that your product strategy is valid. This iterative approach, piloted by Lean Startup, helps you acquire the new knowledge fast and in a goal-oriented, focused manner addressing the key risks or assumptions. It avoids the danger of carrying out too much and too little research, reduces time-to-market, and increases your chances of creating a successful product. 2 Focus on what Matters Most The term product strategy means different things to different people, and strategies come in different shapes and sizes. While that’s perfectly fine, an initial product strategy that forms the basis for subsequent correction and refinement cycles should focus on what matters most: the market, the value proposition, the product’s unique selling points, and the business goals. This is where my Vision Board comes in. I have designed it as the simplest thing that could possibly work to capture the vision and the product strategy. You can download it from romanpichler.com/tools/vision-board for free. For an introduction to the Vision Board, please see my post “The Product Vision Board”. 3 Create the Product Strategy Collaboratively A great way to create your product strategy is to employ a collaborative workshop. Invite the key people required to develop, market, sell and service your product and the senior management sponsor. Such a workshop generates early buy-in, creates shared ownership, and leverages the collective knowledge and creativity of the group. Selling an existing vision and product strategy can be challenging. Co-creation is often the better option. Your initial Vision Board has to be good enough to create a shared understanding of your vision and initial strategy and to identify the biggest risk so you can start re-working your board. But don’t spend too much time on it and don’t try to make it perfect. Your board will change as you correct, improve and refine it. 4 Let your Vision Guide you The product vision is the very reason for creating your product: It describes your overarching goal. The vision also forms the basis of your product strategy as the path to reach your overall goal. As the vision is so important, you should capture it before you describe your strategy. Here are four tips to help you capture your vision: Make sure that your vision does not restate your product idea but goes beyond it. For instance, the idea for this post is to write about creating an agile product strategy, but my vision is to help you develop awesome and successful products. Choose a broad vision, a vision that engages people and that enables you to pivot – to change the strategy while staying true to your vision. Make your vision statement concise; capture it in one or two sentences; and ensure that it is clear and easy to understand. Try to come up with a motivating and inspiring vision that helps unite everyone working on the product. Choosing an altruistic vision, a vision that focuses on the benefits created for others, can help you with this. 5 Put the Users First Once you have captured your vision, work on your strategy by filling in the lower sections of the Vision Board from left to right. Start with the “Target Group”, the people who should use and buy your product rather than thinking about the cool, amazing product features or the smart business model that will monetise the product. While both aspects are important, capturing the users and customers and their needs forms the basis for making the right product and business model decisions. While it’s tempting to think of all the people who could possibly benefit from your product, it is more helpful to choose a clear-cut and narrow target group instead. Describe the users and customers as clearly as you can and state the relevant demographic characteristics. If there are several segments that your product could serve then choose the most promising one. Working with a focused target group makes it easier to test your assumptions, to select the right test group and test method, and to analyse the resulting feedback and data. If it turns out that you have picked the wrong group or made the segment is too small then simply pivot to a new or bigger one. A large or heterogeneous target group is usually difficult to test. What’s more, it leads to many diverse needs, which make it difficult to determine a clear and convincing value proposition and therefore to market and sell the product. 6 Clearly State the Main Problem or Benefit Once you have captured your target users and customers, describe their needs. Consider why they would purchase and use your product. What problem will your product solve, what pain or discomfort will it remove, what tangible benefit will it create? If you identify several needs, then determine the main problem or the main benefit, for instance, by putting it at the top of the section. This helps you test your ideas and create a convincing value proposition. I find that if I am not able to clearly describe the main problem or benefit, I don’t really understand why people would want to use and to buy a product. 7 Describe the Essence of your Product Once you have captured the needs, use the “Product” section to describe your actual product idea. State the three to five key features of your product, those features that make the product desirable and that set it apart from its competitors. When capturing the features consider not only product functionality but also nonfunctional qualities such as performance and interoperability, and the visual design. Don’t make the mistake of turning this section into a product backlog. The point is not to describe the product comprehensively or in a great amount of detail but to identify those features that really matter to the target group. 8 State your Business Goals and Key Business Model Elements Use the “Value” section to state your business goals such as creating a new revenue stream, entering a new market, meeting a profitability goal, reducing cost, developing the brand, or selling another product. Make explicit why it is worthwhile for your company to invest in the product. Prioritise the business goals and state them in the order of their importance. This will guide your efforts and help you choose the right business model. Once you have captured the business goals, state the key elements of your business model including the main revenue sources and cost factors. This is particularly important when you work with a new or significantly changed business model. 9 Extend your Board The Vision Board’s simplicity is one of its assets, but it can sometimes become restricting: The Product and the Value sections can get crowded as the board does not separately capture the competitors, the partners, the channels, the revenue sources, the cost factors, and other business model elements. Luckily there is a simple solution: Extend your board and add further sections, for instance, “Competitors”, “Channels”, “Revenue Streams”, and “Cost Factors”, or download an extended version from my website. But before using an extended Vision Board make sure that you understand who your customers and users are and why they would buy and use the product. There is no point in worrying about the marketing and the sales channels or the technologies if you are not confident that you have identified a problem that’s worthwhile addressing. Additionally, a more complex board usually contains more risks and assumptions. This makes it harder to identify the biggest risk and leap-of-faith assumption. 10 Put it to the Test Capturing your vision and initial product strategy on the Vision Board is great. But it’s only the beginning of a journey in search of a valid strategy, as your initial board is likely to be wrong. After all, you have based the board on what you know now rather than extensive market research work. You should therefore review your initial Vision Board carefully, identify its critical risks or leap-of-faith assumptions, and select the most crucial risk or assumption. Determine the right test group, for instance, selected target users, and the right test method such as problem interviews. Carry out the test, analyse the feedback or data collected, and change your Vision Board with the newly gained knowledge as the picture below shows. If you find that the key risks and assumptions hard to identify then your board may be too vague. If that’s the case then narrow down the target group, select the main problem or benefit, reduce the key features to no more than five, identify the main business benefit, and remove everything else. Your board may significantly change as you iterate over your strategy, and you may have to pivot, to choose a different strategy to make your vision come true. If your Vision Board does not change at all then you should stop and reflect: Are you addressing the right risks in the right way and are you analysing the feedback and data effectively?

Recently I’ve been butting heads with some people on the subject of Ownership, Responsibility and Accountability. There seems to be a very unhealthy obsession with these things sometimes, and I think this is indicative of a less-than-ideal culture. I don’t want to say that they’re “anti-agile” because that just sounds a bit weak, and because I also think they’re not just bad for agile, they’re bad for pretty much any system. I’m not sure how familiar most people are with the “RACI matrix” concept, but in my eyes it’s downright evil in the wrong hands, and I’ve been hearing “RACI Matrix” a lot recently (it’s now on my Bullshit Bingo card). I’ll start off by clarifying what I mean. I’ve got nothing against people owning actions or being accountable for certain particular (usually small) things, but I do take offence when pretty much everything has to be given an owner, someone accountable and someone to “take responsibility”. It’s divisive and results in lots of finger pointing, in my experience. I much prefer the concept of shared ownership, and collective accountability. As a software delivery team, we should all feel responsible for the quality of the product, as well as the performance and the feature richness. These things shouldn’t be assigned for ownership to individuals, as it’ll create an attitude of “well it’s not my problem” among the other team members. Here’s an example: I’ve worked in a team where one person was made the “owner” of the build system. They busied themselves making sure all the builds passed and that the system was regularly ticking over. Of course, the builds often failed and nobody cared except this one person, who then had to try to get people to fix their broken builds. It almost seemed as if people didn’t care about the fact that their software wasn’t capable of being compiled, or that the tests were failing, and in truth they didn’t. They cared about writing code and checking it in, because they didn’t “own” the build system. One message that I always try to drive home with software delivery teams is that our objective is to make software that works for our users, not just write code. I know how easy it is for developers to just focus on checking in code, or perhaps just make sure it passes the tests in the CI system, but beyond that, their focus drops off. I know because I was once one of those developers :-) These days I try to encourage everyone to care about things such as: How your code builds How the tests execute How good the tests are How good the code is How easy it is to deploy How easy it is to maintain How easy it is to monitor Because it takes all of these things to produce good software that users can enjoy, which means we get paid. Here’s another example of how “ownership” has hurt a product: A large system I once worked on was deployed into production using a complicated system of bash and perl scripts, which were cobbled together by a sysadmin who did the deployments. He became the de facto “owner” of the deployment system. There were untold issues with the running of the application because of permissions, paths etc and so forth. The deployment process was creaky and relatively untested. Since the “ownership” of this system was assigned to the sysadmin, rather than devolved or collectively shared throughout the delivery team, the “deployability” was seen as a second class citizen within the delivery team, because everybody felt like it was “owned” by one person who just happened to be on the periphery of the team at best. So here’s what I think: The ability to monitor, maintain, deploy, test, build and create software should all be treated as first class citizens and should be the collective responsibility of everyone in the team. They should all own it, and they should all be accountable. I would extend this out further, to include supporting systems such as environments, build systems, testing frameworks and so-on. Sure, each team might have an SME or two who focuses more on one of these things than any other, but that doesn’t make that one person accountable, responsible or the owner any more than any particular developer is the “owner” of any particular class, method or function. If I write some code that depends on a method that someone else has written, and that method is failing, I don’t just down tools, shrug my shoulders and say “well I’m not accountable for that”. That would be hugely unhelpful and I’d make no friends either. In the same way, we shouldn’t treat our supporting functions and systems as someone else’s responsibility. If we need it in order to make our software work for the end user, then it’s our collective responsibility, no matter what “it” is.

one of the most dreadful problem of unit testing is slow testing. if your whole suite of tests runs in 10 minutes, it is normal for developers not to run the whole suite at each build. one of the most common question is how can i deal with slow unit tests? here is my actual scenario: in a project i’m working in, we have some multilingual full text search done in elastic search and we have a battery of unit tests that verify that searches work as expected. since each test deletes all documents, insert a bunch of new documents and finally commits lucene index, execution times is high compared to the rest of tests. each test need almost 2 seconds to run on my workstation, where i have really fast ssd and plenty of ram. this kind of tests cannot be run in memory or with some fancy trick to make then run quickly. actually we have about 30 tests that executes in less than one seconds, and another 13 tests that runs in about 23 seconds, this is clearly unacceptable . after few hours of work, we already reached the point where running the whole suite becomes annoying. the solution this is a real common problem and it is quite simple to fix. first of all visual studio test runner actually tells you execution time for each unit test, so you can immediately spot slow tests. when you identify slow tests you can mark them with a specific category, i use slowtest 1 2 3 4 [testfixture] [category("elasticsearch")] [category("slowtest")] public class essearcherfixture : basetestfixturewithhelpers since i know in advance that this test are slow i immediately mark the entire class with the attribute slowtest. if you have no idea what of your tests are slow, i suggest grouping test by duration in visual studio test runner. figure 1: group tests by duration the result is interesting, because visual studio consider every test that needs more than one second to be slow. i tend to agree with this distinction. figure 2: test are now grouped by duration this permits you to immediately spot slow tests, so you can add the category slowtest to them. if you keep your unit tests organized and with a good usage of categories, you can simply ask vs test runner to exclude slow test with filter – traits:”slowtest” figure 3: thanks to filtering i can now execute continuously only test that are not slow. i suggest you to do a periodic check to verify that every developers is using the slowtest category wisely, just group by duration, filters out the slowtest and you should not have no tests that are marked slow. figure 4: removing the slowtest category and grouping by duration should list no slow test. the nice part is that i’m using nunit, because visual studio test runner supports many unit tests frameworks thanks to the concepts of test adapters. if you keep your tests well organized you will gain maximum benefit from them :).

When I am trying to install third party apk using ADB tool, I have faced "Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]" error. To resolve the issue, I have followed few steps. Open command prompt; Go to your debug.keystore location. For eg: You can find the debug.keystore file in the following location C:\Documents and Settings\User\.android 1.Using Zip align copied apk. zipalign -v 4 D:\Test.apk D:\Testc.apk 2.keytool -genkey -v -keystore debug.keystore -alias sampleName -keyalg RSA -keysize 2048 -validity 20000 Now a prompt will ask for Password First and lastname Name of Organization unit Name of Organization City State Country After entering these fields we get our Certificate 3. jarsigner -verbose -keystore debug.keystore D:\Testc.apk sampleName In some cases we need add -sigalg SHA1withRSA -digestalg SHA1 arguments to work out the step 3 jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore debug.keystore D:\Testc.apk sampleNameNow it will ask for the password and then it will replace the apk with the signed one. To check whether it is working or not, you can check using the following command. jarsigner -verify D:\Testc.apk Then I have installed apk using ADB. Adb install D:\Testc.apk

In an earlier blog entry I had touched on a fictitious rube goldberg flow for capitalizing a string through a complicated series of steps, the premise of the article was to introduce Spring Integration Java DSL as an alternative to defining integration flows through xml configuration files. I learned a few new things after writing that blog entry, thanks to Artem Bilan and wanted to document those learnings here: So, first my original sample, here I have the following flow(the one's in bold): Take in a message of this type - "hello from spring integ" Split it up into individual words(hello, from, spring, integ) Send each word to a ActiveMQ queue Pick up the word fragments from the queue and capitalize each word Place the response back into a response queue Pick up the message, re-sequence based on the original sequence of the words Aggregate back into a sentence("HELLO FROM SPRING INTEG") and Return the sentence back to the calling application. EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); jmsOutboundGateway.setReplyChannel(sequenceChannel()); return jmsOutboundGateway; } It turns out, based on Artem Bilan's feedback, that a few things can be optimized here. First notice how I have explicitly defined two direct channels, "requestChannel" for starting the flow that takes in the string message and the "sequenceChannel" to handle the message once it returns back from the jms message queue, these can actually be totally removed and the flow made a little more concise this way: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); return jmsOutboundGateway; } "requestChannel" is now being implicitly created just by declaring a name for it. The sequence channel is more interesting, quoting Artem Bilan - do not specify outputChannel for AbstractReplyProducingMessageHandler and rely on DSL , what it means is that here jmsOutboundGateway is a AbstractReplyProducingMessageHandler and its reply channel is implicitly derived by the DSL. Further, two methods which were earlier handling the flows for sending out the message to the queue and then continuing once the message is back, is collapsed into one. And IMHO it does read a little better because of this change. The second good change and the topic of this article is the introduction of the Jms namespace factories, when I had written the previous blog article, DSL had support for defining the AMQ inbound/outbound adapter/gateway, now there is support for Jms based inbound/adapter adapter/gateways also, this simplifies the flow even further, the flow now looks like this: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(Jms.outboundGateway(connectionFactory) .requestDestination("amq.outbound")) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } The inbound Jms part of the flow also simplifies to the following: @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(Jms.inboundGateway(connectionFactory) .destination("amq.outbound")) .transform((String s) -> s.toUpperCase()) .get(); } Thus, to conclude, Spring Integration Java DSL is an exciting new way to concisely configure Spring Integration flows. It is already very impressive in how it simplifies the readability of flows, the introduction of the Jms namespace factories takes it even further for JMS based flows.

Closely approaching the one year mark of when I first joined MongoLab (and the MongoDB community), I had the pleasure of attending the inaugural MongoDB World conference put together by the incredible MongoDB team. Second only to the excitement around major MongoDB feature announcements was the collective disbelief that this was MongoDB’s first multi-day conference ever. A big congratulations to all those that worked hard to put on such a massive (did you see the Intrepid!?) event. All this planning would have been for naught if MongoDB leaders and engineers failed to deliver announcements and features that would meet and exceed expectations. From major public cloud announcements to the reveal of document-level locking in version 2.8, developers and conference goers had plenty to be excited about. There was a lot to digest from the conference… we’ll cover the major highlights in case you missed them. Big announcements in public cloud Our time at the MongoLab booth yielded many high-quality conversations, predominantly those about offloading previously internal processes and workloads to the public cloud. It was remarkable to see and hear so many enterprise teams with the exact same message: the public cloud is the future, and the future is now. It’s no surprise then that MongoDB, Inc. released not one, but two press releases around MongoDB solutions for the public cloud. Fully-managed MongoDB on the Microsoft Azure Store Nearly one year ago, MongoDB, Inc. chose to partner with the MongoLab team to build a production-ready MongoDB solution for developers on Microsoft Azure. On the first day of World, MongoDB, Inc. announced the product of our collaboration – a fully-managed highly available MongoDB-as-a-Service Add-On offering on the Microsoft Azure Store. This new service runs MongoDB Enterprise and offers replication, monitoring and support from MongoDB, Inc. It’s also backed by MongoDB Management Service (MMS), allowing for point-in-time recovery of MongoDB deployments. Now, teams without the expertise or resources to manage their MongoDB deployment(s) can outsource all the database operations (monitoring and alerting, backups, performance tuning, etc.) to both MongoLab and MongoDB’s expert support teams. You can check out the MongoDB add-on in the Azure Store: https://azure.microsoft.com/en-us/gallery/store/mongodb/mongodb-inc/ MongoDB solutions on Google Cloud Platform MongoDB, Inc. also announced the arrival of new resources to help Google Cloud Platform customers deploy MongoDB on Google Compute Engine. These resources include a “Click to Deploy” feature and a MongoDB on Google Compute Engine Solutions paper covering MongoDB best practices. If you are looking for a fully-managed solution, with automated provisioning, backups, integrated monitoring and alerting, along with expert support, MongoLab recently announced the arrival of production-ready replica sets on Google. Product Roadmap – MongoDB version 2.8 On the second day of MongoDB World, Eliot Horowitz, MongoDB, Inc. CTO & Co-founder, took center stage and announced two huge changes to the MongoDB core project: document-level locking and pluggable storage engines. These features not only reflect improvements to the core project, but also signal to the community that the MongoDB team is listening to its users and is capable of delivering the software needed to power the workloads of tomorrow. Document-level locking The slides above from Eliot’s keynote point to a current obstacle (database-level locking) in MongoDB that limits overall scalability. With database-level locking, any write operation to the database holds the write lock and prevents subsequent writes from executing on the database until the original operation holding the write lock completes. Eliot’s announcement of document-level locking moves the write lock contention from the database level to the document (MongoDB equivalent to SQL “records”) level. This change will allow users to achieve much higher write throughput (we saw a 10x performance improvement in the live demo) across their MongoDB deployments, improving write scalability. If you’d like to try out document-level locking, the MongoDB team has already pushed the feature to the master branch on GitHub. This should only be used for experimentation, not to be run in production. Pluggable storage engine As MongoDB matures, feature releases like document level locking will continue to allow developers to build robust systems on top of MongoDB. But as the number of use cases grows, different tooling tailored to specific use cases may prove to be extremely beneficial. For example, if Company X decides that they want to use MongoDB to warehouse some of their data, they would likely want to optimize their database for slow-moving data and storage efficiency (compression). With the introduction of pluggable storage engines, many new possibilities are open to the community. Teams can now write their own storage engine for a particular use case, configure replica set nodes with different storage engines for specific situations, or collaborate with the open-source community to architect innovative solutions. This feature not only allows for more granular control of the database, but also encourages the MongoDB community to work together. Takeaways: A maturing and thriving ecosystem Roughly a year ago, MongoLab CTO Todd Dampier recapped MongoSF 2013 and spoke to the health of the MongoDB ecosystem. How far we’ve come! After attending the inaugural MongoDB World and chatting with MongoDB Masters, interns, hackathon winners, power users and those new to the community, the enthusiasm is still surging and as positive as ever. This enthusiasm is well placed. Developers and hackers use MongoDB because so much rich data on the web is shared as JSON (think Facebook, Twitter, Google, etc.). As a result, MongoDB is the de-facto database for hackathons and bootstrapped projects. Just learn the API for the site you want to mine, throw the JSON in MongoDB and query your data with the rich query language- it’s that easy. The MongoDB ecosystem is maturing as well. Take a look at the Customer Success Stories and you’ll get a feel for the extent in which enterprises leverage the solution and use it in production. To further drive enterprise adoption, MongoDB, Inc.’s public cloud solutions and product roadmap features aim to help teams run MongoDB in production and give teams the confidence that MongoDB will continue to improve scalability and meet their growing project requirements. Congratulations again to the MongoDB team on their big announcements and for creating such a fantastic forum at which to learn and meet fellow MongoDB users. Our team at MongoLab had a great time making new friends and talking shop; we look forward to meeting more MongoDB users soon (at a MongoDB Days near you)! -Chris@MongoLab

NOTE: Always Agile Consulting now offers an Introduction To Trunk Based Development training course! Trunk Based Development supports Optimistic and Pessimistic Release Branching Trunk Based Development is a style of software development in which all developers commit their changes to a single shared trunk in source control, and every commit yields a production-ready build. It is a prerequisite for Continuous Delivery as it ensures that all code is continuously integrated into a single workstream, that developers always work against the latest code, and that merge/integration pain is minimised. It is important to note that Trunk Based Development does not prohibit branching. Trunk Based Development is compatible with a Release Branching strategy of short-lived release branches that are used for post-development defect fixes. That Release Branching strategy might be optimistic and defer branch creation until a defect occurs, or be pessimistic and immediately incur branch creation. For example, consider an application developed using Trunk Based Development. The most recent commits to trunk were source revisions a and b which yielded application versions 610 and 611 respectively, and version 610 is intended to be the next production release. With Optimistic Branching, the release of version 610 is immediate as there is no upfront branching. If a defect is subsequently found then a decision must be made where to commit the fix, as trunk has progressed since 610 from a to b. If the risk of pulling forward from a to b is acceptable then the simple solution is to commit the fix to trunk as c, and consequently release version 612. However, if the risk of pulling forward from a to b is unacceptable then a 610.x release branch is created from a, with the fix committed to the branch as c and released as version 610.1. That fix is then merged back into trunk as d to produce the next release candidate 612, and the 610.x branch is earmarked for termination. With Pessimistic Branching, the release of version 610 is accompanied by the upfront creation of a 610.x release branch in anticipation of defect(s). If a defect is found in version 610 then as with Optimistic Branching a decision must be made as to where the defect fix should be committed. If the risk of pulling forward from a to b is deemed insignificant then trunk can be pulled forward from a to b and the fix committed to trunk as c for release as version 612. The 610.x branch is therefore terminated without ever being used. If on the other hand the risk is deemed significant then the fix is committed to the 610.x branch as c and released as version 610.1. The fix is merged back into trunk as d and version 612, which will also receive its own branch upon release. The choice between Optimistic Branching and Pessimistic Branching for Trunk Based Development is dependent upon product quality and lead times. If product quality is poor and lead times are long, then the upfront cost of Pessimistic Branching may be justifiable. Alternatively, if post-development defects are rare and production releases are frequent then Optimistic Branching may be preferable.

sqs, simple message queue , is a message-queue-as-a-service offering from amazon web services. it supports only a handful of messaging operations, far from the complexity of e.g. amqp , but thanks to the easy to understand interfaces, and the as-a-service nature, it is very useful in a number of situations. but how fast is sqs? how does it scale? is it useful only for low-volume messaging, or can it be used for high-load applications as well? if you know how sqs works, and want to skip the details on the testing methodology, you can jump straight to the test results . sqs semantics sqs exposes an http-based interface. to access it, you need aws credentials to sign the requests. but that’s usually done by a client library (there are libraries for most popular languages; we’ll use the official java sdk ). the basic message-related operations are: send a message, up to 256 kb in size, encoded as a string. messages can be sent in bulks of up to 10 (but the total size is capped at 256 kb). receive a message. up to 10 messages can be received in bulk, if available in the queue long-polling of messages. the request will wait up to 20 seconds for messages, if none are available initially delete a message there are also some other operations, concerning security, delaying message delivery, and changing a messages’ visibility timeout, but we won’t use them in the tests. sqs offers at-least-once delivery guarantee. if a message is received, then it is blocked for a period called “visibility timeout”. unless the message is deleted within that period, it will become available for delivery again. hence if a node processing a message crashes, it will be delivered again. however, we also run into the risk of processing a message twice (if e.g. the network connection when deleting the message dies, or if an sqs server dies), which we have to manage on the application side. sqs is a replicated message queue, so you can be sure that once a message is sent, it is safe and will be delivered; quoting from the website: amazon sqs runs within amazon’s high-availability data centers, so queues will be available whenever applications need them. to prevent messages from being lost or becoming unavailable, all messages are stored redundantly across multiple servers and data centers. testing methodology to test how fast sqs is and how it scales, we will be running various numbers of nodes, each running various number of threads either sending or receiving simple, 100-byte messages. each sending node is parametrised with the number of messages to send, and it tries to do so as fast as possible. messages are sent in bulk, with bulk sizes chosen randomly between 1 and 10. message sends are synchronous, that is we want to be sure that the request completed successfully before sending the next bulk. at the end the node reports the average number of messages per second that were sent. the receiving node receives messages in maximum bulks of 10. the amazonsqsbufferedasyncclient is used, which pre-fetches messages to speed up delivery. after receiving, each message is asynchronously deleted. the node assumes that testing is complete once it didn’t receive any messages within a minute, and reports the average number of messages per second that it received. each test sends from 10 000 to 50 000 messages per thread. so the tests are relatively short, 2-5 minutes. there are also longer tests, which last about 15 minutes. the full (but still short) code is here: sender , receiver , sqsmq . one set of nodes runs the mqsender code, the other runs the mqreceiver code. the sending and receiving nodes are m3.large ec2 servers in the eu-west region, hence with the following parameters: 2 cores intel xeon e5-2670 v2 7.5 gb rams the queue is of course also created in the eu-west region. minimal setup the minimal setup consists of 1 sending node and 1 receiving node, both running a single thread. the results are, in messages/second: average min max sender 429 365 466 receiver 427 363 463 scaling threads how do these results scale when we add more threads (still using one sender and one receiver node)? the tests were run with 1, 5, 25, 50 and 75 threads. the numbers are an average msg/second throughput. number of threads: 1 5 25 50 75 sender per thread 429,33 407,35 354,15 289,88 193,71 sender total 429,33 2 036,76 8 853,75 14 493,83 14 528,25 receiver per thread 427,86 381,55 166,38 83,92 47,46 receiver total 427,86 1 907,76 4 159,50 4 196,17 3 559,50 as you can see, on the sender side, we get near-to-linear scalability as the number of thread increases, peaking at 14k msgs/second sent (on a single node!) with 50 threads. going any further doesn’t seem to make a difference. the receiving side is slower, and that is kind of expected, as receiving a single message is in fact two operations: receive + delete, while sending is a single operation. the scalability is worse, but still we can get as much as 4k msgs/second received. scaling nodes another (more promising) method of scaling is adding nodes, which is quite easy as we are “in the cloud”. the test results when running multiple nodes, each running a single thread are: number of nodes: 1 2 4 8 sender per node 429,33 370,36 350,30 337,84 sender total 429,33 740,71 1 401,19 2 702,75 receiver per node 427,86 360,60 329,54 306,40 receiver total 427,86 721,19 1 318,15 2 451,23 in this case, both on the sending&receiving side, we get near-linear scalability, reaching 2.5k messages sent&received per second with 8 nodes. scaling nodes and threads the natural next step is, of course, to scale up both the nodes, and the threads! here are the results, when using 25 threads on each node: number of nodes: 1 2 4 8 sender per node&thread 354,15 338,52 305,03 317,33 sender total 8 853,75 16 925,83 30 503,33 63 466,00 receiver per node&thread 166,38 159,13 170,09 174,26 receiver total 4 159,50 7 956,33 17 008,67 34 851,33 again, we get great scalability results, with the number of receive operations about half the number of send operations per second. 34k msgs/second processed is a very nice number! to the extreme the highest results i managed to get are: 108k msgs/second sent when using 50 threads and 8 nodes 35k msgs/second received when using 25 threads and 8 nodes i also tried running longer “stress” tests with 200k messages/thread, 8 nodes and 25 threads, and the results were the same as with the shorter tests. running the tests – technically to run the tests, i built docker images containing the sender / receiver binaries, pushed to docker’s hub, and downloaded on the nodes by chef. to provision the servers, i used amazon opsworks. this enabled me to quickly spin up and provision a lot of nodes for testing (up to 16 in the above tests). for details on how this works, see my “cluster-wide java/scala application deployments with docker, chef and amazon opsworks” blog . the sender / receiver daemons monitored (by checking each second the last-modification date) a file on s3. if a modification was detected, the file was downloaded – it contained the test parameters – and the test started. summing up sqs has good performance and really great scalability characteristics. i wasn’t able to reach the peak of its possibilities – which would probably require more than 16 nodes in total. but once your requirements get above 35k messages per second, chances are you need custom solutions anyway; not to mention that while sqs is cheap, it may become expensive with such loads. from the results above, i think it is clear that sqs can be safely used for high-volume messaging applications, and scaled on-demand. together with its reliability guarantees, it is a great fit both for small and large applications, which do any kind of asynchronous processing; especially if your service already resides in the amazon cloud. as benchmarking isn’t easy, any remarks on the testing methodology, ideas how to improve the testing code are welcome!

The Android system software stack is typically divided into the four areas as the following graphic: Terminology Android Software Development Kit (Android SDK) contains the necessary tools to create, compile and package Android applications Android debug bridge (adb), which is a tool that allows you to connect to a virtual or real Android device Google provides two integrated development environments (IDEs) to develop new applications. Android Developer Tools (ADT) are based on the Eclipse IDE Android Studio based on the IntelliJ IDE Android RunTime (ART) uses Ahead Of Time compilation, and optional runtime for Android 4.4 Android Virtual Device (AVD) - The Android SDK contains an Android device emulator. This emulator can be used to run an Android Virtual Device (AVD), which emulates a real Android phone Dalvik Virtual Machine (Dalvik)- The Android system uses a special virtual machine, Dalvik, to run Java-based applications. Dalvik uses a custom bytecode format which is different from Java bytecode. Therefore you cannot run Java class files on Android directly; they need to be converted into the Dalvik bytecode format.

This post exists to help with an MSDN Magazine article that I am authoring It provides some of the low-level details for the article How to install Mono and root certificates on a raspberry pi How to create an Azure mobile service How to create a Custom API inside Azure mobile services that the raspberry pi can call into How to create an Azure storage account MONO - HOW TO INSTALL ON A RASPBERRY PI Why Mono? How to install Mono on a raspberry pi Installing trusted root certificates on to the raspberry pi http://www.mono-project.com/Main_Page An open source, cross-platform, implementation of C# and the CLR that is binary compatible with Microsoft.NET Mono is a free and open source project led by Xamarin (formerly by Novell) that provides a .NET Framework-compatible set of tools including, among others, a C# compiler and a Common Language Runtime WHY MONO? Because it lets us write .net code compiled on Windows We can simply copy the binary files from Windows to Linux and run it as is From a raspberry pi device, it is possible to use a .net application to take a photo and upload it to Windows Azure storage HOW TO INSTALL ON A RASPBERRY PI RUNNING LINUX You will issue the following commands: pi@raspberrypi ~ $ sudo apt-get update pi@raspberrypi ~ $ sudo apt-get install mono-complete The first command makes sure all the local package index are up to date with the changes made in repositories. Second command installs the complete Mono tooling and runtime. MAKING SURE THAT YOUR MONO APPLICATIONS CAN MAKE A HTTPS REST-BASED CALLS This command downloads the trusted root certificates from the Mozilla LXR web site into the Mono certificate store. Once complete, the Raspberry PI will be capable of making web requests using HTTPS requests within Mono. pi@raspberrypi ~ $ mozroots --import --ask-remove --machine CREATING A NEW AZURE MOBILE SERVICES ACCOUNT The mobile services account is needed to host a Node.js application that provides shared access signatures to raspberry pi devices The shared access signature is needed by the raspberry pi, so that it can directly and securely upload photos to Azure storage STEPS TO CREATE AN AZURE MOBILE SERVICE The steps below will create an Azure mobile service The service will be used to host a Node.js application interacting with a raspberry pi devices We will provision a SQL database, although it will not be used initially FOLLOW THESE STEPS TO CREATE THE MOBILE SERVICE Login into the Azure Portal Select MOBILE SERVICES from the left menu pane at the Azure Portal. In the lower left corner select "+NEW" to create a new Azure Mobile Service. Make sure you've selected, "COMPUTE / MOBILE SERVICE / CREATE." You will now enter a url. We will call this service raspberrymobileservice. For the DATABASE, we will choose "Create a new SQL database instance." The REGION we chose is "West US." The BACKEND is "JavaScript." Click the "->" arrow to proceed to the next screen. In this screen you will "Specify database settings." The NAME of your database will based on the URL you entered previously. In this case, the database is called "raspberrymobileservice_db." You will need to choose a SERVER. We will choose "New SQL database server" from the drop-down list. You will need to provide a SERVER LOGIN NAME and a SERVER LOGIN PASSWORD. Take note of the login you provided as it will be needed later CREATING A CUSTOM API Azure mobile services allows you to create a custom API written in JavaScript that can be called from a raspberry pi device using REST This custom API is really just a Node.js application running in the server CREATING THE API TO RESPOND TO THE DEVICE TRYING TO UPLOAD PHOTOS Now that the service is established, we will turn our attention to creating an API that the device can call into to upload a photo. Login into the Azure Portal Your mobile service will take a few minutes to complete, and you should see the "Ready" flag as the "Status" for your service. Once it is ready you can drill into your service to customize its behavior. Just to the right of the service name, click the right arrow key "->" to drill into the service details. The top menu bar will offer many options, but we are interested in the one titled "API." The API allows you to create a series of node.JS API calls that a device can call into using rest-based approaches. Click on "API." from there, select "CREATE A CUSTOM API." You will be asked to provide an API name. Type in "photos" for the API name. Below you will see a series of drop-down combo boxes that relate to permission. We will keep the default value of "Anybody with the application key." This might not be the best option for all scenarios. You can read more about this here. http://msdn.microsoft.com/en-us/library/azure/jj193161.aspx. Click the checkmark to complete the process. The name of the AP you just created, "Photos," should be visible on the portal interface. To drill into the photos API click on the right arrow key "->". The right arrow key will be just to the right of the name of the API "Photos". At this point you should see a basic script that has been provided by default. We will overwrite this default script with our own script as described in the MSDN Magazine article. CREATING A STORAGE ACCOUNT TO STORE THE PHOTOS Navigate to the portal and create a storage account Create a container for the photos Obtain the: Storage Account Name (you will provide a name) Storage Account Access key (generated for you) Container Name (you will create) CREATING A STORAGE ACCOUNT We will need a storage account so that we can upload photos to it. The steps are well documented here: http://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account/ In our case we call the storage account raspberrystorage. This means that the URL that the device will use to upload photos is https://raspberrystorage.blob.core.windows.net/. As you complete these steps make sure that you choose the storage account location to be the same location as was used for your mobile services account. This avoids any unnecessary latency or bandwidth costs between data centers. Once the storage account is created, we will need to create a container within it. Photos or any blob for that matter, are always stored within a container. To create a container drill into your newly created storage account and select CONTAINERS from the top menu. From there, select CREATE A CONTAINER. The new container dialog box will ask for a name for your container. Take note of the name you provide. We are calling our container ?photocontainer.? When the raspberry pi device uploads photos to the storage account, it will target a specific container, such as the one we just created. You will next be asked to indicate ACCESS rights. To keep things simple we will select access rights of Public Blob. ENTERING APP SETTINGS Rather than hard-code storage account information inside your JavaScript/Node.js applications, you should consider using apps settings inside of the Azure mobile services portal This post also discusses it well: http://blogs.msdn.com/b/carlosfigueira/archive/2013/12/09/application-settings-in-azure-mobile-services.aspx ?The idea of application settings is a set of key-value pairs which can be set for the mobile service (either via the portal or via the command-line interface), and those values could be then read in the service runtime.? NAVIGATING TO APP SETTINGS Navigate to the Azure Mobile Services section of the portal. Drill into the specific service by hitting the arrow below Select from the Configure Menu at the top Scroll down to the very bottom to see app settings Note that we need to enter: - We need to get this from Azure Storage - PhotoContainerName - AccountName - AccountKey We get this information from the Azure Storage Section of the Portal. Note that you need to have provisioned a Storage Account to have this information. How to get the AccountKey with Azure Storage Services Now you can get the access keys HOW NODE.JS WILL ACCESS THE APP SETTINGS You will create a Node.js application inside of Azure Mobile Services See previous steps THE NODE.JS APPLICATION READING APP SETTINGS You will starting by going back to Azure Mobile Services and drill down into your newly minted service We called ours raspberrymobileservice Once you click API, you should see: Notice the app settings are being read on lines 12 to 14.

One of the great improvement in Java EE 5 and beyond it is the introduction of CDI (Context and Dependency Injection). CDI is used for injecting dependencies among a lot of other things like events, interceptors, … and can be used in POJOs. In some cases instead of injecting other objects (as a dependency injection), you want to inject a value from a properties file into a class that needs to be configured externally. I have written an example in Tomitribe community zone: https://github.com/tomitribe/community/tree/master/injecting-properties We keep learning, Alex.

A unit test is a set of methods launched frequently to validate your code. It is usually a good idea to write code in this order: Write a class API Write a method to test the API Implement the API Launch the unit tests Why write unit tests? They validate current and future implementations. They measure code quality. They force you to write testable, loosely coupled code. They’re cheaper than manual regression testing. They build confidence in your code. They help teamwork. Why use a checklist? Unit testing can be harder than actual implementation. Unit testing forces you to really think things through. But unit tests should be simple, direct, and easy to read and maintain. You also need to know when to stop writing tests and start writing the implementation. Use this checklist to be sure your tests are really useful and to the point. Remember: the checklist helps you avoid big mistakes, but you need to make sure of the following: □ My test class is testing only one class. o You are testing a class API to be sure the public contract is respected. □ My methods are testing only one method at a time. o Be sure not to test private methods! They are hidden implementation details, not API. □ My variables and method names are explicit. o For example, store an expected value in an expectedFoo variable instead of just foo. If you test many combinations, use composed variable names like inputValue_NotNull, inputValue_ZeroData, inputValue_PastDate, etc. (according to your application’s coding convention). □ My test cases are easy to read by humans. o Future maintainers should be able to read your tests before reading the implementation. This will help them understand a class API before tweaking or debugging it. □ My tests respect the usual clean code standards. o There should be no flow control in a test method (switch, if, etc.). A good test is just a very straightforward sequence of setup/validate instructions. If necessary, use sub-methods to factorize and make your tests easier to read. In case of multiple scenarios, use multiple test methods (one for each case). o For example, a test method should fit on screen without scrolling – 1 to 20 lines long. If the method is longer, consider writing multiple test methods for each case instead of jamming them together. □ My tests are also testing expected exceptions. o In java, use @Test(expected=MyException.class). □ My tests don’t need access to a database. o Or if a test does need database access, then it must be a mocked, “fire and forget” temporary database that you fill with test cases for every new test method (use the Setup/Teardown methods to prepare it). □ My tests don’t need access to network resources. o You can’t rely on third parties like network or device presence to validate a method (use mocks). □ My tests control side effects, limit values (max, min) and null variables (even when they throw exceptions). o You want to make sure these problem cases never occur, even when the test won’t be used during maintenance. □ My tests can be run at any time, at any place without needing configuration or human intervention. □ My tests pass for the current implementation and are easy to evolve. o Tests really exist to support code evolution. If they are too hard to maintain or too light to refine the code, then they are a useless burden. (Many developers avoid unit testing for this reason.) □ My tests are concrete. o In Java: don’t use Date() as input for a method you are testing, but build a concrete date out of Calendar (don’t forget to force the timezone). Other example: use name = “Smith”; instead of name = “name”; or name = “test”; □ My tests use a mock to simulate/stub complex class structure or methods. o Remember to test only one class API at a time. o Never test third-party libraries through your own classes. Libraries should come with their own tests (this is actually a good way to choose a library). □ My tests are never @ignored or commented out. Never. Ever. □ My tests help me validate my architecture. o If you can’t test a method or a class, your design is not agile enough. □ My tests can run on any supported platform, not just the targeted platform. o Don’t expect a particular device or hardware configuration. Otherwise, your tests will make migration tougher and you will be incentivized to disable them. □ My tests are lightning fast! o Slow tests shouldn’t drag you down. Speed encourages you to launch your tests often. It also helps to reduce building time on Continuous Integration systems. o Use a test runner that allows you to launch one test at a time while you are writing it. Use “delay” or “sleep” with caution – i.e., only in some edge cases, like waiting for notifications or clock-based methods.