[This article was written by John Wetherill.] This is a continuation of the Microservices and PaaS - Part I blog post I wrote last week, which was an attempt to distil the wealth of information presented at the microservices meetup hosted by Cisco, with Adrian Cockcroft and others presenting. Part I provided a brief background on microservices, with a summary of some lessons learned by microservices pioneers. In this installment I will cover a number of practices related to microservices that were discussed during the meetup. A followup article will dive into the advantages that Platform as a Service brings to microservice development. Microservices Practices I'm calling these "Microservice Practices," not "Microservices Best Practices" because microservices-based architectures are still evolving, with new practices, techniques, tools, and patterns emerging constantly. At the meetup a number of practices were highlighted that Netflix and other microservices pioneers have spearheaded in their efforts to adopt a microservices mentality across their organizations. Break Things Deliberately According to Netflix: "We have found that the best defense against major unexpected failures is to fail often." Netflix has brought us "Chaos Monkey" which is a powerful tool the sole purpose of which is to break things, often and randomly. They use this tool continuously on their production systems to bring down essential services, to ensure that doing so doesn't disrupt the user experience or their overall service. It's much better to deliberately break the system in the middle of the morning when all teams are assembled and sufficient caffeine has been consumed, than to be informed of a breakage by a page at 3am. No Manual "Anything" In a world where microservices come and go, grow and shrink, and migrate around racks and data centers in seconds - there's absolutely no room for manual intervention. All aspects of deployment, monitoring, testing, and recovery must be fully automated. For example, monitoring a service should occur instantly and automatically by virtue of it being deployed, not requiring a separate manual step. Similarly failure discovery and rerouting to old code, as described in Part I of this blog, must be fully automated, no human intervention required. Respect Human Attention Span Speaking of humans, a typical human's attention span, say when filling out a shopping cart, is around 10 seconds. If a failure occurs when deploying an updated shopping cart microservice, it's important that the time between the failure, reporting, and rerouting to existing, working code is kept under around this 10 second range. Obviously this shouldn't happen too often, but the occasional 10 second gap in response will probably not lose the customer. A five minute, or 5 hour lag, resulting from manual intervention and rollback, will. Denormalize like Crazy Refactor database schemas, and de-normalize everything, to allow complete separation and partitioning of data. That is, do not use underlying tables that serve multiple microservices. There should be no sharing of underlying tables that span multiple microservices, and no sharing of data. Instead, if several services need access to the same data, it should be shared via a service API (such as a published REST or a message service interface). Polyglot Persistence Each microservice can have its own persistence layer. Gone are the days of a single monolithic database instance that's shared across all parts of an application. Databases are getting cheaper and easier. As an example, Neo4J allows you to embed an industry-strength self-contained graph database in your microservice at the cost of a few megabytes in a jarfile, with startup time on the order of milliseconds. That's essentially free. Even better, any PaaS worth its salt will provide multiple database services that can be spawned and accessed at the drop of a hat. With technology like this at our disposal, it makes sense to use the persistence layer that fits, both to the problem being solved, and to the expertise - and passions - of the team that's solving the problem. Avoid Trunk Conflicts The old mindset had all code for a large project contained in a single source repository. This can be slightly easier to setup and manage, but it ties the microservices together and makes it much more difficult to evolve them independently. Instead each microservice should have its own scm repository so it can truly be updated and enhanced independent of other services. One Service, One Manifest Each microservice must have its own manifest and dependencies, instead of maintaining a global dependency list for all services. This allows, for example, one microservice to depend on Spring v3.2, while another can require Spring 4.1. The dependencies for one microservice can change over time with no effect on the dependencies of other microservices. Contain Everything All microservices should run in a container, such as Tomcat, Docker, or in whatever container system is provided by the PaaS (you are running a PaaS aren't you?). Do not run microservices on bare metal, or directly on a VM. Containerization brings countless advantages, particularly a consistent, isolated runtime environment that can easily migrate around the datacenter or around the globe. With Docker and other modern containerization approaches, there is very little overhead in running in a container, and considerable upside. No State Do not build stateful services. Instead, maintain state in a dedicated persistence service, or elsewhere. This is a well-known practice brought to us by the cloud. When an application instance maintains state, it can't easily be moved, scaling is more complex, and it's more likely to cause problems when it fails. This practice applies even more to microservices which in general should be light-weight, instantly replaceable on failure, and should be able to hop around data-centers. Don't Name your Chickens People who raise chickens soon learn that naming chickens is a bad idea: after naming a chicken you get attached to it, at least the kids do, and it can be uncomfortable to have to explain at the dinner table that the chicken pot pie is really "Molly." Instead, number your chickens, so you can say "that was chicken #38" or even better, "that was chicken 586ec9bd." Makes for a much more enjoyable meal. The same can be said of computer systems. Do not name systems after planets, or animals, or philosophers, or prisons, as was common practice in the UNIX world for decades. Instead, assign them guid's, and don't attach any sort of significance to them, like assigning them specific roles or purposes. Systems should be commodities, like McDonalds Franchises. Each McDonalds is eerily similar, with the advantage that if one shuts down you can just walk an extra few blocks and be served the exact same burger at the same price in the same amount of time. Create and Curate Access Libraries Microservices are accessed by externally published APIs or protocols. This allows the microservice implementation to completely change with no effect on its consumers, as long as the API remains constant. But just publishing an API is not enough. The microservice provider should also be responsible for building and stewarding client libraries used to access the service. If this is not done, the construction of these libraries will be left to third parties, and will likely result in fragmentation where various implementations might have slight differences, or implementors may incorrectly interpret the spec and introduce inconsistencies which then stick. Optimize the Interaction One downside of a microservices architecture is the "fanout" problem where a single request to the overall application results in 10 or 20 requests bubbling throughout the various microservices the application relies on. This dramatic increase in network traffic calls for more optimal communication between microservices. Instead of transmitting the standard text/html REST content type, consider using something like Google Protocol Buffers, Simple Binary Encoding, or Apache Thrift, to decrease the size of the payload and optimize the inter-microservice communications. Release the Monkeys Netflix has released what they call the "Simian Army," a suite of tools including Chaos Monkey, mentioned above, whose purpose is to help an organization build resilient, scalable, fault-tolerant software. The suite includes such tools as Janitor Monkey, to reclaim unused resources, Security Monkey which looks for security vulnerabilities, Latency Monkey, which induces artificial delays in the REST layer to scare out latency issues, and many more. As Phil described last week in his blog Devops: Tools vs. Culture, most organizations don't have the resources or luxury of being able to build their own toolsets when evolving to a microservices and devops culture. Instead they must leverage existing tools, and fortunately lots of tools are constantly appearing. It's worth spending the effort searching and researching these tools, and incorporating them into your overall development process when they make sense. To be continued... Again I originally intended to cover last week's microservices meetup in a single blog post, which then expanded to two. I have yet to address the power of PaaS in microservices architectures, and I'm out of space already. So I will continue this Microservices and PaaS theme next week, finally getting into PaaS, and discuss how Platform as a Service can significantly streamline the microservices development process.

[This article was written by Koby Nachmany.] If you are familiar with configuration management (aka CM) and automation, you probably know a thing or two about Puppet, and the amazing and rich collection of modules it offers. Puppet Forge contains a wealth of third party modules that enable us to do some pretty nifty stuff with almost no effort. Puppet helps deal with the messy parts of CM, like installing binaries and running installation scripts that are tedious to do manually. Tools such as Puppet were originally created for IT operations people, that are for the most part infrastructure-centric, and are best suited for setup and maintenance of hosts in a physical data center. Dealing with applications and certainly managing applications on an elastic virtualized or even cloudified environment, brings a set of new challenges despite the agility and other benefits it provides. Now imagine we can have this goodness coupled with an intelligent orchestration framework for an entire deployment? In this blog post I'd like to demonstrate how a cloud application orchestrator can complement already existing automation processes powered by configuration management tools, in this case we will demonstrate with Puppet. I will use the nodecellar application and the popular WordPress content management framework as examples. This will hopefully provide a good introduction to Cloudify blueprints. Overview So we've seen how Cloudify 3 allows us to easily orchestrate the "nodecellar" application Read about it Cloudify blueprints here. With the "nodecellar" example, Cloudify deploys a complex application using workflows that map deployment lifecycle events to bash scripts using Cloudify's bash runner plugin. Cloudify's Puppet integration now makes this pretty easy. Cloudify 3.0 - Taking Puppet to the Next Level of Orchestration. Check it out. Go The synergy between Cloudify and Puppet not only allows you to enjoy the benefits of your Puppet environment, but it also amplifies its usability by introducing unique advantages that will answer the following common challenges involved with configuration management tools: Agent Installation: Provision your service VMs, install a Puppet agent (if you like) and wires them up with the Puppet Master. Or, if you choose to run standalone, you can install the agent with the appropriate manifests needed for that service, as well. Order of Dependencies: Define the dependencies between application stacks, services and infrastructure resources. Which will then be launched based on that order. Remote Execution and Updates: Other than the basic install/uninstall, Cloudify enables customized application workflows that allow you to execute tools like remote shell scripts on a group of instances that belong to a particular service, or to a specific instance in a group. This feature is useful to run maintenance operations, such as snapshots in the case of a database, or code pushes in a continuous deployment model. In addition, you can run puppet apply whenever you feel it's right for your service. Post Deployment: Once your application is up, Cloudify will be able to glue your monitoring tool of choice, or you can choose to use the built-in one. A robust policy engine, enables auto-healing and even auto-scaling according to your service's required SLA. I'm now going to take a deep dive on my experience with a WordPress example that I feel is a very good representation of how Puppet and Cloudify work in sync. Let's say we want to deploy the popular WordPress application stack on two VMs . Something as follows: The flow is quite simple: -server 3.5.1 with the basic following modules installed: |-- hunner-wordpress (v0.6.0) |-- puppetlabs-apache (v1.0.1) - with php mods enabled |-- puppetlabs-mysql (v2.1.0) Your site.pp file should resemble something like this: node /^apache_web.*/ { include apache class { 'wordpress': create_db => false, create_db_user => false, } } node /^mysql.*/ { class { '::mysql::server': root_password => 'password', override_options => { 'mysqld' => { 'bind_address' => '0.0.0.0' } } } include mysql::client include wordpress } As we can see, we have an Apache PHP application that will likely require a database connection string (IP, port, user and password). This is where Cloudify facilitates the "gluing" of all the pieces together, by allowing us to inject dynamic/static custom facts to the dependent node (Apache server). Cloudify supports both standalone agents and PuppetMaster environments. Step 2: Tweaking the Original WordPress Module. Some minor adaptations to the wordpress init class of the WordPress module will allow us to embed these facts during Puppet agent invocation. Below is a code snippet (With defaults truncated): class wordpress ( $db_host_ip = $cloudify_related_host_ip, $db_user, = $cloudify_properties_db_user, $db_password = $cloudify_properties_db_pw, . . ) And some tweaking to the templates/wp-config.php.erb: /** MySQL hostname */ define('DB_HOST', ''); Let's add some tags for finer control of manifest execution: The MySQL node will not require the application part to run on it, so I've excluded it using a Puppet "tag" (read more about Puppet tags). Cloudify, of course, supports this and will provide the appropriate tags during agent invocation. -> class { 'wordpress::app': tag => ['postconfigure'], install_dir => $install_dir, install_url => $install_url, version => $version, db_name => $db_name, . .} Step 3: Creating the Blueprint In a similar way to the "nodecellar" blueprint, first lets create a folder with the name of "wp_puppet" and create a blueprint.yaml file within it. This file will then serve as the blueprint file. Now let's declare the name of this blueprint. blueprint: name: wp_puppet nodes: Now we can start creating the topology. Step 4: Creating VM Nodes Since, in this case I use the OpenStack provider to create the nodes, let's import the "OpenStack types" plugin. imports: - http://www.getcloudify.org/spec/openstack-plugin/1.0/plugin.yaml Since the VMs are the same, I declared a generic template for a VM host: vm_host: derived_from: cloudify.openstack.server properties: - install_agent: true - worker_config: user: ubuntu port: 22 # example for ssh key file (see `key_name` below) # this file matches the agent key configured during the bootstrap key: ~/.ssh/agent.key # Uncomment and update `management_network_name` when working a n neutron enabled openstack - management_network_name: cfy-mng-network - server: image: 8c096c29-a666-4b82-99c4-c77dc70cfb40 flavor: 102 key_name: cfy-agnt-kp security_groups: ['cfy-agent-default', 'wp_security_group'] # This is how we inject the puppet server's ip userdata: | #!/bin/bash -ex grep -q puppet /etc/hosts || echo "x.x.x.x puppet" | sudo -A tee -a /etc/hosts Create the MySQL and Apache VMs: - name: mysql_db_vm type: vm_host instances: deploy: 1 - name: apache_web_vm type: vm_host instances: deploy: 1 Step 5: Declaring Apache and MySQL Servers Since we are using the Puppet plugin to create those servers, first we have to import it: plugins: puppet_plugin: derived_from: cloudify.plugins.agent_plugin properties: url: https://github.com/cloudify-cosmo/cloudify-puppet-plugin/archive/nightly.zip The plugin defines server types as follows: middleware_server, app_server, db_server, web_server, message_bus_server, app_module. They are virtually the same, but serve the purpose of enabling better readability for the user and GUI visualization A Puppet server type is derived_from: cloudify.types.server type, but includes some puppet-specific properties and lifecycle events. For documentation see: Puppet Types So we now will go ahead and declare the server types: cloudify.types.puppet.web_server: derived_from: cloudify.types.web_server properties: # All Puppet related configuration goes inside # the "puppet_config" property. - puppet_config interfaces: cloudify.interfaces.lifecycle: # Specifically "start" operation. Otherwise tags must be # provided. - start: puppet_plugin.operations.operation cloudify.types.puppet.app_module: derived_from: cloudify.types.app_module properties: - puppet_config interfaces: cloudify.interfaces.lifecycle: - configure: puppet_plugin.operations.operation cloudify.types.puppet.db_server: derived_from: cloudify.types.db_server properties: - puppet_config interfaces: cloudify.interfaces.lifecycle: - start: puppet_plugin.operations.operation Step 6: Instantiating the Apache and MySQL nodes: Here we provide the Puppet configuration and tags and define the relationships between the nodes. Cloudify's agent will use those relationships in order to decide the appropriate facts to inject. - name: apache_web_server type: cloudify.types.puppet.web_server properties: port: 8080 puppet_config: server: puppet environment: wordpress_env relationships: - type: cloudify.relationships.contained_in target: apache_web_vm - name: wordpress_app type: cloudify.types.puppet.app_module properties: db_user: wordpress db_pass: passwd puppet_config: server: puppet tags: ['postconfigure'] environment: wordpress_env relationships: - type: cloudify.relationships.contained_in target: apache_web_server - type: wp_connected_to_mysql target: mysql_db_server - name: mysql_db_server type: cloudify.types.puppet.db_server properties: db_user: wordpress db_pass: passwd puppet_config: server: puppet environment: wordpress_env relationships: - type: cloudify.relationships.contained_in target: mysql_db_vm Step 7: Upload the Blueprint and Create the Deployment (via CLI or GUI) Then execute your deployment (via CLI or GUI). ubuntu@koby-n-cfy3-cli:~/cosmo_cli$ cfy blueprints upload -b wp9 wordpress/blueprint.yaml ubuntu@koby-n-cfy3-cli:~/cosmo_cli$ cfy deployments create -b wp9 -d WordPress_Deployment_1 Step 8: Take a Quick Coffee Break. Step 9: Enjoy your Orchestrated WordPress Stack!

1 Start with Personas The first step towards writing the right user stories is to understand your target users and customers. After all, user stories want to tell a story about the users using the product. If you don’t know who the users are and what problem we want to solve then it’s impossible to write the right stories and you end up with a long wish list rather than a description of the relevant product functionality. Personas offer a great way to capture the users and the customers with their needs. They are fictional characters that have a name and picture; relevant characteristics such as a role, activities, behaviours, and attitudes; and a goal, which is the problem that has to be addressed or the benefit that should be provided. Let’s look at an example. Say we want to create a game for children, which is fun to play and which educates the kids about music and dancing. We would then create at least two personas, one to represent the children, and one for the parents, as the following picture illustrates. The two sample personas above use my simple yet effective persona template. It encourages you to keep your personas concise, to focus on what really matters and to leave out the rest. You can download the template from romanpichler.com/tools/persona-template where more information on writing personas and using the template is available. Once you have created a cast of characters, select a primary persona, the persona you are mainly designing and building the product for. This helps you make the right product decision and get the user experience (UX) right. In the example above, I have chosen Yasmin as the primary persona. 2 Derive Epics from the Persona Goals Once you have created your personas, use their goals personas to identify the product functionality. Ask yourself what the product should do to address the personas’ problems or to create the desired benefits for them, as the following picture shows. Start with your primary persona and capture the functionality as epics, as coarse-grained, high-level stories. Write all the epics necessary to meet the persona goals but keep them rough and sketchy at this stage. For the dance game, we could write the epics below assuming that the game will be initially launched as an iPad app: As the epics above show, the game should allow the players to select different characters, to make them dance, to choose different dance floors and music tracks, to play the game with their friends, and to post a snapshot of their game on Facebook. While epics are great to sketch the product’s functionality, there is more to your product than epics and stories: You should also capture the user interaction and the sequences in which the epics are used, the visual design of your product, and the important nonfunctional qualities such as interoperability and performance. Use, for instance, workflow diagrams, story maps, storyboards, sketches, mock-ups, and constraint cards to describe them. You can find out more about describing the different product aspects in my post “User Stories are Not Enough to Create a Great User Experience”. 3 Progressively Decompose the Epics into User Stories With a holistic but coarse-grained description of your product in place start progressively decomposing your epics. Rather than detailing all epics and writing all user stories in one go, you derive your stories step by step as the following picture shows. As long as there are some significant risks present and you are figuring out what the product should look like and do, it’s best to derive just enough user stories just in time for the next sprint. Use your sprint goal or hypothesis to determine which epics to decompose and which stories to write as the following diagram illustrates. The approach depicted above minimises the amount of detailed items in your product backlog. This makes it easier to integrate new insights derived from exposing product increments or minimum viable products (MVPs) to users and customers. Say that we want to address the risk of creating the wrong game characters by developing an executable prototype that allows us to run a usability test with selected children. We could then write the following user stories: The stories above are derived from the epics “Choose character” and “Play with character”. The resulting prototype only partially implements the two epics – just to the extent of being able to test if the characters resonate with the users. Once you understand better how to meet the customer and user needs, you can start pre-writing user stories and have a larger inventory of detailed items on your product backlog as you are unlikely to experience bigger changes to your epics and your overall backlog. 4 Get the Stories Ready Before the development team starts working on the stories, check that each user story is ready: clear, feasible, and testable. A story is clear if there is a shared understanding between the product owner and the team about its meaning. It is feasible if it can be delivered in the next sprint according to the Definition of Done. This implies that the story is small enough to fit into the sprint but also that the necessary user interface design, test, and documentation work can be carried out. In the case of the sample stories above, we would have to add acceptance criteria, ensure that the stories are small enough to fit into the next sprint, and consider creating some very rough design sketches to indicate what the characters look like. For instance, to get the story “Yas chooses the little girl” ready, we could create the following rough sketch: The sketch above complement the user story and allows the team to implement the entire story including the visual design in the next sprint. With ready user stories in place the development team is in a good position to progress your product in an effective manner. For more details on getting user stories ready please take a look at my post “The Definition of Ready in Scrum”. Learn More You can learn more about writing the right epics and user stories by attending my Writing Great User Stories training course. If you want to learn more about the creating the UX artefacts mentioned in this post, then attend my Agile UX and Scrum training course. Please contact me if you are interested in having the courses delivered at your office. The persona pictures and the Manga girl sketch were created by Ole Størksen. Thanks Ole!

This is a post in a series discussing using spring-boot and docker for deployment. Refer to the end of the first post for a table of contents. Shortly after you start building docker containers you will realize that you need some place to publish your images. You could push to the central docker registry. However, the central registry is public. Not a great idea if you are working on a private project. If this is your case, you can simply run a local docker registry. To install and run your private registry run $ docker run -p 5000:5000 -d registry Surprise!!! It is ran in a docker container. You can now start pushing to your local repository. As an example, I will pull the latest postgres image and push version 9.4 to my local registry. $ docker pull postgres $ docker tag postgres:9.4 localhost:5000/postgres:9.4 $ docker push localhost:5000/postgres Outputs: The push refers to a repository [localhost:5000/postgres] (len: 1) Sending image list Pushing repository localhost:5000/postgres (1 tags) 511136ea3c5a: Image successfully pushed ec3443b7b068: Image successfully pushed 06af7ad6cff1: Image successfully pushed 37eae31ff4e9: Image successfully pushed 83e30bf01299: Image successfully pushed 499da968a652: Image successfully pushed bf09bd07d760: Image successfully pushed 1eee820e762b: Image successfully pushed 7bf9287ccfce: Image successfully pushed 288b8d534217: Image successfully pushed f20dbf0acb45: Image successfully pushed bd511e81a5ed: Image successfully pushed 8fe7eb38aea1: Image successfully pushed 464263a50f65: Image successfully pushed 1f58a67adecd: Image successfully pushed a99fb4ee814d: Image successfully pushed 6112f975feab: Image successfully pushed 6dff1b5c2259: Image successfully pushed Pushing tag for rev [6dff1b5c2259] on {http://localhost:5000/v1/repositories/postgres/tags/9.4} Looking at the current images, you will notice that the version tagged with localhost and the official images have the same information. Notice that I had to retag the image with the location of the repository. I thought the requirement to put the location address as part of the image name was a little odd. However, after using docker longer, it makes sense. It ensures you know where the image was originally pulled. $ docker images postgres 9.4 6dff1b5c2259 5 days ago 244.4 MB localhost:5000/postgres 9.4 6dff1b5c2259 5 days ago 244.4 MB Since docker tags are not permanent, and newer version of the postgres:9.4 image could be pushed to the public registry. When you self-host images, you are in control of when updates are pushed to any base image that you have extended. Someday I intend to learn how to build an image completely from scratch. Docker-ize All the Things!

I have a small Spring boot based application that uses a Postgres database as a datastore. I wanted to document the steps involved in deploying this sample application to Cloud Foundry. Some of the steps are described in the Spring Boot reference guide, however the guides do not sufficiently explain how to integrate with the datastore provided in a cloud based environment. Spring-cloud provides the glue to connect Spring based applications deployed on a Cloud to discover and connect to bound services, so the first step is to pull in the Spring-cloud libraries into the project with the following pom entries: org.springframework.cloud spring-cloud-spring-service-connector 1.0.0.RELEASE org.springframework.cloud spring-cloud-cloudfoundry-connector 1.0.0.RELEASE Once this dependency is pulled in, connecting to a bound service is easy, just define a configuration along these lines: @Configuration public class PostgresCloudConfig extends AbstractCloudConfig { @Bean public DataSource dataSource() { return connectionFactory().dataSource(); } } Spring-Cloud understands that the application is deployed on a specific Cloud(currently Cloud Foundry and Heroku by looking for certain characteristics of the deployed Cloud platform), discovers the bound services, recognizes that there is a bound service using which a Postgres based datasource can be created and returns the datasource as a Spring bean. This application can now deploy cleanly to a Cloud Foundry based Cloud. The sample application can be tried out in a version of Cloud Foundry deployed with bosh-lite, these are how the steps in my machine looks like once Cloud Foundry is up and running with bosh-lite: The following command creates a user provided service in Cloud Foundry: cf create-user-provided-service psgservice -p '{"uri":"postgres://postgres:[email protected]:5432/hotelsdb"}' Now, push the app, however don't start it up. We can do that once the service above is bound to the app: cf push spring-boot-mvc-test -p target/spring-boot-mvc-test-1.0.0-SNAPSHOT.war --no-start Bind the service to the app and restart the app: cf bind-service spring-boot-mvc-test psgservice cf restart spring-boot-mvc-test That is essentially it, Spring Cloud should ideally take over at the point and cleanly parse the credentials from the bound service which within Cloud Foundry translates to an environment variable called VCAP_SERVICES, and create the datasource from it. There is however an issue with this approach - once the datasource bean is created using spring-cloud approach, it does not work in a local environment anymore. The potential fix for this is to use Spring profiles, assume that there is a different "cloud" Spring profile available in Cloud environment where the Spring-cloud based datasource gets returned: @Profile("cloud") @Configuration public class PostgresCloudConfig extends AbstractCloudConfig { @Bean public DataSource dataSource() { return connectionFactory().dataSource(); } } and let Spring-boot auto-configuration create a datasource in the default local environment, this way the configuration works both local as well as in Cloud. Where does this "cloud" profile come from, it can be created using a ApplicationContextInitializer, and looks this way: public class SampleWebApplicationInitializer implementsApplicationContextInitializer { private static final Log logger = LogFactory.getLog(SampleWebApplicationInitializer.class); @Override public void initialize(AnnotationConfigEmbeddedWebApplicationContext applicationContext) { Cloud cloud = getCloud(); ConfigurableEnvironment appEnvironment = applicationContext.getEnvironment(); if (cloud!=null) { appEnvironment.addActiveProfile("cloud"); } logger.info("Cloud profile active"); } private Cloud getCloud() { try { CloudFactory cloudFactory = new CloudFactory(); return cloudFactory.getCloud(); } catch (CloudException ce) { return null; } } } This initializer makes use of the Spring-cloud's scanning capabilities to activate the "cloud" profile. One last thing which I wanted to try was to make my local behave like Cloud atleast in the eyes of Spring-Cloud and this can be done by adding in some environment variables using which Spring-Cloud makes the determination of the type of cloud where the application is deployed, the following is my startup script in local for the app to pretend as if it is deployed in Cloud Foundry: read -r -d '' VCAP_APPLICATION <<'ENDOFVAR' {"application_version":"1","application_name":"spring-boot-mvc-test","application_uris":[""],"version":"1.0","name":"spring-boot-mvc-test","instance_id":"abcd","instance_index":0,"host":"0.0.0.0","port":61008} ENDOFVAR export VCAP_APPLICATION=$VCAP_APPLICATION read -r -d '' VCAP_SERVICES <<'ENDOFVAR' {"postgres":[{"name":"psgservice","label":"postgresql","tags":["postgresql"],"plan":"Standard","credentials":{"uri":"postgres://postgres:[email protected]:5432/hotelsdb"}]} ENDOFVAR export VCAP_SERVICES=$VCAP_SERVICES mvn spring-boot:run This entire sample is available at this github location:https://github.com/bijukunjummen/spring-boot-mvc-test Conclusion Spring Boot along with Spring-Cloud project now provide an excellent toolset to create Spring-powered cloud ready applications, and hopefully these notes are useful in integrating Spring Boot with Spring-Cloud and using these for seamless local and Cloud deployments.

if you want evidence that the software development industry is susceptible to fashion, just go and take a look at all of the hype around microservices. it's everywhere! for some people microservices is "the next big thing", whereas for others it's simply a lightweight evolution of the big soap service-oriented architectures that we saw 10 years ago "done right". i do like a lot of what the current microservice architectures are doing, but it's by no means a silver bullet. okay, i know that sounds obvious, but i think many people are jumping on them for the wrong reason. i often show this slide in my conference talks, and i've blogged about this before , but basically there are different ways to build software systems. on the one side we have traditional monolithic systems, where everything is bundled up inside a single deployable unit. this is probably where most of the industry is. caveats apply, but monoliths can be built quickly and are easy to deploy, but they provide limited agility because even tiny changes require a full redeployment. we also know that monoliths often end up looking like a big ball of mud because of the way that software often evolves over time. for example, many monolithic systems are built using a layered architecture, and it's relatively easy for layered architectures to be abused (e.g. skipping "around" a service to call the repository/data access layer directly). on the other side we have service-based architectures, where a software system is made up of many separately deployable services. again, caveats apply but, if done well, service-based architectures buy you a lot of flexibility and agility because each service can be developed, tested, deployed, scaled, upgraded and rewritten separately, especially if the services are decoupled via asynchronous messaging. the downside is increased complexity because your software system now has many more moving parts than a monolith. as robert says, the complexity is still there, you're just moving it somewhere else . there is, of course, a mid-ground here. we can build monolithic systems that are made up of in-process components, each of which has an explicit well-defined interface and set of responsibilities. this is old-school component-based design that talks about high cohesion and low coupling, but i usually sense some hesitation when i talk about it. and this seems odd to me. before i explain why, let me quote something from a blog post that i read earlier this morning about the rationale behind a team adopting a microservices approach. when we started building karma, we decided to split the project into two main parts: the backend api, and the frontend application. the backend is responsible for handling orders from the store, usage accounting, user management, device management and so forth, while the frontend offers a dashboard for users which accesses this api. along the way we noticed that if the whole backend api is monolithic it doesn't work very well because everything gets entangled. the blog post also mentions scaling, versioning and multiple languages/frameworks as other reasons to choose microservices. again, there are no silver bullets here, everything is a trade-off. anyway, "everything getting entangled" is not a reason to switch from monoliths to microservices. if you're building a monolithic system and it's turning into a big ball of mud, perhaps you should consider whether you're taking enough care of your software architecture. do you really understand what the core structural abstractions are in your software? are their interfaces and responsibilities clear too? if not, why do you think moving to a microservices architecture will help? sure, the physical separation of services will force you to not take some shortcuts, but you can achieve the same separation between components in a monolith. a little design thinking and an architecturally-evident coding style will help to achieve this without the baggage of going distributed. many of the teams i've spoken to are building monolithic systems and don't want to look at component-based design. the mid-ground seems to be a hard-sell. i ran a software architecture sketching workshop with a team earlier this year where we diagrammed one of their software systems. the diagram started as a strictly layered architecture (presentation, business services, data access) with all arrows pointing downwards and each layer only ever calling the layer directly beneath it. the code told a different story though and the eventual diagram didn't look so neat anymore. we discussed how adopting a package by component approach could fix some of these problems, but the response was, "meh, we like building software using layers". it seems as if teams are jumping on microservices because they're sexy, but the design thinking and decomposition strategy required to create a good microservices architecture are the same as those needed to create a well structured monolith. if teams find it hard to create a well structured monolith, i don't rate their chances of creating a well structured microservices architecture. as michael feathers recently said, " there's a bit of overhead involved in implementing each microservice. if they ever become as easy to create as classes, people will have a freer hand to create trouble - hulking monoliths at a different scale. ". i agree. a world of distributed big balls of mud worries me.

Spring Integration (SI) is a framework enabling a collection of individual applications to integrate together to deliver a business enterprise system. The framework is essentially a lightweight messaging system that enables spring based applications to communicate with one another and supports integration with external systems via declarative adaptors. It is based on the 'filters and pipes' design architecture. A key feature of it is that it achieves this integration in a minimally intrusive way. The framework is built on 3 main components: Messages Encapsulate the data to be transferred from one place to another. They comprise of a header (holds meta data such as message-id, timestamp, etc) and a payload (your data typically in the form of a POJO). Channels Provide a mechanism to transport messages from one endpoint to another. Represents the pipes in the pipes & filters architecture. SI offers two types of channels, namely Pollable and Subscribable Channels. The former rely on consumers to periodically check for messages whereas the latter is directly responsible for notifying registered consumers when messages become available. Endpoints Consumer/Producer of messages. Performs some action based on the payload. Endpoints come in various flavours, each performing a different function. These include Transformers (transform data), Routers (route data), Filters (filter data), Splitter (splits messages), Aggregator (aggregates group of messages into single message), Service Activator (connecting messages to Services) and Channel Adapters (connect channels to external applications). The basic idea behind the SI framework is that applications communicate with each other by sending/receiving messages. These messages would typically contain the information (payload) required by the next application in the process pipeline. The transport of messages from one application to another is performed by Channel components. The Endpoints perform some action based on the payload. This could be routing the messages to another endpoint or processing the payload itself. The objective of this post is to provide an introduction to Spring Integration. To help achieve this, I developed a sample application which will be discussed below. The source for this sample application is available at here. The project was built and run using spring-integration-4.0.0, maven 3.2.1 and jdk1.6. The main dependency is for the relevant spring-integration jar as declared in the pom.xml: org.springframework.integration spring-integration-stream 4.0.0.RC1 I ran the application using the maven exec plugin. This allows me to clean, package and run the application by invoking mvn clean package exec:java -P OnlineShop from the command line. Developing a sample application: Tabernus My goal as usual was to build something very simple which would help me to become familiar with key concepts of this framework and to this end I've knocked up a simple app which does not connect up individual systems but rather invokes methods on a POJO. Extending this to actual working applications shouldn't be too difficult. The scenario I'm going to model revolves around purchasing items from an online store (Tabernus). This store only sells 3 types of items: Books, Music CDs, and software. During a Sale, the owners have decided to apply different discounts based on the item type. In this instance books, music and software benefit from discounts of 5%, 10%, and 15% respectively. The following diagram shows our domain entities. The class diagram shows that a Customer can place an Order comprising of a number of OrderItems which are of type Book, MusicCD or Software. The problem I need to solve is to design a system which can interrogate each Order and apply the correct discount based on the item type. Subsequently it should be able to compute the total cost of the order once the discounts have been applied. To model this using Spring Integration we need the following pipeline The above diagram shows various components most of which can be divided into 2 categories, channels (blue cylinder shapes) and endpoints (rectangular boxes). The exception to this is the Poller component whose purpose is to enable the various endpoints to function correctly and discussion of it will be given later. We'll start off by briefly covering the various stages in this pipeline as indicated by the numbers in red. Following this we will delve deeper into how we build this pipeline using the SI framework. The pipeline is comprised of 6 major stages as reflected by the numbers in the diagram, The Gateway component represents the entry point to the messaging system. All new Orders will be submitted to this component which will in turn wrap them as messages and place them into the channel appropriately named ordersChannel. Using the Splitter component - each Order is decomposed into a collection of it's constituent OrderItem instances. Each of these is wrapped in a Message and placed in the orderItemsChannel. The Router component considers each OrderItem in turn and places it in the relevant channel, e.g. Book items will be placed in the bookItemsChannel etc. This allows us to consider the different item types separately. The ServiceActivator needs to consider messages within each of the 3 channels and calculate the correct discount based on the channel. After completing the calculation for each OrderItem, it will place the OrderItem in the processedItemsChannel. The Aggregator component will collect all OrderItem instances placed in the processedItemsChannel and reconstruct the original Order. This will subsequently be placed in the deliveriesChannel, which represents the end of the pipeline. The Poller Component is required to configure how often the various endpoints will interrogate their respective input channels for messages. To implement the pipeline shown above using the SI framework, we need to implement the various end points. configure the pipeline in an xml file (Shop.xml) - identifying the various channels and endpoints and how they wire up together. At this point I should mention that SI offers 2 approach to configuring your process pipeline, annotations based and xml. In this article I'll be using the latter. Let's start to look at some code. We'll consider each stage described above and show the java implementation of the endpoint and xml configuration required to wire up the components. Step 1 - Gateway To begin with, we need to implement the Client that will invoke the Gateway component to place the Order. The client (OnlineShop.java) is shown below, public class OnlineShop { public static void main(String[] args) { AbstractApplicationContext context = new ClassPathXmlApplicationContext("/META-INF/com/prodcod/shop.xml", OnlineShop.class); Shop shop = (Shop) context.getBean("shop"); final Order order = createOrder(); shop.placeOrder(order); context.close(); } The logic here is quite simple. The client creates a dummy Order and passes this as an argument when it invokes the placeOrder() method on the gateway component. The gateway component referred here as Shop is injected by Spring. The Gateway component looks like: // Gateway component public interface Shop { @Gateway(requestChannel="ordersChannel") void placeOrder(Order order); } As you can see, this is simply an interface, whose implementation will be provided by Spring when it is injected into the client application. This is achieved by the use of the @Gateway annotation which informs Spring that this is a Gateway component and it needs to provide the implementation. Additionally the annotation accepts an attribute, requestChannel which defines the channel on which the Order instance will be placed. The framework does this by simply wrapping our instance of Order within a Message instance and placing it in the channel, 'ordersChannel'. The Gateway component and the 'ordersChannel' are declared as follows in the file shop.xml Step 2 - Splitter The next end point is the Splitter component. Appropriately named, it's role is to take a single message containing a payload of a collection of items and splitting it into a number of messages, each of which contains a single element from the collection. In our case, we want to decompose the Order into it's constituent OrderItem instances. It does this by taking a Message containing the payload of Order from 'ordersChannel' and then processing it before sending messages (each containing an OrderItem instance) to the 'orderItemsChannel'. Our implementation of the splitter is called OrderSplitter and is defined as below, public class OrderSplitter extends AbstractMessageSplitter{ @Override protected Object splitMessage(Message message) { return ((Order)message.getPayload()).getOrderItems(); } } Implementing a splitter is quite easy and involves extending the AbstractMessageSplitter class and overriding the splitMessage() method. This simply takes a message containing the payload of Order and returns it's collection of OrderItems. Step 3 - Router Having decomposed the Order into it's constituent OrderItems, we now need to separate them into groups of Books, MusicCD, and Software. This is achieved using a router. Our implementation of the Router looks like, public class OrderItemRouter { public String routeOrder(OrderItem orderItem) { String channel = ""; if(isBook(orderItem)) { channel = "bookItemsChannel"; } else if(isMusic(orderItem)) { channel = "musicItemsChannel"; } else if(isSoftware(orderItem)) { channel = "softwareItemsChannel"; } return channel; } ..................... ..................... } Nothing too complicated here. For each OrderItem, the method routeOrder() will determine it's item type and return the name of the channel that this message should be sent to. The channel name is returned by the method. Spring will then ensure that the message containing the OrderItem is relayed to the named channel. The configuration for OrderItemRouter looks like, The config identifies that the class OrderItemRouter is a Router component which will consume messages from the orderItemsChannel. Further Spring needs to invoke the method routeOrder() which contains the logic to perform the routing. The channels for each item type are declared as follows Step 4 - ServiceActivator The next step is to calculate the discounted price for each item type and this is performed by a ServiceActivator component. This is implemented as follows public class Shopkeeper { private static final BigDecimal BOOK_DISCOUNT = new BigDecimal(0.05); private static final BigDecimal MUSIC_DISCOUNT = new BigDecimal(0.10); private static final BigDecimal SOFTWARE_DISCOUNT = new BigDecimal(0.15); /** * Performs discount on books * @param bookOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processBooks(OrderItem bookOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(bookOrderItem, BOOK_DISCOUNT); bookOrderItem.setDiscountedPrice(finalPrice); return bookOrderItem; } /** * Performs discount on music * @param musicOrderItem OrderItem comprising of a music item * @return OrderItem with discount price newly calculated */ public OrderItem processMusic(OrderItem musicOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(musicOrderItem, MUSIC_DISCOUNT); musicOrderItem.setDiscountedPrice(finalPrice); return musicOrderItem; } /** * Performs discount on software * @param softwareOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processSoftware(OrderItem softwareOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(softwareOrderItem, SOFTWARE_DISCOUNT); softwareOrderItem.setDiscountedPrice(finalPrice); return softwareOrderItem; } } This class exposes 3 methods to compute the new discounted price for each item type. Each method returns the OrderItem instance with the new price. The ServiceActivator is configured as follows: This tells Spring that the Shopkeeper class is a ServiceActivator and will consume messages from any of the 3 channels defined in the input-channel attribute. When a message appears in one of these channels, Spring will invoke the appropriate method on the ServiceActivator class as specfied by the attribute method. Anything returned from all three methods will be placed in the processedItems channel, ready for the next step of the processing pipeline. Step 5 - Aggregator The final stage is to take the individual OrderItems with their newly computed discounted prices and reconstruct the Order. This is achieved using an aggregator. Our implementation of an aggregator is listed below public class OrderCompleter { public Order prepareDelivery(List orderItems) { final Order order = new Order(); order.setOrderItems(orderItems); return order; } } The aggregator exposes a method that takes a collection of OrderItem objects. These will come from the processedItems channel declared as Recall this is the output channel for the service activator class as discussed above. The aggregator is configured in the xml file as The configuration tells Spring that the aggregator component will consume messages from the processedItems channel. These will be processed by the method prepareDelivery on the class OrderCompleter. Any output from this class will be relayed to the channel-adaptor deliveries, which is declared as The stdout-channel-adapter component writes to the systems STDOUT output stream. Step 6 - Poller To complete the setup we have to configure a poller component. This is required to enable the channels to work correctly. All our channels are of a queue type and so their respective consumers need to know when to query them. This is achieved using a poller mechanism. It is configured in the following way In this case, we have declared a global poller (as indicated by the default attribute). This will be used by the various end points to determine when they should interrogate their respective input-channels for messages. The second attribute fixed-delay is used to configure the polling interval. Running the Application Building and running the app shows the following output: The logging shows that the Customer submitted an Order for 3 items, one of each type. All items cost £100 each. The Order was then split into 3 OrderItems each of which was routed to the correct processing channel based on the item type. The ServiceActivator (Shopkeeper) then calculated the discount for each item and this was set on the OrderItem instance. The OrderItems were then aggregated using the OrderCompleter class which displays the final discounted price of £270 to be paid by the Customer. Note that the messages are logged to be in different stages of the processing pipeline despite starting off in the same order. This completes the tutorial on the Spring Integration Framework. Any comments relating to corrections, omissions, etc are welcome.

You should by now have worked your way through Part 1: Getting Started with Ansible. If you haven't, go and do that now. In this article, I'll be demonstrating a very simple application deployment workflow, deploying an insanely simple node.js application from a github repository, and configuring it to start with supervisord, and be reverse-proxied with Nginx. As with last time, we'll be using Parallax as the starting point for this. I've actually gone through and put the config in there already (if you don't feel like doing it yourself ;) - name: Install all the packages and stuff required for a demobox hosts: demoboxes user: user sudo: yes roles: - redis - nginx - nodejs - zeromq # - deploy_thingy In the 9c818d0b8f version, you'll be able to see that I've created a new role, inventively called "deploy_thingy". **Updated** I've been recommended that my __template__ role be based on the output of ansible-galaxy init $rolename So I've recreated the __template__ role to be based on an ansible-galaxy role template. There's not that many changes, but it does include a new directory 'default/' containing the Galaxy metadata required if you wish to push back to the public galaxy role index. In an attempt to make creating new roles easier, I put a __template__ role into the file tree when I first created Parallax, so that all you do to create a new role is execute: cp -R __template__ new_role_name in the roles/ directory. . ├── files │ ├── .empty │ ├── thingy.nginx.conf │ └── thingy.super.conf ├── handlers │ ├── .empty │ └── main.yml ├── meta │ ├── .empty │ └── main.yml ├── tasks │ └── main.yml └── templates └── .empty In this role, we define some dependencies in meta/main.yml, there's two files in the files/ directory, and there's a set of tasks defined in tasks/main.yml. There's also some handlers defined in handlers/main.yml. Let's have a quick glance at the meta/main.yml file. --- dependencies: - { role: nodejs } - { role: nginx } This basically sets the requirement that this role, deploy_thingy depends on services installed by the roles: nginx and nodejs. Although these roles are explicitly stated to be installed in site.yml, this gives us a level of belt-and-braces configuration, in case the deploy_thingy role were ever included without the other two roles being explicitly stated, or if it were configured to run before its dependencies had explicitly been set to run. tasks/main.yml is simple. --- - name: Create directory under /srv for thingy file: path=/srv/thingy state=directory mode=755 - name: Git checkout from github git: repo=https://github.com/tomoconnor/shiny-octo-computing-machine.git dest=/srv/thingy - name: Drop Config for supervisord into the conf.d directory copy: src=thingy.super.conf dest=/etc/supervisor/conf.d/thingy.conf notify: reread supervisord - name: Drop Reverse Proxy Config for Nginx copy: src=thingy.nginx.conf dest=/etc/nginx/sites-enabled/thingy.conf notify: restart nginx We'll create somewhere for it to live, check the code out of my git repository [1], Then drop two config files in place, one to configure supervisor(d), and one to configure Nginx. Because the command to configure supervisor(d) and nginx change the configuration of those services, there are notify: handlers to reload the configuration, or restart the service. Let's have a quick peek at those handlers now: --- - name: reread supervisord shell: /usr/bin/supervisorctl reread && /usr/bin/supervisorctl update - name: restart nginx service: name=nginx state=restarted When the supervisor config changes (and we add something to /etc/supervisor/conf.d), we need to tell supervisord to re-read it's configuration files, at which point, it will see the new services, and then run supervisorctl update, which will set the state of the newly added items from 'available' to 'started'. When we change the nginx configuration, we'll hit nginx with a restart. It's possible to do softer actions, like reload here, but I've chosen service restart for simplicity. I've also changed the basic Ansible config, and configuration of roles/common/files/insecure_sudoers so that it will still ask you for a sudo password in light of some minor criticism. I've found that if you're developing Ansible playbooks on an isolated system, then there's no great harm in disabling SSH Host Key Checking (in ansible.cfg), similarly how there's no great problems in disabling sudo authentication, so it's effectively like NOPASSWD use. However, Micheil made a very good point that in live environments it's a bit dodgy to say the least. So I've commented those lines out of the playbook in Parallax, so that it should give users a reasonable level of basic security. At the end of the day, it's up to you how you use Parallax, and if you find that disabling security works for you, then fine. It's not like you haven't been warned. But I digress. The next thing to do is to edit site.yml, and ensure that the new role we've created gets mapped to a hostgroup in the play configuration. In the latest version of Parallax this is already done for you, but as long as the role name in the list matches the directory in roles/, it should be ready to go. Now if we run: ansible-playbook -k -K -i playbooks/example/hosts playbooks/example/site.yml It should go through the playbook, installing stuff, then finally do the git clone from github, deploy the configuration files, and trigger a reread of supervisord, and a restart of nginx. If I now test that it's working, with: curl -i http://192.168.20.151/ HTTP/1.1 200 OK Server: nginx/1.4.1 (Ubuntu) Date: Mon, 27 Jan 2014 14:51:29 GMT Content-Type: text/html; charset=utf-8 Content-Length: 170 Connection: keep-alive X-Powered-By: Express ETag: "1827834703" That X-Powered-By: Express line shows that Nginx is indeed working, and that the node.js application is running too. You can get more information about stuff that supervisord is controlling by running: sudo supervisorctl status on the target host. $ sudo supervisorctl status thingy RUNNING pid 19756, uptime 0:00:06 If the Nginx side is configured, but the node.js application isn't running, you'd get a HTTP 502 error, as follows: curl -i http://192.168.20.151/ HTTP/1.1 502 Bad Gateway Server: nginx/1.4.1 (Ubuntu) Date: Mon, 27 Jan 2014 14:59:34 GMT Content-Type: text/html Content-Length: 181 Connection: keep-alive So, that's it. A very simple guide to deploying a very simple application with Ansible. Of course, it should be obvious that you can deploy *anything* from a git repository, it really boils down to the configuration of supervisord. For that matter, it doesn't have to be supervisord. I consider configuring supervisord for process controlling to be outside of the scope of this article, but I might touch on it in future in more detail. Next up, Part 3: Ansible and Amazon Web Services. 1: It's really simple, and I'm not very node-savvy, so I'm sorry if it sucks.

In this blog, we will discuss one particular data grid platform from Redhat namely JBoss Data Grid (JDG). We will firstly cover how to access and install this data grid platform and then we will demonstrate how to develop and deploy a simple remote client/server data grid application which utilises the HotRod protocol. We will be using the latest release JDG 6.2 from Redhat in this article. Installation Overview To start using JDG, firstly log on to the redhat site https://access.redhat.com/home and download the software from the Downloads section of the site. We wish to download JDG 6.2 server by clicking on the appropriate links in the Downloads section. For future reference, it is also useful to download the quickstart and maven repository zip files. To install JDG, we simply unzip the JDG server package into an appropriate directory in your environment. JDG Overview In this section, we will provide a brief overview of the contents of the JDG installation package and the most notable configuration options available to users. Out of the box, users are provided with two runtime options either to run JDG in standalone or clustered mode. We can start JDG in either mode by invoking the stanadalone or clustered start up scripts in the / bin directory. To configure the JDG in either mode we need to configure the files standalone.xml and clustered.xml. In our case we will creating a distributed cache which will run on 3 node JDG cluster so we will be utilizing the clustered startup script. In order to set up and add new cache instances to JDG, we modify the infinispan subsystems in the appropriate xml configuration file above. We should also note the principal difference between the standalone and clustered configuration file is that in the clustered configuration file there is a JGroups subsystem configured element which allows for communication and messaging between configured cache instances running in a JDG cluster. Development Environment Setup and Configuration In this section, we will detail how to develop and configure a simple datagrid application which will be deployed to a 3 node JDG cluster. We will demonstrate how to configure and deploy a distributed cache in JDG and also show how to develop a HotRod Java client application which will be used to insert, update and display entries in the distributed cache. We will firstly discuss setting a new distributed cache on a 3 node JDG cluster. In this example, we will run our JDG cluster on a single machine by running each JDG instance on different ports. Firstly, we will create 3 instances of JDG by creating 3 directories (server1, server2, server3) on our host machine and unzipping each JDG installation into each directory. We will now configure each node in our cluster by copying and renaming the clustered.xml configuration file in the \server1\jboss-datagrid-6.2.0-server\standalone\configuration directory. We will name each of the cluster configuration files as "clustered1.xml", "clustered2.xml" and "clustered3.xml" for the JDG instances denoted by "server1", "server2" and "server3" respectively. We will now set up a new distributed cache on our JDG cluster by modifying the infinispan subsystem element in each clustered.xml file. We will demonstrate this for the node denoted "server1" here by modifying the file "clustered1.xml". The cache configuration shown here will be the same across all 3 nodes. To setup a new distributed cache named "directory-dist-cache", we configure the following elements in the file named "clustered1.xml" ......... ...... .............. ...... ...... /socket-binding-group> We will discuss the key elements and attributes relating to the configuration above. In the infinispan endpoint subsystem, we will configure hotrod clients to connect to the JDG server instance on socket 11222. The name of the cache container to host each of the cache instances will be held in the container named "clusteredcache". We have configured the infinispan core subsystem to the default cache container named "clusteredcacahe" whereby we will allow for jmx statistics to be collected relating the configured cache entries i.e statistics="true" We have created a new distributed cache named "directory-dist-cache" whereby there will be two copies of each cache entry held on two of the 3 cluster nodes. We have also set up an eviction policy whereby should there be more than 20 entries in our cache then cache entries will be removed using the LRU algorithm We should have configured nodes "server2" and "server3" to start up with a port offset of 100 and 200 respectively by configuring the socketing binding group element appropriately. Please view the socket bindings noted below. To set the socket binding element with a port offset of 100 on "server2", we configure "clustered2.xml" with the following entry: ...... ...... /socket-binding-group> To set the socket binding element with a port offset of 200 on "server3", we configure "clustered3.xml" with the following entry: ...... ...... /socket-binding-group> Before discussing the setup and configuration of our Hotrod client which will be used to interact with our JDG clustered HotRod server, we will start up each server instance to ensure our newly configured JDG distributed cache starts up correctly. Open up 3 Windows or Linux consoles and execute the following start up commands: Console 1: 1) Navigate to \server1\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the first instance of our JDG cluster denoted "server1": clustered -c=clustered1.xml -Djboss.node.name=server1 Console 2: 1) Navigate to \server2\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the second instance of our JDG cluster denoted "server2": clustered -c=clustered2.xml -Djboss.node.name=server2 Console 3: 1) Navigate to \server3\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the third instance of our JDG cluster denoted "server3": clustered -c=clustered3.xml -Djboss.node.name=server3 Providing all 3 JDG instances have started up correctly, you should see output in the console window whereby we can see there are 3 JDG instances in the JGroups view: HotRod Client Development Setup Now that the Hotrod server is up and running, we need to develop a Hotrod Java client which will interact with the clustered server application. The development environment consists of the following tools. 1) JDK Hotspot 1.7.0_45 2) IDE - Eclipse Kepler Build id: 20130919-0819 The HotRod client application is a simple application consisting of two Java classes. The application allows users to retrieve a reference to the distributed cache from the JDG server and then perform these actions: a) add new cinema objects. b) add and remove shows to each cinema object. c) print the list of all cinemas and shows stored in our distributed cache. The source code can be downloaded from github @ https://github.com/davewinters/JDG. We could use maven here to build and execute our application by configuring the maven settings.xml to point to the maven repository files we downloaded earlier and set up a maven project file (pom.xml) to build and execute the client application. In this article we will build our application using the Eclipse IDE and run the client application on the command line. To create a HotRod client application and execute the sample application, one should complete the following steps: 1) Create a new Java Project in Eclipse 2) Create a new package named uk.co.c2b2.jdg.hotrod and import the source code that has been downloaded from Github mentioned previously. 3) Now we need to configure the build path in Eclipse to contain the appropriate JDG client jar files which are required to compile the application. You should include all the client jar files in the project build path. These jar files are contained in the JDG installation zip file. For example on my machine these jar files are located in the directory: \server1\jboss-datagrid-6.2.0-server\client\hotrod\java 4. Providing the Eclipse build path has been configured appropriately, the application source should compile without issue. 5. We will need to execute the Hotrod application by opening the console window and executing the following command. Note the path specified here will differ depending on where the JDG client jar files and application class files are located in your environment: java -classpath ".;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\commons-pool-1.6-redhat-4.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-client-hotrod-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-commons-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-query-dsl-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-remote-query-client-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-logging-3.1.2.GA-redhat-1.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-marshalling-1.4.2.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-marshalling-river-1.4.2.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\protobuf-java-2.5.0.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\protostream-1.0.0.CR1-redhat-1.jar" uk/co/c2b2/jdg/hotrod/CinemaDirectory 6. The Hotrod client at runtime provides the end user with a number of different options to interact with the distributed cache as we can view from the console window below. Client Application Principal API Details We will not provide a detailed overview of the Hotrod application code however we will describe the principal API and code details briefly. In order to interact with the distributed cache on the JDG cluster using the Hotrod protocol, we will use the RemoteCacheManager Object which will allow us to retrieve a remote reference to the distributed cache. We have initialised a Properties object with the list of JDG instances and the associated with HotRod server port on each instance. We can add Cinema objects into the distributed cache using the RemoteCache.put() method. private RemoteCacheManager cacheManager; private RemoteCache cache; ..... Properties properties = new Properties(); properties.setProperty(ConfigurationProperties.SERVER_LIST, "127.0.0.1:11222;127.0.0.1:11322;127.0.0.1:11422"); cacheManager = new RemoteCacheManager(properties); cache = cacheManager.getCache("directory-dist-cache"); ..... cache.put(cinemaKey, cinemalist); In the webinar below, I describe in further detail how to set up a JDG cluster and how to develop and run the JDG application discussed above. For further details on JDG please visit: http://www.redhat.com/products/jbossenterprisemiddleware/data-grid/ Webinar: Introduction to JBoss Data Grid -- Installation, Configuration and Development In this webinar we will look at the basics of setting up JBoss Data Grid covering installation, configuration and development. We will look at practical examples of storing data, viewing the data in the cache and removing it. We will also take a look at the different clustered modes and what effect these have on the storage of your data:

a powerful multi-format xml schema (xsd) documentation generator and a tool for rapid development of custom xsd documentation generators according to user needs. about docflex/xml "xsddoc" template set template processor template designer integrations generation of xsd diagrams apache ant & maven links about docflex/xml docflex/xml is a java-based software system for development and execution of high performance template-driven documentation generators from any data stored in xml files. the actual doc/report generators are programmed in the form of special templates using a graphic template designer , which represents the templates visually in a form resembling the output they generate. further, the templates are interpreted by a template processor , which takes on input the xml files and produces by them the result documentation. this article describes an application of docflex/xml for the task of generation of high-quality xml schema documentation. that includes the following features of docflex/xml system: " xsddoc " template set that implements the ready-to-use xml schema documentation generator itself. template processor makes the templates works. currently, it provides three interchangeable output generators for html, rtf, txt (plain text) formats. template designer provides a high quality gui to design/modify templates. if you need a special xml schema doc generator, the simplest way to create it is to modify the standard xsddoc templates. the template designer enables you to do that. integrations with altova xmlspy and oxygen xml editor . if you are a user of one of those popular xml editors, you can turn it also into a dynamically linked diagramming engine for docflex, so that to include automatically the xsd diagrams generated by xmlspy/oxygenxml into the xml schema documentation generated by docflex (with the full support of hyperlinks). "xsddoc" template set it is the implementation of xml schema documentation itself, which provides the following functionality: generation of single documentation by any number of xml schema (xsd) files together, in particular: highly navigable framed (javadoc-like) html documentation single-file html documentation rtf documentation (further convertible to pdf) processing of any referenced xml schemas, in particular: correct processing of all , , elements found across all involved xsd files. automatic loading and processing (i.e. inclusion in the documentation scope) all directly/indirectly referenced xsd files. sophisticated documenting of xsd components , including: component diagrams (with hyperlinks to everything depicted on them; see also integrations ) xml representation summary (a textual alternative to diagrams) lists of related components. for elements this includes also the list of possible containing elements . (such a list is never present in the output generated by xslt-based doc generators). list of usage locations support of any xml schema design patterns . this comes down mainly to the following: special treatment of local elements (see below) support and documenting of substitution groups support of importing, inclusion and redefinition of schema files special documenting of local elements . local elements are those components that are declared locally within other xsd components. w3c xml schema spec allows you to declare any number of local elements that may share the same name but have different content. that's because their meaning is local and there will be no collisions with other declarations. that, however, creates a problem for documenting, because in a documentation both global and local elements may appear simultaneously in various lists according to their common properties. if each element component is identified only by its name, you will get the lists with multiple repeating names but little clue what they mean. moreover, some xml schemas may contain lots of identical local element declarations (that is, they have the same both name and content). so, you'll get in those lists a mess of repeating names, some of which referencing to effectively the same entities, whereas others to complete different ones. in xsddoc , those problems are solved in two ways: adding extensions to local element names. the extension provides more information about the element (e.g. where it can be inserted or its global type or where it is defined). that makes the whole string identifying the element unique. here is how it looks. the grey text is the name extension: unifying local elements by type. on the left you can see a documentation generated with such unification. on the right, all local elements are documented straight as they are. click on each screenshot to view the docs: we believe the first documentation (on the left) is easier to understand and use. processing of xhtml markup . you can format your xml schema annotations with xhtml tags, which will be recognized and rendered with the appropriate formatting in both html and rtf output, as shown on the following screenshots (click to see more details): here, on the left you can see the xml source of an xml schema, whose annotations are heavily laden with xhtml markup (including insertion of images). the next is the html documentation generated by that schema. on the right is a page of rtf documentation also generated by that schema. possibility of unlimited customization : xsddoc is controlled by more than 400 parameters, which allow you to adjust the generated documentation within huge range of included details. template parameters serve the same role as options in traditional doc generators. the difference is that docflex template architecture makes the support/implementation of template parameters very cheap (typically, the most of efforts takes writing their descriptions). so, there may be hundreds of parameters controlling a large template application. if parameters are not enough, you can modify the templates themselves using the template designer . in case of html output, you can also apply your own css styles to change how the generated documentation looks. template processor the template processor (also called simply "generator") makes everything work. it consists of two logical parts: 1. template interpreter 2. output generator the output generator actually has three different implementations for each currently supported output format: html, rtf, txt (plain text). the plain-text output can be used to generate documentation in formats not supported directly by docflex. the template processor is started directly from java command line with the following arguments: ● main template ● template parameters ● initial xsd files to be processed (documented) ● xml catalogs (to redirect physical location of input files) ● destination directory/file ● output format (this selects which output generator will be used) ● output format options (specify settings to control the selected output generator) actually, the number of settings may be so large that the template processor provides a special gui to specify everything interactively (click to enlarge): template designer although docflex templates are stored as plain-text files (with an xml-like format), they are not supposed for editing manually. rather, a special graphic template designer must be used, which visualizes the templates in the form of template components they are made of. those components are the actual constructs of the template language (not some textual statements, operators, blocks etc.) the following screenshots show templates open in the template designer (click to see a lot more): that approach has a number of advantages, among them: the processing structures represented by template components may be displayed in a way that visually expresses what a component does (for instance, it may resemble the output it generates). that representation may be both expressive and compact (after all, it is not just a text), which allows you easily to navigate a template, understand what it does and modify anything you need. as template components are visual and interactive, they may have very complex internal structure, for instance, contain lots of properties and nested components. at that, you don't need to scroll and navigate some kind of enormous text, which encodes all of this (as it would be in case of a script). rather, you just need to invoke some property dialogs and expand/collapse some component sections. a template component may be easily copied, pasted and deleted as a whole. at that, you don't need to bother that the template syntax is restored after that. the template designer will also ensure that each component is created, copied or moved only in the allowed place. the highly structured nature of templates eliminates the need for most of various named identifiers. many connections between different template components are also maintained by the template designer (i.e. modified automatically when necessary). as template files are stored and read only programmatically, there is no need to know and understand their syntax. there will be no syntax errors either. the actual syntax of template files may be optimized not for human programmers, but for faster loading and processing of templates by the template processor . there is no need in a compilation phase. the separation of template semantics from the particular structure of template files helps for faster and easier evolution of the template language. the obsolete constructs of older template versions can be automatically converted into new structures. both old and new templates will look and work up-to-date. integrations generation of xsd diagrams docflex/xml is able to work with any kind of diagrams (i.e. inserting them automatically in the generated output). that is supported on the level of templates, along with the generation of hypertext imagemaps, as shown on the following screenshot (click to see a lot more): docflex/xml provides no diagramming engine of its own. instead, it includes integrations with two most popular xml editors that do generate xsd diagrams: ● altova xmlspy ● oxygen xml editor effectively, the third-party software is used as dynamically linked diagramming engine. the advantage of such integrations is that when you are the user of one of those xml editors, you will get in the documentation generated by docflex the same diagrams as you see in your xml editor. here is how such a documentation with diagrams looks (click on a screenshot to view the real html): apache ant & maven as a pure java application, docflex/xml can be run in any environment that runs java itself. the template processor can be easily integrated with ant (that can be specified just in the ant build file). in case of maven, docflex/xml includes a simple maven plugin. it is possible also to use all diagraming integrations with both ant and maven. links docflex/xml (home page): http://www.filigris.com/docflex-xml/ docflex/xml xsddoc: http://www.filigris.com/docflex-xml/xsddoc/ xsddoc examples: http://www.filigris.com/docflex-xml/xsddoc/examples/ xmlspy integration: http://www.filigris.com/docflex-xml/xmlspy/ oxygenxml integration: http://www.filigris.com/docflex-xml/oxygenxml/ free downloads: http://www.filigris.com/downloads/ this original article: http://www.filigris.com/ann/docflex-xsd/

This is a sample that I have wanted to try for sometime - A Websocket application to tail the contents of a file. The following is the final view of the web-application: There are a few parts to this application: Generating a File to tail: I chose to use a set of 100 random quotes as a source of the file content, every few seconds the application generates a quote and writes this quote to the temporary file. Spring Integration is used for wiring this flow for writing the contents to the file: Just a quick note, Spring Integration flows can now also be written using a Java Based DSL, and this flow using Java is available here Tailing the file and sending the content to a broker The actual tailing of the file itself can be accomplished by OS specific tail command or by using a library like Apache Commons IO. Again in my case I decided to use Spring Integration which provides Inbound channel adapters to tail a file purely using configuration, this flow looks like this: and its working Java equivalent There is a reference to a "fileContentRecordingService" above, this is the component which will direct the lines of the file to a place where the Websocket client will subscribe to. Websocket server configuration Spring Websocket support makes it super simple to write a Websocket based application, in this instance the entire working configuration is the following: @Configuration @EnableWebSocketMessageBroker public class WebSocketDefaultConfig extends AbstractWebSocketMessageBrokerConfigurer { @Override public void configureMessageBroker(MessageBrokerRegistry config) { //config.enableStompBrokerRelay("/topic/", "/queue/"); config.enableSimpleBroker("/topic/", "/queue/"); config.setApplicationDestinationPrefixes("/app"); } @Override public void registerStompEndpoints(StompEndpointRegistry registry) { registry.addEndpoint("/tailfilesep").withSockJS(); } } This may seem a little over the top, but what these few lines of configuration does is very powerful and the configuration can be better understood by going through the reference here. In brief, it sets up a websocket endpoint at '/tailfileep' uri, this endpoint is enhanced with SockJS support, Stomp is used as a sub-protocol, endpoints `/topic` and `/queue` is configured to a real broker like RabbitMQ or ActiveMQ but in this specific to an in-memory one. Going back to the "fileContentRecordingService" once more, this component essentially takes the line of the file and sends it this in-memory broker, SimpMessagingTemplate facilitates this wiring: public class FileContentRecordingService { @Autowired private SimpMessagingTemplate simpMessagingTemplate; public void sendLinesToTopic(String line) { this.simpMessagingTemplate.convertAndSend("/topic/tailfiles", line); } } Websocket UI configuration The UI is angularjs based, the client controller is set up this way and internally uses the javascript libraries for sockjs and stomp support: var tailFilesApp = angular.module("tailFilesApp",[]); tailFilesApp.controller("TailFilesCtrl", function ($scope) { function init() { $scope.buffer = new CircularBuffer(20); } $scope.initSockets = function() { $scope.socket={}; $scope.socket.client = new SockJS("/tailfilesep); $scope.socket.stomp = Stomp.over($scope.socket.client); $scope.socket.stomp.connect({}, function() { $scope.socket.stomp.subscribe("/topic/tailfiles", $scope.notify); }); $scope.socket.client.onclose = $scope.reconnect; }; $scope.notify = function(message) { $scope.$apply(function() { $scope.buffer.add(angular.fromJson(message.body)); }); }; $scope.reconnect = function() { setTimeout($scope.initSockets, 10000); }; init(); $scope.initSockets(); }); The meat of this code is the "notify" function which the callback acting on the messages from the server, in this instance the new lines coming into the file and showing it in a textarea. This wraps up the entire application to tail a file. A complete working sample without any external dependencies is available at this github location, instructions to start it up is also available at that location. Conclusion Spring Websockets provides a concise way to create Websocket based applications, this sample provides a good demonstration of this support. I had presented on this topic recently at my local JUG (IndyJUG) and a deck with the presentation is available here

In this post, I will show how to integrate Apache Tiles 3 with Spring MVC.

Recently I’ve been butting heads with some people on the subject of Ownership, Responsibility and Accountability. There seems to be a very unhealthy obsession with these things sometimes, and I think this is indicative of a less-than-ideal culture. I don’t want to say that they’re “anti-agile” because that just sounds a bit weak, and because I also think they’re not just bad for agile, they’re bad for pretty much any system. I’m not sure how familiar most people are with the “RACI matrix” concept, but in my eyes it’s downright evil in the wrong hands, and I’ve been hearing “RACI Matrix” a lot recently (it’s now on my Bullshit Bingo card). I’ll start off by clarifying what I mean. I’ve got nothing against people owning actions or being accountable for certain particular (usually small) things, but I do take offence when pretty much everything has to be given an owner, someone accountable and someone to “take responsibility”. It’s divisive and results in lots of finger pointing, in my experience. I much prefer the concept of shared ownership, and collective accountability. As a software delivery team, we should all feel responsible for the quality of the product, as well as the performance and the feature richness. These things shouldn’t be assigned for ownership to individuals, as it’ll create an attitude of “well it’s not my problem” among the other team members. Here’s an example: I’ve worked in a team where one person was made the “owner” of the build system. They busied themselves making sure all the builds passed and that the system was regularly ticking over. Of course, the builds often failed and nobody cared except this one person, who then had to try to get people to fix their broken builds. It almost seemed as if people didn’t care about the fact that their software wasn’t capable of being compiled, or that the tests were failing, and in truth they didn’t. They cared about writing code and checking it in, because they didn’t “own” the build system. One message that I always try to drive home with software delivery teams is that our objective is to make software that works for our users, not just write code. I know how easy it is for developers to just focus on checking in code, or perhaps just make sure it passes the tests in the CI system, but beyond that, their focus drops off. I know because I was once one of those developers :-) These days I try to encourage everyone to care about things such as: How your code builds How the tests execute How good the tests are How good the code is How easy it is to deploy How easy it is to maintain How easy it is to monitor Because it takes all of these things to produce good software that users can enjoy, which means we get paid. Here’s another example of how “ownership” has hurt a product: A large system I once worked on was deployed into production using a complicated system of bash and perl scripts, which were cobbled together by a sysadmin who did the deployments. He became the de facto “owner” of the deployment system. There were untold issues with the running of the application because of permissions, paths etc and so forth. The deployment process was creaky and relatively untested. Since the “ownership” of this system was assigned to the sysadmin, rather than devolved or collectively shared throughout the delivery team, the “deployability” was seen as a second class citizen within the delivery team, because everybody felt like it was “owned” by one person who just happened to be on the periphery of the team at best. So here’s what I think: The ability to monitor, maintain, deploy, test, build and create software should all be treated as first class citizens and should be the collective responsibility of everyone in the team. They should all own it, and they should all be accountable. I would extend this out further, to include supporting systems such as environments, build systems, testing frameworks and so-on. Sure, each team might have an SME or two who focuses more on one of these things than any other, but that doesn’t make that one person accountable, responsible or the owner any more than any particular developer is the “owner” of any particular class, method or function. If I write some code that depends on a method that someone else has written, and that method is failing, I don’t just down tools, shrug my shoulders and say “well I’m not accountable for that”. That would be hugely unhelpful and I’d make no friends either. In the same way, we shouldn’t treat our supporting functions and systems as someone else’s responsibility. If we need it in order to make our software work for the end user, then it’s our collective responsibility, no matter what “it” is.

When I am trying to install third party apk using ADB tool, I have faced "Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]" error. To resolve the issue, I have followed few steps. Open command prompt; Go to your debug.keystore location. For eg: You can find the debug.keystore file in the following location C:\Documents and Settings\User\.android 1.Using Zip align copied apk. zipalign -v 4 D:\Test.apk D:\Testc.apk 2.keytool -genkey -v -keystore debug.keystore -alias sampleName -keyalg RSA -keysize 2048 -validity 20000 Now a prompt will ask for Password First and lastname Name of Organization unit Name of Organization City State Country After entering these fields we get our Certificate 3. jarsigner -verbose -keystore debug.keystore D:\Testc.apk sampleName In some cases we need add -sigalg SHA1withRSA -digestalg SHA1 arguments to work out the step 3 jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore debug.keystore D:\Testc.apk sampleNameNow it will ask for the password and then it will replace the apk with the signed one. To check whether it is working or not, you can check using the following command. jarsigner -verify D:\Testc.apk Then I have installed apk using ADB. Adb install D:\Testc.apk

In an earlier blog entry I had touched on a fictitious rube goldberg flow for capitalizing a string through a complicated series of steps, the premise of the article was to introduce Spring Integration Java DSL as an alternative to defining integration flows through xml configuration files. I learned a few new things after writing that blog entry, thanks to Artem Bilan and wanted to document those learnings here: So, first my original sample, here I have the following flow(the one's in bold): Take in a message of this type - "hello from spring integ" Split it up into individual words(hello, from, spring, integ) Send each word to a ActiveMQ queue Pick up the word fragments from the queue and capitalize each word Place the response back into a response queue Pick up the message, re-sequence based on the original sequence of the words Aggregate back into a sentence("HELLO FROM SPRING INTEG") and Return the sentence back to the calling application. EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); jmsOutboundGateway.setReplyChannel(sequenceChannel()); return jmsOutboundGateway; } It turns out, based on Artem Bilan's feedback, that a few things can be optimized here. First notice how I have explicitly defined two direct channels, "requestChannel" for starting the flow that takes in the string message and the "sequenceChannel" to handle the message once it returns back from the jms message queue, these can actually be totally removed and the flow made a little more concise this way: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); return jmsOutboundGateway; } "requestChannel" is now being implicitly created just by declaring a name for it. The sequence channel is more interesting, quoting Artem Bilan - do not specify outputChannel for AbstractReplyProducingMessageHandler and rely on DSL , what it means is that here jmsOutboundGateway is a AbstractReplyProducingMessageHandler and its reply channel is implicitly derived by the DSL. Further, two methods which were earlier handling the flows for sending out the message to the queue and then continuing once the message is back, is collapsed into one. And IMHO it does read a little better because of this change. The second good change and the topic of this article is the introduction of the Jms namespace factories, when I had written the previous blog article, DSL had support for defining the AMQ inbound/outbound adapter/gateway, now there is support for Jms based inbound/adapter adapter/gateways also, this simplifies the flow even further, the flow now looks like this: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(Jms.outboundGateway(connectionFactory) .requestDestination("amq.outbound")) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } The inbound Jms part of the flow also simplifies to the following: @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(Jms.inboundGateway(connectionFactory) .destination("amq.outbound")) .transform((String s) -> s.toUpperCase()) .get(); } Thus, to conclude, Spring Integration Java DSL is an exciting new way to concisely configure Spring Integration flows. It is already very impressive in how it simplifies the readability of flows, the introduction of the Jms namespace factories takes it even further for JMS based flows.

Closely approaching the one year mark of when I first joined MongoLab (and the MongoDB community), I had the pleasure of attending the inaugural MongoDB World conference put together by the incredible MongoDB team. Second only to the excitement around major MongoDB feature announcements was the collective disbelief that this was MongoDB’s first multi-day conference ever. A big congratulations to all those that worked hard to put on such a massive (did you see the Intrepid!?) event. All this planning would have been for naught if MongoDB leaders and engineers failed to deliver announcements and features that would meet and exceed expectations. From major public cloud announcements to the reveal of document-level locking in version 2.8, developers and conference goers had plenty to be excited about. There was a lot to digest from the conference… we’ll cover the major highlights in case you missed them. Big announcements in public cloud Our time at the MongoLab booth yielded many high-quality conversations, predominantly those about offloading previously internal processes and workloads to the public cloud. It was remarkable to see and hear so many enterprise teams with the exact same message: the public cloud is the future, and the future is now. It’s no surprise then that MongoDB, Inc. released not one, but two press releases around MongoDB solutions for the public cloud. Fully-managed MongoDB on the Microsoft Azure Store Nearly one year ago, MongoDB, Inc. chose to partner with the MongoLab team to build a production-ready MongoDB solution for developers on Microsoft Azure. On the first day of World, MongoDB, Inc. announced the product of our collaboration – a fully-managed highly available MongoDB-as-a-Service Add-On offering on the Microsoft Azure Store. This new service runs MongoDB Enterprise and offers replication, monitoring and support from MongoDB, Inc. It’s also backed by MongoDB Management Service (MMS), allowing for point-in-time recovery of MongoDB deployments. Now, teams without the expertise or resources to manage their MongoDB deployment(s) can outsource all the database operations (monitoring and alerting, backups, performance tuning, etc.) to both MongoLab and MongoDB’s expert support teams. You can check out the MongoDB add-on in the Azure Store: https://azure.microsoft.com/en-us/gallery/store/mongodb/mongodb-inc/ MongoDB solutions on Google Cloud Platform MongoDB, Inc. also announced the arrival of new resources to help Google Cloud Platform customers deploy MongoDB on Google Compute Engine. These resources include a “Click to Deploy” feature and a MongoDB on Google Compute Engine Solutions paper covering MongoDB best practices. If you are looking for a fully-managed solution, with automated provisioning, backups, integrated monitoring and alerting, along with expert support, MongoLab recently announced the arrival of production-ready replica sets on Google. Product Roadmap – MongoDB version 2.8 On the second day of MongoDB World, Eliot Horowitz, MongoDB, Inc. CTO & Co-founder, took center stage and announced two huge changes to the MongoDB core project: document-level locking and pluggable storage engines. These features not only reflect improvements to the core project, but also signal to the community that the MongoDB team is listening to its users and is capable of delivering the software needed to power the workloads of tomorrow. Document-level locking The slides above from Eliot’s keynote point to a current obstacle (database-level locking) in MongoDB that limits overall scalability. With database-level locking, any write operation to the database holds the write lock and prevents subsequent writes from executing on the database until the original operation holding the write lock completes. Eliot’s announcement of document-level locking moves the write lock contention from the database level to the document (MongoDB equivalent to SQL “records”) level. This change will allow users to achieve much higher write throughput (we saw a 10x performance improvement in the live demo) across their MongoDB deployments, improving write scalability. If you’d like to try out document-level locking, the MongoDB team has already pushed the feature to the master branch on GitHub. This should only be used for experimentation, not to be run in production. Pluggable storage engine As MongoDB matures, feature releases like document level locking will continue to allow developers to build robust systems on top of MongoDB. But as the number of use cases grows, different tooling tailored to specific use cases may prove to be extremely beneficial. For example, if Company X decides that they want to use MongoDB to warehouse some of their data, they would likely want to optimize their database for slow-moving data and storage efficiency (compression). With the introduction of pluggable storage engines, many new possibilities are open to the community. Teams can now write their own storage engine for a particular use case, configure replica set nodes with different storage engines for specific situations, or collaborate with the open-source community to architect innovative solutions. This feature not only allows for more granular control of the database, but also encourages the MongoDB community to work together. Takeaways: A maturing and thriving ecosystem Roughly a year ago, MongoLab CTO Todd Dampier recapped MongoSF 2013 and spoke to the health of the MongoDB ecosystem. How far we’ve come! After attending the inaugural MongoDB World and chatting with MongoDB Masters, interns, hackathon winners, power users and those new to the community, the enthusiasm is still surging and as positive as ever. This enthusiasm is well placed. Developers and hackers use MongoDB because so much rich data on the web is shared as JSON (think Facebook, Twitter, Google, etc.). As a result, MongoDB is the de-facto database for hackathons and bootstrapped projects. Just learn the API for the site you want to mine, throw the JSON in MongoDB and query your data with the rich query language- it’s that easy. The MongoDB ecosystem is maturing as well. Take a look at the Customer Success Stories and you’ll get a feel for the extent in which enterprises leverage the solution and use it in production. To further drive enterprise adoption, MongoDB, Inc.’s public cloud solutions and product roadmap features aim to help teams run MongoDB in production and give teams the confidence that MongoDB will continue to improve scalability and meet their growing project requirements. Congratulations again to the MongoDB team on their big announcements and for creating such a fantastic forum at which to learn and meet fellow MongoDB users. Our team at MongoLab had a great time making new friends and talking shop; we look forward to meeting more MongoDB users soon (at a MongoDB Days near you)! -Chris@MongoLab

NOTE: Always Agile Consulting now offers an Introduction To Trunk Based Development training course! Trunk Based Development supports Optimistic and Pessimistic Release Branching Trunk Based Development is a style of software development in which all developers commit their changes to a single shared trunk in source control, and every commit yields a production-ready build. It is a prerequisite for Continuous Delivery as it ensures that all code is continuously integrated into a single workstream, that developers always work against the latest code, and that merge/integration pain is minimised. It is important to note that Trunk Based Development does not prohibit branching. Trunk Based Development is compatible with a Release Branching strategy of short-lived release branches that are used for post-development defect fixes. That Release Branching strategy might be optimistic and defer branch creation until a defect occurs, or be pessimistic and immediately incur branch creation. For example, consider an application developed using Trunk Based Development. The most recent commits to trunk were source revisions a and b which yielded application versions 610 and 611 respectively, and version 610 is intended to be the next production release. With Optimistic Branching, the release of version 610 is immediate as there is no upfront branching. If a defect is subsequently found then a decision must be made where to commit the fix, as trunk has progressed since 610 from a to b. If the risk of pulling forward from a to b is acceptable then the simple solution is to commit the fix to trunk as c, and consequently release version 612. However, if the risk of pulling forward from a to b is unacceptable then a 610.x release branch is created from a, with the fix committed to the branch as c and released as version 610.1. That fix is then merged back into trunk as d to produce the next release candidate 612, and the 610.x branch is earmarked for termination. With Pessimistic Branching, the release of version 610 is accompanied by the upfront creation of a 610.x release branch in anticipation of defect(s). If a defect is found in version 610 then as with Optimistic Branching a decision must be made as to where the defect fix should be committed. If the risk of pulling forward from a to b is deemed insignificant then trunk can be pulled forward from a to b and the fix committed to trunk as c for release as version 612. The 610.x branch is therefore terminated without ever being used. If on the other hand the risk is deemed significant then the fix is committed to the 610.x branch as c and released as version 610.1. The fix is merged back into trunk as d and version 612, which will also receive its own branch upon release. The choice between Optimistic Branching and Pessimistic Branching for Trunk Based Development is dependent upon product quality and lead times. If product quality is poor and lead times are long, then the upfront cost of Pessimistic Branching may be justifiable. Alternatively, if post-development defects are rare and production releases are frequent then Optimistic Branching may be preferable.

The Android system software stack is typically divided into the four areas as the following graphic: Terminology Android Software Development Kit (Android SDK) contains the necessary tools to create, compile and package Android applications Android debug bridge (adb), which is a tool that allows you to connect to a virtual or real Android device Google provides two integrated development environments (IDEs) to develop new applications. Android Developer Tools (ADT) are based on the Eclipse IDE Android Studio based on the IntelliJ IDE Android RunTime (ART) uses Ahead Of Time compilation, and optional runtime for Android 4.4 Android Virtual Device (AVD) - The Android SDK contains an Android device emulator. This emulator can be used to run an Android Virtual Device (AVD), which emulates a real Android phone Dalvik Virtual Machine (Dalvik)- The Android system uses a special virtual machine, Dalvik, to run Java-based applications. Dalvik uses a custom bytecode format which is different from Java bytecode. Therefore you cannot run Java class files on Android directly; they need to be converted into the Dalvik bytecode format.

This post exists to help with an MSDN Magazine article that I am authoring It provides some of the low-level details for the article How to install Mono and root certificates on a raspberry pi How to create an Azure mobile service How to create a Custom API inside Azure mobile services that the raspberry pi can call into How to create an Azure storage account MONO - HOW TO INSTALL ON A RASPBERRY PI Why Mono? How to install Mono on a raspberry pi Installing trusted root certificates on to the raspberry pi http://www.mono-project.com/Main_Page An open source, cross-platform, implementation of C# and the CLR that is binary compatible with Microsoft.NET Mono is a free and open source project led by Xamarin (formerly by Novell) that provides a .NET Framework-compatible set of tools including, among others, a C# compiler and a Common Language Runtime WHY MONO? Because it lets us write .net code compiled on Windows We can simply copy the binary files from Windows to Linux and run it as is From a raspberry pi device, it is possible to use a .net application to take a photo and upload it to Windows Azure storage HOW TO INSTALL ON A RASPBERRY PI RUNNING LINUX You will issue the following commands: pi@raspberrypi ~ $ sudo apt-get update pi@raspberrypi ~ $ sudo apt-get install mono-complete The first command makes sure all the local package index are up to date with the changes made in repositories. Second command installs the complete Mono tooling and runtime. MAKING SURE THAT YOUR MONO APPLICATIONS CAN MAKE A HTTPS REST-BASED CALLS This command downloads the trusted root certificates from the Mozilla LXR web site into the Mono certificate store. Once complete, the Raspberry PI will be capable of making web requests using HTTPS requests within Mono. pi@raspberrypi ~ $ mozroots --import --ask-remove --machine CREATING A NEW AZURE MOBILE SERVICES ACCOUNT The mobile services account is needed to host a Node.js application that provides shared access signatures to raspberry pi devices The shared access signature is needed by the raspberry pi, so that it can directly and securely upload photos to Azure storage STEPS TO CREATE AN AZURE MOBILE SERVICE The steps below will create an Azure mobile service The service will be used to host a Node.js application interacting with a raspberry pi devices We will provision a SQL database, although it will not be used initially FOLLOW THESE STEPS TO CREATE THE MOBILE SERVICE Login into the Azure Portal Select MOBILE SERVICES from the left menu pane at the Azure Portal. In the lower left corner select "+NEW" to create a new Azure Mobile Service. Make sure you've selected, "COMPUTE / MOBILE SERVICE / CREATE." You will now enter a url. We will call this service raspberrymobileservice. For the DATABASE, we will choose "Create a new SQL database instance." The REGION we chose is "West US." The BACKEND is "JavaScript." Click the "->" arrow to proceed to the next screen. In this screen you will "Specify database settings." The NAME of your database will based on the URL you entered previously. In this case, the database is called "raspberrymobileservice_db." You will need to choose a SERVER. We will choose "New SQL database server" from the drop-down list. You will need to provide a SERVER LOGIN NAME and a SERVER LOGIN PASSWORD. Take note of the login you provided as it will be needed later CREATING A CUSTOM API Azure mobile services allows you to create a custom API written in JavaScript that can be called from a raspberry pi device using REST This custom API is really just a Node.js application running in the server CREATING THE API TO RESPOND TO THE DEVICE TRYING TO UPLOAD PHOTOS Now that the service is established, we will turn our attention to creating an API that the device can call into to upload a photo. Login into the Azure Portal Your mobile service will take a few minutes to complete, and you should see the "Ready" flag as the "Status" for your service. Once it is ready you can drill into your service to customize its behavior. Just to the right of the service name, click the right arrow key "->" to drill into the service details. The top menu bar will offer many options, but we are interested in the one titled "API." The API allows you to create a series of node.JS API calls that a device can call into using rest-based approaches. Click on "API." from there, select "CREATE A CUSTOM API." You will be asked to provide an API name. Type in "photos" for the API name. Below you will see a series of drop-down combo boxes that relate to permission. We will keep the default value of "Anybody with the application key." This might not be the best option for all scenarios. You can read more about this here. http://msdn.microsoft.com/en-us/library/azure/jj193161.aspx. Click the checkmark to complete the process. The name of the AP you just created, "Photos," should be visible on the portal interface. To drill into the photos API click on the right arrow key "->". The right arrow key will be just to the right of the name of the API "Photos". At this point you should see a basic script that has been provided by default. We will overwrite this default script with our own script as described in the MSDN Magazine article. CREATING A STORAGE ACCOUNT TO STORE THE PHOTOS Navigate to the portal and create a storage account Create a container for the photos Obtain the: Storage Account Name (you will provide a name) Storage Account Access key (generated for you) Container Name (you will create) CREATING A STORAGE ACCOUNT We will need a storage account so that we can upload photos to it. The steps are well documented here: http://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account/ In our case we call the storage account raspberrystorage. This means that the URL that the device will use to upload photos is https://raspberrystorage.blob.core.windows.net/. As you complete these steps make sure that you choose the storage account location to be the same location as was used for your mobile services account. This avoids any unnecessary latency or bandwidth costs between data centers. Once the storage account is created, we will need to create a container within it. Photos or any blob for that matter, are always stored within a container. To create a container drill into your newly created storage account and select CONTAINERS from the top menu. From there, select CREATE A CONTAINER. The new container dialog box will ask for a name for your container. Take note of the name you provide. We are calling our container ?photocontainer.? When the raspberry pi device uploads photos to the storage account, it will target a specific container, such as the one we just created. You will next be asked to indicate ACCESS rights. To keep things simple we will select access rights of Public Blob. ENTERING APP SETTINGS Rather than hard-code storage account information inside your JavaScript/Node.js applications, you should consider using apps settings inside of the Azure mobile services portal This post also discusses it well: http://blogs.msdn.com/b/carlosfigueira/archive/2013/12/09/application-settings-in-azure-mobile-services.aspx ?The idea of application settings is a set of key-value pairs which can be set for the mobile service (either via the portal or via the command-line interface), and those values could be then read in the service runtime.? NAVIGATING TO APP SETTINGS Navigate to the Azure Mobile Services section of the portal. Drill into the specific service by hitting the arrow below Select from the Configure Menu at the top Scroll down to the very bottom to see app settings Note that we need to enter: - We need to get this from Azure Storage - PhotoContainerName - AccountName - AccountKey We get this information from the Azure Storage Section of the Portal. Note that you need to have provisioned a Storage Account to have this information. How to get the AccountKey with Azure Storage Services Now you can get the access keys HOW NODE.JS WILL ACCESS THE APP SETTINGS You will create a Node.js application inside of Azure Mobile Services See previous steps THE NODE.JS APPLICATION READING APP SETTINGS You will starting by going back to Azure Mobile Services and drill down into your newly minted service We called ours raspberrymobileservice Once you click API, you should see: Notice the app settings are being read on lines 12 to 14.

this post will show how to automate the deployment process of a android application using jenkins continuous integration – to build the project, run the unit tests (if any), archive the built artifacts and run the android lint reports. 1. install jenkins as a windows service navigate to jenkins-ci.org website using an internet browser and download the windows native package (the link is underlined for easy identification) as shown from the right side pane of the download jenkins tab. once the download is complete, uncompress the zip file and click on the jenkins-1.xxx.msi file. proceed through the configuration steps to install the jenkins as a windows service. 2. modify default jenkins port by default jenkins runs on the port 8080. in order to avoid conflict with other applications, the default port can be modified by editing the jenkins.xml found under c:\program files (x86)\jenkins location. as shown below, modify the httpport to 8082. jenkins jenkins this service runs jenkins continuous integration system. %base%\jre\bin\java -xrs -xmx256m -dhudson.lifecycle=hudson.lifecycle.windowsservicelifecycle -jar "%base%\jenkins.war" --httpport=8082 rotate once the modification is saved in jenkins.xml file, restart the jenkins service from the windows task manager->services and right clicking on the jenkins service and choose stop service to stop the service as shown below. once the status of the service changes to stopped, restart the service by right clicking on the jenkins service and choose start service to start the service again. navigate to localhost:8082 to verify if the jenkins restart was successful as shown below – jenkins dashboard will be displayed. note that it takes a while before the jenkins service becomes available. 3. install plugins on the jenkins dashboard, navigate to manage jenkins –> manage plugins as shown in the snapshot below. install the following plugins and restart jenkins for the changes to take effect. git plugin (for integrating git with jenkins) gradle plugin (for integrating gradle with jenkins) android lint plugin (for integration lint with jenkins) 4. configure system on the jenkins dashboard, navigate to manage jenkins –> configure system as shown in the snapshot below. navigate to the global properties section and click on add to add an environment variable android_home as shown in the snapshot below. enter the name as android_home and enter the path of the location where the android sdk is stored on windows. navigate to the jdk section and click on “add jdk” to add the jdk installation as shown in the snapshot below. specify a jdk name, choose the jdk version to install and follow the on-screen instructions to save the oracle login credentials. save the changes. next, proceed to the git section and click on “add git” to add the git installation as shown in the snapshot below. specify git name, specify the path to git executable and save the changes. next, proceed to the gradle section and click on “add gradle” to add the gradle installation as shown in the snapshot below. specify gradle name, choose the appropriate version (at the time of writing, i used gradle 1.10) and save the changes. next, proceed to the email notification section and enter the smtp server details as shown below. click on the advanced button to add the further details required and save the changes. click on “test configuration by sending test e-mail”, enter the test e-mail recipient and click on “test configuration” to see if the email is successfully sent. 5. create a new jenkins job from the jenkins dashboard, click on “new job” to create a new job. enter a name for the job and choose “build a free-style software project” as option and click on ok as shown below. from the new job configuration screen, proceed to the source code management section. save the git credentials by clicking on “add” as shown below and entering the details in the following dialog. save the changes by clicking on “add” as shown below. specify the git repository url for the project, choose the saved credentials from the drop-down list as shown in the snapshot below. save the changes. next, from the build triggers section, select the options desired as shown below and save the changes. proceed to the build section, choose “invoke gradle script” from the drop-down list of choices for “add build step”. choose the appropriate gradle version which is configured, enter the tasks to be built and select the options as desired. save the changes. proceed to the post-build actions section, click on “publish android lint results” from the drop-down list of choices for “add post-build action” and specify the location where the lint results should be stored in the jenkins workspace for the job. similarly, click on “archive the artifacts” from the drop-down list of choices for “add post-build action” and the specify the format of apk files to be archived after every build. additionally, options from advanced section such as “discard all but the last successful/stable artifact to save disk space” could be enabled for saving disk space. click on “e-mail notification” from the drop-down list of choices for “add post-build action” and enter the values for the email recipients as shown below. save the changes. 6. build now once the above configuration steps are complete, click on “build now” under the jenkins –> build android application (or the respective job name) to build the project based on the configuration. the console output has the detailed logs of what steps were initiated by the configuration and the outcome of the entire build. clicking on any successful build outcome shows the artifacts that were archived as part of the build, the change that started the build and the lint results as shown below. thus the entire process of building the project an android application project whenever a scm change is triggered or under another condition, running lint reports, archiving the artifacts built, publishing lint reports and triggering emails to the recipients can be automated with a click of a button through jenkins.