By Alex Komyagin, Technical Services Engineer at MongoDB Setting up Kerberos authentication and SSL encryption in a MongoDB Java application is not as simple as other languages. In this post, I’m going to show you how to create a Kerberos and SSL enabled Java application that communicates with MongoDB. My original setup consists of the following: 1) KDC server: kdc.mongotest.com kerberos config file (/etc/krb5.conf): [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] default_realm = MONGOTEST.COM dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true [realms] MONGOTEST.COM = { kdc = kdc.mongotest.com admin_server = kdc.mongotest.com } [domain_realm] .mongotest.com = MONGOTEST.COM mongotest.com = MONGOTEST.COM KDC has the following principals: [email protected] - user principle (for java app) mongodb/[email protected] - service principle (for mongodb server) 2) MongoDB server: rhel64.mongotest.com MongoDB version: 2.6.0 MongoDB config file: dbpath= logpath= fork=true auth = true setParameter = authenticationMechanisms=GSSAPI sslOnNormalPorts = true sslPEMKeyFile = /etc/ssl/mongodb.pem This server also has the global environment variable $KRB5_KTNAME set to the keytab file exported from KDC. Application user is configured in the admin database like this: { "_id" : "[email protected]", "user" : "[email protected]", "db" : "$external", "credentials" : { "external" : true }, "roles" : [ { "role" : "readWrite", "db" : "test" } ] } Download the Java driver: wget http://central.maven.org/maven2/org/mongodb/mongo-java-driver/2.12.1/mongo-java-driver-2.12.1.jar Install java and jdk: sudo yum install java-1.7.0 sudo yum install java-1.7.0-devel Create a certificate store for Java and store the server certificate there, so that Java knows who it should trust: keytool -importcert -file mongodb.crt -alias mongoCert -keystore firstTrustStore (mongodb.crt is just a public certificate part of mongodb.pem) Copy kerberos config file to the application server: /etc/krb5.conf or ““C:\WINDOWS\krb5.ini“` (otherwise you’ll have to specify kdc and realm as Java runtime options) Use kinit to store the principal password on the application server: kinit [email protected] As an alternative to kinit, you can use JAAS to cache kerberos credentials. Compile and run the Java program javac -cp ../mongo-java-driver-2.12.1.jar SSLApp.java java -cp .:../mongo-java-driver-2.12.1.jar -Djavax.net.ssl.trustStore=firstTrustStore -Djavax.net.ssl.trustStorePassword=changeme -Djavax.security.auth.useSubjectCredsOnly=false SSLApp It is important to specify useSubjectCredsOnly=false, otherwise you’ll get the “No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)” exception from Java. As we discovered, this is not strictly necessary in all cases, but it is if you are relying on kinit to get the service ticket. The Java driver needs to construct MongoDB service principal name in order to request the Kerberos ticket. The service principal is constructed based on the server name you provide (unless you explicitly asked to canonicalize server name). For example, if I change rhel64.mongotest.com to the host IP address in the connection URI, I would be getting Kerberos exceptions No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - UNKNOWN_SERVER)]. So be sure you specify the same server host name as you used in the Kerberos principal (). Adding -Dsun.security.krb5.debug=true to Java runtime options helps a lot in debugging kerberos auth issues. These steps should help simplify the process of connecting Java applications with SSL. Before deploying any application with MongoDB, be sure to read through our Security Checklist which outlines recommended security measures to protect your MongoDB installation. More information on configuring MongoDB Security can be found in the MongoDB Manual. For further questions, feel free to reach out to the MongoDB team through google-groups.

Learn How to Enable Swagger in your Spring Boot Web Application

Some weeks ago I started creating a small JUnit Runner (Oleaster) that allows you to use the Jasmine way of writing unit tests in JUnit. I learned that creating custom JUnit Runners is actually quite simple. In this post I want to show you how JUnit Runners work internally and how you can use custom Runners to modify the test execution process of JUnit. So what is a JUnit Runner? A JUnit Runner is class that extends JUnit's abstract Runner class. Runners are used for running test classes. The Runner that should be used to run a test can be set using the @RunWith annotation. @RunWith(MyTestRunner.class) public class MyTestClass { @Test public void myTest() { .. } } JUnit tests are started using the JUnitCore class. This can either be done by running it from command line or using one of its various run() methods (this is what your IDE does for you if you press the run test button). JUnitCore.runClasses(MyTestClass.class); JUnitCore then uses reflection to find an appropriate Runner for the passed test classes. One step here is to look for a @RunWith annotation on the test class. If no other Runner is found the default runner (BlockJUnit4ClassRunner) will be used. The Runner will be instantiated and the test class will be passed to the Runner. Now it is Job of the Runner to instantiate and run the passed test class. How do Runners work? Lets look at the class hierarchy of standard JUnit Runners: Runner is a very simple class that implements the Describable interface and has two abstract methods: public abstract class Runner implements Describable { public abstract Description getDescription(); public abstract void run(RunNotifier notifier); } The method getDescription() is inherited from Describable and has to return a Description.Descriptions contain the information that is later being exported and used by various tools. For example, your IDE might use this information to display the test results. run() is a very generic method that runs something (e.g. a test class or a test suite). I think usually Runner is not the class you want to extend (it is just too generous). In ParentRunner things get a bit more specific. ParentRunner is an abstract base class for Runners that have multiple children. It is important to understand here, that tests are structured and executed in a hierarchical order (think of a tree). For example: You might run a test suite which contains other test suites. These test suites then might contain multiple test classes. And finally each test class can contain multiple test methods. ParentRunner has the following three abstract methods: public abstract class ParentRunner extends Runner implements Filterable, Sortable { protected abstract List getChildren(); protected abstract Description describeChild(T child); protected abstract void runChild(T child, RunNotifier notifier); } Subclasses need to return a list of the generic type T in getChildren(). ParentRunner then asks the subclass to create a Description for each child (describeChild()) and finally to run each child (runChild()). Now let's look at two standard ParentRunners: BlockJUnit4ClassRunner and Suite. BlockJUnit4ClassRunner is the default Runner that is used if no other Runner is provided. So this is the Runner that is typically used if you run a single test class. If you look at the source ofBlockJUnit4ClassRunner you will see something like this: public class BlockJUnit4ClassRunner extends ParentRunner { @Override protected List getChildren() { // scan test class for methonds annotated with @Test } @Override protected Description describeChild(FrameworkMethod method) { // create Description based on method name } @Override protected void runChild(final FrameworkMethod method, RunNotifier notifier) { if (/* method not annotated with @Ignore */) { // run methods annotated with @Before // run test method // run methods annotated with @After } } } Of course this is overly simplified, but it shows what is essentially done in BlockJUnit4ClassRunner. The generic type parameter FrameworkMethod is basically a wrapper aroundjava.lang.reflect.Method providing some convenience methods. In getChildren() the test class is scanned for methods annotated with @Test using reflection. The found methods are wrapped in FrameworkMethod objects and returned. describeChildren() creates aDescription from the method name and runChild() finally runs the test method. BlockJUnit4ClassRunner uses a lot of protected methods internally. Depending on what you want to do exactly, it can be a good idea to check BlockJUnit4ClassRunner for methods you can override. You can have a look at the source of BlockJUnit4ClassRunner on GitHub. The Suite Runner is used to create test suites. Suites are collections of tests (or other suites). A simple suite definition looks like this: @RunWith(Suite.class) @Suite.SuiteClasses({ MyJUnitTestClass1.class, MyJUnitTestClass2.class, MyOtherTestSuite.class }) public class MyTestSuite {} A test suite is created by selecting the Suite Runner with the @RunWith annotation. If you look at the implementation of Suite you will see that it is actually very simple. The only thing Suite does, is to create Runner instances from the classes defined using the @SuiteClasses annotation. So getChildren() returns a list of Runners and runChild() delegates the execution to the corresponding runner. Examples With the provided information it should not be that hard to create your own JUnit Runner (at least I hope so). If you are looking for some example custom Runner implementations you can have a look at the following list: Fabio Strozzi created a very simple and straightforward GuiceJUnitRunner project. It gives you the option to inject Guice components in JUnit tests. Source on GitHub Spring's SpringJUnit4ClassRunner helps you test Spring framework applications. It allows you to use dependency injection in test classes or to create transactional test methods. Source on GitHub Mockito provides MockitoJUnitRunner for automatic mock initialization. Source on GitHub Oleaster's Java 8 Jasmine runner. Source on GitHub (shameless self promotion) Conclusion JUnit Runners are highly customizable and give you the option to change to complete test execution process. The cool thing is that can change the whole test process and still use all the JUnit integration points of your IDE, build server, etc. If you only want to make minor changes it is a good idea to have a look at the protected methods of BlockJUnit4Class runner. Chances are high you find an overridable method at the right location. In case you are interested in Olaester, you should have a look at my blog post: An alternative approach of writing JUnit tests.

"Lambda Architecture" (introduced by Nathan Marz) has gained a lot of traction recently. Fundamentally, it is a set of design patterns of dealing with Batch and Real time data processing workflow that fuel many organization's business operations. Although I don't realize any novice ideas has been introduced, it is the first time these principles are being outlined in such a clear and unambiguous manner. In this post, I'd like to summarize the key principles of the Lambda architecture, focus more in the underlying design principles and less in the choice of implementation technologies, which I may have a different favors from Nathan. One important distinction of Lambda architecture is that it has a clear separation between the batch processing pipeline (ie: Batch Layer) and the real-time processing pipeline (ie: Real-time Layer). Such separation provides a means to localize and isolate complexity for handling data update. To handle real-time query, Lambda architecture provide a mechanism (ie: Serving Layer) to merge/combine data from the Batch Layer and Real-time Layer and return the latest information to the user. Data Source Entry At the very beginning, data flows in Lambda architecture as follows ... Transaction data starts streaming in from OLTP system during business operations. Transaction data ingestion can be materialized in the form of records in OLTP systems, or text lines in App log files, or incoming API calls, or an event queue (e.g. Kafka) This transaction data stream is replicated and fed into both the Batch Layer and Realtime Layer Here is an overall architecture diagram for Lambda. Batch Layer For storing the ground truth, "Master dataset" is the most fundamental DB that captures all basic event happens. It stores data in the most "raw" form (and hence the finest granularity) that can be used to compute any perspective at any given point in time. As long as we can maintain the correctness of master dataset, every perspective of data view derived from it will be automatically correct. Given maintaining the correctness of master dataset is crucial, to avoid the complexity of maintenance, master dataset is "immutable". Specifically data can only be appended while update and delete are disallowed. By disallowing changes of existing data, it avoids the complexity of handling the conflicting concurrent update completely. Here is a conceptual schema of how the master dataset can be structured. The center green table represents the old, traditional-way of storing data in RDBMS. The surrounding blue tables illustrates the schema of how the master dataset can be structured, with some key highlights Data are partitioned by columns and stored in different tables. Columns that are closely related can be stored in the same table NULL values are not stored Each data record is associated with a time stamp since then the record is valid Notice that every piece of data is tagged with a time stamp at which the data is changed (or more precisely, a change record that represents the data modification is created). The latest state of an object can be retrieved by extracting the version of the object with the largest time stamp. Although master dataset stores data in the finest granularity and therefore can be used to compute result of any query, it usually take a long time to perform such computation if the processing starts with such raw form. To speed up the query processing, various data at intermediate form (called Batch View) that aligns closer to the query will be generated in a periodic manner. These batch views (instead of the original master dataset) will be used to serve the real-time query processing. To generate these batch views, the "Batch Layer" use a massively parallel, brute force approach to process the original master dataset. Notice that since data in master data set is timestamped, the data candidate can be identified simply from those that has the time stamp later than the last round of batch processing. Although less efficient, Lambda architecture advocates that at each round of batch view generation, the previous batch view should just be simply discarded and the new batch view is computed from master dataset. This simple-mind, compute-from-scratch approach has some good properties in stopping error propagation (since error cannot be accumulated), but the processing may not be optimized and may take a longer time to finish. This can increase the "staleness" of the batch view. Real time Layer As discussed above, generating the batch view requires scanning a large volume of master dataset that takes few hours. The batch view will therefore be stale for at least the processing time duration (ie: between the start and end of the Batch processing). But the maximum staleness can be up to the time period between the end of this Batch processing and the end of next Batch processing (ie: the batch cycle). The following diagram illustrate this staleness. Even the batch view is stale period, business operates as usual and transaction data will be streamed in continuously. To answer user's query with the latest, up-to-date information. The business transaction records need to be captured and merged into the real-time view. This is the responsibility of the Real-time Layer. To reduce the latency of latest information availability close to zero, the merge mechanism has to be done in an incremental manner such that no batching delaying the processing will be introduced. This requires the real time view update to be very different from the batch view update, which can tolerate a high latency. The end goal is that the latest information that is not captured in the Batch view will be made available in the Realtime view. The logic of doing the incremental merge on Realtime view is application specific. As a common use case, lets say we want to compute a set of summary statistics (e.g. mean, count, max, min, sum, standard deviation, percentile) of the transaction data since the last batch view update. To compute the sum, we can simply add the new transaction data to the existing sum and then write the new sum back to the real-time view. To compute the mean, we can multiply the existing count with existing mean, adding the transaction sum and then divide by the existing count plus one. To implement this logic, we need to READ data from the Realtime view, perform the merge and WRITE the data back to the Realtime view. This requires the Realtime serving DB (which host the Realtime view) to support both random READ and WRITE. Fortunately, since the realtime view only need to store the stale data up to one batch cycle, its scale is limited to some degree. Once the batch view update is completed, the real-time layer will discard the data from the real time serving DB that has time stamp earlier than the batch processing. This not only limit the data volume of Realtime serving DB, but also allows any data inconsistency (of the realtime view) to be clean up eventually. This drastically reduce the requirement of sophisticated multi-user, large scale DB. Many DB system support multiple user random read/write and can be used for this purpose. Serving Layer The serving layer is responsible to host the batch view (in the batch serving database) as well as hosting the real-time view (in the real-time serving database). Due to very different accessing pattern, the batch serving DB has a quite different characteristic from the real-time serving DB. As mentioned in above, while required to support efficient random read at large scale data volume, the batch serving DB doesn't need to support random write because data will only be bulk-loaded into the batch serving DB. On the other hand, the real-time serving DB will be incrementally (and continuously) updated by the real-time layer, and therefore need to support both random read and random write. To maintain the batch serving DB updated, the serving layer need to periodically check the batch layer progression to determine whether a later round of batch view generation is finished. If so, bulk load the batch view into the batch serving DB. After completing the bulk load, the batch serving DB has contained the latest version of batch view and some data in the real-time view is expired and therefore can be deleted. The serving layer will orchestrate these processes. This purge action is especially important to keep the size of the real-time serving DB small and hence can limit the complexity for handling real-time, concurrent read/write. To process a real-time query, the serving layer disseminates the incoming query into 2 different sub-queries and forward them to both the Batch serving DB and Realtime serving DB, apply application-specific logic to combine/merge their corresponding result and form a single response to the query. Since the data in the real-time view and batch view are different from a timestamp perspective, the combine/merge is typically done by concatenate the results together. In case of any conflict (same time stamp), the one from Batch view will overwrite the one from Realtime view. Final Thoughts By separating different responsibility into different layers, the Lambda architecture can leverage different optimization techniques specifically designed for different constraints. For example, the Batch Layer focuses in large scale data processing using simple, start-from-scratch approach and not worrying about the processing latency. On the other hand, the Real-time Layer covers where the Batch Layer left off and focus in low-latency merging of the latest information and no need to worry about large scale. Finally the Serving Layer is responsible to stitch together the Batch View and Realtime View to provide the final complete picture. The clear demarcation of responsibility also enable different technology stacks to be utilized at each layer and hence can tailor more closely to the organization's specific business need. Nevertheless, using a very different mechanism to update the Batch view (ie: start-from-scratch) and Realtime view (ie: incremental merge) requires two different algorithm implementation and code base to handle the same type of data. This can increase the code maintenance effort and can be considered to be the price to pay for bridging the fundamental gap between the "scalability" and "low latency" need. Nathan's Lambda architecture also introduce a set of candidate technologies which he has developed and used in his past projects (e.g. Hadoop for storing Master dataset, Hadoop for generating Batch view, ElephantDB for batch serving DB, Cassandra for realtime serving DB, STORM for generating Realtime view). The beauty of Lambda architecture is that the choice of technologies is completely decoupled so I intentionally do not describe any of their details in this post. On the other hand, I have my own favorite which is different and that will be covered in my future posts.

In this article we will discuss how we can perform CRUD operations on a backbone model using a REST based HTTP service. Background Earlier we have discussed about the benefits of using backbone.js and we also looked at the backbone models. Link to complete series: BackBone Tutorial – Part 1: Introduction to Backbone.Js BackBone Tutorial – Part 2: Understanding the basics of Backbone Models BackBone Tutorial – Part 3: More about Backbone Models BackBone Tutorial – Part 4: CRUD Operations on BackboneJs Models using HTTP REST Service[^] BackBone Tutorial – Part 5: Understanding Backbone.js Collections[^] BackBone Tutorial – Part 6: Understanding Backbone.js Views[^] BackBone Tutorial – Part 7: Understanding Backbone.js Routes and History[^] In this article we will look at performing the CRUD operations on backbone models using a REST based web service. Using the code The first thing we will do is that we will create a simple REST based web api that can be used to save the data on the server using our simple backbone application. For this I have created a simple database with a single table as: The ID field is configured to auto increment and this is the primary key of the table. so while creating a new model we don’t have to provide this to the server. Now on top of this model, I have written a simple ASP.NET web api that will provide us the RESTful api. This API is configured to run on my local machine at: http://localhost:51377/. The API details are as follows: Create: POST http://localhost:51377/api/values Read: GET http://localhost:51377/api/values/{id} Update: PUT http://localhost:51377/api/values/{id} Delete: DELETE http://localhost:51377/api/values/{id} Once we have the API running, we can start working on our backbone model. We had create the backbone model in our previous article as: var Book = Backbone.Model.extend({ defaults: { ID: "", BookName: "" }, idAttribute: "ID", initialize: function () { console.log('Book has been initialized'); this.on("invalid", function (model, error) { console.log("Houston, we have a problem: " + error) }); }, constructor: function (attributes, options) { console.log('Book\'s constructor had been called'); Backbone.Model.apply(this, arguments); }, validate: function (attr) { if (!attr.BookName) { return "Invalid BookName supplied." } } }); The backbone models inherently supports saving on the server using a restful web api. To save the model using a HTTP REST service, we need to specify the urlRoot in the backbone model. To actually save the model, we can call the save on the backbone model.The save method will trigger the validations and if the validations are successful, it will try to identify the action to be performed i.e. create or update and based on that action, it will use urlRoot and call the appropriate REST API to perform the operation. Let us specify the URL root to enable this model to use our web api service. var Book = Backbone.Model.extend({ defaults: { ID: "", BookName: "" }, idAttribute: "ID", initialize: function () { console.log('Book has been initialized'); this.on("invalid", function (model, error) { console.log("Houston, we have a problem: " + error) }); }, constructor: function (attributes, options) { console.log('Book\'s constructor had been called'); Backbone.Model.apply(this, arguments); }, validate: function (attr) { if (!attr.BookName) { return "Invalid BookName supplied." } }, urlRoot: 'http://localhost:51377/api/Books' }); Now let us try to perform CRUD operations on this model. Create To create a new entity on the server, we need to populate the non identity fields in the model (other than ID in this case) and then call the Save method on the model. // Lets perform a create operation [CREATE] var book = new Book({ BookName: "Backbone Book 43" }); book.save({}, { success: function (model, respose, options) { console.log("The model has been saved to the server"); }, error: function (model, xhr, options) { console.log("Something went wrong while saving the model"); } }); Read To read a single book entity, we need to create the book entity with the identity attribute populated, i.e., the ID of the book we want to read. Then we need to call the fetch method on the model object. // Now let us try to retrieve a book [READ] var book1 = new Book({ ID: 40 }); book1.fetch({ success: function (bookResponse) { console.log("Found the book: " + bookResponse.get("BookName")); } }); Update Now let’s say we want to update the name of the book retrieved in the earlier fetch call. All we need to do is set the attributes we need to update and call the save method again. // Lets try to update a book [UPDATE] var book1 = new Book({ ID: 40 }); book1.fetch({ success: function (bookResponse) { console.log("Found the book: " + bookResponse.get("BookName")); // Let us update this retreived book now (doing it in the callback) [UPDATE] bookResponse.set("BookName", bookResponse.get("BookName") + "_updated"); bookResponse.save({}, { success: function (model, respose, options) { console.log("The model has been updated to the server"); }, error: function (model, xhr, options) { console.log("Something went wrong while updating the model"); } }); } }); Delete Now to delete a Model, we just need to call the destroy method of the model object. // Let us delete the model with id 13 [DELETE] var book2 = new Book({ ID: 40 }); book2.destroy({ success: function (model, respose, options) { console.log("The model has deleted the server"); }, error: function (model, xhr, options) { console.log("Something went wrong while deleting the model"); } }); Custom URLs to perform CRUD operation on models There are few scenarios where we might want to have provide custom URLs for the individual operations. This can be achieved by overriding the sync function and providing custom URL for each action. Let us create one more model BookEx to see how this can be done. var BookEx = Backbone.Model.extend({ defaults: { ID: "", BookName: "" }, idAttribute: "ID", // Lets create function which will return the custom URL based on the method type getCustomUrl: function (method) { switch (method) { case 'read': return 'http://localhost:51377/api/Books/' + this.id; break; case 'create': return 'http://localhost:51377/api/Books'; break; case 'update': return 'http://localhost:51377/api/Books/' + this.id; break; case 'delete': return 'http://localhost:51377/api/Books/' + this.id; break; } }, // Now lets override the sync function to use our custom URLs sync: function (method, model, options) { options || (options = {}); options.url = this.getCustomUrl(method.toLowerCase()); // Lets notify backbone to use our URLs and do follow default course return Backbone.sync.apply(this, arguments); } }); Now we can perform the CRUD operations on this model in the same way as we did for the previous model. Point of interest In this article we have looked at how to perform CRUD operations on backbone models using HTTP based REST service. This has been written from a beginner’s perspective. I hope this has been informative. Download sample Web API code: WebAPISample Download sample backbone app code: backboneSample

This is a post in a series discussing using spring-boot and docker for deployment. Refer to the end of the first post for a table of contents. Shortly after you start building docker containers you will realize that you need some place to publish your images. You could push to the central docker registry. However, the central registry is public. Not a great idea if you are working on a private project. If this is your case, you can simply run a local docker registry. To install and run your private registry run $ docker run -p 5000:5000 -d registry Surprise!!! It is ran in a docker container. You can now start pushing to your local repository. As an example, I will pull the latest postgres image and push version 9.4 to my local registry. $ docker pull postgres $ docker tag postgres:9.4 localhost:5000/postgres:9.4 $ docker push localhost:5000/postgres Outputs: The push refers to a repository [localhost:5000/postgres] (len: 1) Sending image list Pushing repository localhost:5000/postgres (1 tags) 511136ea3c5a: Image successfully pushed ec3443b7b068: Image successfully pushed 06af7ad6cff1: Image successfully pushed 37eae31ff4e9: Image successfully pushed 83e30bf01299: Image successfully pushed 499da968a652: Image successfully pushed bf09bd07d760: Image successfully pushed 1eee820e762b: Image successfully pushed 7bf9287ccfce: Image successfully pushed 288b8d534217: Image successfully pushed f20dbf0acb45: Image successfully pushed bd511e81a5ed: Image successfully pushed 8fe7eb38aea1: Image successfully pushed 464263a50f65: Image successfully pushed 1f58a67adecd: Image successfully pushed a99fb4ee814d: Image successfully pushed 6112f975feab: Image successfully pushed 6dff1b5c2259: Image successfully pushed Pushing tag for rev [6dff1b5c2259] on {http://localhost:5000/v1/repositories/postgres/tags/9.4} Looking at the current images, you will notice that the version tagged with localhost and the official images have the same information. Notice that I had to retag the image with the location of the repository. I thought the requirement to put the location address as part of the image name was a little odd. However, after using docker longer, it makes sense. It ensures you know where the image was originally pulled. $ docker images postgres 9.4 6dff1b5c2259 5 days ago 244.4 MB localhost:5000/postgres 9.4 6dff1b5c2259 5 days ago 244.4 MB Since docker tags are not permanent, and newer version of the postgres:9.4 image could be pushed to the public registry. When you self-host images, you are in control of when updates are pushed to any base image that you have extended. Someday I intend to learn how to build an image completely from scratch. Docker-ize All the Things!

the article represents steps required to configure tomcat 8 with eclipse kepler. download tomcat 8 and place it within any local folder. download eclipse java ee kepler as of date, tomcat 8 is not supported in eclipse javeee kepler. however, you could add the tomcat 8 by doing following: go to the wtp downloads page, select the latest version (currently 3.6), and download the zip. here’s the current link . copy the all of the files in features and plugins directories of the downloaded wtp into the corresponding eclipse directories in your eclipse folder (overwriting the existing files). start eclipse and click on “servers” tab in the workbench. go ahead and try adding a new server. you would find option for tomcat 8 available for selection as shown below. after clicking finish, you would see a new server added with the name as “tomcat v8.0 server at localhost”. start the server. check http://localhost:8080 (provided you installed tomcat 8 and set http port as 8080) interestingly, you would not see the welcome page, but the 404 error page. to get rid of that, double click on ”tomcat v8.0 server at localhost”. in the window that opens up, select “use tomcat installation” and, change deploy path from wtpwebapps to webapps. look at the figure below. restart the server and access http://localhost:8080 . you are all set.

Actuate recently released a new, free BIRT server called the BIRT iHub F-Type. It incorporates all the functionality of BIRT iHub and is limited only by the capacity of output it can deliver on a daily basis. It is ideal for departmental and smaller scale applications. When BIRT F-Type reaches its maximum output capacity, additional capacity can be purchased on a subscription based model. Some of the key features of BIRT iHub F-Type that will help improve your BIRT content applications are: Interactivity – Allow end-users to modify and personalize reports, and answer questions themselves. Scheduling – Automate report generation based on rules and calendar, and then notify users. Sharing – Secure document management and distribution that allows users to only access content/data they are entitled to. Excel Emitter – Export as native Excel (not CSV) with formulas/pivot tables/worksheets/charts. Integration – JavaScript API to embed dynamic reports and visualizations in your web app. Downloading BIRT iHub F-Type Before we get started with the installation process, we need to download BIRT iHub F-Type. There are three downloads available: Windows, Linux, and a VMware image. This blog will cover the Windows installation. If you’re installing either of the other types, you’ll find links to guides for them at the bottom of this blog post. Once you click on your chosen download, you’ll be asked to register. If you’ve already registered, click the “Click to Login” button. If not, fill out the short registration form to get started. Next, read and accept the license agreement. Once you’ve done that, click the checkbox, and a link for the download will appear. Click that to start your download. At this point, you should also receive an email with an activation code. Be sure to check your spam folder if you don’t see it in your inbox. Installing BIRT iHub F-Type After the download is complete, launch the executable file named ActuateBIRTiHubFType.exe. A welcome message will appear. Press Next to continue. You must read and accept the license agreement on the next screen. Choose a destination folder for the installation. The default is C:\Actuate\BIRTiHub. If you have existing BIRT designs that depend on a JDBC database driver, you can optionally specify the folder where these drivers are located. Press Next to continue. Once the installation has finished, press Finish to launch the BIRT iHub F-Type. A desktop shortcut is also created that points to the iHub F-Type URL at http://localhost:8700/iportal. The first time you launch the BIRT iHub F-Type, you will need to activate it. Enter the activation code that you should have received in an e-mail. After entering a valid activation code, you should receive a message that the code was accepted and the BIRT iHub F-Type should start initializing services. Once that has completed, you will be presented with the login screen. The default user name is “administrator” and the password is blank for your first log in. You’ll be able to change this after you have logged in. Press “Log In” to continue. The first time you launch the BIRT iHub F-Type, you will be in tutorial mode which will help you get started loading your BIRT content and required resources. You can bypass the tutorial mode at any time by pressing the “Exit Tutorial” button at the top right. Select a BIRT design (*.rptdesign) file and press the Upload button. If you don’t have a BIRT design, you can download a sample from the link on the same page. The BIRT design file is automatically inspected and if there are any dependent files needed, like images, data files, BIRT report libraries, CSS styles, or other linked BIRT designs, you will be asked to upload those files as well. Once your BIRT design and dependent files are uploaded, your BIRT report will be displayed in the BIRT iHub F-Type and is now ready to explore. Thanks for reading. Now, it’s time to unleash the full power of BIRT into your application. If you have any questions or comments, please feel free to use the comments section below or visit the BIRT iHub F-Type forum. -Virgil For more blogs in the “Introducing BIRT iHub F-Type” series, see the list below: Installing iHub F-Type: Linux | VMWare Image - See more at: http://blogs.actuate.com/introducing-birt-ihub-f-type-installing-on-windows/#sthash.QPJhv2gw.dpuf

I have a small Spring boot based application that uses a Postgres database as a datastore. I wanted to document the steps involved in deploying this sample application to Cloud Foundry. Some of the steps are described in the Spring Boot reference guide, however the guides do not sufficiently explain how to integrate with the datastore provided in a cloud based environment. Spring-cloud provides the glue to connect Spring based applications deployed on a Cloud to discover and connect to bound services, so the first step is to pull in the Spring-cloud libraries into the project with the following pom entries: org.springframework.cloud spring-cloud-spring-service-connector 1.0.0.RELEASE org.springframework.cloud spring-cloud-cloudfoundry-connector 1.0.0.RELEASE Once this dependency is pulled in, connecting to a bound service is easy, just define a configuration along these lines: @Configuration public class PostgresCloudConfig extends AbstractCloudConfig { @Bean public DataSource dataSource() { return connectionFactory().dataSource(); } } Spring-Cloud understands that the application is deployed on a specific Cloud(currently Cloud Foundry and Heroku by looking for certain characteristics of the deployed Cloud platform), discovers the bound services, recognizes that there is a bound service using which a Postgres based datasource can be created and returns the datasource as a Spring bean. This application can now deploy cleanly to a Cloud Foundry based Cloud. The sample application can be tried out in a version of Cloud Foundry deployed with bosh-lite, these are how the steps in my machine looks like once Cloud Foundry is up and running with bosh-lite: The following command creates a user provided service in Cloud Foundry: cf create-user-provided-service psgservice -p '{"uri":"postgres://postgres:[email protected]:5432/hotelsdb"}' Now, push the app, however don't start it up. We can do that once the service above is bound to the app: cf push spring-boot-mvc-test -p target/spring-boot-mvc-test-1.0.0-SNAPSHOT.war --no-start Bind the service to the app and restart the app: cf bind-service spring-boot-mvc-test psgservice cf restart spring-boot-mvc-test That is essentially it, Spring Cloud should ideally take over at the point and cleanly parse the credentials from the bound service which within Cloud Foundry translates to an environment variable called VCAP_SERVICES, and create the datasource from it. There is however an issue with this approach - once the datasource bean is created using spring-cloud approach, it does not work in a local environment anymore. The potential fix for this is to use Spring profiles, assume that there is a different "cloud" Spring profile available in Cloud environment where the Spring-cloud based datasource gets returned: @Profile("cloud") @Configuration public class PostgresCloudConfig extends AbstractCloudConfig { @Bean public DataSource dataSource() { return connectionFactory().dataSource(); } } and let Spring-boot auto-configuration create a datasource in the default local environment, this way the configuration works both local as well as in Cloud. Where does this "cloud" profile come from, it can be created using a ApplicationContextInitializer, and looks this way: public class SampleWebApplicationInitializer implementsApplicationContextInitializer { private static final Log logger = LogFactory.getLog(SampleWebApplicationInitializer.class); @Override public void initialize(AnnotationConfigEmbeddedWebApplicationContext applicationContext) { Cloud cloud = getCloud(); ConfigurableEnvironment appEnvironment = applicationContext.getEnvironment(); if (cloud!=null) { appEnvironment.addActiveProfile("cloud"); } logger.info("Cloud profile active"); } private Cloud getCloud() { try { CloudFactory cloudFactory = new CloudFactory(); return cloudFactory.getCloud(); } catch (CloudException ce) { return null; } } } This initializer makes use of the Spring-cloud's scanning capabilities to activate the "cloud" profile. One last thing which I wanted to try was to make my local behave like Cloud atleast in the eyes of Spring-Cloud and this can be done by adding in some environment variables using which Spring-Cloud makes the determination of the type of cloud where the application is deployed, the following is my startup script in local for the app to pretend as if it is deployed in Cloud Foundry: read -r -d '' VCAP_APPLICATION <<'ENDOFVAR' {"application_version":"1","application_name":"spring-boot-mvc-test","application_uris":[""],"version":"1.0","name":"spring-boot-mvc-test","instance_id":"abcd","instance_index":0,"host":"0.0.0.0","port":61008} ENDOFVAR export VCAP_APPLICATION=$VCAP_APPLICATION read -r -d '' VCAP_SERVICES <<'ENDOFVAR' {"postgres":[{"name":"psgservice","label":"postgresql","tags":["postgresql"],"plan":"Standard","credentials":{"uri":"postgres://postgres:[email protected]:5432/hotelsdb"}]} ENDOFVAR export VCAP_SERVICES=$VCAP_SERVICES mvn spring-boot:run This entire sample is available at this github location:https://github.com/bijukunjummen/spring-boot-mvc-test Conclusion Spring Boot along with Spring-Cloud project now provide an excellent toolset to create Spring-powered cloud ready applications, and hopefully these notes are useful in integrating Spring Boot with Spring-Cloud and using these for seamless local and Cloud deployments.

if you want evidence that the software development industry is susceptible to fashion, just go and take a look at all of the hype around microservices. it's everywhere! for some people microservices is "the next big thing", whereas for others it's simply a lightweight evolution of the big soap service-oriented architectures that we saw 10 years ago "done right". i do like a lot of what the current microservice architectures are doing, but it's by no means a silver bullet. okay, i know that sounds obvious, but i think many people are jumping on them for the wrong reason. i often show this slide in my conference talks, and i've blogged about this before , but basically there are different ways to build software systems. on the one side we have traditional monolithic systems, where everything is bundled up inside a single deployable unit. this is probably where most of the industry is. caveats apply, but monoliths can be built quickly and are easy to deploy, but they provide limited agility because even tiny changes require a full redeployment. we also know that monoliths often end up looking like a big ball of mud because of the way that software often evolves over time. for example, many monolithic systems are built using a layered architecture, and it's relatively easy for layered architectures to be abused (e.g. skipping "around" a service to call the repository/data access layer directly). on the other side we have service-based architectures, where a software system is made up of many separately deployable services. again, caveats apply but, if done well, service-based architectures buy you a lot of flexibility and agility because each service can be developed, tested, deployed, scaled, upgraded and rewritten separately, especially if the services are decoupled via asynchronous messaging. the downside is increased complexity because your software system now has many more moving parts than a monolith. as robert says, the complexity is still there, you're just moving it somewhere else . there is, of course, a mid-ground here. we can build monolithic systems that are made up of in-process components, each of which has an explicit well-defined interface and set of responsibilities. this is old-school component-based design that talks about high cohesion and low coupling, but i usually sense some hesitation when i talk about it. and this seems odd to me. before i explain why, let me quote something from a blog post that i read earlier this morning about the rationale behind a team adopting a microservices approach. when we started building karma, we decided to split the project into two main parts: the backend api, and the frontend application. the backend is responsible for handling orders from the store, usage accounting, user management, device management and so forth, while the frontend offers a dashboard for users which accesses this api. along the way we noticed that if the whole backend api is monolithic it doesn't work very well because everything gets entangled. the blog post also mentions scaling, versioning and multiple languages/frameworks as other reasons to choose microservices. again, there are no silver bullets here, everything is a trade-off. anyway, "everything getting entangled" is not a reason to switch from monoliths to microservices. if you're building a monolithic system and it's turning into a big ball of mud, perhaps you should consider whether you're taking enough care of your software architecture. do you really understand what the core structural abstractions are in your software? are their interfaces and responsibilities clear too? if not, why do you think moving to a microservices architecture will help? sure, the physical separation of services will force you to not take some shortcuts, but you can achieve the same separation between components in a monolith. a little design thinking and an architecturally-evident coding style will help to achieve this without the baggage of going distributed. many of the teams i've spoken to are building monolithic systems and don't want to look at component-based design. the mid-ground seems to be a hard-sell. i ran a software architecture sketching workshop with a team earlier this year where we diagrammed one of their software systems. the diagram started as a strictly layered architecture (presentation, business services, data access) with all arrows pointing downwards and each layer only ever calling the layer directly beneath it. the code told a different story though and the eventual diagram didn't look so neat anymore. we discussed how adopting a package by component approach could fix some of these problems, but the response was, "meh, we like building software using layers". it seems as if teams are jumping on microservices because they're sexy, but the design thinking and decomposition strategy required to create a good microservices architecture are the same as those needed to create a well structured monolith. if teams find it hard to create a well structured monolith, i don't rate their chances of creating a well structured microservices architecture. as michael feathers recently said, " there's a bit of overhead involved in implementing each microservice. if they ever become as easy to create as classes, people will have a freer hand to create trouble - hulking monoliths at a different scale. ". i agree. a world of distributed big balls of mud worries me.

Spring Integration (SI) is a framework enabling a collection of individual applications to integrate together to deliver a business enterprise system. The framework is essentially a lightweight messaging system that enables spring based applications to communicate with one another and supports integration with external systems via declarative adaptors. It is based on the 'filters and pipes' design architecture. A key feature of it is that it achieves this integration in a minimally intrusive way. The framework is built on 3 main components: Messages Encapsulate the data to be transferred from one place to another. They comprise of a header (holds meta data such as message-id, timestamp, etc) and a payload (your data typically in the form of a POJO). Channels Provide a mechanism to transport messages from one endpoint to another. Represents the pipes in the pipes & filters architecture. SI offers two types of channels, namely Pollable and Subscribable Channels. The former rely on consumers to periodically check for messages whereas the latter is directly responsible for notifying registered consumers when messages become available. Endpoints Consumer/Producer of messages. Performs some action based on the payload. Endpoints come in various flavours, each performing a different function. These include Transformers (transform data), Routers (route data), Filters (filter data), Splitter (splits messages), Aggregator (aggregates group of messages into single message), Service Activator (connecting messages to Services) and Channel Adapters (connect channels to external applications). The basic idea behind the SI framework is that applications communicate with each other by sending/receiving messages. These messages would typically contain the information (payload) required by the next application in the process pipeline. The transport of messages from one application to another is performed by Channel components. The Endpoints perform some action based on the payload. This could be routing the messages to another endpoint or processing the payload itself. The objective of this post is to provide an introduction to Spring Integration. To help achieve this, I developed a sample application which will be discussed below. The source for this sample application is available at here. The project was built and run using spring-integration-4.0.0, maven 3.2.1 and jdk1.6. The main dependency is for the relevant spring-integration jar as declared in the pom.xml: org.springframework.integration spring-integration-stream 4.0.0.RC1 I ran the application using the maven exec plugin. This allows me to clean, package and run the application by invoking mvn clean package exec:java -P OnlineShop from the command line. Developing a sample application: Tabernus My goal as usual was to build something very simple which would help me to become familiar with key concepts of this framework and to this end I've knocked up a simple app which does not connect up individual systems but rather invokes methods on a POJO. Extending this to actual working applications shouldn't be too difficult. The scenario I'm going to model revolves around purchasing items from an online store (Tabernus). This store only sells 3 types of items: Books, Music CDs, and software. During a Sale, the owners have decided to apply different discounts based on the item type. In this instance books, music and software benefit from discounts of 5%, 10%, and 15% respectively. The following diagram shows our domain entities. The class diagram shows that a Customer can place an Order comprising of a number of OrderItems which are of type Book, MusicCD or Software. The problem I need to solve is to design a system which can interrogate each Order and apply the correct discount based on the item type. Subsequently it should be able to compute the total cost of the order once the discounts have been applied. To model this using Spring Integration we need the following pipeline The above diagram shows various components most of which can be divided into 2 categories, channels (blue cylinder shapes) and endpoints (rectangular boxes). The exception to this is the Poller component whose purpose is to enable the various endpoints to function correctly and discussion of it will be given later. We'll start off by briefly covering the various stages in this pipeline as indicated by the numbers in red. Following this we will delve deeper into how we build this pipeline using the SI framework. The pipeline is comprised of 6 major stages as reflected by the numbers in the diagram, The Gateway component represents the entry point to the messaging system. All new Orders will be submitted to this component which will in turn wrap them as messages and place them into the channel appropriately named ordersChannel. Using the Splitter component - each Order is decomposed into a collection of it's constituent OrderItem instances. Each of these is wrapped in a Message and placed in the orderItemsChannel. The Router component considers each OrderItem in turn and places it in the relevant channel, e.g. Book items will be placed in the bookItemsChannel etc. This allows us to consider the different item types separately. The ServiceActivator needs to consider messages within each of the 3 channels and calculate the correct discount based on the channel. After completing the calculation for each OrderItem, it will place the OrderItem in the processedItemsChannel. The Aggregator component will collect all OrderItem instances placed in the processedItemsChannel and reconstruct the original Order. This will subsequently be placed in the deliveriesChannel, which represents the end of the pipeline. The Poller Component is required to configure how often the various endpoints will interrogate their respective input channels for messages. To implement the pipeline shown above using the SI framework, we need to implement the various end points. configure the pipeline in an xml file (Shop.xml) - identifying the various channels and endpoints and how they wire up together. At this point I should mention that SI offers 2 approach to configuring your process pipeline, annotations based and xml. In this article I'll be using the latter. Let's start to look at some code. We'll consider each stage described above and show the java implementation of the endpoint and xml configuration required to wire up the components. Step 1 - Gateway To begin with, we need to implement the Client that will invoke the Gateway component to place the Order. The client (OnlineShop.java) is shown below, public class OnlineShop { public static void main(String[] args) { AbstractApplicationContext context = new ClassPathXmlApplicationContext("/META-INF/com/prodcod/shop.xml", OnlineShop.class); Shop shop = (Shop) context.getBean("shop"); final Order order = createOrder(); shop.placeOrder(order); context.close(); } The logic here is quite simple. The client creates a dummy Order and passes this as an argument when it invokes the placeOrder() method on the gateway component. The gateway component referred here as Shop is injected by Spring. The Gateway component looks like: // Gateway component public interface Shop { @Gateway(requestChannel="ordersChannel") void placeOrder(Order order); } As you can see, this is simply an interface, whose implementation will be provided by Spring when it is injected into the client application. This is achieved by the use of the @Gateway annotation which informs Spring that this is a Gateway component and it needs to provide the implementation. Additionally the annotation accepts an attribute, requestChannel which defines the channel on which the Order instance will be placed. The framework does this by simply wrapping our instance of Order within a Message instance and placing it in the channel, 'ordersChannel'. The Gateway component and the 'ordersChannel' are declared as follows in the file shop.xml Step 2 - Splitter The next end point is the Splitter component. Appropriately named, it's role is to take a single message containing a payload of a collection of items and splitting it into a number of messages, each of which contains a single element from the collection. In our case, we want to decompose the Order into it's constituent OrderItem instances. It does this by taking a Message containing the payload of Order from 'ordersChannel' and then processing it before sending messages (each containing an OrderItem instance) to the 'orderItemsChannel'. Our implementation of the splitter is called OrderSplitter and is defined as below, public class OrderSplitter extends AbstractMessageSplitter{ @Override protected Object splitMessage(Message message) { return ((Order)message.getPayload()).getOrderItems(); } } Implementing a splitter is quite easy and involves extending the AbstractMessageSplitter class and overriding the splitMessage() method. This simply takes a message containing the payload of Order and returns it's collection of OrderItems. Step 3 - Router Having decomposed the Order into it's constituent OrderItems, we now need to separate them into groups of Books, MusicCD, and Software. This is achieved using a router. Our implementation of the Router looks like, public class OrderItemRouter { public String routeOrder(OrderItem orderItem) { String channel = ""; if(isBook(orderItem)) { channel = "bookItemsChannel"; } else if(isMusic(orderItem)) { channel = "musicItemsChannel"; } else if(isSoftware(orderItem)) { channel = "softwareItemsChannel"; } return channel; } ..................... ..................... } Nothing too complicated here. For each OrderItem, the method routeOrder() will determine it's item type and return the name of the channel that this message should be sent to. The channel name is returned by the method. Spring will then ensure that the message containing the OrderItem is relayed to the named channel. The configuration for OrderItemRouter looks like, The config identifies that the class OrderItemRouter is a Router component which will consume messages from the orderItemsChannel. Further Spring needs to invoke the method routeOrder() which contains the logic to perform the routing. The channels for each item type are declared as follows Step 4 - ServiceActivator The next step is to calculate the discounted price for each item type and this is performed by a ServiceActivator component. This is implemented as follows public class Shopkeeper { private static final BigDecimal BOOK_DISCOUNT = new BigDecimal(0.05); private static final BigDecimal MUSIC_DISCOUNT = new BigDecimal(0.10); private static final BigDecimal SOFTWARE_DISCOUNT = new BigDecimal(0.15); /** * Performs discount on books * @param bookOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processBooks(OrderItem bookOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(bookOrderItem, BOOK_DISCOUNT); bookOrderItem.setDiscountedPrice(finalPrice); return bookOrderItem; } /** * Performs discount on music * @param musicOrderItem OrderItem comprising of a music item * @return OrderItem with discount price newly calculated */ public OrderItem processMusic(OrderItem musicOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(musicOrderItem, MUSIC_DISCOUNT); musicOrderItem.setDiscountedPrice(finalPrice); return musicOrderItem; } /** * Performs discount on software * @param softwareOrderItem OrderItem comprising of a book item * @return OrderItem with discount price newly calculated */ public OrderItem processSoftware(OrderItem softwareOrderItem){ final BigDecimal finalPrice = calculateDiscountedPrice(softwareOrderItem, SOFTWARE_DISCOUNT); softwareOrderItem.setDiscountedPrice(finalPrice); return softwareOrderItem; } } This class exposes 3 methods to compute the new discounted price for each item type. Each method returns the OrderItem instance with the new price. The ServiceActivator is configured as follows: This tells Spring that the Shopkeeper class is a ServiceActivator and will consume messages from any of the 3 channels defined in the input-channel attribute. When a message appears in one of these channels, Spring will invoke the appropriate method on the ServiceActivator class as specfied by the attribute method. Anything returned from all three methods will be placed in the processedItems channel, ready for the next step of the processing pipeline. Step 5 - Aggregator The final stage is to take the individual OrderItems with their newly computed discounted prices and reconstruct the Order. This is achieved using an aggregator. Our implementation of an aggregator is listed below public class OrderCompleter { public Order prepareDelivery(List orderItems) { final Order order = new Order(); order.setOrderItems(orderItems); return order; } } The aggregator exposes a method that takes a collection of OrderItem objects. These will come from the processedItems channel declared as Recall this is the output channel for the service activator class as discussed above. The aggregator is configured in the xml file as The configuration tells Spring that the aggregator component will consume messages from the processedItems channel. These will be processed by the method prepareDelivery on the class OrderCompleter. Any output from this class will be relayed to the channel-adaptor deliveries, which is declared as The stdout-channel-adapter component writes to the systems STDOUT output stream. Step 6 - Poller To complete the setup we have to configure a poller component. This is required to enable the channels to work correctly. All our channels are of a queue type and so their respective consumers need to know when to query them. This is achieved using a poller mechanism. It is configured in the following way In this case, we have declared a global poller (as indicated by the default attribute). This will be used by the various end points to determine when they should interrogate their respective input-channels for messages. The second attribute fixed-delay is used to configure the polling interval. Running the Application Building and running the app shows the following output: The logging shows that the Customer submitted an Order for 3 items, one of each type. All items cost £100 each. The Order was then split into 3 OrderItems each of which was routed to the correct processing channel based on the item type. The ServiceActivator (Shopkeeper) then calculated the discount for each item and this was set on the OrderItem instance. The OrderItems were then aggregated using the OrderCompleter class which displays the final discounted price of £270 to be paid by the Customer. Note that the messages are logged to be in different stages of the processing pipeline despite starting off in the same order. This completes the tutorial on the Spring Integration Framework. Any comments relating to corrections, omissions, etc are welcome.

This is the third blog in C2B2 series looking at Glassfish 4. The previous two are available here: Part 1 - Getting started with Glassfish 4 Part 2 - Glassfish 4 - Features For High Availability In this blog I will look at 3 areas: Performance Tuning, where I will look at some of the areas to look at when setting up a system for production. Monitoring, where I will look at some of the tools we use for monitoring a system both during performance testing and tuning and once a system is up and running. Troubleshooting, where I will look at some of the tools you can use to help diagnose and detect performance issues. Performance Tuning Glassfish out of the box (as with most app servers) is optimised for development purposes. Developers want the ability to deploy and undeploy continuously, create and remove resources, debug, etc. However, this configuration is not suitable for a production system. When configuring any application server you have to take into account what you are trying to achieve and what is best suited for the applications you intend to run. One size does not fit all! It can be a long and complex process and I'm afraid I can't give you a one-stop solution. However, I can give you some pointers to some of the things you can do to prepare your system for production. So, what kind of things do we look at when we are looking to performance tune a Glassfish system. Some of the most common things are: JVM Settings Garbage Collection Glassfish Settings Logging JVM Settings The standard JVM defaults are not suitable for a production system. One of the simplest changes that can be made is to use the -server flag, rather than the default -client. Although the Server and Client VMs are similar, the Server VM has been specially tuned to maximise peak operating speed. It is intended for executing long-running server applications, which need the fastest possible operating speed more than a fast start-up time or smaller runtime memory footprint. Allocate more memory to the JVM by modifying the value of the -Xmx flag. How much depends on the size and complexity of your enterprise application and how much memory you have available. In addition we also want to make sure we allocate all of the memory on startup. This is done with the -Xms flag. We set the minimum and maximum perm gen to the same value in order to avoid allocation failures & subsequent full garbage collections. Garbage Collection There are a number of settings that can be tweaked regarding Garbage Collection. I'm not going to cover GC tuning as that is a whole topic all of it's own but here are some of the settings we would always recommend regarding GC in a production environment: Firstly we want to ensure we log all Garbage Collection information as this can prove extremely useful in diagnosing issues. -verbose:gc Next we want to make sure we log GC information to a file. This will make it easier to separate the GC from other details in the log files. -Xloggc:/path_to_log_file/gc.log We also want to ensure we have as much detail as possible. -XX:+PrintGCDetails and that the information is timestamped for easier diagnosis of long running errors and to be able to ascertain what normal levels are over time. -XX:+PrintGCDateStamps Finally, we want to ensure that developers aren't making explicit calls to System.gc(). Hopefully they don’t anyway and if they are you need to look into why (doing so is a bad idea since this forces major collections) but this will disable it just in case. -XX:+DisableExplicitGC Heap Dumps Heap dumps can be extremely useful for diagnosing memory issues. There are two settings we would definitely recommend. These tell the JVM to generate a heap dump when an allocation from the Java heap or the permanent generation cannot be satisfied. There is no overhead in running with these options but they can be useful for production systems where OutOfMemoryErrors can take a long time to surface. -XX:-HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/dumps/glassfish.hprof Configuring Glassfish There are three ways to configure Glassfish: Through the admin console By directly editing the config files Using the asadmin tool Although making changes through the admin console can often be the easiest way to make changes we’d recommend where possible to script all changes so you have a repeatable production server build. Also you should ensure copies of all config files are kept in Config Control so you know you have a working copy and can roll back to a previous version when needed. Turn off development features Turn off auto-deploy and dynamic application reloading. Both of these features are great for development, but can affect performance. Configure the JSP servlet not to check JSP files for changes on every request. Also, set the parameter genStrAsCharArray to true. This will ensure all String values are declared as static char arrays. One reason for this is that the array has less memory overhead than String. These changes will mean you cannot change JSP pages on your production server without redeploying the application, but on a production system this is generally what you want. Acceptor Threads and Request Threads There are two main thread values we would recommend setting, acceptor threads and request threads. Acceptor threads are used to accept new connections to the server and to schedule existing connections when a new request comes in. Set this value equal to the number of CPU cores in your server. So, if you have two quad core CPUs, this value should be set to eight. Request threads run HTTP requests. You want enough of these to keep the machine busy, but not so many that they compete for CPU resources which would cause your throughput to suffer greatly. Static resources By default, GlassFish does not tell the client to cache static resources. It is recommended to cache static resources, like CSS files and images particularly if you have a lot of them. Thread pools Max thread pool and min pool size should be set to the same value. Specifying the same value will allow GlassFish to use a slightly more optimised thread pool. This configuration should be considered unless the load on the server varies significantly. Increasing this value will reduce HTTP response latency times. What to set these values to depends heavily on what your application is doing. In order to get this value right you should look to incrementally increase the thread count and to monitor performance after each incremental increase. When performance stops improving stop increasing the thread count. Logging You should look to turn off as much logging as possible. In a production environment we would generally recommend logging at WARN and above. This includes the logging done by Glassfish as well as your own applications. Monitoring The fewer monitoring options that are enabled, the better the server's performance. All Glassfish monitoring is turned off by default. Switching monitoring on can be very useful when diagnosing issues and when doing initial system testing and performance tuning for monitoring what changes. What to monitor Used Heap Size - Compare this number with the maximum allowed heap size to see what portion of the heap is in use. If the used heap size nears the max heap size, the garbage collector urgently attempts to free memory and this is something that should be avoided where possible. Number of loaded classes - Useful for detecting performance and application development trends. JVM Threads - Important for performance tuning and for troubleshooting JVM crashes. Some of the most essential indicators are the current active JVM thread count and the peak values. Thread pools - You should compare a pools current usage with the maximum number allowed. Problems can start to occur when the current count nears the max threads number. JVM Tools for Monitoring The following is a list of a a few of the tools that come with the JDK that are useful for monitoring information from the JVM. jstat - This tool displays performance statistics regarding usage of the perm gen, new gen and old gen. It also provides class loading and compilation statistics jmap - Gives you visibility of memory usage, can produce a class histogram and can dump the memory to a file jconsole/jvisualvm - These tools can display all the previously mentioned monitoring indicators and graph them over time. This allows you to spot trends and to get a better overall picture of your normal performance levels and changes over time. Note - These should NOT be left running permanently on a production system! Troubleshooting Unfortunately, no matter how much tuning and testing you do all systems WILL go wrong from time to time. So, what should you do when your production server bursts into flames? Well, in that situation you should call the fire service but for more general problems: Gather data - get as much data as you can, there is no such thing as too much! Analyse that data - Data is worthless when you don’t know what it means. Visualise where possible – graphs and charts reveal trends and patterns over time Make educated decisions - Only make decisions based on data. If you go with your “gut instinct” and what “feels right” you will probably make things worse Gathering data First up, for most of the JVM tools you will need the process ID of the server. You can get this information in various ways. Two of the simplest are: jps -v This will list all current running Java processes. The -v flag is for verbose output. ps aux | grep glassfish The ps command with the options aux will show all processes from all users. This will display a LOT of information so pipe it through grep to filter for the glassfish process As mentioned earlier the jstat tool can be used for gathering info on JVM performance. Other useful tools include: jstack This will produce thread stack dumps for all threads running in the JVM. This can be very useful for discovering stuck threads or long running threads. jmap This tool can be used to create a heap dump. It outputs to a file in .hprof format which can be read by a number of analysis tools jrcmd and jrmc These tools are only available with the jRockit JDK. I won't go into any detail here as I have previously blogged about jrcmd here: http://blog.c2b2.co.uk/2012/11/troubleshooting-jrockit-using-jrcmd.html and my colleague has blogged about jrmc here: http://blog.c2b2.co.uk/2012/10/weblogic-troubleshooting-with-jrockit.html Glassfish asadmin The Glassfish asadmin tool has a built in command which will provide similar functionality to the above tools but without the need for the PID. asadmin generate-jvm-report --type=[type] Analysing the data There are various tools available for analysing performance data. The following are some of the most useful: IBM Support Assistant is a free troubleshooting application that helps you research, analyze, and resolve problems using various support features and tools. It contains a Garbage Collection and Memory Visualiser as well as a Heap Analyser. It will also provide a report telling you where issues might exist, and listing red flags with advice on what to change in your applications jRockit Mission Control is a very powerful tool which can be used to monitor live systems or analyse historical data in the form of flight recordings. JVisualVM GCViewer is an optional plugin for jVisualVM which can transform a tool which is already great for live monitoring into a powerful analysis tool jhat is a Java Heap Analysis Tool. It processes heap dump files and produces HTML reports. There are better analysis tools, but it’s always freely available if you’re running a JDK. Others There are many open source and freely available tools and projects to help you, here we’ve covered some very common and widely used ones, but our list is by no means exhaustive! Conclusion Remember, Glassfish out of the box (or out of the zip file!) is not designed to be run 'as is'. You should also note that there is no ideal configuration that will work for all systems. It will take time and effort to get the best configuration for what you require. Hopefully in this blog I have given you some useful guidelines and pointers. You should take time to work out what you want in terms of services, then strip back your config to match that. You should test, test and test again to ensure that your configuration matches the requirements with regards to the applications you will be running on your server. You should tune your JVM to ensure you have the best settings for your particular configuration. You should ensure you have monitoring in place to keep a check on everything and ensure that if your server does crash you have as much information as possible at hand to diagnose what caused it. The next blog in this series looks at Migrating to Glassfish 4: http://blog.c2b2.co.uk/2013/07/glassfish-4-migrating-to-glassfish.html

In this blog, we will discuss one particular data grid platform from Redhat namely JBoss Data Grid (JDG). We will firstly cover how to access and install this data grid platform and then we will demonstrate how to develop and deploy a simple remote client/server data grid application which utilises the HotRod protocol. We will be using the latest release JDG 6.2 from Redhat in this article. Installation Overview To start using JDG, firstly log on to the redhat site https://access.redhat.com/home and download the software from the Downloads section of the site. We wish to download JDG 6.2 server by clicking on the appropriate links in the Downloads section. For future reference, it is also useful to download the quickstart and maven repository zip files. To install JDG, we simply unzip the JDG server package into an appropriate directory in your environment. JDG Overview In this section, we will provide a brief overview of the contents of the JDG installation package and the most notable configuration options available to users. Out of the box, users are provided with two runtime options either to run JDG in standalone or clustered mode. We can start JDG in either mode by invoking the stanadalone or clustered start up scripts in the / bin directory. To configure the JDG in either mode we need to configure the files standalone.xml and clustered.xml. In our case we will creating a distributed cache which will run on 3 node JDG cluster so we will be utilizing the clustered startup script. In order to set up and add new cache instances to JDG, we modify the infinispan subsystems in the appropriate xml configuration file above. We should also note the principal difference between the standalone and clustered configuration file is that in the clustered configuration file there is a JGroups subsystem configured element which allows for communication and messaging between configured cache instances running in a JDG cluster. Development Environment Setup and Configuration In this section, we will detail how to develop and configure a simple datagrid application which will be deployed to a 3 node JDG cluster. We will demonstrate how to configure and deploy a distributed cache in JDG and also show how to develop a HotRod Java client application which will be used to insert, update and display entries in the distributed cache. We will firstly discuss setting a new distributed cache on a 3 node JDG cluster. In this example, we will run our JDG cluster on a single machine by running each JDG instance on different ports. Firstly, we will create 3 instances of JDG by creating 3 directories (server1, server2, server3) on our host machine and unzipping each JDG installation into each directory. We will now configure each node in our cluster by copying and renaming the clustered.xml configuration file in the \server1\jboss-datagrid-6.2.0-server\standalone\configuration directory. We will name each of the cluster configuration files as "clustered1.xml", "clustered2.xml" and "clustered3.xml" for the JDG instances denoted by "server1", "server2" and "server3" respectively. We will now set up a new distributed cache on our JDG cluster by modifying the infinispan subsystem element in each clustered.xml file. We will demonstrate this for the node denoted "server1" here by modifying the file "clustered1.xml". The cache configuration shown here will be the same across all 3 nodes. To setup a new distributed cache named "directory-dist-cache", we configure the following elements in the file named "clustered1.xml" ......... ...... .............. ...... ...... /socket-binding-group> We will discuss the key elements and attributes relating to the configuration above. In the infinispan endpoint subsystem, we will configure hotrod clients to connect to the JDG server instance on socket 11222. The name of the cache container to host each of the cache instances will be held in the container named "clusteredcache". We have configured the infinispan core subsystem to the default cache container named "clusteredcacahe" whereby we will allow for jmx statistics to be collected relating the configured cache entries i.e statistics="true" We have created a new distributed cache named "directory-dist-cache" whereby there will be two copies of each cache entry held on two of the 3 cluster nodes. We have also set up an eviction policy whereby should there be more than 20 entries in our cache then cache entries will be removed using the LRU algorithm We should have configured nodes "server2" and "server3" to start up with a port offset of 100 and 200 respectively by configuring the socketing binding group element appropriately. Please view the socket bindings noted below. To set the socket binding element with a port offset of 100 on "server2", we configure "clustered2.xml" with the following entry: ...... ...... /socket-binding-group> To set the socket binding element with a port offset of 200 on "server3", we configure "clustered3.xml" with the following entry: ...... ...... /socket-binding-group> Before discussing the setup and configuration of our Hotrod client which will be used to interact with our JDG clustered HotRod server, we will start up each server instance to ensure our newly configured JDG distributed cache starts up correctly. Open up 3 Windows or Linux consoles and execute the following start up commands: Console 1: 1) Navigate to \server1\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the first instance of our JDG cluster denoted "server1": clustered -c=clustered1.xml -Djboss.node.name=server1 Console 2: 1) Navigate to \server2\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the second instance of our JDG cluster denoted "server2": clustered -c=clustered2.xml -Djboss.node.name=server2 Console 3: 1) Navigate to \server3\jboss-datagrid-6.2.0-server\bin 2) Execute this command to start the third instance of our JDG cluster denoted "server3": clustered -c=clustered3.xml -Djboss.node.name=server3 Providing all 3 JDG instances have started up correctly, you should see output in the console window whereby we can see there are 3 JDG instances in the JGroups view: HotRod Client Development Setup Now that the Hotrod server is up and running, we need to develop a Hotrod Java client which will interact with the clustered server application. The development environment consists of the following tools. 1) JDK Hotspot 1.7.0_45 2) IDE - Eclipse Kepler Build id: 20130919-0819 The HotRod client application is a simple application consisting of two Java classes. The application allows users to retrieve a reference to the distributed cache from the JDG server and then perform these actions: a) add new cinema objects. b) add and remove shows to each cinema object. c) print the list of all cinemas and shows stored in our distributed cache. The source code can be downloaded from github @ https://github.com/davewinters/JDG. We could use maven here to build and execute our application by configuring the maven settings.xml to point to the maven repository files we downloaded earlier and set up a maven project file (pom.xml) to build and execute the client application. In this article we will build our application using the Eclipse IDE and run the client application on the command line. To create a HotRod client application and execute the sample application, one should complete the following steps: 1) Create a new Java Project in Eclipse 2) Create a new package named uk.co.c2b2.jdg.hotrod and import the source code that has been downloaded from Github mentioned previously. 3) Now we need to configure the build path in Eclipse to contain the appropriate JDG client jar files which are required to compile the application. You should include all the client jar files in the project build path. These jar files are contained in the JDG installation zip file. For example on my machine these jar files are located in the directory: \server1\jboss-datagrid-6.2.0-server\client\hotrod\java 4. Providing the Eclipse build path has been configured appropriately, the application source should compile without issue. 5. We will need to execute the Hotrod application by opening the console window and executing the following command. Note the path specified here will differ depending on where the JDG client jar files and application class files are located in your environment: java -classpath ".;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\commons-pool-1.6-redhat-4.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-client-hotrod-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-commons-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-query-dsl-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\infinispan-remote-query-client-6.0.1.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-logging-3.1.2.GA-redhat-1.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-marshalling-1.4.2.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\jboss-marshalling-river-1.4.2.Final-redhat-2.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\protobuf-java-2.5.0.jar;C:\Users\David\Installs\jbossdatagrids62\server1\jboss-datagrid-6.2.0-server\client\hotrod\java\protostream-1.0.0.CR1-redhat-1.jar" uk/co/c2b2/jdg/hotrod/CinemaDirectory 6. The Hotrod client at runtime provides the end user with a number of different options to interact with the distributed cache as we can view from the console window below. Client Application Principal API Details We will not provide a detailed overview of the Hotrod application code however we will describe the principal API and code details briefly. In order to interact with the distributed cache on the JDG cluster using the Hotrod protocol, we will use the RemoteCacheManager Object which will allow us to retrieve a remote reference to the distributed cache. We have initialised a Properties object with the list of JDG instances and the associated with HotRod server port on each instance. We can add Cinema objects into the distributed cache using the RemoteCache.put() method. private RemoteCacheManager cacheManager; private RemoteCache cache; ..... Properties properties = new Properties(); properties.setProperty(ConfigurationProperties.SERVER_LIST, "127.0.0.1:11222;127.0.0.1:11322;127.0.0.1:11422"); cacheManager = new RemoteCacheManager(properties); cache = cacheManager.getCache("directory-dist-cache"); ..... cache.put(cinemaKey, cinemalist); In the webinar below, I describe in further detail how to set up a JDG cluster and how to develop and run the JDG application discussed above. For further details on JDG please visit: http://www.redhat.com/products/jbossenterprisemiddleware/data-grid/ Webinar: Introduction to JBoss Data Grid -- Installation, Configuration and Development In this webinar we will look at the basics of setting up JBoss Data Grid covering installation, configuration and development. We will look at practical examples of storing data, viewing the data in the cache and removing it. We will also take a look at the different clustered modes and what effect these have on the storage of your data:

a powerful multi-format xml schema (xsd) documentation generator and a tool for rapid development of custom xsd documentation generators according to user needs. about docflex/xml "xsddoc" template set template processor template designer integrations generation of xsd diagrams apache ant & maven links about docflex/xml docflex/xml is a java-based software system for development and execution of high performance template-driven documentation generators from any data stored in xml files. the actual doc/report generators are programmed in the form of special templates using a graphic template designer , which represents the templates visually in a form resembling the output they generate. further, the templates are interpreted by a template processor , which takes on input the xml files and produces by them the result documentation. this article describes an application of docflex/xml for the task of generation of high-quality xml schema documentation. that includes the following features of docflex/xml system: " xsddoc " template set that implements the ready-to-use xml schema documentation generator itself. template processor makes the templates works. currently, it provides three interchangeable output generators for html, rtf, txt (plain text) formats. template designer provides a high quality gui to design/modify templates. if you need a special xml schema doc generator, the simplest way to create it is to modify the standard xsddoc templates. the template designer enables you to do that. integrations with altova xmlspy and oxygen xml editor . if you are a user of one of those popular xml editors, you can turn it also into a dynamically linked diagramming engine for docflex, so that to include automatically the xsd diagrams generated by xmlspy/oxygenxml into the xml schema documentation generated by docflex (with the full support of hyperlinks). "xsddoc" template set it is the implementation of xml schema documentation itself, which provides the following functionality: generation of single documentation by any number of xml schema (xsd) files together, in particular: highly navigable framed (javadoc-like) html documentation single-file html documentation rtf documentation (further convertible to pdf) processing of any referenced xml schemas, in particular: correct processing of all , , elements found across all involved xsd files. automatic loading and processing (i.e. inclusion in the documentation scope) all directly/indirectly referenced xsd files. sophisticated documenting of xsd components , including: component diagrams (with hyperlinks to everything depicted on them; see also integrations ) xml representation summary (a textual alternative to diagrams) lists of related components. for elements this includes also the list of possible containing elements . (such a list is never present in the output generated by xslt-based doc generators). list of usage locations support of any xml schema design patterns . this comes down mainly to the following: special treatment of local elements (see below) support and documenting of substitution groups support of importing, inclusion and redefinition of schema files special documenting of local elements . local elements are those components that are declared locally within other xsd components. w3c xml schema spec allows you to declare any number of local elements that may share the same name but have different content. that's because their meaning is local and there will be no collisions with other declarations. that, however, creates a problem for documenting, because in a documentation both global and local elements may appear simultaneously in various lists according to their common properties. if each element component is identified only by its name, you will get the lists with multiple repeating names but little clue what they mean. moreover, some xml schemas may contain lots of identical local element declarations (that is, they have the same both name and content). so, you'll get in those lists a mess of repeating names, some of which referencing to effectively the same entities, whereas others to complete different ones. in xsddoc , those problems are solved in two ways: adding extensions to local element names. the extension provides more information about the element (e.g. where it can be inserted or its global type or where it is defined). that makes the whole string identifying the element unique. here is how it looks. the grey text is the name extension: unifying local elements by type. on the left you can see a documentation generated with such unification. on the right, all local elements are documented straight as they are. click on each screenshot to view the docs: we believe the first documentation (on the left) is easier to understand and use. processing of xhtml markup . you can format your xml schema annotations with xhtml tags, which will be recognized and rendered with the appropriate formatting in both html and rtf output, as shown on the following screenshots (click to see more details): here, on the left you can see the xml source of an xml schema, whose annotations are heavily laden with xhtml markup (including insertion of images). the next is the html documentation generated by that schema. on the right is a page of rtf documentation also generated by that schema. possibility of unlimited customization : xsddoc is controlled by more than 400 parameters, which allow you to adjust the generated documentation within huge range of included details. template parameters serve the same role as options in traditional doc generators. the difference is that docflex template architecture makes the support/implementation of template parameters very cheap (typically, the most of efforts takes writing their descriptions). so, there may be hundreds of parameters controlling a large template application. if parameters are not enough, you can modify the templates themselves using the template designer . in case of html output, you can also apply your own css styles to change how the generated documentation looks. template processor the template processor (also called simply "generator") makes everything work. it consists of two logical parts: 1. template interpreter 2. output generator the output generator actually has three different implementations for each currently supported output format: html, rtf, txt (plain text). the plain-text output can be used to generate documentation in formats not supported directly by docflex. the template processor is started directly from java command line with the following arguments: ● main template ● template parameters ● initial xsd files to be processed (documented) ● xml catalogs (to redirect physical location of input files) ● destination directory/file ● output format (this selects which output generator will be used) ● output format options (specify settings to control the selected output generator) actually, the number of settings may be so large that the template processor provides a special gui to specify everything interactively (click to enlarge): template designer although docflex templates are stored as plain-text files (with an xml-like format), they are not supposed for editing manually. rather, a special graphic template designer must be used, which visualizes the templates in the form of template components they are made of. those components are the actual constructs of the template language (not some textual statements, operators, blocks etc.) the following screenshots show templates open in the template designer (click to see a lot more): that approach has a number of advantages, among them: the processing structures represented by template components may be displayed in a way that visually expresses what a component does (for instance, it may resemble the output it generates). that representation may be both expressive and compact (after all, it is not just a text), which allows you easily to navigate a template, understand what it does and modify anything you need. as template components are visual and interactive, they may have very complex internal structure, for instance, contain lots of properties and nested components. at that, you don't need to scroll and navigate some kind of enormous text, which encodes all of this (as it would be in case of a script). rather, you just need to invoke some property dialogs and expand/collapse some component sections. a template component may be easily copied, pasted and deleted as a whole. at that, you don't need to bother that the template syntax is restored after that. the template designer will also ensure that each component is created, copied or moved only in the allowed place. the highly structured nature of templates eliminates the need for most of various named identifiers. many connections between different template components are also maintained by the template designer (i.e. modified automatically when necessary). as template files are stored and read only programmatically, there is no need to know and understand their syntax. there will be no syntax errors either. the actual syntax of template files may be optimized not for human programmers, but for faster loading and processing of templates by the template processor . there is no need in a compilation phase. the separation of template semantics from the particular structure of template files helps for faster and easier evolution of the template language. the obsolete constructs of older template versions can be automatically converted into new structures. both old and new templates will look and work up-to-date. integrations generation of xsd diagrams docflex/xml is able to work with any kind of diagrams (i.e. inserting them automatically in the generated output). that is supported on the level of templates, along with the generation of hypertext imagemaps, as shown on the following screenshot (click to see a lot more): docflex/xml provides no diagramming engine of its own. instead, it includes integrations with two most popular xml editors that do generate xsd diagrams: ● altova xmlspy ● oxygen xml editor effectively, the third-party software is used as dynamically linked diagramming engine. the advantage of such integrations is that when you are the user of one of those xml editors, you will get in the documentation generated by docflex the same diagrams as you see in your xml editor. here is how such a documentation with diagrams looks (click on a screenshot to view the real html): apache ant & maven as a pure java application, docflex/xml can be run in any environment that runs java itself. the template processor can be easily integrated with ant (that can be specified just in the ant build file). in case of maven, docflex/xml includes a simple maven plugin. it is possible also to use all diagraming integrations with both ant and maven. links docflex/xml (home page): http://www.filigris.com/docflex-xml/ docflex/xml xsddoc: http://www.filigris.com/docflex-xml/xsddoc/ xsddoc examples: http://www.filigris.com/docflex-xml/xsddoc/examples/ xmlspy integration: http://www.filigris.com/docflex-xml/xmlspy/ oxygenxml integration: http://www.filigris.com/docflex-xml/oxygenxml/ free downloads: http://www.filigris.com/downloads/ this original article: http://www.filigris.com/ann/docflex-xsd/

This is a sample that I have wanted to try for sometime - A Websocket application to tail the contents of a file. The following is the final view of the web-application: There are a few parts to this application: Generating a File to tail: I chose to use a set of 100 random quotes as a source of the file content, every few seconds the application generates a quote and writes this quote to the temporary file. Spring Integration is used for wiring this flow for writing the contents to the file: Just a quick note, Spring Integration flows can now also be written using a Java Based DSL, and this flow using Java is available here Tailing the file and sending the content to a broker The actual tailing of the file itself can be accomplished by OS specific tail command or by using a library like Apache Commons IO. Again in my case I decided to use Spring Integration which provides Inbound channel adapters to tail a file purely using configuration, this flow looks like this: and its working Java equivalent There is a reference to a "fileContentRecordingService" above, this is the component which will direct the lines of the file to a place where the Websocket client will subscribe to. Websocket server configuration Spring Websocket support makes it super simple to write a Websocket based application, in this instance the entire working configuration is the following: @Configuration @EnableWebSocketMessageBroker public class WebSocketDefaultConfig extends AbstractWebSocketMessageBrokerConfigurer { @Override public void configureMessageBroker(MessageBrokerRegistry config) { //config.enableStompBrokerRelay("/topic/", "/queue/"); config.enableSimpleBroker("/topic/", "/queue/"); config.setApplicationDestinationPrefixes("/app"); } @Override public void registerStompEndpoints(StompEndpointRegistry registry) { registry.addEndpoint("/tailfilesep").withSockJS(); } } This may seem a little over the top, but what these few lines of configuration does is very powerful and the configuration can be better understood by going through the reference here. In brief, it sets up a websocket endpoint at '/tailfileep' uri, this endpoint is enhanced with SockJS support, Stomp is used as a sub-protocol, endpoints `/topic` and `/queue` is configured to a real broker like RabbitMQ or ActiveMQ but in this specific to an in-memory one. Going back to the "fileContentRecordingService" once more, this component essentially takes the line of the file and sends it this in-memory broker, SimpMessagingTemplate facilitates this wiring: public class FileContentRecordingService { @Autowired private SimpMessagingTemplate simpMessagingTemplate; public void sendLinesToTopic(String line) { this.simpMessagingTemplate.convertAndSend("/topic/tailfiles", line); } } Websocket UI configuration The UI is angularjs based, the client controller is set up this way and internally uses the javascript libraries for sockjs and stomp support: var tailFilesApp = angular.module("tailFilesApp",[]); tailFilesApp.controller("TailFilesCtrl", function ($scope) { function init() { $scope.buffer = new CircularBuffer(20); } $scope.initSockets = function() { $scope.socket={}; $scope.socket.client = new SockJS("/tailfilesep); $scope.socket.stomp = Stomp.over($scope.socket.client); $scope.socket.stomp.connect({}, function() { $scope.socket.stomp.subscribe("/topic/tailfiles", $scope.notify); }); $scope.socket.client.onclose = $scope.reconnect; }; $scope.notify = function(message) { $scope.$apply(function() { $scope.buffer.add(angular.fromJson(message.body)); }); }; $scope.reconnect = function() { setTimeout($scope.initSockets, 10000); }; init(); $scope.initSockets(); }); The meat of this code is the "notify" function which the callback acting on the messages from the server, in this instance the new lines coming into the file and showing it in a textarea. This wraps up the entire application to tail a file. A complete working sample without any external dependencies is available at this github location, instructions to start it up is also available at that location. Conclusion Spring Websockets provides a concise way to create Websocket based applications, this sample provides a good demonstration of this support. I had presented on this topic recently at my local JUG (IndyJUG) and a deck with the presentation is available here

In this post, I will show how to integrate Apache Tiles 3 with Spring MVC.

this blog post presents rolling time window counting and rate limiting in redis. you can apply it to activate login captcha on your site only when it is needed. for the syntax highlighted python source code please see the original blog post . table of contents 1. about redis 2. rollingwindow.py: 3. problematic captchas 4. captchas and different login situations 5. mitigating botnet-driven login attack with on-situation captcha 6. captchamode.py 1. about redis redis is a key-value store and persistent cache. besides normal get/set functionality it offers more complex data structures like lists, hashes and sorted sets. if you are familiar with memcached think redis as memcached with steroids. often redis is used for rate limiting purposes . usually the rate limit recipes are count how many times something happens on a certain second or a certain minute. when the clock ticks to the next minute, rate limit counter is reset back to the zero. this might be problematic if you are looking to limit rates where hits per integration time window is very low. if you are looking to limit to the five hits per minute, in one time window you get just one hit and six in another, even though the average over two minutes is 3.5. this posts presents an python example how to do a rolling time window based counting, so that rate counting does not reset itself back to the zero in any point, but counts hits over x seconds to the past. this is achieved using redis sorted sets . 2. rollingwindow.py: if you know any better way to do this with redis – please let me know – i am no expert here. this is the first implementation i figured out. """ redis rolling time window counter and rate limit. use redis sorted sets to do a rolling time window counters and limiters. http://redis.io/commands/zadd """ import time def check(redis, key, window=60, limit=50): """ do a rolling time window counter hit. :param redis: redis client :param key: redis key name we use to keep counter :param window: rolling time window in seconds :param limit: allowed operations per time window :return: true is the maximum limit has been reached for the current time window """ # expire old keys (hits) expires = time.time() - window redis.zremrangebyscore(key, '-inf', expires) # add a hit on the very moment now = time.time() redis.zadd(key, now, now) # if we currently have more keys than limit, # then limit the action if redis.zcard(key) > limit: return true return false def get(redis, key): """ get the current hits per rolling time window. :param redis: redis client :param key: redis key name we use to keep counter :return: int, how many hits we have within the current rolling time window """ return redis.zcard(key) 3. problematic captchas everybody of us hates captchas . they are two-edged swords. on one hand, you need to keep bots out from your site. on the other, captchas are turn off for your site visitors and they drive away potential users. even though the most popular captcha-as-a-service, google’s recaptcha, has made substantial progress to make captchas for real visitors and hard for bots , captchas still present a usability problem. also in the case of recaptcha, javascript and image assets are loaded from google front end services and they tend to get blocked in china, disabling your site for chinese visitors . 4. captchas and different login situations there are three cases where you want the user to complete captcha for login somebody is bruteforcing a single username (targeted attack): you need to count logins per usename and not let the login proceed if this user is getting too many logins. somebody is going through username/password combinations for a single ip: you count logins per ip. somebody is going through username/password combinations and the attack comes from very large ip pool. usually these are botnet-driven attacks and the attacker can easily have tens of thousands of ip addresses to burn. the botnet-driven login attack is tricky to block. there might be only one login attempt from each ip. the only way to effectively stop the attack is to present pre-login captcha i.e. the user needs to solve the captcha even before the login can be attempted. however pre-login captcha is very annoying usability wise – it prevents you to use browser password manager for quick logins and sometimes gives you extra headache of two minutes before you get in to your favorite site. even services like cloudflare do not help you here. because there is only one request per single ip, they cannot know beforehand if the request is going to be legitimate or not (though they have some global heurestics and ip blacklists for sure). you can flip on the “challenge” on your site, so that every visitors must complete the captcha before they can access your site and this is usability let down again. 5. mitigating botnet-driven login attack with on-situation captcha you can have the best of the both worlds: no login captcha and still mitigate botnet-driven login atttacks. this can be done by monitoring your site login rate in normal situation do not have pre-login captcha when there is clearly an abnormal login rate, which means there might be an attack going on, enable the pre-login captcha for certain time below is an pseudo-python example how this can be achieved with using rollingwindow python module from the above. 6. captchamode.py from redis_cache import get_redis_connection import rollingwindow #: redis sorted set key counting login attempts redis_login_attempts_counter = "login_attempts" #: key telling that captcha become activated due to #: high login attempts rate redis_captcha_activated = "captcha_activated" #: captcha mode expires in 120 minutes (attack cooldown) captcha_timeout = 120 * 60 #: are you presented captcha when logging in first time #: disabled in unit tests. login_attempts_challenge_threshold = 500 # per minute def clear(): """ resets the challenge system state, per system or per ip. """ redis = get_redis_connection("redis") redis.delete(redis_captcha_activated) redis.delete(redis_login_attempts_counter) def get_login_rate(): """ :return: system global login rate per minute for metrics """ redis = get_redis_connection("redis") return rollingwindow.get(redis, redis_login_attempts_counter) def check_captcha_needed(redis): """ check if we need to enable login captcha globally. increase login page load/submit counter. :return: true if our threshold for login page loads per minute is exceeded """ # count a hit towards login rate threshold_exceeded = rollingwindow.check(redis, redis_login_attempts_counter, limit=login_attempts_challenge_threshold) # are we in attack mode if not redis.get(redis_captcha_activated): if not threshold_exceeded: # no login rate threshold exceeded, # and currently captcha not activated -> # allow login without captcha return false # login attempt threshold exceeded, # we might be under attack, # activate captcha mode redis.setex(redis_captcha_activated, "true", captcha_timeout) return true def login(request): redis = get_redis_connection("redis") if check_captcha_needed(request): # ... we need to captcha before this login can proceed .. else: # ... allow login to proceed without captcha ...

Originally written by Scott Jarr for VoltDB. In post one of this series, we introduced the ideas that a Corporate Data Architecture was taking shape and that working with Fast Data is different from working with Big Data. In the second post we looked at examples of Fast Data and what is required of applications that interact with Fast Data. In this post, I will illustrate how I envision the corporate architecture that will enable companies to achieve the data dream that integrates Fast and Big. The following diagram depicts a basic view of how the “Big” side of the picture is starting to fill out. At the center is a Data Lake, or pool or reservoir or…. there is no shortage of clever names and debate over what to call it. What is clear is this is the spot in which the enterprise will dump ALL of its data. This component is not necessarily unique because of its design or functionality, but because it is an enormously cost effective system to store everything. Essentially, it is a distributed file system on cheap commodity machines. There may or may not be a single winning technology here. It may be HDFS or some other store (maybe S3 if you’re on Amazon), but the point is, this is where all data will go. This platform will: 1. Store data that will be sent to other data management products, and 2. Support frameworks for executing jobs directly against the data in the file system. Moving around the outside of our Data Lake are the complementary pieces of technology that allow people to gain insight and value from the data stored in the Data Lake. Starting at 12 o’clock in the diagram above and moving clockwise: BI – Reporting: Data warehouses do an excellent job of reporting, and will continue to offer this capability. Some data will be exported to those systems and temporarily stored there, while other data will be accessed directly from the Data Lake in a hybrid fashion. These data warehouse systems were specifically designed to run complex report analytics, and do this well. SQL on Hadoop: There is a lot of innovation here. The goal of many of these products is to displace the data warehouse. Advances have been made with the likes of Hawq and Impala. But make no mistake, there is a long way to go for these systems to get near the speed and efficiency of the data warehouses, especially those with columnar designs. SQL-on-Hadoop systems exist for a couple of important reasons: 1) SQL is still the best way to get at data, and 2) Processing can occur without moving big chunks of data around. Exploratory Analytics: This is the realm of the data scientist. These tools offer the ability to “find” things in data – patterns, obscure relationships, statistical rules, etc. Mahout and R are popular tools in this category. MapReduce: This is a lazily-named group of all the job scheduling and management tasks that often occur on Hadoop (I really should come up with something more accurate). Many Hadoop use cases today involve pre-processing or cleaning data prior to the use of the analytics tools described above. These are the tools and interfaces that allow that to happen. ETL of Enterprise Apps: Last at 6 o’clock is the ETL process that will help get all the legacy data from our trusty enterprise applications into our data lake that stores everything. These applications will slowly migrate to full-fledged Fast+Big Data apps in time, which I will discuss in a future post. But suffice it to say: once I add sensors to a manufacturing line, I have a Fast+Big Data problem. OK, we now have analytics … so what? Why do we do analytics in the first place? Simple. We want: Better decisions Better personalization Better detection Better …. Interaction. Interaction is what the application is responsible for, and the most valuable improvements come when you can do these interactions accurately and in real-time. This brings us to the second half of the architecture where we deal with Fast Data to make better, faster real-time applications, depicted in the diagram below. The first thing to notice is that there is a tight coupling of Fast and Big, although they are separate systems. They have to be, at least at scale. The database system designed to work with millions of event decisions per second is wholly different from the system designed to hold Petabytes of data and generate extensive reports. The nature of Fast Data produces a number of critical requirements to get the most out of it. These include the ability to: Ingest / interact with the data feed Make decisions on each event in the feed Provide visibility into fast-moving data with real-time analytics Seamlessly integrate into the systems designed to store Big Data Ability to serve analytic results and knowledge from the Big Data systems quickly to users and applications, closing the data loop. There is no better technology to meet these requirements than an operational database. The challenge we have faced is that there hasn’t been an operational database that can manage this kind of throughput. As a result, there have been a number of Band-Aids people have used to attempt to meet their needs, often giving up capabilities and always adding complexity. In a next post, I will detail the capabilities I see customers looking for to support their Fast Data applications. Then we will take a look at the results of attempting this solution with a popular alternative, stream processing. Originally written by Scott Jarr for VoltDB.

Spring Security offers an authentication replacement feature, often referred to as Run-As, that can replace the current user's authentication (and thus permissions) during a single secured object invocation. Using this feature makes sense when a backend system invoked during request processing requires different privileges than the current application. For example, an application might want to expose a financial transaction log to the currently logged in user, but the backend system that provides it only permits this action to the members of a special "auditor" role. The application can not simply assign this role to the user as that would potentially permit them to execute other restricted actions. Instead, the user can be given this right exclusively for viewing their transaction log. Only two classes are used to implement this feature. Instances of RunAsManager are tasked with producing the actual replacement authentication tokens. A sensible default implementation is already provided by Spring Security. As with other types of authentication, it is also necessary to register an instance of an appropriate AuthenticationProvider. Tokens produced by runAsManager are signed with the provided key (my_run_as_key in the example above) and are later checked against the same key by runAsAuthenticationProvider, in order to mitigate the risk of fake tokens being provided. These keys can have any value, but need to be the same in both objects. Otherwise, runAsAuthenticationProvider will reject the produced tokens as invalid. If an instance is registered, RunAsManager will be invoked by AbstractSecurityInterceptor for every intercepted object invocation for which the user has already been given access. If RunAsManager returns a token, this token will be used be used instead of the original one for the duration of the invocation, thus granting the user different privileges. There are two key points here. In order for the authentication replacement feature to do anything, the call has to actually be secured (and thus intercepted), and the user has to already have been granted access. To register a RunAsManager instance with the method security interceptor, something similar to the following is needed: Now, all methods secured by the @Secured annotation will be able to trigger RunAsManager. One important point here is that global-method-security will only work in the Spring context in which it is defined. In Spring MVC applications, there usually are two Spring contexts: the parent context, attached to ContextLoaderListener, and the child context, attached toDispatcherServlet. To secure Controller methods in this way, global-method-security must be added to DispatcherServlet's context. To secure methods in beans not in this context, global-method-security should also be added to ContextLoaderListener's context. Otherwise, security annotations will be ignored. The default implementation of RunAsManager (RunAsManagerImpl) will inspect the secured object's configuration and if it finds any attributes prefixed with RUN_AS_, it will create a token identical to the original, with the addition of one new GrantedAuthorty per RUN_AS_ attribute found. The new GrantedAuthority will be a role (prefixed by ROLE_ by default) named like the found attribute without the RUN_AS_ prefix. So, if a user with a role ROLE_REGISTERED_USER invokes a method annotated with @Secured({"ROLE_REGISTERED_USER","RUN_AS_AUDITOR"}), e.g. @Controller public class TransactionLogController { @Secured({"ROLE_REGISTERED_USER","RUN_AS_AUDITOR"}) //Authorities needed for method access and authorities added by RunAsManager prefixed with RUN_AS_ @RequestMapping(value = "/transactions", method = RequestMethod.GET) //Spring MVC configuration. Not related to security @ResponseBody //Spring MVC configuration. Not related to security public List getTransactionLog(...) { ... //Invoke something in the backend requiring ROLE_AUDITOR { ... //User does not have ROLE_AUDITOR here } the resulting token created by RunAsManagerImpl with be granted ROLE_REGISTERED_USER and ROLE_AUDITOR. Thus, the user will also be allowed actions, normally reserved for ROLE_AUDITOR members, during the current invocation, permitting them, in this case, to access the transaction log.To enable runAsAuthenticationProvider, register it as usual: ... other authentication-providers used by the application ... This is all that is necessary to have the default implementation activated. Still, this setting will not work for methods secured by @PreAuthorize and @PostAuthorize annotations as their configuration attributes are differently evaluated (they are SpEL expressions and not a simple list or required authorities like with @Secured) and will not be recognized by RunAsManagerImpl. For this scenario to work, a custom RunAsManager implementation is required, as, at least at the time of writing, no applicable implementation is provided by Spring. A custom RunAsManager implementation for use with @PreAuthorize/@PostAuthorize A convenient implementation relying on a custom annotation is provided below: public class AnnotationDrivenRunAsManager extends RunAsManagerImpl { @Override public Authentication buildRunAs(Authentication authentication, Object object, Collection attributes) { if(!(object instanceof ReflectiveMethodInvocation) || ((ReflectiveMethodInvocation)object).getMethod().getAnnotation(RunAsRole.class) == null) { return super.buildRunAs(authentication, object, attributes); } String roleName = ((ReflectiveMethodInvocation)object).getMethod().getAnnotation(RunAsRole.class).value(); if (roleName == null || roleName.isEmpty()) { return null; } GrantedAuthority runAsAuthority = new SimpleGrantedAuthority(roleName); List newAuthorities = new ArrayList(); // Add existing authorities newAuthorities.addAll(authentication.getAuthorities()); // Add the new run-as authority newAuthorities.add(runAsAuthority); return new RunAsUserToken(getKey(), authentication.getPrincipal(), authentication.getCredentials(), newAuthorities, authentication.getClass()); } } This implementation will look for a custom @RunAsRole annotation on a protected method (e.g. @RunAsRole("ROLE_AUDITOR")) and, if found, will add the given authority (ROLE_AUDITOR in this case) to the list of granted authorities. RunAsRole itself is just a simple custom annotation: @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.METHOD) public @interface RunAsRole { String value(); } This new implementation would be instantiated in the same way as before: And registered in a similar fashion: The expression-handler is always required for pre-post-annotations to work. It is a part of the standard Spring Security configuration, and not related to the topic described here. Both pre-post-annotations and secured-annotations can be enabled at the same time, but should never be used in the same class. The protected controller method from above could now look like this: @Controller public class TransactionLogController { @PreAuthorize("hasRole('ROLE_REGISTERED_USER')") //Authority needed to access the method @RunAsRole("ROLE_AUDITOR") //Authority added by RunAsManager @RequestMapping(value = "/transactions", method = RequestMethod.GET) //Spring MVC configuration. Not related to security @ResponseBody //Spring MVC configuration. Not related to security public List getTransactionLog(...) { ... //Invoke something in the backend requiring ROLE_AUDITOR { ... //User does not have ROLE_AUDITOR here }

In an earlier blog entry I had touched on a fictitious rube goldberg flow for capitalizing a string through a complicated series of steps, the premise of the article was to introduce Spring Integration Java DSL as an alternative to defining integration flows through xml configuration files. I learned a few new things after writing that blog entry, thanks to Artem Bilan and wanted to document those learnings here: So, first my original sample, here I have the following flow(the one's in bold): Take in a message of this type - "hello from spring integ" Split it up into individual words(hello, from, spring, integ) Send each word to a ActiveMQ queue Pick up the word fragments from the queue and capitalize each word Place the response back into a response queue Pick up the message, re-sequence based on the original sequence of the words Aggregate back into a sentence("HELLO FROM SPRING INTEG") and Return the sentence back to the calling application. EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); jmsOutboundGateway.setReplyChannel(sequenceChannel()); return jmsOutboundGateway; } It turns out, based on Artem Bilan's feedback, that a few things can be optimized here. First notice how I have explicitly defined two direct channels, "requestChannel" for starting the flow that takes in the string message and the "sequenceChannel" to handle the message once it returns back from the jms message queue, these can actually be totally removed and the flow made a little more concise this way: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); return jmsOutboundGateway; } "requestChannel" is now being implicitly created just by declaring a name for it. The sequence channel is more interesting, quoting Artem Bilan - do not specify outputChannel for AbstractReplyProducingMessageHandler and rely on DSL , what it means is that here jmsOutboundGateway is a AbstractReplyProducingMessageHandler and its reply channel is implicitly derived by the DSL. Further, two methods which were earlier handling the flows for sending out the message to the queue and then continuing once the message is back, is collapsed into one. And IMHO it does read a little better because of this change. The second good change and the topic of this article is the introduction of the Jms namespace factories, when I had written the previous blog article, DSL had support for defining the AMQ inbound/outbound adapter/gateway, now there is support for Jms based inbound/adapter adapter/gateways also, this simplifies the flow even further, the flow now looks like this: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(Jms.outboundGateway(connectionFactory) .requestDestination("amq.outbound")) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } The inbound Jms part of the flow also simplifies to the following: @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(Jms.inboundGateway(connectionFactory) .destination("amq.outbound")) .transform((String s) -> s.toUpperCase()) .get(); } Thus, to conclude, Spring Integration Java DSL is an exciting new way to concisely configure Spring Integration flows. It is already very impressive in how it simplifies the readability of flows, the introduction of the Jms namespace factories takes it even further for JMS based flows.