i feel like writing some posts about cdi (contexts and dependency injection). so this is the first one of a series of x posts ( 0 javax.enterprise cdi-api 1.0 provided an empty beans.xml will do to enable cdi you must have a beans.xml file in your project (under the meta-inf or web-inf). that’s because cdi needs to identify the beans in your classpath (this is called bean discovery) and build its internal metamodel. with the beans.xml file cdi knows it has beans to discover. so, for all the following examples i’ll make it simple and will leave this file completely empty. java ee 6 containers let’s start with the easiest possible environment : java ee 6 containers . why is it the simplest ? well, because you don’t have to do anything : cdi is part of java ee 6 as well as the web profile 1.0 so you don’t need to manually bootstrap it. let’s see how to inject a cdi bean within an ejb 3.1 and a servlet 3.0 . ejb 3.1 since ejb 3.1 you can use the ejbcontainer api to get an in-memory embedded ejb container and you can easily unit test your ejbs. so let’s write an ejb and a test class. first let’s have a look at the code of the ejb. as you can see, with version 3.1 an ejb is just a pojo : no inheritance, no interface, just one @stateless annotation. it gets a reference of the hello bean buy using the @inject annotation and uses it in the saysomething() method. @stateless public class mainejb31 { @inject hello hello; public string saysomething() { return hello.sayhelloworld(); } } you can now package the mainejb31, hello and world classes with the empty beans.xml file into a jar, deploy it to glassfish 3.x , and it will work. but if you don’t want to bother deploying it to glassfish and just unit test it, this is what you need to do : public class mainejbtest { private static ejbcontainer ec; private static context ctx; @beforeclass public static void initcontainer() throws exception { map properties = new hashmap(); properties.put(ejbcontainer.modules, new file("target/classes")); ec = ejbcontainer.createejbcontainer(properties); ctx = ec.getcontext(); } @afterclass public static void closecontainer() throws exception { if (ec != null) ec.close(); } @test public void shoulddisplayhelloworld() throws exception { // looks up the ejb mainejb31 mainejb = (mainejb31) ctx.lookup("java:global/classes/mainejb!org.antoniogoncalves.cdi.helloworld.mainejb"); assertequals("should say hello world !!!", "hello world !!!", mainejb.saysomething()); } } in the code above the method initcontainer() initializes the ejbcontainer. the shoulddisplayhelloworld() looks up the ejb (using the new portable jndi name ), invokes it and makes sure the saysomething() method returns hello world !!!. green test. that was pretty easy too. servlet 3.0 servlet 3.0 is part of java ee 6, so again, there is no needed configuration to bootstrap cdi. let’s use the new @webservlet annotation and write a very simple one that injects a reference of hello and displays an html page with hello world !!!. this is what the servlet looks like : @webservlet(urlpatterns = "/mainservlet") public class mainservlet30 extends httpservlet { @inject hello hello; @override protected void service(httpservletrequest req, httpservletresponse resp) throws servletexception, ioexception { resp.setcontenttype("text/html"); printwriter out = resp.getwriter(); out.println(""); out.println(""); out.println(""); out.println(saysomething()); out.println(""); out.println(""); out.close(); } public string saysomething() { return hello.sayhelloworld(); } } thanks to the @webservlet i don’t need any web.xml (it’s optional in servlet 3.0) to map the mainservlet30 to the /mainservlet url. you can now package the mainservlet30, hello and world classes with the empty beans.xml and no web.xml into a war, deploy it to glassfish 3.x , go to http://localhost:8080/bootstrapping-servlet30-1.0/mainservlet and it will work. unfortunately servlet 3.0 doesn’t have an api for the container (such as ejbcontainer). there is no servletcontainer api that would let you use an embedded servlet container in a standard way and, why not, easily unit test it. application client container not many people know it, but java ee (or even older j2ee versions) comes with an application client container (acc). it’s like an ejb or servlet container but for plain pojos. for example you can develop a swing application (yes, i’m sure that some of you still use swing), run it into the acc and get some extra services given by the container (security, naming, certain annotations…). glassfish v3 has an acc that you can launch in a command line : appclient -jar . so i thought, great, i can use cdi with acc the same way i use it within ejb or servlet container, no need to bootstrap anything, it’s all out of the box. i was wrong . as per the cdi specification (section 12.1), cdi is not required to support application client bean archives. so the glassfish application client container doesn’t support it. i haven’t tried the jboss acc , maybe it works. other containers the beauty of cdi is that it doesn’t require java ee 6 . you can use cdi with simple pojos in a java se environment, as well as some servlet 2.5 containers. of course it’s not as easy to bootstrap because you need a bit of configuration. but it then works fine (not always but). java se 6 ok, so until now there was nothing to do to bootstrap cdi. it is already bundled with the ejb 3.1 and servlet 3.0 containers of java ee 6 (and web profile). so the idea here is to use cdi in a simple java se environment. coming back to our hello and world classes, we need a pojo with an entry point that will bootstrap cdi so we can use injection to get those classes. in standard java se when we say entry point , we think of a public static void main(string[] args) method. well, we need something similar… but different. weld is the reference implementation of cdi. that means it implements the specification, the standard apis (mostly found in javax.inject and javax.enterprise.context packages) but also some proprietary code (in org.jboss.weld package). bootstrapping cdi in java se is not specified so you will need to use specific weld features. you can do that in two different flavors: by observing the containerinitialized event or using the programatic bootstrap api consisting of the weld and weldcontainer classes. the following code uses the containerinitialized event. as you can see, it uses the @observes annotation that i’ll explain in a future post. but the idea is that this class is listening to the event and processes the code once the event is triggered. import org.jboss.weld.environment.se.events.containerinitialized; import javax.enterprise.event.observes; import javax.inject.inject; public class mainjavase6 { @inject hello hello; public void saysomething(@observes containerinitialized event) { system.out.println(hello.sayhelloworld()); } } but who trigers the containerinitialized event ? well, it’s the org.jboss.weld.environment.se.startmain class. i’m using maven so a nice trick is to use the exec-maven-plugin to run the startmain class. download the code , have a look at the pom.xml and give it a try. the other possibility is to programmatically bootstrap the weld container. this can be handy in unit testing. the code below initializes the weld container (with new weld().initialize()) and then looks for the hello class (using weld.instance().select(hello.class).get()). import org.jboss.weld.environment.se.weld; import org.jboss.weld.environment.se.weldcontainer; import org.junit.beforeclass; import org.junit.test; import static junit.framework.assert.assertequals; public class hellotest { @test public void shoulddisplayhelloworld() { weldcontainer weld = new weld().initialize(); hello hello = weld.instance().select(hello.class).get(); assertequals("should say hello world !!!", "hello world !!!", hello.sayhelloworld()); } } execute the test with mvn test and it should be green. as you can see, there is a bit more work using cdi in a java se environment, but it’s not that complicated. tomcat 6.x ok, and what about your legacy servlet 2.5 containers ? the first one that comes in mind is tomcat 6.x ( note that tomcat 7.x will implement servlet 3.0 but is still in beta version at the time of writing this post ). weld provides support for tomcat but you need to configure it a bit to make cdi work. first of all, this is a servlet 2.5, not a 3.0. so the code of the servlet is slightly different from the one seen before (no annotation allowed) and of course, you need your good old web.xml file : public class mainservlet25 extends httpservlet { @inject hello hello; @override protected void service(httpservletrequest req, httpservletresponse resp) throws servletexception, ioexception { resp.setcontenttype("text/html"); printwriter out = resp.getwriter(); out.println(""); out.println(""); out.println(""); out.println(saysomething()); out.println(""); out.println(""); out.close(); } public string saysomething() { return hello.sayhelloworld(); } } because we don’t have a @webservlet annotation in servlet 2.5, we need to declare and map it in the web.xml (using the servlet and servlet-mapping tags). then, you need to explicitly specify the servlet listener to boot weld and control its interaction with requests (org.jboss.weld.environment.servlet.listener). tomcat has a read-only jndi, so weld can’t automatically bind the beanmanager extension spi. to bind the beanmanager into jndi, you should populate meta-inf/context.xml and make the beanmanager available to your deployment by adding it to your web.xml: mainservlet25 org.antoniogoncalves.cdi.bootstrapping.servlet.mainservlet25 mainservlet25 /mainservlet org.jboss.weld.environment.servlet.listener beanmanager javax.enterprise.inject.spi.beanmanager the meta-inf/context.xml file is an optional file which contains a context for a single tomcat web application. this can be used to define certain behaviours for your application, jndi resources and other settings. package all the files (mainservlet25, hello, world, meta-inf/context.xml, beans.xml and web.xml) into a war and deploy it into tomcat 6.x. go to http://localhost:8080/bootstrapping-servlet25-tomcat-1.0/mainservlet and you will see your hello world page. jetty 6.x another famous servlet 2.5 containers is jetty 6.x (at codehaus) and jetty 7.x ( note that jetty 8.x will implement servlet 3.0 but it’s still in experimental stage at the time of writing this post ). if you look at the weld documentation, there is actually support for jetty 6.x and 7.x . the code is the same one as tomcat (because it’s a servlet 2.5 container), but the configuration changes. with jetty you need to add two files under web-inf : jetty-env.xml and jetty-web.xml : beanmanager javax.enterprise.inject.spi.beanmanager org.jboss.weld.resources.managerobjectfactory true package all the files (mainservlet25, hello, world, web-inf/jetty-env.xml, web-inf/jetty-web.xml, beans.xml and web.xml) into a war and deploy it into jetty 6.x. go to http://localhost:8080/bootstrapping-servlet25-jetty6/mainservlet and you will see your hello world page. there was a mistake in the weld documentation so i couldn’t make it work. i started a thread on the weld forum and thanks to dan allen , pete muir and all the weld team, this was fixed and i managed to make it work. simple as posting an email to the forum . thanks for your help guys. spring 3.x here is the tricky part. spring 3.x implements the jsr 330 : dependency injection for java , which means that @inject works out of the box. but i didn’t find a way to integrate cdi with spring 3.x . the weld documentation mentions that because of its extension points, “ integration with third-party frameworks such as spring (…) was envisaged by the designers of cdi “. i did find this blog that simulates cdi features by enabling spring ones. what i didn’t find is a clear statement or roadmap on springsource about supporting cdi or not in future releases. the last trace of this topic is a comment on a long tss flaming thread . at that time (16 december 2009), juergen huller said “ with respect to implementing cdi on top of spring (…) trying to hammer it into the semantic frame of another framework such as cdi would be an exercise that is certainly achievable (…) but ultimately pointless “. but if you have any fresh news about it, let me know. conclusion as i said, this post is not about explaining cdi, i’ll do that in future posts. i just wanted to focus on how to bootstrap it in several environments so you can try by yourself. as you saw, it’s much simpler to use cdi within an ejb 3.1 or servlet 3.0 container in java ee 6. i’ve used glassfish 3.x but it should also work with other java ee 6 or web profile containers such as jboss 6 or resin . when you don’t use java ee 6, there is a bit more work to do. depending on your environment or servlet container you need some configuration to bootstrap weld. by the way, i’ve used weld because it’s the reference implementation, the one bunddled with glassfish and jboss. but you could also use openwebbeans , another cdi implementation. download the code , give it a try, and give me some feedback. from http://agoncal.wordpress.com/2011/01/12/bootstrapping-cdi-in-several-environments/

In this multi-part post, I’m going to share my personal experiences while working with user stories for gathering, tracking and planning requirements. It currently consists out of three parts: What are they and why do you need them? Who writes them and how do you control scope? Where do you start and what about the planning? You can also download all parts as one comprehensive PDF for easy printing or e-reading. Where do you start? Suppose that after intensive discussions and tough scoping sessions you ended up with a list of user stories and are about to start building the system. The first story not only needs to realize some particular feature, but also involves building a skeleton implementation of the system’s architecture. How do you avoid spending way too much time on plumbing and other general purpose stuff you need for the rest of the stories? The article Managing the Bootstrap Story by Jennitta Andrea addressed this challenge in more detail and offers some alternative solutions. One of these solutions is to find and define a user story with the product owner that offers minimal functionality yet still has project value. Such a story is often referred to as the backbone story because you realize the backbone of your system in it. It’s quite common to use the backbone story to realize a proof-of-concept (PoC) that verifies the chosen architecture. Since a working PoC can give the product owner confidence that the team is able to build such a product, that fact alone may be enough project value for the product owner. More storyotypes? You might have suspected it already, but that backbone story is just an example of another storyotype. In fact, after I started looking for an approach to capture the non-functional requirements of a project or system, I ran into a slide deck that mentioned a whole set of additional storyotypes. Dan Rawsthorne, the author, tried to define a storyotype for virtually every possible thing you might need to do in a project. Personally I think he went a bit too far, but a small set of additional storyotypes proved to be very useful anyway. Storyotype Description Compound Epic A composite user story that groups a number of stories in a logical sense. Complex Epic A user story whose content and impact must be determined later in the project, but for which it is clear that it involves a significant amount of work. Setup A story that is used to setup the project environment, including a source control environment, a project website, a build server. Technical A story that involves making a technical improvement or adjustment. Examples include introducing a coding standard, refactoring a poor design, executing a performance test. Documentation A story for writing a user manual, installation manual, etc. Training A story for developing and/or hosting a training, or having a workshop with end users. Quality Improvements A story which objective is to fix a collection of related bugs, or spent a fixed amount of time to improve the quality of the code base. Spike A story that aims to do a technical investigation to determine the usability of a specific technology, or for trying an alternative technical solution. When is the story complete? So how do you know that a user story has been successfully realized? Well, if all is good, all stories will conform with INVEST and are associated with a number of acceptance criteria (typically written down as the how-to demo) specified by the product owner. That should be enough to determine if it is functionally sound. But what you still miss is a way of explaining the stakeholders, including the product owner, when the team treats the story as finished. That may differ by team, but usually includes some or more of the following criteria. The code compiles and there are no warnings or errors. The code meets the coding standards setup by the project or the organization. The code is reviewed by a peer developer. All automated unit and integration tests have completed successfully. Visual Studio’s static code analysis tool does not report any violations. ReSharper reports no potential errors (a.k.a. everything is 'green'). The daily integration build has completed successfully. The functionality was tested by another member of the team (anybody but the developer). The feature or functionality has been signed off using the project checklist. The system functionality is tested by a tester. The visual look and feel is has been approved by an employee of the communications department. Together with the story’s how-to demo these criteria are commonly referred to as the definition-of-done. Usually, a team or project will have a default definition-of-done that applies to all stories and only mentions the particulars of that story if necessary. Then what about the planning? User stories are an excellent unit for tracking progress within your project. However, purists within the Agile community will tell you that an Agile project will have no long term plan. Instead, the functionality is realized iteratively according to the priority defined by the product owner. I agree with the latter and believe that its iterative nature is essential for dealing with the changing requirements that are common in all projects. It allows deferring decisions to the last responsible moment, and that’s always a good thing. But in reality you often can’t escape from providing at least a rough schedule to your management. How should you deal with that? What I often do to get all stakeholders to join me in a number of workshops. Using use case diagrams to illustrate the context of the discussions, I try to get enough stories on paper to represent the entire scope of the project. You need to beware though that you don’t write down too much details or have too much in-depth discussions. That would give the stakeholders a false sense of precision, and consequently, will cause them to see the stories as a formal functional design. Also, if you run into some high-level chunk of functionality for which nobody really knows what it will look like, add an epic story for it and include a spike to elaborate on the epic later on in the project. Then organize a number of shorter meetings with the team or, if the team hasn’t been formed yet, with a few experienced developers. Let them discuss every story one by one and then try to estimate the size of each story in so called story points. Some people from the Agile community say you should estimate using relative sizes only. In other words, a story that seems to require twice as much work as another story should also have twice as many story points. The story point as a unit does not have value. It’s the relative differences that are important. What works for me is that every story point corresponds to the ideal day of an experienced senior software developer. In other words, one story point means that an experienced developer familiar with the chosen architecture, technology and project methodology needs to work for 8 hours without being disturbed by telephone, email, coffee breaks, or any other distractions. Mike Cohn, author of User Stories Applied, has dedicated many chapters to this estimation technique. Ideally, each story is between 1 and 8 story points, but at the beginning of the project you still may have some epics to break up. After finishing those meetings you should have an estimate of the total size of the project. Now, in order to get from those story points to a total number of hours you need to estimate the expected productivity of the team. Mike Cohn does this by creating a table with the expected roles, their availability (to deal with part time employees), and the expected productivity compared to the ideal senior developer (as a percentage). By calculating the average productivity and multiplying it with the number of story points you’ll end up with the total number of estimated man-hours. It’s only an estimate and both the productivity can be disappointing as well as the estimate in story points may appear to be wrong. But it still gives you an initial estimate that can be used for global planning and budget discussions. Obviously it is important to ensure that you keep on continuously measuring the actual productivity. Wow, now what? By now, it should be clear that a user story is not an independent concept but something that closely resonates with many of the aspects of our work in the software industry. In this multi-part post I have tried to explain a number of those aspects and to clarify the relationship between them. But even though I’ve not touched everything as detailed as possible, I still hope I've managed to convince you about the power and potential of user stories. Last but not least, if you have any questions or comments, please do not hesitate to email me at [email protected] or tweet me at my Twitter ID ddoomen.

There has been a lot of buzz lately on high-availability and clustering. Most developers don't care and why should they? These features should be transparent to the application architecture and not something of concern to the developers of that application. But knowledge never hurts, so I emerged myself into the world of load balancing, heartbeats and virtual IP addresses. And you know what? Next time we need a infrastructure like this, I can at least sit down with the guys from the infrastructure department and at least know what the hell they are talking about. So what exactly is a high-availability clustered infrastructure (HACI, as I'll call it from now on) ? In essence, it should be a zero-downtime infrastructure (or at least perceived as one by the end user, which means never ever returning a default browser 404 page), capable of horizontal scaling when the need for it arises and without a single point of failure. It's the SLA writer's dream. A basic HACI setup looks like this: The users enters through a virtual IP address, assigned to one of the two load balancers. Only one of the load-balancers is active (the active master, LB1), the other one is there in the event LB1 fails ((LB2, a passive slave). The two load balancers are redundant, ie. having the exact same configuration. The load balancers redirect all traffic to the real servers. This can be done through round-robin assignment or through other means like sticky sessions, where the same user is redirected to the same server each and every time within a session. Servers can be added at any moment and configured on the load balancers. Ideally, the load balancer configuration is aware of the hardware specification and balances the load accordingly, but that's beyond the scope of this article (it involves adding weights). If all servers balanced by the load balancer fail, a backup server should be used to redirect all traffic coming from the load balancer. This can be a very lightweight server, which purpose is only to provide a sensible error page to the user (something like 'Sorry, we are performing maintenance'). Again, perception and immediate feedback to the user is key. You don't want to show the user a plain 404 page. Off course, if the backup server goes down too, you're in trouble (off course, by that time, warning bells should have gone off on every level in the hierarchy). So how to achieve this with as little effort as possible? If you want to try this out, I suggest you start by installing a virtual machine like VirtualBox or VMWare. This way you can try out the configuration yourself. In this example, I'll be load-balancing 3 Tomcat servers using sticky sessions using 2 load balancers in active-passive mode. I'm assuming all 3 Tomcat servers share the same hardware configuration, so they are all able to handle the same amount of traffic each. I'm also throwing in a backup server, in case all 3 Tomcat servers go down (serving a custom 503 page kindly informing the user of a catastrophic failure, instead of dropping the standard 404 bomb). You want to start off by assigning IP addresses to the servers. This will make your life a bit easier. We'll need 7 addresses: 3 for the tomcat server, 1 for the backup server, 2 for the loadbalancers and 1 virtual IP address to be shared between the load balancers (and which will be the entry point for your users). So our assignment will be: Virtual IP 10.0.5.99 www.haci.local LB1 10.0.5.100 lb1.haci.local #MASTER LB2 10.0.5.101 lb2.haci.local #SLAVE WEB1 10.0.5.102 web1.haci.local WEB2 10.0.5.103 web2.haci.local WEB3 10.0.5.104 web3.haci.local BACKUP 10.0.5.105 backup.haci.local Setting up the web servers is easy. You just install Tomcat on each server and create a simple JSP file to be served to users (make a small change, like the background color, on each server to distinguish the servers). I won't be covering session replication between the Tomcat servers, as it'll take me too far. If you want, you can configure the appropriate session replication and storage (using multicast or JDBC for example). The backup server I'm using is a basic LAMP server that returns a simple 503 page on every request it gets. The 503 error code is important, because it reflects the current state of the system: currently unavailable. For the loadbalancers I'll be using 2 applications: HAProxy and keepalived. HAProxy is going to handle load balancing, while keepalived will handle the failover between the two load balancers. First, we're going to configure HAProxy for both LB1 and LB2. Installing HAProxy is quite easy on an ubuntu system. Just do a sudo apt-get install haproxy and you're off. After the install, backup the current HAProxy config and start editing away. cp /etc/haproxy.cfg /etc/haproxy.cfg_orig cat /dev/null > /etc/haproxy.cfg vi /etc/haproxy.cfg The content of the config to reflect our setup should become something like this (same config on LB1 and LB2): global log 127.0.0.1 local0 log 127.0.0.1 local1 notice #log loghost local0 info maxconn 4096 #debug #quiet user haproxy group haproxy defaults log global mode http option httplog option dontlognull retries 3 redispatch maxconn 2000 contimeout 5000 clitimeout 50000 srvtimeout 50000 frontend http-in bind 10.0.5.99:80 default_backend servers backend servers mode http stats enable stats auth someuser:somepassword balance roundrobin cookie JSESSIONID prefix option httpclose option forwardfor option httpchk HEAD /check.txt HTTP/1.0 server web1 10.0.5.102:80 cookie haci_web1 check server web2 10.0.5.103:80 cookie haci_web2 check server web3 10.0.5.104:80 cookie haci_web3 check server webbackup 10.0.5.105:80 backup After this, enable HAProxy on both LB1 and LB2 by editing /etc/defaults/haproxy # Set ENABLED to 1 if you want the init script to start haproxy. ENABLED=1 # Add extra flags here. #EXTRAOPTS="-de -m 16" So far for the HAProxy configuration. We can't start it up yet, as LB1 and LB2 aren't listening yet on the virtual IP address. Next we'll configure the failover of the loadbalancers using keepalived. Installing it on Ubuntu is as easy as it was for HAProxy: sudo apt-get install keepalived. But its configuration is slightly different on both load balancers. First, we need to configure the both servers to be able to listen to the shared IP address. Add the following line to /etc/sysctl.conf: net.ipv4.ip_nonlocal_bind=1 And run sysctl -p Now, we configure keepalived so that LB1 is configured as the main load balancer and binds to the shared IP address, while LB2 is on standby, ready to take over whenever LB1 goes down. The configuration for LB1 looks like this (edit /etc/keepalived/keepalived.conf): vrrp_script chk_haproxy { # Requires keepalived-1.1.13 script "killall -0 haproxy" # cheaper than pidof interval 2 # check every 2 seconds weight 2 # add 2 points of prio if OK } vrrp_instance VI_1 { interface eth0 state MASTER virtual_router_id 51 priority 101 # 101 on master, 100 on backup virtual_ipaddress { 10.0.5.99 } track_script { chk_haproxy } } Start up keepalived and check whether it is listening to the virtual IP address. /etc/init.d/keepalived start ip addr sh eth0 It should return something like this, indicating it is listening to the virtual IP address 2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:a5:5b:93 brd ff:ff:ff:ff:ff:ff inet 10.0.5.100/24 brd 10.0.5.255 scope global eth0 inet 10.0.5.99/32 scope global eth0 inet6 fe80::20c:29ff:fea5:5b93/64 scope link valid_lft forever preferred_lft forever Next, we configure LB2. The configuration is almost the same, exception for the priority. vrrp_script chk_haproxy { # Requires keepalived-1.1.13 script "killall -0 haproxy" # cheaper than pidof interval 2 # check every 2 seconds weight 2 # add 2 points of prio if OK } vrrp_instance VI_1 { interface eth0 state MASTER virtual_router_id 51 priority 100 # 101 on master, 100 on backup virtual_ipaddress { 10.0.5.99 } track_script { chk_haproxy } } Start up keepalived and check the network interface. /etc/init.d/keepalived start ip addr sh eth0 It should return something like this, indicating it is not listening to the virtual IP address 2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000 link/ether 00:0c:29:a5:5b:93 brd ff:ff:ff:ff:ff:ff inet 10.0.5.101/24 brd 10.0.5.255 scope global eth0 inet6 fe80::20c:29ff:fea5:5b93/64 scope link valid_lft forever preferred_lft forever Now, start up HAProxy on both LB1 and LB2. /etc/init.d/haproxy start Now you can issue requests to 10.0.5.99 (or www.haci.local), which will go to LB1, which in turn will load-balance the request to either WEB1, WEB2 and WEB3. You can test the load balancing by turning off WEB1 (or the server you're currently on). You can also the backup server by turning all main webservers (WEB1, WEB2 and WEB3). And you can test the loadbalancer failover by turning off LB1. At that point LB2 will kick in and act as the master, loadbalancing all requests. When you turn LB1 back on, it'll take over the master role once again. HAProxy allows you to add extra servers very easily, reloading the configuration without breaking existing sessions. See the HAProxy documentation for more info or on ServerFault. (http://serverfault.com/questions/165883/is-there-a-way-to-add-more-backend-server-to-haproxy-without-restarting-haproxy). Cheap and effective. While most enterprise shops have hardware load balancers, which also have these possibilities and more, if you're on a tight budget or need to simulate a HACI environment for development purposes (a lesson here: always simulate your production environment when you're testing during development), this might be the sane option. To finish, I'll quickly explain how to set up the backup server (a simple LAMP server). Create a vhost configuration on the apache for www.haci.local or any other domain pointing to the virtual IP address and set up mod_rewrite for it: RewriteEngine On RewriteCond %{REQUEST_URI} !\.(css|gif|ico|jpg|js|png|swf|txt)$ [NC] RewriteConf %{REQUEST_URI} !/503.php RewriteRule .* /503.php [L] Then create the 503.php file and add this to the top of it: Sorry, our servers are currently undergoing maintenance. Please check back with us in a while. Thank you for your patience. You can decorate the 503.php file any way you like. You can even use CSS, JavaScript and image files in the php file. Now, back to my IDE. I'm getting withdrawal symptoms.

Creating a statically linked version of a unix software saves you in all the circumstances where you can’t install the software as root, when you don’t have development tools on the target machine, when you cannot find a prepackaged version for the specific server, when the libraries available on a server are conflicting with the ones required by your software, etc. Examples: The unix server that hosts this blog, provides me a restricted shell to manage mysql and public web files with minimal access privileges. On this server I periodically update WordPress and other things, and I like to keep the changes under control and backed up with git. The problem is that on this server I haven’t git binaries, and I cannot install software and libraries required. Having the files of my WordPress installation under git version control saved me a lot of time in the past when somebody hacked into the server and changed the php files to do nasty stuff. I was able to easily check what was exactly changed since last legitimate update with “git status” command, and restore things as before. I own a little NAS (network attached storage) which runs a mini distribution of Linux which doesn’t have a prepackaged version of git to install, nor I cannot use any package management tool like apt rpm or yum etc. So I thought that I can compile by myself the git binaries and have them deployed on the target machine. It is possible, but there is an important note: when you build a software on a unix server, the resulting binaries are referencing the libraries installed there, so you cannot easily port the binaries between servers since they have dependencies. What you need is a “statically linked” version of the software, which fortunately it’s not so difficult to achieve. Binaries which are statically linked are usually bigger in size and will possibly require more memory to execute, but they won’t require the specific libraries to be present on the executing computer, since the libraries code is contained in the binaries themselves. Here is how I built a static version of git on an ubuntu virtual machine, that can be ported to other unix servers: # let's make sure we have all we need to proceed $ sudo apt-get install libexpat1-dev asciidoc libz-dev gettext curl # let's create the directory to host the built artifacts $ sudo mkdir /opt/git-1.7.4.1-static # we are ready to download and unpack latest version of git sources $ curl http://kernel.org/pub/software/scm/git/git-1.7.4.1.tar.bz2 | tar xvj $ cd git-1.7.4.1/ # then compile and install the files in the target directory we created $ ./configure --prefix=/opt/git-1.7.4.1-static CFLAGS="${CFLAGS} -static" NO_OPENSSL=1 NO_CURL=1 $ sudo make install $ sudo make install-doc On the above commands, the thing to notice is the CFLAGS=”${CFLAGS} -static” which is used to specify that the libraries must be statically linked with the binaries. The last thing to do is create a tarball of /opt/git-1.7.4.1-static folder and copy that on the target machine; adding the /opt/git-1.7.4.1-static/bin directory to the PATH variable and the /opt/git-1.7.4.1-static/share/man to the MANPATH variable. If possible, it’s a good idea to keep the same installation path on target machine (/opt/git-1.7.4.1-static) since this path gets hardcoded in some git scripts during the build process. But it shouldn’t give too many problems, anyway. From http://en.newinstance.it/2011/02/27/how-to-create-a-statically-linked-version-of-git-binaries/

I've blogged a few times now about Git (which I pronounce with a hard 'g' a la "get", as it's supposed to be named for Linus Torvalds, a self-described git, but which I've also heard called pronounced with a soft 'g' like "jet"). Either way, I'm finding it way more efficient and less painful than either CVS or SVN combined. So, to continue this series ([1], [2], [3]), here is how (and why) to pull an SVN repo down as a Git repo, but with the omission of old (irrelevant) revisions and branches. Using SVN for SVN repos In days of yore when working with the JBoss Tools and JBoss Developer Studio SVN repos, I would keep a copy of everything in trunk on disk, plus the current active branch (most recent milestone or stable branch maintenance). With all the SVN metadata, this would eat up substantial amounts of disk space but still require network access to pull any old history of files. The two repos were about 2G of space on disk, for each branch. Sure, there's tooling to be able to diff and merge between branches w/o having both branches physically checked out, but nothing beats the ability to place two folders side by side OFFLINE for deep comparisons. So, at times, I would burn as much as 6-8G of disk simply to have a few branches of source for comparison and merging. With my painfullly slow IDE drive, this would grind my machine to a halt, especially when doing any SVN operation or counting files / disk usage. Using Git for SVN repos naively Recently, I started using git-svn to pull the whole JBDS repo into a local Git repo, but it was slow to create and still unwieldy. And the JBoss Tools repo was too large to even create as a Git repo - the operation would run out of memory while processing old revisions of code to play forward. At this point, I was stuck having individual Git repos for each JBoss Tools component (major source folder) in SVN: archives, as, birt, bpel, build, etc. It worked, but replicating it when I needed to create a matching repo-collection for a branch was painful and time-consuming. As well, all the old revision information was eating even more disk than before: jbosstools' trunk as multiple git-svn clones: 6.1G devstudio's trunk as single git-svn clone: 1.3G So, now, instead of a couple Gb per branch, I was at nearly 4x as much disk usage. But at least I could work offline and not deal w/ network-intense activity just to check history or commit a change. Still, far from ideal. Cloning SVN with standard layout & partial history This past week, I discovered two ways to make the git-svn experience at least an order of magnitude better: Standard layout (-s) - this allows your generated Git repo to contain the usual trunk, branches/* and tags/* layout that's present in the source SVN repo. This is a win because it means your repo will contain the branch information so you can easily switch between branches within the same repo on disk. No more remote network access needed! Revision filter (-r) - this allows your generated Git repo to start from a known revision number instead of starting at its birth. Now instead of taking hours to generate, you can get a repo in minutes by excluding irrelevant (ancient) revisions. So, why is this cool? Because now, instead of having 2G of source+metadata to copy when I want to do a local comparison between branches, the size on disk is merely: jbosstools' trunk as single git-svn clone w/ trunk and single branch: 1.3G devstudio's trunk as single git-svn clone w/ trunk and single branch: 0.13G So, not only is the footprint smaller, but the performance is better and I need never do a full clone (or svn checkout) again - instead, I can just copy the existing Git repo, and rebase it to a different branch. Instead of hours, this operation takes seconds (or minutes) and happens without the need for a network connection. Okay, enough blather. Show me the code! Check out the repo, including only the trunk & most recent branch # Figure out the revision number based on when a branch was created, then # from r28571, returns -r28571:HEAD rev=$(svn log --stop-on-copy \ http://svn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x \ | egrep "r[0-9]+" | tail -1 | sed -e "s#\(r[0-9]\+\).\+#-\1:HEAD#") # now, fetch repo starting from the branch's initial commit git svn clone -s $rev http://svn.jboss.org/repos/jbosstools jbosstools_GIT Now you have a repo which contains trunk & a single branch git branch -a # list local (Git) and remote (SVN) branches * master remotes/jbosstools-3.2.x remotes/trunk Switch to the branch git checkout -b local/jbosstools-3.2.x jbosstools-3.2.x # connect a new local branch to remote one Checking out files: 100% (609/609), done. Switched to a new branch 'local/jbosstools-3.2.x' git svn info # verify now working in branch URL: http://svn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x Repository Root: http://svn.jboss.org/repos/jbosstools Switch back to trunk git checkout -b local/trunk trunk # connect a new local branch to remote trunk Switched to a new branch 'local/trunk' git svn info # verify now working in branch URL: http://svn.jboss.org/repos/jbosstools/trunk Repository Root: http://svn.jboss.org/repos/jbosstools Rewind your changes, pull updates from SVN repo, apply your changes; won't work if you have local uncommitted changes git svn rebase Fetch updates from SVN repo (ignoring local changes?) git svn fetch Create a new branch (remotely with SVN) svn copy \ http://svn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x \ http://svn.jboss.org/repos/jbosstools/branches/some-new-branch From http://divby0.blogspot.com/2011/01/howto-partially-clone-svn-repo-to-git.html

we've all seen them on various websites. crappy search utilities. they are a constant reminder that search is not something you should take lightly when building a website or application. search is not just google's game anymore. when a java library called lucene was introduced into the apache ecosystem, and then solr was built on top of that, open source developers began to wield some serious power when it came to customizing search features. in this article you'll be introduced to apache solr and a wealth of applications that have been built with it. the content is divided as follows: introduction setup solr applications summary 1. introduction apache solr is an open source search server. it is based on the full text search engine called apache lucene . so basically solr is an http wrapper around an inverted index provided by lucene. an inverted index could be seen as a list of words where each word-entry links to the documents it is contained in. that way getting all documents for the search query "dzone" is a simple 'get' operation. one advantage of solr in enterprise projects is that you don't need any java code, although java itself has to be installed. if you are unsure when to use solr and when lucene, these answers could help. if you need to build your solr index from websites, you should take a look into the open source crawler called apache nutch before creating your own solution. to be convinced that solr is actually used in a lot of enterprise projects, take a look at this amazing list of public projects powered by solr . if you encounter problems then the mailing list or stackoverflow will help you. to make the introduction complete i would like to mention my personal link list and the resources page which lists books, articles and more interesting material. 2. setup solr 2.1. installation as the very first step, you should follow the official tutorial which covers the basic aspects of any search use case: indexing - get the data of any form into solr. examples: json, xml, csv and sql-database. this step creates the inverted index - i.e. it links every term to its documents. querying - ask solr to return the most relevant documents for the users' query to follow the official tutorial you'll have to download java and the latest version of solr here . more information about installation is available at the official description . next you'll have to decide which web server you choose for solr. in the official tutorial, jetty is used, but you can also use tomcat. when you choose tomcat be sure you are setting the utf-8 encoding in the server.xml . i would also research the different versions of solr, which can be quite confusing for beginners: the current stable version is 1.4.1. use this if you need a stable search and don't need one of the latest features. the next stable version of solr will be 3.x the versions 1.5 and 2.x will be skipped in order to reach the same versioning as lucene. version 4.x is the latest development branch. solr 4.x handles advanced features like language detection via tika, spatial search , results grouping (group by field / collapsing), a new "user-facing" query parser ( edismax handler ), near real time indexing, huge fuzzy search performance improvements, sql join-a like feature and more. 2.2. indexing if you've followed the official tutorial you have pushed some xml files into the solr index. this process is called indexing or feeding. there are a lot more possibilities to get data into solr: using the data import handler (dih) is a really powerful language neutral option. it allows you to read from a sql database, from csv, xml files, rss feeds, emails, etc. without any java knowledge. dih handles full-imports and delta-imports. this is necessary when only a small amount of documents were added, updated or deleted. the http interface is used from the post tool, which you have already used in the official tutorial to index xml files. client libraries in different languages also exist. (e.g. for java (solrj) or python ). before indexing you'll have to decide which data fields should be searchable and how the fields should get indexed. for example, when you have a field with html in it, then you can strip irrelevant characters , tokenize the text into 'searchable terms', lower case the terms and finally stem the terms . in contrast, if you would have a field with text in it that should not be interpreted (e.g. urls) you shouldn't tokenize it and use the default field type string. please refer to the official documentation about field and field type definitions in the schema.xml file. when designing an index keep in mind the advice from mauricio : "the document is what you will search for. " for example, if you have tweets and you want to search for similar users, you'll need to setup a user index - created from the tweets. then every document is a user. if you want to search for tweets, then setup a tweet index; then every document is a tweet. of course, you can setup both indices with the multi index options of solr. please also note that there is a project called solr cell which lets you extract the relevant information out of several different document types with the help of tika. 2.3. querying for debugging it is very convenient to use the http interface with a browser to query solr and get back xml. use firefox and the xml will be displayed nicely: you can also use the velocity contribution , a cross-browser tool, which will be covered in more detail in the section about 'search application prototyping' . to query the index you can use the dismax handler or standard query handler . you can filter and sort the results: q=superman&fq=type:book&sort=price asc you can also do a lot more ; one other concept is boosting. in solr you can boost while indexing and while querying. to prefer the terms in the title write: q=title:superman^2 subject:superman when using the dismax request handler write: q=superman&qf=title^2 subject check out all the various query options like fuzzy search , spellcheck query input , facets , collapsing and suffix query support . 3. applications now i will list some interesting use cases for solr - in no particular order. to see how powerful and flexible this open source search server is. 3.1. drupal integration the drupal integration can be seen as generic use case to integrate solr into php projects. for the php integration you have the choice to either use the http interface for querying and retrieving xml or json. or to use the php solr client library . here is a screenshot of a typical faceted search in drupal : for more information about faceted search look into the wiki of solr . more php projects which integrates solr: open source typo3- solr module magento enterprise - solr module . the open source integration is out dated. oxid - solr module . no open source integration available. 3.2. hathi trust the hathi trust project is a nice example that proves solr's ability to search big digital libraries. to quote directly from the article : "... the index for our one million book index is over 200 gigabytes ... so we expect to end up with a two terabyte index for 10 million books" other examples for libraries: vufind - aims to replace opac internet archive national library of australia 3.3. auto suggestions mainly, there are two approaches to implement auto-suggestions (also called auto-completion) with solr: via facets or via ngramfilterfactory . to push it to the extreme you can use a lucene index entirely in ram. this approach is used in a large music shop in germany. live examples for auto suggestions: kaufda.de 3.4. spatial search applications when mentioning spatial search, people have geographical based applications in mind. with solr, this ordinary use case is attainable . some examples for this are : city search - city guides yellow pages kaufda.de spatial search can be useful in many different ways : for bioinformatics, fingerprints search, facial search, etc. (getting the fingerprint of a document is important for duplicate detection). the simplest approach is implemented in jetwick to reduce duplicate tweets, but this yields a performance of o(n) where n is the number of queried terms. this is okay for 10 or less terms, but it can get even better at o(1)! the idea is to use a special hash set to get all similar documents. this technique is called local sensitive hashing . read this nice paper about 'near similarity search and plagiarism analysis' for more information. 3.5. duckduckgo duckduckgo is made with open source and its "zero click" information is done with the help of solr using the dismax query handler: the index for that feature contains 18m documents and has a size of ~12gb. for this case had to tune solr: " i have two requirements that differ a bit from most sites with respect to solr: i generally only show one result, with sometimes a couple below if you click on them. therefore, it was really important that the first result is what people expected. false positives are really bad in 0-click, so i needed a way to not show anything if a match wasn't too relevant. i got around these by a) tweaking dismax and schema and b) adding my own relevancy filter on top that would re-order and not show anything in various situations. " all the rest is done with tuned open source products. to quote gabriel again: "the main results are a hybrid of a lot of things, including external apis, e.g. bing, wolframalpha, yahoo, my own indexes and negative indexes (spam removal), etc. there are a bunch of different types of data i'm working with. " check out the other cool features such as privacy or bang searches . 3.6. clustering support with carrot2 carrot2 is one of the "contributed plugins" of solr. with carrot2 you can support clustering : " clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. " see some research papers regarding clustering here . here is one visual example when applying clustering on the search "pannous" - our company : 3.7. near real time search solr isn't real time yet, but you can tune solr to the point where it becomes near real time, which means that the time ('real time latency') that a document takes to be searchable after it gets indexed is less than 60 seconds even if you need to update frequently. to make this work, you can setup two indices. one write-only index "w" for the indexer and one read-only index "r" for your application. index r refers to the same data directory of w, which has to be defined in the solrconfig.xml of r via: /pathto/indexw/data/ to make sure your users and the r index see the indexed documents of w, you have to trigger an empty commit every 60 seconds: wget -q http://localhost:port/solr/update?stream.body=%3ccommit/%3e -o /dev/null everytime such a commit is triggered a new searcher without any cache entries is created. this can harm performance for visitors hitting the empty cache directly after this commit, but you can fill the cache with static searches with the help of the newsearcher entry in your solrconfig.xml. additionally, the autowarmcount property needs to be tuned, which fills the cache with a newsearcher from old entries. also, take a look at the article 'scaling lucene and solr' , where experts explain in detail what to do with large indices (=> 'sharding') and what to do for high query volume (=> 'replicating'). 3.8. loggly = full text search in logs feeding log files into solr and searching them at near real-time shows that solr can handle massive amounts of data and queries the data quickly. i've setup a simple project where i'm doing similar things , but loggly has done a lot more to make the same task real-time and distributed. you'll need to keep the write index as small as possible otherwise commit time will increase too great. loggly creates a new solr index every 5 minutes and includes this when searching using the distributed capabilities of solr ! they are merging the cores to keep the number of indices small, but this is not as simple as it sounds. watch this video to get some details about their work. 3.9. solandra = solr + cassandra solandra combines solr and the distributed database cassandra , which was created by facebook for its inbox search and then open sourced. at the moment solandra is not intended for production use. there are still some bugs and the distributed limitations of solr apply to solandra too. tthe developers are working very hard to make solandra better. jetwick can now run via solandra just by changing the solrconfig.xml. solandra also has the advantages of being real-time (no optimize, no commit!) and distributed without any major setup involved. the same is true for solr cloud. 3.10. category browsing via facets solr provides facets , which make it easy to show the user some useful filter options like those shown in the "drupal integration" example. like i described earlier , it is even possible to browse through a deep category tree. the main advantage here is that the categories depend on the query. this way the user can further filter the search results with this category tree provided by you. here is an example where this feature is implemented for one of the biggest second hand stores in germany. a click on 'schauspieler' shows its sub-items: other shops: game-change 3.11. jetwick - open twitter search you may have noticed that twitter is using lucene under the hood . twitter has a very extreme use case: over 1,000 tweets per second, over 12,000 queries per second, but the real-time latency is under 10 seconds! however, the relevancy at that volume is often not that good in my opinion. twitter search often contains a lot of duplicates and noise. reducing this was one reason i created jetwick in my spare time. i'm mentioning jetwick here because it makes extreme use of facets which provides all the filters to the user. facets are used for the rss-alike feature (saved searches), the various filters like language and retweet-count on the left, and to get trending terms and links on the right: to make jetwick more scalable i'll need to decide which of the following distribution options to choose: use solr cloud with zookeeper use solandra move from solr to elasticsearch which is also based on apache lucene other examples with a lot of facets: cnet reviews - product reviews. electronics reviews, computer reviews & more. shopper.com - compare prices and shop for computers, cell phones, digital cameras & more. zappos - shoes and clothing. manta.com - find companies. connect with customers. 3.12. plaxo - online address management plaxo.com , which is now owned by comcast, hosts web addresses for more than 40 million people and offers smart search through the addresses - with the help of solr. plaxo is trying to get the latest 'social' information of your contacts through blog posts, tweets, etc. plaxo also tries to reduce duplicates . 3.13. replace fast or google search several users report that they have migrated from a commercial search solution like fast or google search appliance (gsa) to solr (or lucene). the reasons for that migration are different: fast drops linux support and google can make integration problems. the main reason for me is that solr isn't a black box —you can tweak the source code, maintain old versions and fix your bugs more quickly! 3.14. search application prototyping with the help of the already integrated velocity plugin and the data import handler it is possible to create an application prototype for your search within a few hours. the next version of solr makes the use of velocity easier. the gui is available via http://localhost:port/solr/browse if you are a ruby on rails user, you can take a look into flare. to learn more about search application prototyping, check out this video introduction and take a look at these slides. 3.15. solr as a whitelist imagine you are the new google and you have a lot of different types of data to display e.g. 'news', 'video', 'music', 'maps', 'shopping' and much more. some of those types can only be retrieved from some legacy systems and you only want to show the most appropriated types based on your business logic . e.g. a query which contains 'new york' should result in the selection of results from 'maps', but 'new yorker' should prefer results from the 'shopping' type. with solr you can set up such a whitelist-index that will help to decide which type is more important for the search query. for example if you get more or more relevant results for the 'shopping' type then you should prefer results from this type. without the whitelist-index - i.e. having all data in separate indices or systems, would make it nearly impossible to compare the relevancy. the whitelist-index can be used as illustrated in the next steps. 1. query the whitelist-index, 2. decide which data types to display, 3. query the sub-systems and 4. display results from the selected types only. 3.16. future solr is also useful for scientific applications, such as a dna search systems. i believe solr can also be used for completely different alphabets so that you can query nucleotide sequences - instead of words - to get the matching genes and determine which organism the sequence occurs in, something similar to blast . another idea you could harness would be to build a very personalized search. every user can drag and drop their websites of choice and query them afterwards. for example, often i only need stackoverflow, some wikis and some mailing lists with the expected results, but normal web search engines (google, bing, etc.) give me results that are too cluttered. my final idea for a future solr-based app could be a lucene/solr implementation of desktop search. solr's facets would be especially handy to quickly filter different sources (files, folders, bookmarks, man pages, ...). it would be a great way to wade through those extra messy desktops. 4. summary the next time you think about a problem, think about solr! even if you don't know java and even if you know nothing about search: solr should be in your toolbox. solr doesn't only offer professional full text search, it could also add valuable features to your application. some of them i covered in this article, but i'm sure there are still some exciting possibilities waiting for you!

Maven profiles, like chainsaws, are a valuable tool, with whose power you can easily get carried away, wielding them upon problems to which they are unsuited. Whilst you're unlikely to sever a leg misusing Maven profiles, I thought it worthwhile to share some suggestions about when and when not to use them. These three best practices are all born from real-world mishaps: The build must pass when no profile has been activated Never use Use profiles to manage build-time variables, not run-time variables and not (with rare exceptions) alternative versions of your artifact I'll expand upon these recommendations in a moment. First, though, let's have a brief round-up of what Maven profiles are and do. Maven Profiles 101 A Maven profile is a sub-set of POM declarations that you can activate or disactivate according to some condition. When activated, they override the definitions in the corresponding standard tags of the POM. One way to activate a profile is to simply launch Maven with a -P flag followed by the desired profile name(s), but they can also be activated automatically according to a range of contextual conditions: JDK version, OS name and version, presence or absence of a specific file or property. The standard example is when you want certain declarations to take effect automatically under Windows and others under Linux. Almost all the tags that can be placed directly in a POM can also be enclosed within a tag. The easiest place to read up further about the basics is the Build Profiles chapter of Sonatype's Maven book. It's freely available, readable, and explains the motivation behind profiles: making the build portable across different environments. The build must pass when no profile has been activated (Thanks to for this observation.) Why? Good practice is to minimise the effort required to make a successful build. This isn't hard to achieve with Maven, and there's no excuse for a simple mvn clean package not to work. A maintainer coming to the project will not immediately know that profile wibblewibble has to be activated for the build to succeed. Don't make her waste time finding it out. How to achieve it It can be achieved simply by providing sensible defaults in the main POM sections, which will be overridden if a profile is activated. Never use Why not? This flag activates the profile if no other profile is activated. Consequently, it will fail to activate the profile if any other profile is activated. This seems like a simple rule which would be hard to misunderstand, but in fact it's surprisingly easy to be fooled by its behaviour. When you run a multimodule build, the activeByDefault flag will fail to operate when any profile is activated, even if the profile is not defined in the module where the activeByDefault flag occurs. (So if you've got a default profile in your persistence module, and a skinny war profile in your web module... when you build the whole project, activating the skinny war profile because you don't want JARs duplicated between WAR and EAR, you'll find your persistence layer is missing something.) activeByDefault automates profile activation, which is a good thing; activates implicitly, which is less good; and has unexpected behaviour, which is thoroughly bad. By all means activate your profiles automatically, but do it explicitly and automatically, with a clearly defined rule. How to avoid it There's another, less documented way to achieve what aims to achieve. You can activate a profile in the absence of some property: !foo.bar This will activate the profile "nofoobar" whenever the property foo.bar is not defined. Define that same property in some other profile: nofoobar will automatically become active whenever the other is not. This is admittedly more verbose than , but it's more powerful and, most importantly, surprise-free. Use profiles to adapt to build-time context, not run-time context, and not (with rare exceptions) to produce alternative versions of your artifact Profiles, in a nutshell, allow you to have multiple builds with a single POM. You can use this ability in two ways: Adapt the build to variable circumstances (developer's machine or CI server; with or without integration tests) whilst still producing the same final artifact, or Produce variant artifacts. We can further divide the second option into: structural variants, where the executable code in the variants is different, and variants which vary only in the value taken by some variable (such as a database connection parameter). If you need to vary the value of some variable at run-time, profiles are typically not the best way to achieve this. Producing structural variants is a rarer requirement -- it can happen if you need to target multiple platforms, such as JDK 1.4 and JDK 1.5 -- but it, too, is not recommended by the Maven people, and profiles are not the best way of achieving it. The most common case where profiles seem like a good solution is when you need different database connection parameters for development, test and production environments. It is tempting to meet this requirement by combining profiles with Maven's resource filtering capability to set variables in the deliverable artifact's configuration files (e.g. Spring context). This is a bad idea. Why? It's indirect: the point at which a variable's value is determined is far upstream from the point at which it takes effect. It makes work for the software's maintainers, who will need to retrace the chain of events in reverse It's error prone: when there are multiple variants of the same artifact floating around, it's easy to generate or use the wrong one by accident. You can only generate one of the variants per build, since the profiles are mutually exclusive. Therefore you will not be able to use the Maven release plugin if you need release versions of each variant (which you typically will). It's against Maven convention, which is to produce a single artifact per project (plus secondary artifacts such as documentation). It slows down feedback: changing the variable's value requires a rebuild. If you configured at run-time you would only need to restart the application (and perhaps not even that). One should always aim for rapid feedback. Profiles are there to help you ensure your project will build in a variety of environments: a Windows developer's machine and a CI server, for instance. They weren't intended to help you build variant artifacts from the same project, nor to inject run-time configuration into your project. How to achieve it If you need to get variable runtime configuration into your project, there are alternatives: Use JNDI for your database connections. Your project only contains the resource name of the datasource, which never changes. You configure the appropriate database parameters in the JNDI resource on the server. Use system properties: Spring, for example, will pick these up when attempting to resolve variables in its configuration. Define a standard mechanism for reading values from a configuration file that resides outside the project. For example, you could specify the path to a properties file in a system property. Structural variants are harder to achieve, and I confess I have no first-hand experience with them. I recommend you read this explanation of how to do them and why they're a bad idea, and if you still want to do them, take the option of multiple JAR plugin or assembly plugin executions, rather than profiles. At least that way, you'll be able to use the release plugin to generate all your artifacts in one build, rather than a single one at a time. Further reading Profiles chapter from the Sonatype Maven book. Deploying to multiple environments (prod, test, dev): Stackoverflow.com discussion; see the first and top-rated answer. Short of creating a specific project for the run-time configuration, you could simply use run-time parameters such as system properties. Creating multiple artifacts from one project: How to Create Two JARs from One Project (…and why you shouldn’t) by Tim O'Brien of Sonatype (the Maven people) Blog post explaining the same technique Maven best practices (not specifically about profiles): http://mindthegab.com/2010/10/21/boost-your-maven-build-with-best-practices/ http://blog.tallan.com/2010/09/16/maven-best-practices/ This article is a completely reworked version of a post from my blog.

Both approaches have some advantages and disadvantages and so it’s a question of the likelihood of particular problems and business requirements and priorities.

Parameterized types - the <> expressions that can be used in Java as of JDK 5 are not just for collections. I find myself frequently using them in APIs I design. They really do let you write things which are more generic in the non-Java sense of the word - and the result is more reusable code, which means less code overall, which means fewer bugs and things to test. The verbosity, and some of the weirdness of type-erasure are less than ideal, but used right, the benefits are worth the complexity. The standard (and somewhere recommended) naming convention for parameterized types is to use a single-letter name. That works fine in signatures that have only one such type. But in practice, single-letter names make code less self-describing, and if you're defining a class with more than one parameterized type, it can be confusing and hard to read. People other than me will have to call, understand and maintain my code - the more self-describing I can make it, the better. So I am looking for a naming convention that makes it obvious that something is a parameterized type, but allows for descriptive names. I am wondering if anybody else has run into this problem, and if there is any emerging consensus on naming generics. Do you work on a project that uses generics a lot? If so, what do you do? Here's an example. At the moment, I'm writing a generic (in both senses) class which simply limits the number of threads which can access some resource. It's basically a wrapper around a Semaphore which uses a Runnable-like object to ensure that the Semaphore is accessed correctly, and does some non-blocking statistic gathering about thread contention. So to access the scarce resource, you pass in a ResourceAccessor: public interface ResourceAccessor { public Result run (ProtectedResource resource, Argument argument); } The problem is that, when somebody looks at this interface, they will instantly get the idea that there are really classes they need to go find, which are called ProtectedResource, Argument and Result - and of course, no such classes exist - these are just names for generic types. The standard-naming-convention is worse: public interface ResourceAccessor { public S run (T resource, R argument); } Here, nobody could possibly figure out what on earth this class is for without extensive documentation - this is a really horrible idea. So I've concluded that the standard recommendations for generic type names are simply wrong for any non-trivial usage (I.e. Collection is fine, since there is one type and Collections are well-understood). You simply can't do this on a non-collection code structure you have invented, or people will just be confused and not use it. The best suggestion I've heard thus far is using $ as a prefix: public interface ResourceAccessor <$ProtectedResource, $Argument, $Result> { public $Result run ($ProtectedResource resource, $Argument argument); } I don't find this pretty, but I don't have any better ideas, and at least it makes it crystal-clear that there is something different about these names. Any thoughts? What do you do in this situation?

This post will walk through how to resize an ExtJS Panel, Grid, Component on Window Resize without using Ext.Viewport. Problem: You have a legacy page and you want to change an html grid for an ExtJS DataGrid, because it has so many cool features. Or you have a page with some design and you are going to use only one ExtJS Component. In both cases, you also want to render your ExtJS Component to a specific DIV. Also, you want you component to be resized in case you resize the browser window. How can you do that if resize a single component in an HTML page it is not the default behavior of an ExtJS Component (except if you use Ext.Viewport)? Solution: Condor (from ExtJS Community Support Team) developed a plugin that can do that for you. I had to spend some time to understand how the plugin works, and I finally got it working as I wanted. Well, I recommend you to spend some time reading this thread: http://www.sencha.com/forum/showthread.php?28318 (if you have any issues or questions, please publish it on the thread, so other members can give you the support you need). Requirements to make the plugin work: Your have to apply the following style to the DIV (the width is up to you, the other styles are mandatory, otherwise it will not work): If you have any border around your ExtJS component, you have to set a HEIGHT. And you will also have to set a height to your ExtJS component. In this case, autoHeight will not work. If you DO NOT have any border or other design on the ExtJS component side, you do not need to set height and you can use autoHeight. In my case, I put a border on the external DIV, so I have to set Height: HTML code (all DIVs): And you need to add the plugin to the component (In this case, I’m using an ExtJS DataGrid): var grid = new Ext.grid.GridPanel({ store: store, columns: [ {header: 'Company', width: 160, sortable: true, dataIndex: 'company'}, {header: 'Price', width: 75, sortable: true, renderer: 'usMoney', dataIndex: 'price'}, {header: 'Change', width: 75, sortable: true, renderer: change, dataIndex: 'change'}, {header: '% Change', width: 75, sortable: true, renderer: pctChange, dataIndex: 'pctChange'}, {header: 'Last Updated', width: 85, sortable: true, renderer: Ext.util.Format.dateRenderer('m/d/Y'), dataIndex: 'lastChange'} ], stripeRows: true, autoExpandColumn: 'company', height: 490, autoWidth:true, title: 'Array Grid', // config options for stateful behavior stateful: true, stateId: 'grid' ,viewConfig:{forceFit:true} ,renderTo: 'reportTabContent' // render the grid to the specified div in the page ,plugins: [new Ext.ux.FitToParent("reportTabContent")] }); And done! Now you can resize the browser and the component will resize itself! I tested it on Firefox, Chrome and IE6. You can download my sample project from my GitHub: http://github.com/loiane/extjs-fit-to-parent PS.: If you want to use the full browser window, use a Viewport. Happy coding!

This tutorial outlines steps required to install Nexus (Maven Repository Manager) under Tomcat, or another webapp container. It shows you practical configuration and includes code snippets that go in your pom.xml and settings.xml in order to read and publish artifacts to your Nexus server. Step 1: Download Download Nexus from here (at the time of writing, latest is 1.6.0) Step 2: Install Copy the war to TOMCAT_HOME/webapps/nexus.war Though not required, it is a generally good idea to restart tomcat after installing a new war /etc/init.d/tomcat restart /etc/init.d/tomcat restart Step 3: Configure security a) Change default admin password: The default admin username/password is admin/admin123. Login as admin and change the password to a secure password. Login -> [admin, admin123] -> Left Menu -> Security -> Change Password -> click “Change Password” b) Anonymous Access: By default Nexus is open to the public. If you want to secure access to nexus, disable ‘Nexus anonymous user’ Admin -> Left Menu -> Users -> ‘Nexus anonymous user’ -> Status=Disabled c) Deployment user: Change password for deployment user Admin -> Left menu -> Users -> Deployment user -> Change email address Admin -> Left menu -> Users -> Right click on ‘Deployment user’ in the user list -> Set Password -> click ‘Set password’ to finish Step 4: Set SMTP server It is a good idea to configure SMTP server, so that you can receive emails from Nexus. Admin login -> Left menu -> Administration -> Server ->SMTP Settings -> (host localhost, port 25, no login, no password mostly works on a linux machine) Step 5: Change Base Url If you are running Nexus behind Apache using mod_jk or mod_proxy, change your base url here. Admin login -> Left menu -> Administration -> Server -> Application Server Settings -> Base url Step 6: Add a task to periodically remove old snapshots If you or your CI server publishes snapshots to Nexus several times a day, then you should consider adding a task to delete duplicate/old snapshots for the same GAV (group, artifact, version). If you don’t do this, you will notice that the Nexus disk usage will increase with time. Admin login -> Left menu -> Administration -> Scheduled tasks -> Add… -> name=”Remove old snapshots”, Repository/Group=Snapshots (Repo), Minimum Snapshot Count=1, Snapshot Retention(days)=3, Recurrence=Daily, Recurring time=2:00 -> click ‘Save’ Step 7: Using Nexus: reading and publishing artifacts If you want to deploy your artifacts to your Nexus, you need to configure 2 files: pom.xml and settings.xml a) pom.xml – for each project which wishes to publish to Nexus, add your repo to the pom.xml vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/releases vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/snapshots vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/releases vineetmanohar-nexus vineetmanohar nexus dav:http://nexus.vineetmanohar.com/nexus/content/repositories/snapshots vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true vineetmanohar-nexus vineetmanohar http://nexus.vineetmanohar.com/nexus/content/groups/public true true b) settings.xml – If you have disabled anonymous access to Nexus, add the deployment password to your ~/.m2/repository/settings.xml file vineetmanohar-nexus deployment password_goes_here From http://www.vineetmanohar.com/2010/06/getting-started-with-nexus-maven-repo-manager

FlexMonkey is a free and open source Adobe AIR application used for testing Flex and AIR based applications. It can record, playback, and verify Flex UI interactions. FlexMonkey also generates ActionScript-based testing scripts that you can easily include within a continuous integration environment. Gorilla Logic is the company that builds FlexMonkey, and its CEO, Stuart Stern, recently spoke with DZone about their launch of FlexMonkey 4, which supports all of the new Spark components in Flex 4. For more info on FlexMonkey, see our interview with Stuart Stern at Adobe Max 2009. DZone: First thing's first. What's new in FlexMonkey 4? Stuart Stern: Before we talk about the updates to FlexMonkey, let me give you a bit of background for those who have not used any of the previous versions. We (Gorilla Logic) built and open sourced the first version of FlexMonkey in late 2008 because we needed a serious Flex testing solution for our enterprise customers. Basically, FlexMonkey allows developers and QA people to create comprehensive tests for their Flex applications by easily recording real interactions with the user interface, and by letting the test creator add verification checks on both data and visual snapshots of the UI. Once the interactions have been recorded the test can be played back through the FlexMonkey console or through generated test code in Fluint / FlexUnit. The generated code can be extended to create complex, data-driven test scenarios, and can be easily run within build and continuous integration environments. In our software consulting engagements, we have found that FlexMonkey reduces the overall numbers of tests that developers need to create, since driving testing from the user interface can exercise the entire application stack, top-to-bottom and even front-to-back. Let’s be clear though, api-level testing and tools like FlexUnit are still an essential part of Flex development, especially in testing non ui components. Where FlexMonkey is a better fit for testing is around visual components, which are difficult, if not impossible, to test as a ‘unit.’ On our typical applications, we tend to end up with about 80% of our developer created tests constructed through FlexMonkey, with the other 20% being created as more traditional unit tests. As far as FlexMonkey 4, the goals were pretty simple; the community has been beating down our door for Spark Component (Flex 4) support. So, we’ve added full support for the new component library recently released by Adobe. This is key for enterprise Flex development projects that have come to depend on FlexMonkey for regression and QA testing, and that are ready to move to Flex 4. We've also simplified the setup for FlexMonkey 4, so it's easier for new users to get up and running quickly. DZone: What were some of the difficulties in implementing support for all of Flex 4's Spark components? Stuart: From a FlexMonkey perspective, there is no difference between Spark and Halo components. However, one of the things that makes FlexMonkey so powerful is that it records "semantic" events such as "open combobox" rather than "click at this screen coordinate". So FlexMonkey needs to "understand" every Flex component, and we had to tell it some new things about the new Spark components and their events. . DZone: Are there any trends your seeing in how developers are using FlexMonkey in their UI design workflow? Stuart: FlexMonkey was initially envisioned as a tool for developers. Because developers test code that is still under development, it is important for a test automation tool to be able to express tests in a largely logical fashion. Tests that are too tied to the precise look of a screen at a particular point in time are two brittle for use by developers. FlexMonkey tests are typically robust across application skinning, since tests can be written independent of the exact positions or styling of the components on the screen, and can pinpoint specific functionality. In this way developers can automate testing of portions of an application even before the UI design is fully finalized. Although we designed it for developer testing, it's ability to record tests automatically, add verification logic by pointing and clicking, and do fuzzy bitmap comparisons on select portions of the screen, make FlexMonkey highly effective for QA testing purposes as well. Additionally, when developers and testers use the same tools, they can share some of the same tests, with QA using developer tests as a starting point, and developers incorporating some QA tests into continuous integration builds. DZone: Tell me about the next tool you'll be focusing on: FlexMonkium. Stuart: FlexMonkium is a plugin for Selenium IDE and Selenium RC. It adds FlexMonkey recording and playback capability to Selenium so you can create tests for applications that mix HTML and Flex. We recently completed development and are now doing final testing and documentation. We expect it make it publicly available any day now. FlexMonkium makes all of FlexMonkey's functionality available within the Selenium IDE, and generates JUnit-based tests that can be run with Selenium RC. DZone: Are there any interesting or exciting things you see down the road for the Flash platform ecosystem? How do you think the platform will fare against emerging UI design technologies like HTML5, CSS3, etc.? Stuart: The recent attacks on the Flash platform by Apple have certainly put the ‘HTML 5 vs. Flash’ battle on everyone’s radar. At Gorilla, we build both native browser applications (HTML 5, etc.) and Flex applications -- and even native iPhone applications -- for our customers. There are pros and cons to each and situations that definitively call for one versus another. Having said that, we are a consulting company that builds serious enterprise software. We embrace Flex because it enables us to do things we cannot do otherwise, and do them quickly. On any given project, we don't ask if we should use Flex, we ask if there is any reason why we can't.

It's amazing how the JBoss Team put together an easy way to do JMS Clustering, out of the box!!. I'll start with an easy example, creating a Queue named "MyClusteredQueue". In this example I'm using JBoss AS 5.1. and two computers connected on the same network, with these IP's: - Computer A: 192.168.0.143 - Computer B: 192.168.0.210 So, here are the steps: 1) Install the JBoss on both computers. We are going to use the "all" configuration for both computers. 2) We create our Queue on both servers. Go to $JBOSS_HOME/server/all/deploy/messaging/ and edit the destinations-service.xml file. Add the MyClusteredQueue before the last server tag. It looks like this: jboss.messaging:service=ServerPeer jboss.messaging:service=PostOffice true This is how you add a Queue to the JBoss, and the people how are familiar with this, the only new thing is to add the attribute "Clustered". This step must be set on both computers. At the end of the article you can find the files. 3) Write the MDB to consume the messages, and deploy it on the two computers. (I'm using an EJB 3 - MDB style). import java.net.InetAddress; import javax.ejb.ActivationConfigProperty; import javax.ejb.MessageDriven; import javax.jms.Message; import javax.jms.MessageListener; import javax.jms.ObjectMessage; import org.apache.log4j.Logger; /** * @author felipeg * */ @MessageDriven(activationConfig = { @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"), @ActivationConfigProperty(propertyName="destination", propertyValue="queue/MyClusteredQueue") }) public class JMSClusterClientHandler implements MessageListener { Logger log = Logger.getLogger(JMSClusterClientHandler.class); @Override public void onMessage(Message message) { try{ if (message instanceof ObjectMessage) { InetAddress addr = InetAddress.getLocalHost(); log.info("########## Processing Host: " + addr.getHostName() + " ##########" ); ObjectMessage objMessage = (ObjectMessage) message; Object obj = objMessage.getObject(); log.info("Object received:" + obj.toString()); } } catch (Exception e) { e.printStackTrace(); } } } 4) Start the jboss with the following options: Computer A: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.143 -Djboss.messaging.ServerPeerID=1 Computer B: $ cd $JBOSS_HOME/bin $ ./run.sh -c all -b 192.168.0.210 -Djboss.messaging.ServerPeerID=2 It is necesary to give an ID to each server and this is accomplished with this directive: -Djboss.messaging.ServerPeerID When you start the jboss on computer A, you should see the logs (server.log) telling you that there is one node ready and listening, and once you start the jboss on computer B, on the log will appear the two nodes, the two IP's ready to consume messages. 5) Now it's time to send a Message to the Queue. To accomplish this it's necessary to change the connection factory to "ClusteredConnectionFactory" (JMSDispatcher.java - See the code below). Also on the jndi.properties (if you are using the default InitialContext) file it's necessary to add the two computers ip's separated by comma to the java.naming.provider.url property. (In my case a create a Properties variable and I set all the necessary properties, JMSDispatcher.java - see the code below). java.naming.provider.url=192.168.0.143:1099,192.168.0.210:1099 The client that I wrote is a web application, that consist in one index.jsp page, which contains a form that prompts you for the name of the queue, the type of messaging (Queue or Topic), the server ip and port, how many times it will send the message and the actual message to be sent; also the web application has a Servlet (JMSClusteredClient.java - see code below) that receives the postback and helper class (JMSDispatcher.java - see code below) that sends the message to the jboss servers. You can to deploy it in any computer. In my case I deployed it on the Computer A. And you can access it through this URL: http://192.168.0.143:8080/JMSWeb/ (just modify the IP where the client war was deployed). If you notice (on the index.jsp - code below) I've already put some default values that reflects the name of the Queue, and the IP's of my two computers. Now, If you increment the number of times that the message will be sent (maybe a 10) and fill out the message box, and click "Send" you should see on the two servers some of the messages being consumed by the MDB. Here are the Files to create the client: index.jsp JMS Clustered - Test Client Server: QueueTopic Times:Message: Servlet: JMSClusteredClient.java public class JMSClusteredClient extends HttpServlet { private static final long serialVersionUID = 1L; /** * @see HttpServlet#service(HttpServletRequest request, HttpServletResponse response) */ protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { PrintWriter out = response.getWriter(); String topicqueue = request.getParameter("topicqueue"); String message = request.getParameter("message"); String server = request.getParameter("server"); String messageType = request.getParameter("messageType"); String times = request.getParameter("times"); int intTimes = Integer.parseInt(times); JMSDispatcher dispatcher = new JMSDispatcher(); dispatcher.setTopicQueueName(topicqueue); dispatcher.setServer(server); dispatcher.setMessageType(messageType); try { for(int count =1; count <= intTimes;count++){ dispatcher.sendMessage( count + " of " + times + " " + message); } out.println("Message [" + message + "] sent successfully to [" + topic + "] to the [" + server + "] server " + times + " times."); } catch (JMSException e) { e.printStackTrace(); out.println("Error:" + e.getMessage()); } catch (NamingException e) { out.println("Error:" + e.getMessage()); e.printStackTrace(); } finally{ out.close(); } } } A utility to send the messages: JMSDispatcher.java public class JMSDispatcher { /** * */ private static final long serialVersionUID = 7105145023422143880L; private static Logger log = Logger.getLogger(JMSDispatcher.class); private final String CONNECTION_FACTORY_CLUSTERED = "ClusteredConnectionFactory"; private final String CONNECTION_FACTORY = "ConnectionFactory"; private final String TOPIC = "TOPIC"; private final String QUEUE = "QUEUE"; private String topicQueueName; private String server; private String messageType; public void setTopicQueueName(String value){ this.topicQueueName = value; } public void setServer(String value){ this.server = value; } public void setMessageType(String value){ this.messageType = value; } public void sendMessage(Object objectMessage) throws JMSException, NamingException{ log.debug("##### Setting up a Queue/Topic Message: #####"); if (TOPIC.equals(messageType)){ sendTopicMessage(objectMessage); } else if (QUEUE.equals(messageType)){ sendQueueMessage(objectMessage); } log.debug("##### Publishing Message: Done #####"); } private void sendQueueMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); QueueConnectionFactory qcf = (QueueConnectionFactory) initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); QueueConnection queueConn = qcf.createQueueConnection(); Queue queue = (Queue) initialContext.lookup(topicQueueName); QueueSession queueSession = queueConn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE); queueConn.start(); QueueSender send = queueSession.createSender(queue); ObjectMessage om = queueSession.createObjectMessage((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Queue: " + queueName + "#####"); send.send(om); send.close(); queueConn.stop(); queueSession.close(); queueConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private void sendTopicMessage(Object objectMessage) throws JMSException, NamingException{ try{ InitialContext initialContext = getInitialContext(); TopicConnectionFactory tcf = (TopicConnectionFactory)initialContext.lookup(CONNECTION_FACTORY_CLUSTERED); TopicConnection topicConn = tcf.createTopicConnection(); Topic topic = (Topic) initialContext.lookup(topicQueueName); TopicSession topicSession = topicConn.createTopicSession(false,TopicSession.AUTO_ACKNOWLEDGE); topicConn.start(); TopicPublisher send = topicSession.createPublisher(topic); ObjectMessage om = topicSession.createObjectMessage(); om.setObject((Serializable)objectMessage); setMessageProperties(om); log.debug("##### Publishing Message to a Topic: " + topicName + "#####"); send.publish(om); send.close(); topicConn.stop(); topicSession.close(); topicConn.close(); }catch(MessageFormatException ex){ log.error("##### The MESSAGE is not Serializable ####"); throw ex; }catch(MessageNotWriteableException ex){ log.error("##### The MESSAGE is not Readable ####"); throw ex; }catch(JMSException ex){ log.error("##### JMS provider fails to set the object due to some internal error. ####"); throw ex; } } private InitialContext getInitialContext() throws NamingException{ Properties jboss = new Properties(); jboss.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory"); jboss.put("java.naming.factory.url.pkgs", "org.jboss.naming:org.jnp.interfaces"); jboss.put("java.naming.provider.url", server); return new InitialContext(jboss); } } And the web.xml JMSWeb index.jsp JMSClusteredClient JMSClusteredClient com.blogspot.felipeg48.jms.web.JMSClusteredClient JMSClusteredClient /JMSClusteredClient Happy Clustering!!

The purpose of this article is to give you the details of our 100 node cluster demo. This demo is recorded and you can watch the 5 minute screencast Hazelcast is an open source clustering and highly scalable data distribution platform for Java. JVMs that are running Hazelcast will dynamically cluster and allow you to easily share and partition your application data across the cluster. Hazelcast is a peer-to-peer solution (there is no master node, every node is a peer) so there is no single point of failure. Communication among cluster members is always TCP/IP with Java NIO beauty. The default configuration comes with 1 backup so if a node fails, no data will be lost (you can specify the backup count). It is as simple as using java.util.{Map, Queue, Set, List}. Just add the hazelcast.jar into your classpath and start coding. When you download the Hazelcast, you will find a test.sh under bin directory. The test.sh runs an application which randomly makes 40% get, 40% put and 20% remove on a distributed map. In this demo the same test application will be used to see how it performs on 100 node cluster. Amazon EC2 and S3 An easy to use and scalable cloud environment was needed for demo so we decided to use Amazon EC2 for server instances (nodes) and S3 service to store demo application zip and configuration files. With its newly announced Java SDK, it is very simple to start/stop server instances and upload files to S3 programatically. Hazelcast AMI & Launcher The challenge here is that we are running an application on 100 nodes and dealing with each and every server in the cluster is a huge task. We don't want to ssh into every server and manually start the application. This part is automated by creating a special server image (AMI). The AMI contains Java Runtime and a launcher application we developed, which will download the demo application from Amazon S3, unzip it, and run the hazelcast/bin/test.sh in it. The Launcher is actually so generic that it can run any application; it doesn't care/know what test.sh contains. Deployer Deployment of the demo application is also automated so that we don't need to login into AWS Management Console and manually start instances. Deployer instantiates any number of Amazon EC2 servers with any AMI and also uploads the demo application zip file to S3. So the idea here is that, the Deployer will store the application into S3 and launch 100 EC2 instances with our image. The Launcher on each instance will download the application from S3 and run it. Demo Details. The smallest EC2 instances (m1.small) are used to run the demo. These are the virtual instances with CPU about 1.0 GHz. Also keep in mind that EC2 platform suffers from considerable amount of network latency. That's why we increased the thread count to 250 in our application. The following steps performed during the demo Download hazelcast-1.8.3.zip from www.hazelcast.com. Unzip the file and move the monitoring war file into tomcat6/webapps directory. Edit the test.sh under the bin directory: Add -Xmx1G -Xms1G Add -Dhazelcast.initial.wait.seconds=100 to make the cluster evenly partition on start so that migration can be avoided for better performance. Add t250 as an argument to the application to set thread count to 250. Remember the latency issue. Run the Deployer from IDE. Check from EC2 Management Console if 100 servers started. Start tomcat. Copy the public DNS name of one of the servers to connect to from monitoring tool. Go to http://localhost:8080/hazelcast-monitor-1.8.3/ (Hazelcast Monitoring Tool). Paste the address and connect to the cluster. Enjoy! Results You should always look for programatic ways of launching applications on the cloud. With these tools we were able to deploy and run the demo application on 100 servers in minutes. The entire Hazelcast cluster was making over 400,000 operations per second on the smallest EC2 instances. In our next demo we will experiment Hazelcast on large data set and even bigger cluster. Watch the screencast

A pipeline in software context is a very well-known architectural style in which a process consists of a series of steps to be followed in order to proceed the data, and the output of one step is the input of another step. This is also called the Pipes and Filters design pattern. The naming comes from the physical pipeline as this architectural style is very similar to a pipeline in which a stream of data comes in and leaves after being processed. The original idea of pipeline in software is implemented in Unix. This pattern is used in many places. Compiler pipeline, ASP.NET HTTP Pipeline, and workflows are three of many examples that I can mention. The pipes and filters style is implemented in various platforms with different techniques and technologies. Recently I was in a situation to implement this pattern and did some research to find more about the possible options to implement this pattern in the .NET Framework. Doing my research, I found many approaches introduced by community members but the most mature technique is the one that Oren Eini has described in his blog post using Generics. There is also an interesting technique described by Jeremy Likness using the yield keyword in C#. In this post I’m going to apply Oren’s approach and expand it to write a simple implementation of the classic KWIC example in Software Engineering. I liked Oren’s code because as he said, it’s comparatively simpler than other solutions introduced for this problem in the .NET Framework. An Overview of KWIC KWIC stands for Key Word in Context and is a classic problem in Software Engineering papers in which you try to create an index of words by sorting and aligning each word in a piece of text. David Parnas has a famous paper on modularity that uses KWIC as an example. There are some basic and advanced implementations of KWIC in different platforms but the main steps are: Reading the input Shifting the words in each line to get a new permutation Sorting the results Writing the output Interestingly, in this case the output of each step is the input of the next step which makes this a good candidate for the Pipes and Filters pattern. Implement the Pipes and Filters Pattern with Generics Oren’s technique for implementing the Pipes and Filters in the .NET Framework is based on a Generic interface and a Generic class. The Generic interface simulates the filter and the Generic class simulates the pipeline. The IOperation interface has a single method called Execute that is the implementation of the filter logic. Each filter should implement this interface. using System.Collections.Generic;namespace KwicPipesFilters{ public interface IOperation { IEnumerable Execute(IEnumerable input); } The use of a generic IEnumerable is a good choice because it leaves a lot of space for the developers to plug in any type that they want and use various types for their filters. The Pipeline class has an Execute and a Register method. Using the Register method, you add different filters to the pipeline and using the Execute method, you start processing the item in all the registered filters. using System.Collections.Generic;namespace KwicPipesFilters{ public class Pipeline { private readonly List> operations = new List>(); public Pipeline Register(IOperation operation) { operations.Add(operation); return this; } public void Execute() { IEnumerable current = new List(); foreach (IOperation operation in operations) { current = operation.Execute(current); } IEnumerator enumerator = current.GetEnumerator(); while (enumerator.MoveNext()); } } The implementation of the Pipeline class is straightforward: it keeps a list of filters and provides a Register function that lets you add new filters to your pipeline, and then use the Execute method to execute all the filters in the list to process an input. Reader The Reader filter reads the input text from a file and returns an IEnumerable list of lines. Of course, for the first filter in the pipe we don’t care about the input as the input is read inside the filter itself. using System;using System.Collections.Generic;using System.IO;namespace KwicPipesFilters{ public class Reader : IOperation { public IEnumerable Execute(IEnumerable input) { Console.Title = "Pipes and Filters Pattern in .NET"; Console.WriteLine("Enter the path of the file:"); return File.ReadLines(Console.ReadLine()); } } Shifter The Shifter filter is where the main logic of the KWIC application is implemented. It shifts the words in each line to find all the possible permutations suitable for the index. using System.Collections.Generic;namespace KwicPipesFilters{ public class Shifter : IOperation { public IEnumerable Execute(IEnumerable input) { List shifts = new List(); foreach (string line in input) { string[] words = line.Split(new char[] { ' ' }); for (int i = 0; i <= words.Length - 1; i++) { shifts.Add(string.Join(" ", words)); string firstWord = words[0]; for (int j = 1; j <= words.Length - 1; j++) { words.SetValue(words[j], j - 1); } words.SetValue(firstWord, words.Length - 1); } } return shifts; } } Here we have a basic implementation of the Shifter filter where we split the line into separate words based on the space between them, then shift all the words to find various permutations. Sorter Before returning the final results in the Writer filter, we need to sort the index alphabetically. This is done in the Sorter filter. using System.Collections.Generic;using System.Linq;namespace KwicPipesFilters{ public class Sorter : IOperation { public IEnumerable Execute(IEnumerable input) { LineComparer lineComparer = new LineComparer(); input.ToList().Sort(lineComparer); return input; } } Here I used a LineComparer class to implement the ICcomparer interface for the string type. using System.Collections.Generic;namespace KwicPipesFilters{ public class LineComparer : IComparer { public int Compare(string x, string y) { return string.Compare(x, y); } } Writer Obviously, the last filter should write the index to the output for the user and that’s the purpose of the Writer filter. using System;using System.Collections.Generic;namespace KwicPipesFilters{ public class Writer : IOperation { public IEnumerable Execute(IEnumerable input) { foreach (string line in input) { Console.WriteLine(); Console.WriteLine(line); } Console.ReadLine(); yield break; } } As you see, this filter uses a yield break to avoid returning any result. Pipeline Having all the filter implemented, I also need to implement the pipeline itself in order to register the filters and make the whole thing work. I do this in my KwicPipeline class with a simple code that it has. namespace KwicPipesFilters{ public class KwicPipeline : Pipeline { public KwicPipeline() { Register(new Reader()); Register(new Shifter()); Register(new Sorter()); Register(new Writer()); } } I inherit from the Pipeline class and register my filters in the public constructor. Putting It Together There is only one step remained and that is putting all these things together to start the pipeline. All I need to do is to create an instance of the KwicPipeline class, call its Execute method, and leave the rest to my pipes and filters. namespace KwicPipesFilters{ class Program { static void Main(string[] args) { KwicPipeline pipeline = new KwicPipeline(); pipeline.Execute(); } } Conclusion In this post I implemented the Pipes and Filters pattern in the .NET Framework using a simple and generalized technique that relies on Generics to implement the KWIC application. In my opinion this is one of the best ways to implement this pattern in the .NET Framework. I have uploaded the sample source code package here. Note that the solution is created using Visual Studio 2010 RC1. There are other techniques to implement this pattern in .NET and one specific technique that I have in mind is using the Windows Workflow Foundation. I may work more on this idea and write about it later.

There are at least four methods that can be used in different combinations to make the process of setting up a complete development environment a lot less painful.

The emergence of version control systems makes both promiscuous and continuous integration merging techniques more attractive. Which is better?

My goal is to compare Sonatype Nexus and JFrog Artifactory,the two leading open source Maven repository managers.

This is the second part of a two-part article introducing Jeff De Luca’s Feature Driven Development (FDD) process. In particular, we are looking at how FDD differs from Scrum and eXtreme Programming-inspired approaches when it comes to working with larger teams and projects. In the first part we briefly introduced the ‘just enough’ upfront activities that FDD uses to support the additional communication that inevitably is needed in a larger project/team. In the second part of the article we cover how FDD leverages the results of those upfront activities within the highly iterative, self-managing, organized-chaos that is the delivery engine room of an FDD project. The Engine Room: Delivering Frequent, Tangible Working Results Once there is an initial overall model (FDD Process #1), an initial overall features list (FDD Process #2), and an initial overall plan (FDD Process #3) in place, an FDD project is ready to start delivering the required software feature by feature. Peter Coad, the Chief Architect on the original FDD project used the phrase ‘Deliver frequent, tangible, working results’ as a mantra to impress upon people the idea of delivering real, completed, client-valued function as often as possible. Scrum and eXtreme Programming do this using fixed length iterations of a calendar month or 2-4 weeks. FDD is different. Each Chief Programmer (lead developer) runs a series of iterations, each of which is normally a matter of a few days, and never longer than two weeks. At the start of each of these iterations, each Chief Programmer selects the next few features that make sense to implement from the backlog of feature sets (activities) that were assigned to him or her in FDD Process #2. The Chief Programmer leads the development of these features through FDD processes #4 and #5, Design by Feature (DBF) and Build By Feature (BBF). Note that iterations through the DBF/BBF processes are not fixed length, and Chief Programmers do not synchronize the start and end of their iterations with each other. In addition, the DBF/BBF processes are always executed as a pair (FDD describes them as two separate processes rather than one combined process for psychological reasons). FDD Process #4: Design By Feature After selecting the features for the iteration, a Chief Programmer needs to form their feature team. Yes, feature teams are formed and disbanded for each iteration through the DBF/BBF process pair. Using the knowledge gained from the modeling process (FDD Process #1), the Chief Programmer identifies the domain classes that are likely to be involved in this iteration, and forms his or her feature team from the owners of those classes. In practice, this means: a feature team is small, typically 3 to 5 people, because features are small. By definition, a feature team comprises of all the class owners who need to modify their classes in the development of the features during that iteration. There is no need to wait for members of other teams to change code. Therefore, there are all the benefits of code ownership and a sense of collective ownership too. Class owners may find themselves a member of multiple feature teams at the same time. This does not happen as frequently as might be supposed because iterations are so short – days not weeks. When it does, it is not a big problem in practice. Chief Programmers work together to resolve any problematic conflicts and, with care, most developers can manage the demands of occasionally belonging to more than one feature team for a short time. Once formed, the Chief Programmer facilitates the collaborative analysis and design of the features for that iteration. Depending on the complexity, this may involve the team walking through the requirements in detail with a domain expert, and studying any existing relevant documents. It also involves agreeing on the interactions and other details that need to be added to the model to support the new features. The final step in the DBF part of the iteration is to review the design. For simple features, this may be a brief sanity check of the design held within the feature team. For more significant features, the Chief Programmer will typically involve other Chief Programmers or class owners so that they are aware and can comment on the impact of the proposed design. For small team projects, the object models are frequently small enough for individual or pairs of developers to create good designs while writing tests for a particular feature or user story. For larger projects, this is not necessarily the case and designs created purely by considering the tests a feature or user story must pass are more likely to be brittle and require significant refactoring. The DBF process in FDD ensures that the overall model also guides the design, helping to maintain its ‘conceptual integrity’ [Brooks]. FDD Process #5: Build By Feature The Build by Feature (BBF) part of the iteration involves the team members coding up the features, testing them at both unit level and feature level, and holding a code inspection before promoting the completed features into the project's regular build process. Testing FDD expects developers to unit test their code. It expects feature teams to test their features. FDD is not overly concerned with how this is achieved. Projects and feature teams are free to adopt the testing tools, frameworks, and level of formality and completeness that are most appropriate. FDD does not mind if tests are written before or after code. What FDD mandates, is that the feature team deliver code that has been appropriately tested and inspected. Only once the new features have passed testing and inspection is the source code allowed into the build process. Code Inspections Most people want to know why FDD mandates code inspections, especially those that have endured sitting through hours of boring, unproductive, ego-polishing/demolishing, point-scoring sessions that formed so-called code reviews, inspections or walkthroughs. The reason FDD mandates code inspections is that research has shown time and again that when done well, inspections find more defects and different kinds of defects than testing [McConnell]. Not only that but by examining the code of the more experienced, knowledgeable developers on the team and having them explain the idioms they use, less experienced developers learn better coding techniques. In addition, knowing that their code will be inspected and not be allowed in the build unless it conforms to the agreed standards encourages developers to pay more attention to conforming to those standards. One of the benefits of working in feature teams is that the whole feature team is on the hot seat during an inspection, not just one individual. This removes much of the intensity and anxiety inherent in inspecting one individuals work. The Chief Programmer decides on the level of formality of each inspection depending on the complexity and impact of the features developed in that iteration. Where the code has little or no impact outside the feature team, an inspection will usually only involve the feature team inspecting each other’s work. Where there is significant impact the Chief Programmer pulls in other Chief Programmers and developers to both verify the code and communicate the impact of the new features. eXtreme Programming acknowledges inspections as a ‘best practice' but promotes pair programming as the logical conclusion of applying this practice. Pair programming is obviously better than individual developers delivering code without any form of inspection. However, while FDD neither mandates nor forbids pair programming, a more-traditional inspection is: fresh eyes looking at the code, catching bad assumptions made by the coder/s a Chief Programmer present to ensure the techniques passed on are good. After all, developers can just as easily teach each other bad habits as well as good habits. a change of pace for developers, a chance to step away from the keyboard and mouse for a short while. With the wide availability of automated source code formatting and static analysis tools, code inspections can now be shorter, concentrating on the logic and coding idioms involved and not getting bogged down in nit-picking such as alignment of braces, etc. The Build FDD assumes some sort of regular build process. Some teams build weekly, others daily and others continuously. FDD avoids mandating any particular build regime. This enables the project team to apply the most applicable. If a continuous integration environment makes sense, then the team is free to employ the best there is. Progress Reports Agile projects like highly visible progress information. FDD projects are no exception. In fact, because larger projects frequently have higher profiles within an organization, presenting meaningful, accurate, timely project information appropriately at the different levels of leadership/management is even more important. Conventionally, FDD projects track the development of each feature through its DBF/BBF iteration against six milestones: domain walkthrough, design, design inspection, coding, testing and inspection, and promoted to build. For each feature, Chief Programmers record the actual date a milestone is reached. Tracking each feature through these six milestones enables the project to keep an eye on how much work is 'in progress'. Too many features at a particular milestone indicate a process problem. Those promoting Kanban and other Limited Work In Progress methods have formalized this idea to strictly define what is meant by 'too many' for each of their development iteration milestones/statuses. They then refuse to move an item to a new milestone/status if the limit on the number of items at that status has been reached. This forces a team to keep items moving forward through the process [Kanban]. FDD is not so formal, leaving the Chief Programmers and Development Manager to keep an eye informally on the amount of work in progress. The Big Wallchart, Burn-Down/Up Charts, Etc For general visibility of progress within a project, the team typically lists all the features in the project complete with their owning Chief Programmer, feature team members, and the dates of each milestone achieved on a suitable wall. In addition, features can be colored to show if they are started, in-progress, completed or blocked. This allows people to stand back from the wall and get a good visual feel for the overall status of the project. They can then walk up to the wall to zoom in on particular areas and activities in more detail. Recording the date each milestone is achieved enables a team to produce burn-down or burn-up charts analogous to those produced in Scrum and XP. Chief Programmers and Project Managers can determine from these if the underlying rate of feature completion is increasing, decreasing, or stable, etc. One of the best ways to achieve this is to have the Chief Programmers regularly (typically once a week) communicate progress to either the project manager or someone dedicated to the task. That person then produces whatever roll-up and burn-down charts desired. Having an administrative person, the equivalent of the Tracker role in eXtreme Programming, perform these report formatting duties frees the Chief Programmers to spend more time on making progress rather than formatting reports about it. Parking Lot Charts For reporting to senior management, the level of individual features is often too granular. Here, FDD projects typically use a graphical report format that known as the Parking Lot chart. In a Parking Lot chart, each group of ‘parking lots’ represents one of the subject areas from the features list. Each parking lot represents one of the activities within that subject area, and displays the name of that set of features, the number of features within it, and the percentage of those features that have been completed (typically both in text and using a progress bar). The parking lots are also colored to indicate whether the features in that activity have been started, completed, or have significant blockages. The FDD parking lot format has become so popular that Mike Cohn included it in his book, Agile Planning and Estimating [Cohn]. (click for larger image) Figure 1: Example Parking Lot Chart Conclusion Feature-Driven Development combines the key advantages of other popular agile approaches with model-centric techniques and other best practices that scale to much larger teams and projects. It defines three upfront activities that provide a conceptual and management framework within which a larger-than-usual agile team can add functionality to the software, feature by feature. It is also just as applicable for smaller teams tackling non-trivial problem domains where it is worth spending just a little time to sketch a map of the journey before dashing off down the agile coding highway. Even if you and your team decide not to adopt FDD as a whole, understanding why FDD is the way it is, can provide insight into scaling traditional agile approaches beyond small, largely independent teams. Finally, I would like to say thank you to Serguei Khramtchenko and Mark Lesk at Nebulon for their corrections and suggestions incorporated in this article. References [Brooks] Frederick P. Brooks, Jr., The Mythical Man-Month, Addison Wesley [Cohn] Cohn, Agile Planning and Estimating, Prentice-Hall PTR [FDD] FDD Community Site, www.featuredrivendevelopment.com/ [Kanban] The home of Kanban software development, www.limitedwipsociety.org/ [McConnell] McConnell, Code Complete, Microsoft [Nebulon] The Latest FDD Processes available from www.nebulon.com/articles/fdd/latestprocesses.html [Palmer-1] Palmer, Felsing, A Practical Guide to Feature-Driven Development, Prentice Hall PTR

We’re using TFS on my current project. Yes, yes, I know. It’s generally good practice to collect all the code under your team’s control in a single uber-solution as described in this Patterns and Practices PDF, Team Development with TFS Guide. If you then configure the TFS build server to build this solution, it’s default behaviour is to place the build output into a single folder, ‘Release’. Any web application projects in your solution will also be output to a folder called _PublishedWebsites\. This is very nice because it means that you can simply robocopy deploy the web application. Unfortunately there’s no similar default behaviour for other project types such as WinForms, console or library. It would be very nice if we could have a _PublishedApplications\ sub folder with the output of any selected project(s). Fortunately it’s not that hard to do. The way _PublishedWebsites works is pretty simple. If you look at the project file of your web application you’ll notice an import near the bottom: On my machine the MSBuildExtensionsPath property evaluates to C:\Program Files\MSBuild, if we open the Microsoft.WebApplication.targets file we can see that it’s a pretty simple MSBuild file that recognises when the build is not a desktop build, i.e. it’s a TFS build, and copies the output to: $(OutDir)_PublishedWebsites\$(MSBuildProjectName) I simply copied the Micrsoft.WebApplication.targets file, put it under source control with a relative path from my project files and changed _PublishedWebsites to _PublishedApplications and renamed the file CI.exe.targets. For each project that I want to output to _PublishedApplications, I simply added this import at the bottom of the project file: You can edit CI.exe.targets (or whatever you want to call it) to do your bidding. In my case, the only change so far is to add a couple of lines to copy the App.config file: There’s a lot of stuff in Microsoft.WebApplication.targets that’s only relevant to web applications and can be stripped out for other project types, but I’ll leave that as an exercise for the reader. There was also a discussion on StackOverflow, with some nice alternative suggestions of how you might want to do this. It’s worth checking out.