We all know how financially important it is for your app’s server architecture to handle peaks of load. This article discusses 5 tips for improving PHP Web performance.

I was running in to a problem with a Java project that occured only in IntelliJ Idea, but not on the command line, when running specific test classes in Maven. The exception stack trace had the following in it: Caused by: com.sun.jersey.api.container.ContainerException: No WebApplication provider is present That seems like an easy problem to fix - it is the exception message that is given when jersey can’t find the provider for JAX-RS. Fixing it is normally just a matter of making sure jersey-core is on the classpath to fulfill SPI requirements for JAX-RS. For some reason though this isn’t happening in IntelliJ Idea. I inspected the log output of the test run and it is quite clear that all of the jersey dependencies are on the classpath. Then it dawns me on the try running mvn dependency:tree from inside of Idea. Here is what I found: [INFO] +- org.mule.modules:mule-module-jersey:jar:3.2.1:provided [INFO] | +- com.sun.jersey:jersey-server:jar:1.6:provided [INFO] | +- com.sun.jersey:jersey-json:jar:1.6:provided [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:provided [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.7.1:provided [INFO] | +- com.sun.jersey:jersey-client:jar:1.6:provided [INFO] | \- org.codehaus.jackson:jackson-jaxrs:jar:1.8.0:provided ... [INFO] +- org.jclouds.driver:jclouds-sshj:jar:1.4.0-rc.3:compile [INFO] | +- org.jclouds:jclouds-compute:jar:1.4.0-rc.3:compile [INFO] | | \- org.jclouds:jclouds-scriptbuilder:jar:1.4.0-rc.3:compile [INFO] | +- org.jclouds:jclouds-core:jar:1.4.0-rc.3:compile [INFO] | | +- net.oauth.core:oauth:jar:20100527:compile [INFO] | | +- com.sun.jersey:jersey-core:jar:1.11:compile [INFO] | | +- com.google.inject.extensions:guice-assistedinject:jar:3.0:compile Notice how I have jersey-core 1.11 coming from jclouds-core but jersey 1.6 everywhere else. That, my friends, is a dependency convergence problem. Maven with its default set of plugins (read: no maven-enforcer-plugin) does not even warn you if something like this happens. In this case, somehow jclouds-core depends directly on jersey-core and happens to resolve the dependency to the version that jclouds-core declared first before jersey-core can be resolved as a transitive dependency on mule-module-jersey. To fix the symptom, all I had to do was add the jersey-core dependency explicitely as a top level dependency in my pom: com.sun.jersey jersey-core ${jersey.version} provided But doing so only fixes the symptom, not the problem. The real problem is that the maven project I’m working on does not presently attempt to detect or resolve dependency convergence problems. This is where the maven-enforcer-plugin comes in handy. You can have the enforcer plugin run the DependencyConvergence rule agaisnt your build and have it fail when you have potential conflicts in your transitive dependencies that you haven’t resolved through exclusions or declaring direct dependencies yet. Binding the maven-enforcer-plugin to your build would look something like this: org.apache.maven.plugins maven-enforcer-plugin 1.0.1 enforce enforce validate ... I chose to bind to the validate phase since that is the first phase to be run in the maven lifecycle. Now my build fails immediately and contains very useful output that looks like the following: Dependency convergence error for org.codehaus.jackson:jackson-jaxrs:1.7.1 paths to dependency are: +-com.nodeable:server:1.0-SNAPSHOT +-org.mule.modules:mule-module-jersey:3.2.1 +-com.sun.jersey:jersey-json:1.6 +-org.codehaus.jackson:jackson-jaxrs:1.7.1 and +-com.nodeable:server:1.0-SNAPSHOT +-org.mule.modules:mule-module-jersey:3.2.1 +-org.codehaus.jackson:jackson-jaxrs:1.8.0 There are many rules you can apply besides DependencyConvergence. However, if the output from the DependencyConvergence rule looks anything like mine does presently, it might take you a while before you get around to getting your maven build to pass and conform to other rules.

the question everybody always asks when they learn about activiti, is as old as software development itself: “how does it perform?”. up till now, when you would ask me that same question, i would tell you about how activiti minimizes database access in every way possible, how we break down the process structure into an ‘execution tree’ which allows for fast queries or how we leverage ten years of workflow framework development knowledge. you know, trying to get around the question without answering it. we knew it is fast, because of the theoretical foundation upon which we have built it. but now we have proof: real numbers …. yes, it’s going to be a lengthy post. but trust me, it’ll be worth your time! disclaimer: performance benchmarks are hard. really hard. different machines, slight different test setup … very small things can change the results seriously. the numbers here are only to prove that the activiti engine has a very minimal overhead, while also integrating very easily into the java eco-system and offering bpmn 2.0 process execution. the activiti benchmark project to test process execution overhead of the activiti engine, i created a little side project on github: https://github.com/jbarrez/activiti-benchmark the project contains currently 9 test processes, which we’ll analyse below. the logic in the project is pretty straightforward: a process engine is created for each test run each of the processes are sequentially executed on this process engine, using a threadpool from 1 up to 10 threads. all the processes are thrown into a bag, of which a number of random executions are drawn. all the results are collected and a html report with some nice charts are generated to run the benchmark, simply follow the instructions on the github page to build and execute the jar. benchmark results the test machine i used for the results is my (fairly old) desktop machine: amd phenom ii x4 940 3.0ghz, 8 gb 800mhz ram and an old-skool 7200 rpm hd running ubuntu 11.10. the database used for the test runs on the same machine on which the tests also run. so keep in mind that in a ‘real’ server environment the results could even be better! the benchmark project i mentioned above, was executed on a default ubuntu mysql 5 database. i just switched to the ‘large.cnf’ setting (which throws more ram at the db and stuff like that) instead the default config. each of the test processes ran for 2500 times, using a threadpool going from one to ten threads . in simpleton language: 2500 process executions using just one thread, 2500 threads using two threads, 2500 process executions using three … yeah, you get it. each benchmark run was done using a ‘default’ activiti process engine. this basically means a ‘regular’ standalone activiti engine, created in plain java. each benchmark run was also done in a ‘spring’ config. here, the process engine was constructed by wrapping it in the factory bean, the datasource is a spring datasource and also the transactions and connection pool is managed by spring (i’m actually using a tweaked bonecp threadpool) each benchmark run was executed with history on the default history level (ie. ‘audit’) and without history enabled (ie. history level ‘none’) . the processes are in detail analyzed in the sections below, but here are the integral results of the test runs already: activiti 5.9 – mysql – default – history enabled activiti 5.9 – mysql – default – history disabled activiti 5.9 – mysql – spring – history enabled activiti 5.9 – mysql – spring – history disabled i ran all the tests using the latest public release of activiti, being activiti 5.9. however, my test runs brought some potential performance fixes to the surface (i also ran the benchmark project through a profiler). it was quickly clear that most of the process execution time was done actually cleaning up when a process ended. basically, more than often queries were fired which were not necessary if we would save some more state in our execution tree. i sat together with daniel meyer from camunda and my colleague frederik heremans, and they’ve managed to commit fixes for this! as such, the current trunk of activiti, being activiti 5.10-snapshot at the moment, is significantly faster than 5.9 . activiti 5.10 – mysql – default – history enabled activiti 5.10 – mysql – default – history disabled activiti 5.10 – mysql – spring – history enabled activiti 5.10 – mysql – spring – history disabled from a high-level perspective (scroll down for detailed analysis), there are a few things to note: i had expected some difference between the default and spring config, due to the more ‘professional’ connection pool being used. however, the results for both environments are quite alike. sometimes the default is faster, sometimes spring. it’s hard to really find a pattern. as such, i omitted the spring results in the detailed analyses below. the best average timings are most of the times found when using four threads to execute the processes . this is probably due to having a quad-core machine. the best throughput numbers are most of the times found when using eight threads to execute the processes. i can only assume that is also has something to do with having a quad-core machine. when the number of threads in the threadpool go up, the throughput (processes executed / second) goes up, both it has a negative effect on the average time. certainly with more than six or seven threads, you see this effect very clear. this basically means that while the processes on itself take a little longer to execute, but due to the multiple threads you can execute more of these ‘slower’ processes in the same amount of time. enabling history does have an impact. often, enabling history will double execution time. this is logical, given that many extra records are inserted when history is on the default level (ie. ‘audit’). there was one last test i ran, just out of curiosity: running the best performing setting on an oracle xe 11.2 database. the oracle xe is a free version of the ‘real’ oracle database. no matter how hard, i tried, i couldn’t get it decently running on ubuntu. as such, i used an old windows xp install on that same machine. however, the os is 32 bit, wich means the system only has 3.2 of the 8gb of ram available. here are the results: activiti 5.10 – oracle on windows – default – history disabled the results speak for itself. oracle blows away any of the (single-threaded) results on mysql (and they are already very fast!). however, when going multi-threaded it is far worse than any of the mysql results. my guess is that these are due to the limitations of the xe version : only one cpu is used, only 1 gb of ram, etc. i would really like to run these test on a real oracle-managed-by-a-real-dba … feel free to contact me if you are interested ! in the next sections, we will take a detailed look into the performance numbers of each of the test processes. an excel sheet containing all the the numbers and charts below can be downloaded for yourself . process 1: the bare micromum (one transaction) the first process is not a very interesting one, business-wise at least. after starting the process, the end is immediately reached. not very useful on itself, but its numbers learn us one essential thing: the bare overhead of the activiti engine. here are the average timings: this process runs in a single transaction, which means that nothing is saved to the database when the history is disabled due to activiti’s optimizations. with history enabled, you’ll basically get the cost for inserting one row into the historical process instance table, which is around 4.44 ms here. it is also clear that our fix for activiti 5.10 has an enormous impact here. in the previous version, 99% of the time was spent in the cleanup check of the process. take a look at the best result here: 0.47 ms when using 4 threads to execute 2500 runs of this process. that’s only half a millisecond ! it’s fair to say that the activiti engine overhead is extremely small. the throughput numbers are equally impressive: in the best case here, 8741 processes are executed. per second. by the time you arrive here reading the post, you could have executed a few millions of this process . you can also see that there is little difference between 4 or 8 threads here. most of the execution time here is cpu time, and no potential collisions such as waiting for a database lock happens here. in these numbers, you can also easily see that the oracle xe doesn’t scale well with multiple threads (which is explained above). you will see the same behavior in the following results. process 2: the same, but a bit longer (one transaction) this process is pretty similar to the previous one. we have again only one transaction. after the process is started, we pass through seven no-op passthrough activities before reaching the end. some things to note here: the best result (again 4 threads, with history disabled) is actually better than the simpler previous process. but also note that the single threaded execution is a tad slower. this means that the process on itself is a bit slower, which is logical as is has more activities. but using more threads and having more activities in the process does allow for more potential interleaving. in the previous case, the thread was barely born before it was killed again. the difference between history enabled/disabled is bigger than the previous process. this is logical, as more history is written here (for each activity one record in the database). again, activiti 5.10 is far more superior to activiti 5.9. the throughput numbers follow these observations: there is more opportunity to use threading here. the best result lingers around 12000 process execution per second . again, it demonstrates the very lightweight execution of the activiti engine. process 3: parallelism in one transaction this process executes a parallel gateway that forks and one that joins in the same transaction. you would expect something along the lines of the previous results, but you’d be surprised: comparing these numbers with the previous process, you see that execution is slower. so why is this process slower, even if it has less activities? the reason lies with how the parallel gateway is implemented, especially the join behavior. the hard part, implementation-wise, is that you need to cope with the situation when multiple executions arrive at the join. to make sure that the behavior is atomic, we internally do some locking and fetch all child executions in the execution tree to find out whether the join activates or not. so it is quite a ‘costly’ operation, compared to the ‘regular’ activities. do mind, we’re talking here about only 5 ms single threaded and 3.59 ms in the best case for mysql . given the functionality that is required for implementing the parallel gateway functionality, this is peanuts if you’d ask me. the throughput numbers: this is the first process which actually contains some ‘logic’. in the best case above, it means 1112 processes can be executed in a second. pretty impressive, if you’d ask me! . process 4: now we’re getting somewhere (one transaction) this process already looks like something you’d see when modeling real business processes. we’re still running it in one database transaction though, as all the activities are automatic passthroughs. here we also have two forks and two joins. take a look at the lowest number: 6.88 ms on oracle when running with one thread. that’s freaking fast , taking in account all that is happening here. the history numbers are at least doubled here (activiti 5.10), which makes sense because there is quite a bit of activity audit logging going on here. you can also see that this causes to have a higher average time for four threads here, which is probably due to the implementation of the joining. if you know a bit about activiti internals, you’ll understand this means there are quite a bit of executions in the execution tree. we have one big concurrent root, but also multiple children which are sometimes also concurrent roots. but while the average time rises, the throughput definitely benefits: running this process with eight threads, allows you to do 411 runs of this process in a single second. there is also something peculiar here: the oracle database performs better with more thread concurrency. this is completely contrary with all other measurements, where oracle is always slower in that environment (see above for explanation). i assume it has something to do with the internal locking and forced update we are applying when forking/joining, which is better handled by oracle it seems. process 5: adding some java logic (single transaction) i added this process to see the influence of adding a java service task in a process. in this process, the first activity generates a random value, stores it as a process variable and then goes up or down in the process depending on the random value. the chance is about 50/50 to go up or down. the average timings are very very good. actually, the results are in the same range as those of process 1 and 2 above (which had no activities or only automatic passthroughs). this means that the overhead of integrating java logic into your process is nearly non-existant (nothing is of course for free). of course, you can still write slow code in that logic, but you can’t blame the activiti engine for that throughput numbers are comparable to those of process 1 and 2: very, very high. in the best case here, more than 9000 processes are executed per second . that indeed also means 9000 invocations of your own java logic. process 6, 7 and 8: adding wait states and transactions the previous processes demonstrated us the bare overhead of the activiti engine. here, we’ll take a look at how wait states and multiple transactions have influence on performance. for this, i added three test processes which contain user tasks. for each user task, the engine commits the current transaction and returns the thread to the client. since the results are pretty much compatible for these processes, we’re grouping them here. these are the processes: here are the average timings results, in order of the processes above. for the first process, containing just one user task: it is clear that having wait states and multiple transaction does have influence on the performance. this is also logical: before, the engine could optimize by not inserting the runtime state into the database, because the process was finished in one transaction. now, the whole state, meaning the pointers to where you are currently, need to be saved into the database. the process could be ‘sleeping’ like this for many days, months, years now …. the activiti engine doesn’t hold it into memory now anymore, and it is freed to give its full attention to other processes. if you check the results of the process with only one user task, you can see that in the best case (oracle, single thread – the 4 threads on mysql is pretty close) this is done in 6.27ms . this is really fast, if you take in account we have a few inserts (the execution tree, the task), a few updates (the execution tree) and deletes (cleaning up) going on here. the second process here, with 7 user tasks: the second chart learns us that logically, more transactions means more time. in the best case here the process is done in 32.12 ms . that is for seven transactions, which gives 4.6 ms for each transactions. so it is clear that average time scales in a linearly way when adding wait states. this makes of course sense, because transactions aren’t free. also note that enabling history does add quite some overhead here. this is due to having the history level set to ‘audit’, which stores all the user task information in the history tables. this is also noticeable from the difference between activiti 5.9 with history disabled and activiti 5.10 with history enabled: this is a rare case where activiti 5.10 with history enabled is slower than 5.9 with history disabled. but it is logical, given the volume of history stored here. and the third process learns us how user tasks and parallel gateways interact: the third chart learns us not much new. we have two user tasks now, and the more ‘expensive’ fork/join (see above). the average timings are how we expected them. the throughput charts are as you would expect given the average timings. between 70 and 250 processes per second. aw yeah! to save some space, you’ll need to click them to enlarge: process 9: so what about scopes? for the last process, we’ll take a look at ‘scopes’. a ‘scope’ is how we call it internally in the engine, and it has to do with variable visibility, relationships between the pointers indicating process state, event catching, etc. bpmn 2.0 has quite some cases for those scopes, for example with embedded subprocesses as shown in the process here. basically, every subprocess can have boundary events (catching an error, a message, etc) that only are applied on its internal activities when it’s scope is active. without going into too much technical details: to get scopes implemented in the correct way, you need some not so trivial logic. the example process here has 4 subprocesses, nested in each other. the inner process is using concurrency, which is a scope on itself again for the activiti engine. there are also two user tasks here, so that means two transactions. so let’s see how it performs: you can clearly see the big difference between activiti 5.9 and 5.10. scopes are indeed an area where the fixes around the ‘process cleanup’ at the end have a huge benefit, as many execution objects are created and persisted to represent the many different scopes. single threaded performance is not so good on activiti 5.9. luckily, as you can see from the gap between the blue and the red bars, those scopes do allow for high concurrency. the numbers of oracle, combined with the multi-threaded results of the 5.10 tests, do prove that scopes are now efficiently handled by the engine. the throughput charts prove that the process nicely scales with more threads, as you can see by the big gap between the red and green line in the second last block. in the best case, 64 processes of this more complex process are handled by the engine. random execution if you have already clicked on the full reports at the beginning of the post, you probably have noticed also random execution is tested for each environment. in this setting, 2500 process executions were done, both the process was randomly chosen. as shown in those reports this meant that over 2500 executions, each process was executed almost the same number of times (normal distribution). this last chart shows the best setting (activiti 5.10, history disabled) and how the throughput of those random process executions goes when adding more threads: as we’ve seen in many of the test above, once passed four threads things don’t change that much anymore. the numbers (167 processes/second) prove that in a realistic situation (ie. multiple processes executing at the same time), the activiti engine nicely scales up. conclusion the average timing charts show two things clearly: the activiti engine is fast and overhead is minimal ! the difference between history enabled or disabled is definitely noticeably. sometimes it comes even down to half the time needed. all history tests were done using the ‘audit’ level, but there is a simpler history level (‘activity’) which might be good enough for the use case. activiti is very flexible in history configuration, and you can tweak the history level for each process specifically. so do think about the level your process needs to have, if it needs to have history at all ! the throughput charts prove that the engine scales very well when more threads are available (ie. any modern application server). activiti is well designed to be used in high-throughput and availability (clustered) architectures . as i said in the introduction, the numbers are what they are: just numbers. my main point which i want to conclude here, is that the activiti engine is extremely lightweight. the overhead of using activiti for automating your business processes is small. in general, if you need to automate your business processes or workflows, you want top-notch integration with any java system and you like all of that fast and scalable … look no further!

Ah!! The name is everywhere, carried with the wind. Apache Hadoop!! The BIG DATA crunching platform! We all know how alien it can be at start too! Phew!! :o Its my personal experience, nearly 11 months before, I was trying to install HBase, I faced few issues! The problem was version compatibility. Ex: "HBase some x.version" with "Hadoop some y.version". This is a real issue because you will never know which package of what version blends well with the other, unless, someone has tested it. This testing again depends on the environment where they have set up and could be another issue. There was a pressing demand for the management of distributions and then comes an open source project which attempts to create a fully integrated and tested Big Data management distribution, "Apache Bigtop". Goals of Apache Bigtop: -Packaging -Deployment -Integration Testing of all the sub-projects of Hadoop. This project aims at system as a whole, than the individual project. I love the way Doug Cutting quoted in the Keynote, back then, wherein he expressed the similarity between Hadoop and Linux kernel,and the corresponding similarity between the big stack of Hadoop ( Hive, Hbase, Pig, Avro, etc.) and the fully operational operating systems with its distributions (RedHat, Ubuntu, Fedora, Debian etc.). This is an awesome analogy! :) Life is made easy with Bigtop: Bigtop Hadoop distribution artifacts won't make you feel that you live in an alien world! After installing, you will get a chance to blend a Hadoop cluster in any mode, with the sub-projects of it. Its all for you to garnish next! :) Setup Of Bigtop and Installing Hadoop: It's time to welcome all your packages home. [I also mean /home/..] ;) I've tested on Ubuntu 11.04 and here goes a quick and easy installation process. Step 1: Installing the GNU Privacy Guard key, a key management system to access all public key directories. wget -O- http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/GPG-KEY-bigtop | sudo apt-key add - Step 2: Get the repo file from the link http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/ubuntu/bigtop.list sudo wget -O /etc/apt/sources.list.d/bigtop.listhttp://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/ubuntu/bigtop.list sudo gedit /etc/apt/sources.list.d/bigtop.list uncomment the mirror link near by. The first link worked for me. deb http://apache.01link.hk/incubator/bigtop/stable/repos/ubuntu/ bigtop contrib Step 3: Updating the apt cache sudo apt-get update Step 4: Checking in the artifacts sudo apt-cache search hadoop Image: Search in the apt cache Step 5: Set your JAVA_HOME export JAVA_HOME=path_to_your_Java export $JAVA_HOME in ~/.bashrc Step 6: Installing the complete Hadoop stack sudo apt-get install hadoop\* Image: (above) Running Hadoop: Step 1: Formatting the namendoe sudo -u hdfs hadoop namenode -format Image : Formatting the namenode Step 2: Starting the Namenode, Datanode, Jobtracker, Tasktracker of Hadoop for i in hadoop-namenode hadoop-datanode hadoop-jobtracker hadoop-tasktracker ; do sudo service $i start ; done Now, the cluster is up and running. Image : Start all the services Step 3: Creating a new directory in hdfs sudo -u hdfs hadoop fs -mkdir /user/bigtop bigtop is the directory name in the user $USER sudo -u hdfs hadoop fs -chown $USER /user/bigtop Image : Create a directory in HDFS Step 4: List the directories in file system hadoop fs -lsr / Image : HDFS directories Step 5: Running a sample pi example hadoop jar /usr/lib/hadoop/hadoop-examples.jar pi 10 1000 Image : Running a sample program Job Completed! Enjoy with your cluster! :) We shall see what more blending could be done with Hadoop (with Hive, Hbase, etc.) in the next post! Until then, Happy Learning!! :):)

JBoss AS 7 is neat but the documentation is still quite lacking (and error messages not as useful as they could be). This post summarizes how you can create your own JavaEE-compliant login module for authenticating users of your webapp deployed on JBoss AS. A working elementary username-password module provided. Why use Java EE standard authentication? Java EE security primer A part of the Java EE specification is security for web and EE applications, which makes it possible both to specify declarative constraints in your web.xml (such as “role X is required to access resources at URLs “/protected/*”) and to control it programatically, i.e. verifying that the user has a particular role (see HttpServletRequest.isUserInRole). It works as follows: You declare in your web.xml: Login configuration – primarily whether to use browser prompt (basic) or a custom login form and a name for the login realm The custom form uses “magic” values for the post action and the fields, starting with j_, which are intercepted and processed by the server The roles used in your application (typically you’d something like “user” and perhaps “admin”) What roles are required for accessing particular URL patterns (default: none) Whether HTTPS is required for some parts of the application You tell your application server how to authenticate users for that login realm, usually by associating its name with one of the available login modules in the configuration (the modules ranging from simple file-based user list to LDAP and Kerberos support). Only rarely do you need to create your own login module, the topic of this post. If this is new for you than I strongly recommend reading The Java EE 5 Tutorial – Examples: Securing Web Applications (Form-Based Authentication with a JSP Page incl. security constraint specification, Basic Authentication with JAX-WS, Securing an Enterprise Bean, Using the isCallerInRole and getCallerPrincipal Methods). Why to bother? Declarative security is nicely decoupled from the business code It’s easy to propagate security information between a webapp and for example EJBs (where you can protect a complete bean or a particular method declaratively via xml or via annotations such as @RolesAllowed) It’s easy to switch to a different authentication mechanism such as LDAP and it’s more likely that SSO will be supported Custom login module implementation options If one of the login modules (part of a security domain) provided out of the box with JBoss, such as UsersRoles, Ldap, Database, Certificate, isn’t sufficient for you then you can adjust one of them or implement your own. You can: Extend one of the concrete modules, overriding one or some of its methods to ajdust to your needs – see f.ex. how to override the DatabaseServerLoginModule to specify your own encryption of the stored passwords. This should be your primary choice, of possible. Subclass UsernamePasswordLoginModule Implement javax.security.auth.spi.LoginModule if you need maximal flexibility and portability (this is a part of Java EE, namely JAAS, and is quite complex) JBoss EAP 5 Security Guide Ch. 12.2. Custom Modules has an excellent description of the basic modules (AbstractServerLoginModule, UsernamePasswordLoginModule) and how to proceed when subclassing them or any other standard module, including description of the key methods to implement/override. You must read it. (The guide is still perfectly applicable to JBoss AS 7 in this regard.) The custom JndiUserAndPass module example, extending UsernamePasswordLoginModule, is also worth reading – it uses module options and JNDI lookup. Example: Custom UsernamePasswordLoginModule subclass See the source code of MySimpleUsernamePasswordLoginModule that extends JBoss’ UsernamePasswordLoginModule. The abstract UsernamePasswordLoginModule (source code) works by comparing the password provided by the user for equality with the password returned from the method getUsersPassword, implemented by a subclass. You can use the method getUsername to obtain the user name of the user attempting login. Implement abstract methods getUsersPassword() Implement getUsersPassword() to lookup the user’s password wherever you have it. If you do not store passwords in plain text then read how to customize the behavior via other methods below getRoleSets() Implement getRoleSets() (from AbstractServerLoginModule) to return at least one group named “Roles” and containing 0+ roles assigned to the user, see the implementation in the source code for this post. Usually you’d lookup the roles for the user somewhere (instead of returning hardcoded “user_role” role). Optionally extend initialize(..) to get access to module options etc. Usually you will also want to extend initialize(Subject subject, CallbackHandler callbackHandler, Map sharedState, Map options) (called for each authentication attempt), To get values of properties declared via the element in the security-domain configuration – see JBoss 5 custom module example To do other initialization, such as looking up a data source via JNDI – see the DatabaseServerLoginModule Optionally override other methods to customize the behavior If you do not store passwords in plain text (a wise choice!) and your hashing method isn’t supported out of the box then you can override createPasswordHash(String username, String password, String digestOption) to hash/encrypt the user-supplied password before comparison with the stored password. Alternatively you could override validatePassword(String inputPassword, String expectedPassword) to do whatever conversion on the password before comparison or even do a different type of comparison than equality. Custom login module deployment options In JBoss AS you can Deploy your login module class in a JAR as a standalone module, independently of the webapp, under /modules/, together with a module.xml – described at JBossAS7SecurityCustomLoginModules Deploy your login module class as a part of your webapp (no module.xml required) In a JAR inside WEB-INF/lib/ Directly under WEB-INF/classes In each case you have to declare a corresponding security-domain it inside JBoss configuration (standalone/configuration/standalone.xml or domain/configuration/domain.xml): The code attribute should contain the fully qualified name of your login module class and the security-domain’s name must match the declaration in jboss-web.xml: form-auth true The code Download the webapp jboss-custom-login containing the custom login module MySimpleUsernamePasswordLoginModule, follow the deployment instructions in the README.

This is a quick introduction to Selenium WebDriver in Python on Ubuntu/Debian systems. WebDriver (part of Selenium 2) is a library for automating browsers, and can be used from a variety of language bindings. It allows you to programmatically drive a browser and interact with web elements. It is most often used for test automation, but can be adapted to a variety of web scraping or automation tasks. To use the WebDriver API in Python, you must first install the Selenium Python bindings. This will give you access to your browser from Python code. The easiest way to install the bindings is via pip. On Ubuntu/Debian systems, this will install pip (and dependencies) and then install the Selenium Python bindings from PyPI: $ sudo apt-get install python-pip $ sudo pip install selenium After the installation, the following code should work: #!/usr/bin/env python from selenium import webdriver browser = webdriver.Firefox() browser.get('http://www.ubuntu.com/') This should open a Firefox browser sessions and navigate to http://www.ubuntu.com/ Here is a simple functional test in Python, using Selenium WebDriver and the unittest framework: #!/usr/bin/env python import unittest from selenium import webdriver class TestUbuntuHomepage(unittest.TestCase): def setUp(self): self.browser = webdriver.Firefox() def testTitle(self): self.browser.get('http://www.ubuntu.com/') self.assertIn('Ubuntu', self.browser.title) def tearDown(self): self.browser.quit() if __name__ == '__main__': unittest.main(verbosity=2) Output: testTitle (__main__.TestUbuntuHomepage) ... ok ---------------------------------------------------------------------- Ran 1 test in 5.931s OK

Here are the most important subjects for software engineering, with brief explanations: 1.Object oriented analysis & design: For better maintainability, reusability and faster development, the most well accepted approach, shortly OOAD and its SOLID principals are very important for software engineering. 2.Software quality factors: Software engineering depends on some very important quality factors. Understanding and applying them is crucial. 3.Data structures & algorithms: Basic data structures like array, list, stack, tree, map, set etc. and useful algorithms are vital for software development. Their logical structure should be known. 4. Big-O notation: Big-O notation indicates the performance of an algorithm/code section. Understanding it is very important for comparing performances. 5.UML notation: UML is the universal and complete language for software design & analysis. If there is lack of UML in a development process, it feels there is no engineering. 6.Software processes and metrics: Software enginnering is not a random process. It requires a high level of systematic and some numbers to monitor those techniques. So, processes and metrics are essential. 7.Design patterns: Design patterns are standard and most effective solutions for specific problems. If you don't want to reinvent the wheel, you should learn them. 8.Operating systems basics: Learning OS basics is very important because all applications runs on it. By learning it, we can have better vision, viewpoints and performance for our applications. 9.Computer organization basics: All applications including OS requires a hardware for physical interaction. So, learning computer organization basics is vital again for better vision, viewpoints and performance. 10.Network basics: Network is related with computer organization, OS and the whole information transfer process. In any case we will face it while software development. So, it is important to learn network basics. 11.Requirement analysis: Requirement analysis is the starting point and one of the most important parts of software engineering. Performing it correctly and practically needs experience but it is very essential. 12.Software testing: Testing is another important part of software engineering. Unit testing, its best practices and techniques like black box, white box, mocking, TDD, integration testing etc. are subjects which must be known. 13.Dependency management: Library (JAR, DLL etc.) management, and widely known tools (Maven, Ant, Ivy etc.) are essential for large projects. Otherwise, antipatterns like Jar Hell are inevitable. 14.Continuous integration: Continuous integration brings easiness and automaticity for testing large modules, components and also performs auto-versioning. Its aim and tools (like Hudson etc.) should be known. 15.ORM (Object relational mapping): ORM and its widely known implementation Hibernate framework is an important technique for mapping objects into database tables. It reduces code length and maintenance time. 16.DI (Dependency Injection): DI or IoC (Inversion of Control) and its widely known implementation Spring framework makes life easy for object creation and lifetime management on big enterprise applications. 17.Version controlling systems: VCS tools (SVN, TFS, CVS etc.) are very important by saving so much time for collaborative works and versioning. Their logical viewpoint and standard cammands should be known. 18.Internationalization (i18n): i18n by extracting strings into external files is the best way of supporting multiple languages in our applications. Its practices on different IDEs and technologies must be known. 19.Architectural patterns: Understanding architectural design patterns (like MVC, MVP, MVVM etc.) is essential for producing a maintainable, clean, extendable and testable source code. 20.Writing clean code: Working code is not enough, it must be readable and maintainable also. So, code formatting and readable code development techniques are needed to be known and applied.

in this article we'll show you how you can use the w3c geolocation api to measure the speed and the heading of your car while driving. this article further uses svg to render the speed gauge and heading compass. what we'll create in this article is the following: here you can see two gauges. one will show the heading you're driving to, and the other shows the speed in kilometers. you can test this out yourself by using the following link: open this in gps enable device . once opened the browser will probably ask you to allow access to your location. if you enable this and start moving, you'll see the two gauges move appropriately. getting all this to work is actually very easy and consists of the following steps: alter the svg images so we can rotate the needle and add to page. use the geolocation api to determine the current speed and heading update the needle based on the current and previous value we'll start with the svg part. alter the svg images so we can rotate the needle and add to page for the images i decided to use svg. svg has the advantage that it can scale without losing detail, and you can easily manipulate and animate the various parts of a svg image. both the svg images were copied from openclipart.org : compass rose speedometer these are vector graphics, both created using illustrator. before we can rotate the needles in these images we need to make a couple of small changes to the svg code. with svg you can apply matrix transformations to each svg element, with this you can easily rotate, skew, scale or translate a component. besides the matrix transformation you can also apply the rotation and translation directly using the translate and rotate keywords. in this example i've used the translaten and rotate functions directly. when working with these functions you have to take into account that the rotate function doesn't rotate around the center of the component, it rotates around point 0,0. so we need to make sure that for our needles the point we want to rotate around is set at 0,0. without diving into too much details, i removed the two needles from the image, and added them as a seperate group to the svg image. i then made sure the needles we're drawn relative to the 0,0 point i wanted to rotate around. for the speedometer the needle is now defined as this: and for the compass the needle is defined like this: if you know how to read svg, you can see that these figures are now drawn around their rotation point (the bottom center for the speedomoter and the center for the compass). as you can see we also added a specific id for both these elements. this way we can reference them directly from our javascript later on and update the transform property from a jquery animation. next we just need to add these to the page. for this i used d3.js , which has all kinds of helper functions for svg and which you can use to load these elements like this: function loadgraphics() { d3.xml("assets/compass.svg", "image/svg+xml", function(xml) { document.body.appendchild(xml.documentelement); }); d3.xml("assets/speed.svg", "image/svg+xml", function(xml) { document.body.appendchild(xml.documentelement); }); } and with this we've got our visualization components ready. use the geolocation api to determine the current speed and heading the next step is using the geolocation api to access the speed and heading properties. you can get this information from the position object that is provided to you by this api: interface position { readonly attribute coordinates coords; readonly attribute domtimestamp timestamp; }; this object has a coordinate object that contains the information we're looking for: interface coordinates { readonly attribute double latitude; readonly attribute double longitude; readonly attribute double? altitude; readonly attribute double accuracy; readonly attribute double? altitudeaccuracy; readonly attribute double? heading; readonly attribute double? speed; }; a lot of useful attributes, but we're only interested in these last two. the heading (from 0 to 360) shows the direction we're moving in, and the speed in meters per second is, as you've probably guessed, the speed we're moving at. there are two different options to get these values. we can poll ourselves for these values (e.g. setinterval) or we can wait wacth our position. in this second case you automatically recieve an update. in this example we use the second approach: function initgeo() { navigator.geolocation.watchposition( geosuccess, geofailure, { enablehighaccuracy:true, maximumage:30000, timeout:20000 } ); //movespeed(30); //movecompassneedle(56); } var count = 0; function geosuccess(event) { $("#debugoutput").text("geosuccess: " + count++ + " : " + event.coords.heading + ":" + event.coords.speed); var heading = event.coords.heading; var speed = event.coords.speed; if (heading != null && speed !=null && speed > 0) { movecompassneedle(heading); } if (speed != null) { // update the speed movespeed(speed); } } with this piece of code, we register a callback function on the watchposition. we also add a couple of properties to the watchposition function. with these properties we tell the api to use gps (enablehighaccuracy) and set some timeout and caching values. whenever we receive an update from the api the geosuccess function is called. this function recieves a position object (shown earlier) that we use to access the speed and the heading. based on the value of the heading and the speed we update the compass and the speedomoter. update the needle based on the current and previous value to update the needles we use jquery animations for the easing. normally you use a jquery animation to animatie css properties of an object, but you can also use this to animate arbitrary properties. to animate the speedomoter we use the following: var currentspeed = {property: 0}; function movespeed(speed) { // we use a svg transform to move to correct orientation and location var translatevalue = "translate(171,157)"; // to is in the range of 45 to 315, which is 0 to 260 km var to = {property: math.round((speed*3.6/250) *270) + 45}; // stop the current animation and run to the new one $(currentspeed).stop().animate(to, { duration: 2000, step: function() { $("#speed").attr("transform", translatevalue + " rotate(" + this.property + ")") } }); } we create a custom object, currentspeed, with a single property. this property is set to the rotate ratio that reflects the current speed. next, this property is used in a jquery animation. note that we stop any existing animations, should we get an update when the current animation is still running. in the step property of the animation we set the transfrom value of the svg element. this will rotate the needle, in two seconds, from the old value to the new value. and to animate the compass we do pretty much the same thing: var currentcompassposition = {property: 0}; function movecompassneedle(heading) { // we use a svg transform to move to correct orientation and location var translatevalue = "translate(225,231)"; var to = {property: heading}; // stop the current animation and run to the new one $(currentcompassposition).stop().animate(to, { duration: 2000, step: function() { $("#compass").attr("transform", translatevalue + " rotate(" + this.property + ")") } }); } there is a smal bug i ran into with this setup. sometimes my phone lost its gps signal (running firefox mobile), and that stopped the dials moving. refreshing the webpage was enough to get things started again however. i might change this to actively pull the information using the getcurrentlocation api call, to see whether that works better. another issue is that there is no way, at least that i found, for you to disable the phone entering sleep mode from the browser. so unless you configure your phone to not go to sleep, the screen will go black.

Recently I needed the ability to use Reportlab’s flowables, but place them in fixed locations. Some of you are probably wondering why I would want to do that. The nice thing about flowables, like the Paragraph, is that they’re easily styled. If I could bold something or center something AND put it in a fixed location, then that would rock! It took a lot of Googling and trial and error, but I finally got a decent template put together that I could use for mailings. In this article, I’m going to show you how to do this too. Getting Started You’ll need to make sure you have Reportlab or you’ll end up with a whole lot of nothing. You can go here to grab it. While you wait for it to download you can continue reading this article or go do something else productive. Are you ready now? Then let’s get this show on the road! Now we just need to come up with an example. Fortunately I was working on something at my job that I’ve been able to dummy up into the following silly and incomplete form letter. Study the code closely because you never know when there will be a test from reportlab.lib.pagesizes import letter from reportlab.lib.styles import getSampleStyleSheet from reportlab.lib.units import mm, inch from reportlab.pdfgen import canvas from reportlab.platypus import Image, Paragraph, Table ######################################################################## class LetterMaker(object): """""" #---------------------------------------------------------------------- def __init__(self, pdf_file, org, seconds): self.c = canvas.Canvas(pdf_file, pagesize=letter) self.styles = getSampleStyleSheet() self.width, self.height = letter self.organization = org self.seconds = seconds #---------------------------------------------------------------------- def createDocument(self): """""" voffset = 65 # create return address address = """ Jack Spratt 222 Ioway Blvd, Suite 100 Galls, TX 75081-4016 """ p = Paragraph(address, self.styles["Normal"]) # add a logo and size it logo = Image("snakehead.jpg") logo.drawHeight = 2*inch logo.drawWidth = 2*inch ## logo.wrapOn(self.c, self.width, self.height) ## logo.drawOn(self.c, *self.coord(140, 60, mm)) ## data = [[p, logo]] table = Table(data, colWidths=4*inch) table.setStyle([("VALIGN", (0,0), (0,0), "TOP")]) table.wrapOn(self.c, self.width, self.height) table.drawOn(self.c, *self.coord(18, 60, mm)) # insert body of letter ptext = "Dear Sir or Madam:" self.createParagraph(ptext, 20, voffset+35) ptext = """ The document you are holding is a set of requirements for your next mission, should you choose to accept it. In any event, this document will self-destruct %s seconds after you read it. Yes, %s can tell when you're done...usually. """ % (self.seconds, self.organization) p = Paragraph(ptext, self.styles["Normal"]) p.wrapOn(self.c, self.width-70, self.height) p.drawOn(self.c, *self.coord(20, voffset+48, mm)) #---------------------------------------------------------------------- def coord(self, x, y, unit=1): """ # http://stackoverflow.com/questions/4726011/wrap-text-in-a-table-reportlab Helper class to help position flowables in Canvas objects """ x, y = x * unit, self.height - y * unit return x, y #---------------------------------------------------------------------- def createParagraph(self, ptext, x, y, style=None): """""" if not style: style = self.styles["Normal"] p = Paragraph(ptext, style=style) p.wrapOn(self.c, self.width, self.height) p.drawOn(self.c, *self.coord(x, y, mm)) #---------------------------------------------------------------------- def savePDF(self): """""" self.c.save() #---------------------------------------------------------------------- if __name__ == "__main__": doc = LetterMaker("example.pdf", "The MVP", 10) doc.createDocument() doc.savePDF() Now you’ve seen the code, so we’ll spend a little time going over how it works. First off we create a Canvas object that we can use without our LetterMaker class. We also create a styles dict and set up a few other class variables. In the createDocument method, we create a Paragraph (an address) using some HTML-like tags to control the font and line breaking behavior. Then we create a logo and size it before putting both items into a Reportlab Table object. You’ll note that I’ve left in a couple commented out lines that show how to place the logo without the table. We use the coord method to help position the flowable. I found it on StackOverflow and thought it was pretty handy. The body of the letter uses a little string substitution and puts the result into another Paragraph. We also use a stored offset to help us position things. I find that storing a couple of offsets for certain portions of the code is very helpful. If you use them carefully then you can just change a couple of offsets to move the content around on the document rather than having to edit the position of each element. If you need to draw lines or shapes, you can do them in the usual way with your canvas object. Wrapping Up I hope this code will help you in your PDF creation endeavors. I have to admit that I’m posting it on here as much for my own future benefit as for your own. I’m a little sad I had to strip out so much from it, but my organization wouldn’t like it very much if I posted the original. Regardless, you now have the tools to create some pretty fancy PDF documents with Python. Now you just have to get out there and do it!

I've seen a lot of discussion about how to monitor Camel based applications. Most people are looking for the following features: ability to view services (contexts, endpoints, routes), to view performance statistics (route throughput, etc) and to perform basic operations (start/stop routes, send messages, etc). This post will breakdown the options (that I know of) that are available today (as of Camel 2.8). If you have used other approaches or know of other ongoing development in this area, please let me know. JMX APIs Camel uses JMX to provide a standardized way to access metadata about contexts/routes/endpoints defined in a given application. Also, you can use JMX to interact with these components (start/stop routes, etc) in some interesting ways. I recently had some very specific Camel/ActiveMQ monitoring requests from a client. After looking at the options, we ended up building a standalone Tomcat web app that used JSPs, jQuery, Ajax and JMX APIs to view route/endpoint statistics, manage Camel routes (stop, start, etc) and monitor/manipulate ActiveMQ queues. It provided some much needed visibility and management features for our Camel/ActiveMQ based message processing application... CamelContext If you have a handle to the CamelContext, there are various APIs that can help describe and manage routes and endpoints. These are used by the existing Camel Web Console and can be used to build custom interface to retrieve and use this information in various ways... here are some of the notable APIs... getRouteDefinitions() getEndpoints() getEndpointsMap() getRouteStatus(routeId) startRoute(routeId) stopRoute(routeId) removeRoute(routeId) addRoutes(routeBuilder) suspendRoute(routeId) resumeRoute(routeId) With a little creativity, you can use these APIs to manage/monitor and re-wire a Camel application dynamically. Camel Web Console This console provides web and REST interfaces to Camel contexts/routes/endpoints and allows you to view/manage endpoints/routes, send messages to endpoints, viewing route statistics, etc. That being said, using this web console with an existing Camel application is tricky at the moment. It's currently deployed as a war file that only has access to the CamelContext defined in its embedded spring XML file. Though the entire camel-web project can be embedded and customized in your application if you desire (and know Scalate). Given my recent client requirements, I opted to build my own basic app using JSPs/JMX as described above. There has been some recent support for deploying this console in OSGI, where it should be able to view any CamelContexts deployed in the container, etc. However, I'm yet to see this work...more on this later. Using Camel APIs There are also a number of Camel technologies/patterns that can be used to add monitoring to existing routes. wire tap - can add message logging (to a file or JMS queue/topic, etc) or other inline processing advicewith - can be used to modify existing routes to apply before/after operations or add/remove operations in a route intercept - can be used to intercept Exchanges while they are in route, can apply to all endpoints, certain endpoints or just starting endpoints BrowsableEndpoint - is an interface which Endpoints may implement to support the browsing of the exchanges which are pending or have been sent on it. That being said, it takes some creativity to use these effectively and caution to not adversely affect the routes you are trying to monitor. Hyperic HQ You can use this tool to monitor Servicemix (or any process), but it more geared towards system monitoring and JVM stats. I didn't find it useful for any Camel specific monitoring. jConsole/VisualVM these are standard JMX based consoles. They aren't web based and can't be customized (easily anyways) to provide anything more than a tree-like view of JMX MBeans. If you know where to look though, you can do a lot with it. Summary These are just some quick notes at this point. As I learn about other ways of monitoring Camel, I'll update this list and give some more detailed comparison. Any comments are welcome...

In retrospect it was probably not a good idea to give the AnimationTimer its name, because it can be used for much more than just animation: measuring the fps-rate, collision detection, calculating the steps of a simulation, the main loop of a game etc. In fact, most of the time I saw AnimationTimer in action was not related to animation at all. Nevertheless there are cases when you want to consider using an AnimationTimer for your animation. This post will explain the class and show an example where AnimationTimer is used to calculate animations. The AnimationTimer provides an extremely simple, but very useful and flexible feature. It allows to specify a method, that will be called in every frame. What this method is used for is not limited and, as already mentioned, does not have anything to do with animation. The only requirement is, that it has to return fast, because otherwise it can easily become the bottleneck of a system. To use it, a developer has to extend AnimationTimer and implement the abstract method handle(). This is the method that will be called in every frame while the AnimationTimer is active. A single parameter is passed to handle(). It contains the current time in nanoseconds, the same as what you would get when calling System.nanoTime(). Why should one use the passed in value instead of calling System.nanoTime() or its little brother System.currentTimeMillis() oneself? There are several reasons, but the most important probably is, that it makes your life a lot easier while debugging. If you ever tried to debug code, that depended on these two methods, you know that you are basically screwed. But the JavaFX runtime goes into a paused state while it is waiting to execute the next step during debugging and the internal clock does not proceed during this pause. In other words no matter if you wait two seconds or two hours before you resume a halted program while debugging, the increment of the parameter will roughly be the same! AnimationTimer has two methods start() and stop() to activate and deactivate it. If you override them, it is important that you call these methods in the super class. The Animation API comes with many feature rich classes, that make defining an animation very simple. There are predefined Transition classes, it is possible to define a key-frame based animation using Timeline, and one can even write a custom Transition easily. But in which cases does it make sense to use an AnimationTimer instead? – Almost always you want to use one of the standard classes. But if you want to specify many simple animations, using an AnimationTimer can be the better choice. The feature richness of the standard animation classes comes with a price. Every single animation requires a whole bunch of variables to be tracked – variables that you often do not need for simple animations. Plus these classes are optimized for speed, not for small memory footprint. Some of the variables are stored twice, once in the format the public API requires and once in a format that helps faster calculation while playing. Below is a simple example that shows a star field. It animates thousands of rectangles flying from the center to the outer edges. Using an AnimationTimer allows to store only the values that are needed. The calculation is extremely simple compared to the calculation within a Timeline for example, because no advanced features (loops, animation rate, direction etc.) have to be considered. package fxsandbox; import java.util.Random; import javafx.animation.AnimationTimer; import javafx.application.Application; import javafx.scene.Group; import javafx.scene.Node; import javafx.scene.Scene; import javafx.scene.paint.Color; import javafx.scene.shape.Rectangle; import javafx.stage.Stage; public class FXSandbox extends Application { private static final int STAR_COUNT = 20000; private final Rectangle[] nodes = new Rectangle[STAR_COUNT]; private final double[] angles = new double[STAR_COUNT]; private final long[] start = new long[STAR_COUNT]; private final Random random = new Random(); @Override public void start(final Stage primaryStage) { for (int i=0; i

Some web applications may need a "Remember Me" functionality. This means that, after a user login, user will have access from same machine to all its data even after session expired. This access will be possible until user does a logout. If you are using Spring and its login form, then you should use "Remember Me" functionality already implemented inside the framework. Some web frameworks also offer a type of SignIn panel which already has "remember me" built-in. But in case you have to implement "Remember Me" functionality by your own, this can be easily achieved using Cookies. Java has a Cookie class named javax.servlet.http.Cookie. Algorithm is straight-forward: your login panel must contain a "Remember Me" check after a succesfull login with "Remember Me" check selected, you can create two cookies: one to keep the value for rememberMe and one to keep a token which has to identify the logged user. For sake of security, this token must never contain user name or user password. The ideea is to generate a random id as token value. And token value aside with user id must be saved in your storage (database) whenever a login is needed, you have to look if there is any cookie saved by you, and if so and your "rememberMe" value is true, you can take the user from storage based on your token and do an automatic login. when a logout is done, you have to delete the cookie that keeps the token To add a cookie, you have to specify the maximum age of the cookie in seconds : HttpServletResponse servletResponse = ...; Cookie c = new Cookie(COOKIE_NAME, encodeString(uuid)); c.setMaxAge(365 * 24 * 60 * 60); // one year servletResponse.addCookie(c); To delete a cookie, you have to find cookie by name and set its maximum age to 0, before adding it to servlet response: HttpServletRequest servletRequest = ...; HttpServletResponse servletResponse = ... ; Cookie[] cookies = servletRequest.getCookies(); for (int i = 0; i < cookies.length; i++) { Cookie c = cookies[i]; if (c.getName().equals(COOKIE_NAME)) { c.setMaxAge(0); c.setValue(null); servletResponse.addCookie(c); } }

As I mentioned in another post about a month ago I’ve been playing around with a neo4j graph in which I have the following relationship between nodes: One thing I wanted to do was work out which node is the most connected on the graph, which would tell me who’s worked with the most people. I started off with the following cypher query: query = " START n = node(*)" query << " MATCH n-[r:colleagues]->c" query << " WHERE n.type? = 'person' and has(n.name)" query << " RETURN n.name, count(r) AS connections" query << " ORDER BY connections DESC" I can then execute that via the neo4j console or through irb using the neography gem like so: > require 'rubygems' > require 'neography' > neo = Neography::Rest.new(:port => 7476) > neo.execute_query query # cut for brevity {"data"=>[["Carlos Villela", 283], ["Mark Needham", 221]], "columns"=>["n.name", "connections"]} That shows me each person and the number of people they’ve worked with but I wanted to be able to see the most connected person in each office . Each person is assigned to an office while they’re working out of that office but people tend to move around so they’ll have links to multiple offices: I put ‘start_date’ and ‘end_date’ properties on the ‘member_of’ relationship and we can work out a person’s current office by finding the ‘member_of’ relationship which doesn’t have an end date defined: query = " START n = node(*)" query << " MATCH n-[r:colleagues]->c, n-[r2:member_of]->office" query << " WHERE n.type? = 'person' and has(n.name) and not(has(r2.end_date))" query << " RETURN n.name, count(r) AS connections, office.name" query << " ORDER BY connections DESC" And now our results look more like this: {"data"=>[["Carlos Villela", 283, "Porto Alegre - Brazil"], ["Mark Needham", 221, "London - UK South"]], "columns"=>["n.name", "connections"]} If we want to restrict that just to return the people for a specific person we can do that as well: query = " START n = node(*)" query << " MATCH n-[r:colleagues]->c, n-[r2:member_of]->office" query << " WHERE n.type? = 'person' and has(n.name) and (not(has(r2.end_date))) and office.name = 'London - UK South'" query << " RETURN n.name, count(r) AS connections, office.name" query << " ORDER BY connections DESC" {"data"=>[["Mark Needham", 221, "London - UK South"]], "columns"=>["n.name", "connections"]} In the current version of cypher we need to put brackets around the not expression otherwise it will apply the not to the rest of the where clause. Another way to get around that would be to put the not part of the where clause at the end of the line. While I am able to work out the most connected person by using these queries I’m not sure that it actually tells you who the most connected person is because it’s heavily biased towards people who have worked on big teams. Some ways to try and account for this are to bias the connectivity in favour of those you have worked longer with and also to give less weight to big teams since you’re less likely to have a strong connection with everyone as you might in a smaller team. I haven’t got onto that yet though!

Manipulating JAXWS header on the client Side like adding WSS username token or logging saop message.

I think like everyone, I go through the same journey whenever I find out about a new technology.. Huh? –> This is really cool –> I use it everywhere –> Hmm, sometimes it’s not so great Remember when people were writing websites with XSLT transforms? Yes, exactly. XML is great for storing a data structure as a string, but you really don’t want to be coding your application’s business logic with it. I’ve gone through a similar journey with Object Relational Mapping tools. After hand-coding my DALs, then code generating them, ORMs seemed like the answer to all my problems. I became an enthusiastic user of NHibernate through a number of large enterprise application builds. Even today I would still use an ORM for most classes of enterprise application. However there are some scenarios where ORMs are best avoided. Let me introduce my easy to use, ‘when to use an ORM’ chart. It’s got two axis, ‘Model Complexity’ and ‘Throughput’. The X-axis, Model Complexity, describes the complexity of your domain model; how many entities you have and how complex their relationships are. ORMs excel at mapping complex models between your domain and your database. If you have this kind of model, using an ORM can significantly speed up and simplify your development time and you’d be a fool not to use one. The problem with ORMs is that they are a leaky abstraction. You can’t really use them and not understand how they are communicating with your relational model. The mapping can be complex and you have to have a good grasp of both your relational database, how it responds to SQL requests, and how your ORM comes to generate both the relational schema and the SQL that talks to it. Thinking of ORMs as a way to avoid getting to grips with SQL, tables, and indexes will only lead to pain and suffering. Their benefit is that they automate the grunt work and save you the boring task of writing all that tedious CRUD column to property mapping code. The Y-axis in the chart, Throughput, describes the transactional throughput of your system. At very high levels, hundreds of transactions per second, you need hard-core DBA foo to get out of the deadlocked hell where you will inevitably find yourself. When you need this kind of scalability you can’t treat your ORM as anything other than a very leaky abstraction. You will have to tweak both the schema and the SQL it generates. At very high levels you’ll need Ayende level NHibernate skills to avoid grinding to a halt. If you have a simple model, but very high throughput, experience tells me that an ORM is more trouble than it’s worth. You’ll end up spending so much time fine tuning your relational model and your SQL that it simply acts as an unwanted obfuscation layer. In fact, at the top end of scalability you should question the choice of a relational ACID model entirely and consider an eventually-consistent event based architecture. Similarly, if your model is simple and you don’t have high throughput, you might be better off using a simple data mapper like SimpleData. So, to sum up, ORMs are great, but think twice before using one where you have a simple model and high throughput.

In Spring 3.1 there is a nice new feature: "@Valid On @RequestBody Controller Method Arguments" From the documentation: "An @RequestBody method argument can be annotated with @Valid to invoke automatic validation similar to the support for @ModelAttribute method arguments. A resulting MethodArgumentNotValidException is handled in the DefaultHandlerExceptionResolver and results in a 400 response code." http://static.springsource.org/spring/docs/current/spring-framework-reference/html/new-in-3.1.html#d0e1654 Based on this new feature I am able to send objects in JSON format, validate them and check for possible errors. Before this feature was introduced my controller looked like this: @Controller @RequestMapping("/user") public class UserController { @RequestMapping(method=RequestMethod.POST) public ResponseEntity create(@Valid User user, BindingResult bindingResult) { if (bindingResult.hasErrors()) { //parse errors and send back to user } ... } But data wasn't sent as a JSON object. It came from simple HTML form. Let's play with Spring 3.1. I've updated my maven depedency to use version 3.1.1.RELEASE and added the Jackson dependency to be able to use the MappingJacksonHttpMessageConverter. cglib cglib-nodep 2.2 org.springframework spring-webmvc 3.1.1.RELEASE org.hibernate hibernate-validator 4.3.0.Final org.codehaus.jackson jackson-mapper-asl 1.9.7 All other configuration is placed in java class instead of xml: @Configuration @EnableWebMvc public class WebappConfig { @Bean public UserController userController() { return new UserController(); } @Bean public MappingJacksonJsonView mappingJacksonJsonView() { return new MappingJacksonJsonView(); } @Bean public ContentNegotiatingViewResolver contentNegotiatingViewResolver() { final ContentNegotiatingViewResolver contentNegotiatingViewResolver = new ContentNegotiatingViewResolver(); contentNegotiatingViewResolver.setDefaultContentType(MediaType.APPLICATION_JSON); final ArrayList defaultViews = new ArrayList(); defaultViews.add(mappingJacksonJsonView()); contentNegotiatingViewResolver.setDefaultViews(defaultViews); return contentNegotiatingViewResolver; } } After that I've changed my create method to: @RequestMapping(method=RequestMethod.POST, consumes = "application/json") public ResponseEntity create(@Valid @RequestBody User user, BindingResult bindingResult) { if (bindingResult.hasErrors()) { //parse errors and send back to user } ... Now I've sent a JSON object but received the following error: "An Errors/BindingResult argument is expected to be immediately after the model attribute argument in the controller method signature" According to the documentation "BindingResult is supported only after @ModelAttribute arguments" After removing BindingResult from my method's arguments it worked. But how to get the errors now ? According to the documentation a MethodArgumentNotValidException will be thrown. So let's declare the necessary ExceptionHandler. @ExceptionHandler(MethodArgumentNotValidException.class) public ResponseEntity handleMethodArgumentNotValidException( MethodArgumentNotValidException error ) { return parseErrors(error.getBindingResult()); } ... And that's all, I'm attaching the full source code as a working example.

While preparing my next exercise I had to first find a way how to connect to a remote SLSB from Spring. It turned out to be childishly simple. See how I did it. Writing SLSB component First I needed a remote SLSB. Using M2Eclipse create a Maven 2 project. From the archetypes selection window I typed mojo in filter field (mojo archetypes are codehaus archetypes, I wrote about them in: Don't use org.apache.maven.archetypes!) and selected ejb3 archetype. I wrote a simple SLSB component: package org.xh.studies.openejb; import javax.ejb.Remote; @Remote public interface Greeter { String sayHello(String to); } package org.xh.studies.openejb; import javax.ejb.Stateless; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; @Stateless public class GreeterImpl implements Greeter { private static final Log log = LogFactory.getLog(GreeterImpl.class); public String sayHello(String to) { log.info("Saying hi to: " + to); return "Hello " + to + "! How are you?"; } } I built it with: mvn package Deploying to OpenEJB I downloaded OpenEJB archive from here: http://openejb.apache.org/download.html and unzipped it. I was done with installation :) I copied built in previous step jar to apps directory, then went to bin directory and ran openejb(.bat). Writing JUnit integration test First I wrote a simple integration test (using failsafe plugin, there are many post on my blog about it, the key one is: Maven2 integration testing with failsafe and jetty plugins) to verify if the SLSB is actually working: public class GreeterIT { private static Context ctx; @BeforeClass public static void setUp() throws NamingException { Properties env = new Properties(); env.put(Context.INITIAL_CONTEXT_FACTORY, "org.apache.openejb.client.RemoteInitialContextFactory"); env.put(Context.PROVIDER_URL, "ejbd://localhost:4201"); ctx = new InitialContext(env); } @AfterClass public static void tearDown() throws NamingException { ctx.close(); } @Test public void sayHelloRemoteEJBTest() throws NamingException { Greeter greeter = (Greeter) ctx.lookup("GreeterImplRemote"); String result = greeter.sayHello("Łukasz"); String expected = "Hello Łukasz! How are you?"; Assert.assertEquals(expected, result); } } I ran the test and it passed. Connecting to EJB from Spring I found out a Spring component called org.springframework.ejb.access.SimpleRemoteStatelessSessionProxyFactoryBean. It uses AOP behinde the scenes to create a proxy for a remote SLSB. All you need is just an interface. First I had to add Spring dependencies to my pom.xml. These were: spring-beans spring-context spring-aop Then I created a file called META-INF/spring/beans.xml and wrote: org.apache.openejb.client.RemoteInitialContextFactory ejbd://localhost:4201 In the above file I created a greeterBean which was a dynamic proxy and, thanks to AOP, would behave like a Greeter object. Writing Spring test I added one more field to my test: private static ApplicationContext springCtx; Then in @BeforeClass class I added ApplicationContext initialisation code: springCtx = new ClassPathXmlApplicationContext("META-INF/spring/beans.xml"); Finally I wrote a new method to test what I intended to do first, connecting to a remote SLSB from Spring: @Test public void sayHelloSpringProxyTest() throws NamingException { Greeter greeter = (Greeter) springCtx.getBean("greeterBean"); String result = greeter.sayHello("Łukasz"); String expected = "Hello Łukasz! How are you?"; Assert.assertEquals(expected, result); } It worked! Source code download A complete source code download can be found here: Connecting-to-EJB-from-Spring.zip. Summary Isn't it beautiful? You get an instance of a remote SLSB as a Spring bean. I definitely like it! thanks, Łukasz

Short blog post today. These are a couple of generic serialize and deserialize methods that can be easily used when needing to serialize and deserialize classes. The methods work with any .Net type. That includes built-in .Net types and custom classes that you might create yourself. These methods will only serialize PUBLIC properties of a class. Also, the XML will be human-readable instead of one long line of text. Serialize Method /// /// Serializes the data in the object to the designated file path /// /// Type of Object to serialize /// Object to serialize /// FilePath for the XML file public static void Serialize(T dataToSerialize, string filePath) { try { using (Stream stream = File.Open(filePath, FileMode.Create, FileAccess.ReadWrite)) { XmlSerializer serializer = new XmlSerializer(typeof(T)); XmlTextWriter writer = new XmlTextWriter(stream, Encoding.Default); writer.Formatting = Formatting.Indented; serializer.Serialize(writer, dataToSerialize); writer.Close(); } } catch { throw; } } Deserialize Method /// /// Deserializes the data in the XML file into an object /// /// Type of object to deserialize /// FilePath to XML file /// Object containing deserialized data public static T Deserialize(string filePath) { try { XmlSerializer serializer = new XmlSerializer(typeof(T)); T serializedData; using (Stream stream = File.Open(filePath, FileMode.Open, FileAccess.Read)) { serializedData = (T)serializer.Deserialize(stream); } return serializedData; } catch { throw; } } Here is some sample code to show the methods in action. Person p = new Person() { Name = "John Doe", Age = 42 }; XmlHelper.Serialize(p, @"D:\text.xml"); Person p2 = new Person(); p2 = XmlHelper.Deserialize(@"D:\text.xml"); Console.WriteLine("Name: {0}", p2.Name); Console.WriteLine("Age: {0}", p2.Age); Console.Read();

If you are using Spring to access/configure resources (DAOs/services), then you might as well add some basic performance monitoring while you are at it. This is a trivial task with Spring AOP and doesn't require any changes to existing code, just some simple configuration. First, you need to include the spring-aop, aspectj and cglib libraries. If you are using Maven, simply include the following dependencies... org.aspectj aspectjweaver 1.5.4 cglib cglib-nodep 2.2 org.springframework spring-aop 2.5.6 Next, identify what needs monitoring and put the AOP hooks in place. Generally, this just requires adding a pointcut and advisor configuration in your existing Spring XML configuration file. This configuration will add method response time logging to all methods in the "com.mycompany.services" package. Note: these classes must be instantiated with the Spring context...otherwise, the AOP hooks will not be executed. Next, you need to setup your logging (log4j, etc) to enable TRACE on the interceptor class. That's it, now when you run your application, you will see the following logging... TRACE PerformanceMonitorInterceptor - StopWatch 'PerfTestService.processRequest': running time (millis) = 1322 TRACE PerformanceMonitorInterceptor - StopWatch 'PerfTestService.processRequest': running time (millis) = 98 TRACE PerformanceMonitorInterceptor - StopWatch 'PerfTestService.processRequest': running time (millis) = 1764 This is a some great raw data, but unfortunately is not very useful on its own. Its for every method call and doesn't provide any other stats. This quickly clutters up the log and without some way to process/aggregate the log entries, its hard to make sense out of it. So, unless you plan of writing some log parsing or using 3rd party software (like Splunk or Cati), then you really should do some processing of the data before writing it to the log file. One easy way to do this is to just write a simple interceptor class to use instead of the Spring default one (PerformanceMonitorInterceptor). Below is an example of this that provides periodic stats (last, average and greatest response time) as well as warning whenever a method response time exceeds a configured threshold. By default, it will log stats every 10 method calls and log a warning message anytime a method response time exceeds 1000ms. public class PerfInterceptor implements MethodInterceptor { Logger logger = LoggerFactory.getLogger(PerfInterceptor.class.getName()); private static ConcurrentHashMap methodStats = new ConcurrentHashMap(); private static long statLogFrequency = 10; private static long methodWarningThreshold = 1000; public Object invoke(MethodInvocation method) throws Throwable { long start = System.currentTimeMillis(); try { return method.proceed(); } finally { updateStats(method.getMethod().getName(),(System.currentTimeMillis() - start)); } } private void updateStats(String methodName, long elapsedTime) { MethodStats stats = methodStats.get(methodName); if(stats == null) { stats = new MethodStats(methodName); methodStats.put(methodName,stats); } stats.count++; stats.totalTime += elapsedTime; if(elapsedTime > stats.maxTime) { stats.maxTime = elapsedTime; } if(elapsedTime > methodWarningThreshold) { logger.warn("method warning: " + methodName + "(), cnt = " + stats.count + ", lastTime = " + elapsedTime + ", maxTime = " + stats.maxTime); } if(stats.count % statLogFrequency == 0) { long avgTime = stats.totalTime / stats.count; long runningAvg = (stats.totalTime-stats.lastTotalTime) / statLogFrequency; logger.debug("method: " + methodName + "(), cnt = " + stats.count + ", lastTime = " + elapsedTime + ", avgTime = " + avgTime + ", runningAvg = " + runningAvg + ", maxTime = " + stats.maxTime); //reset the last total time stats.lastTotalTime = stats.totalTime; } } class MethodStats { public String methodName; public long count; public long totalTime; public long lastTotalTime; public long maxTime; public MethodStats(String methodName) { this.methodName = methodName; } } } Now, you just need to wire this into your app by referencing this class in your Spring xml and logging config. When you run your app, you will see stats like this... WARN PerfInterceptor - method warning: processRequest(), cnt = 10, lastTime = 1072, maxTime = 1937 TRACE PerfInterceptor - method: processRequest(), cnt = 10, lastTime = 1072, avgTime = 1243, runningAvg = 1243, maxTime = 1937 WARN PerfInterceptor - method warning: processRequest(), cnt = 20, lastTime = 1466, maxTime = 1937 TRACE PerfInterceptor - method: processRequest(), cnt = 20, lastTime = 1466, avgTime = 1067, runningAvg = 892, maxTime = 1937 As you can see, these stats can provide valuable feedback about class/method performance with very little effort and without modifying any existing Java code. This information can easily be used to find bottlenecks in your application (generally database or threading related, etc)...good luck

Introduction: Building modern HTTP/RESTful/RPC services has become very easy with the new ASP.NET Web API framework. Using ASP.NET Web API framework, you can create HTTP services which can be accessed from browsers, machines, mobile devices and other clients. Developing HTTP services is now quite easy for ASP.NET MVC developer becasue ASP.NET Web API is now included in ASP.NET MVC. In addition to developing HTTP services, it is also important to return meaningful response to client if a resource(uri) not found(HTTP 404) for a reason(for example, mistyped resource uri). It is also important to make this response centralized so you can configure all of 'HTTP 404 Not Found' resource at one place. In this article, I will show you how to handle 'HTTP 404 Not Found' at one place. Description: Let's say that you are developing a HTTP RESTful application using ASP.NET Web API framework. In this application you need to handle HTTP 404 errors in a centralized location. From ASP.NET Web API point of you, you need to handle these situations, No route matched. Route is matched but no {controller} has been found on route. No type with {controller} name has been found. No matching action method found in the selected controller due to no action method start with the request HTTP method verb or no action method with IActionHttpMethodProviderRoute implemented attribute found or no method with {action} name found or no method with the matching {action} name found. Now, let create a ErrorController with Handle404 action method. This action method will be used in all of the above cases for sending HTTP 404 response message to the client. public class ErrorController : ApiController { [HttpGet, HttpPost, HttpPut, HttpDelete, HttpHead, HttpOptions, AcceptVerbs("PATCH")] public HttpResponseMessage Handle404() { var responseMessage = new HttpResponseMessage(HttpStatusCode.NotFound); responseMessage.ReasonPhrase = "The requested resource is not found"; return responseMessage; } } You can easily change the above action method to send some other specific HTTP 404 error response. If a client of your HTTP service send a request to a resource(uri) and no route matched with this uri on server then you can route the request to the above Handle404 method using a custom route. Put this route at the very bottom of route configuration, routes.MapHttpRoute( name: "Error404", routeTemplate: "{*url}", defaults: new { controller = "Error", action = "Handle404" } ); Now you need handle the case when there is no {controller} in the matching route or when there is no type with {controller} name found. You can easily handle this case and route the request to the above Handle404 method using a custom IHttpControllerSelector. Here is the definition of a custom IHttpControllerSelector, public class HttpNotFoundAwareDefaultHttpControllerSelector : DefaultHttpControllerSelector { public HttpNotFoundAwareDefaultHttpControllerSelector(HttpConfiguration configuration) : base(configuration) { } public override HttpControllerDescriptor SelectController(HttpRequestMessage request) { HttpControllerDescriptor decriptor = null; try { decriptor = base.SelectController(request); } catch (HttpResponseException ex) { var code = ex.Response.StatusCode; if (code != HttpStatusCode.NotFound) throw; var routeValues = request.GetRouteData().Values; routeValues["controller"] = "Error"; routeValues["action"] = "Handle404"; decriptor = base.SelectController(request); } return decriptor; } } Next, it is also required to pass the request to the above Handle404 method if no matching action method found in the selected controller due to the reason discussed above. This situation can also be easily handled through a custom IHttpActionSelector. Here is the source of custom IHttpActionSelector, public class HttpNotFoundAwareControllerActionSelector : ApiControllerActionSelector { public HttpNotFoundAwareControllerActionSelector() { } public override HttpActionDescriptor SelectAction(HttpControllerContext controllerContext) { HttpActionDescriptor decriptor = null; try { decriptor = base.SelectAction(controllerContext); } catch (HttpResponseException ex) { var code = ex.Response.StatusCode; if (code != HttpStatusCode.NotFound && code != HttpStatusCode.MethodNotAllowed) throw; var routeData = controllerContext.RouteData; routeData.Values["action"] = "Handle404"; IHttpController httpController = new ErrorController(); controllerContext.Controller = httpController; controllerContext.ControllerDescriptor = new HttpControllerDescriptor(controllerContext.Configuration, "Error", httpController.GetType()); decriptor = base.SelectAction(controllerContext); } return decriptor; } } Finally, we need to register the custom IHttpControllerSelector and IHttpActionSelector. Open global.asax.cs file and add these lines, configuration.Services.Replace(typeof(IHttpControllerSelector), new HttpNotFoundAwareDefaultHttpControllerSelector(configuration)); configuration.Services.Replace(typeof(IHttpActionSelector), new HttpNotFoundAwareControllerActionSelector()); Summary: In addition to building an application for HTTP services, it is also important to send meaningful centralized information in response when something goes wrong, for example 'HTTP 404 Not Found' error. In this article, I showed you how to handle 'HTTP 404 Not Found' error in a centralized location. Hopefully you will enjoy this article too.