DrupalCon Munich is next week, and I am lucky enough to be going. As part of preparing for the conference, I thought it would be worthwhile to see just how easy (or difficult) it would be to migrate an existing Drupal site to Windows Azure Web Sites. So, in this post, I’ll do just that. Fortunately, because Windows Azure Web Sites supports both PHP and MySQL, the migration process is relatively straightforward. And, because Drupal and PHP run on any platform, the process I’ll describe should work for moving Drupal to Windows Azure Web Sites regardless of what platform you are moving from. Of course, Drupal installations can vary widely, so YMMV. I tested the instructions below on relatively small (and simple) Drupal installation running on CentOS 5. (Unfortunately, I won’t be using Drush since it isn’t supported on Windows Azure Websites.) If you are considering moving a large and complex Drupal application, may want to consider moving to Windows Azure Cloud Services (more information about that here: Migrating a Drupal Site from LAMP to Windows Azure). Before getting started, it’s worth noting that Windows Azure Websites lets you run up to 10 Web Sites for free in a multitenant environment. And, you can seamlessly upgrade to private, reserved VM instances as your traffic grows. To sign up, try the Windows Azure 90-day free trial. 1. Create a Windows Azure Web Site and MySQL database There is a step-by-step tutorial on http://www.windowsazure.com that walks you through creating a new website and a MySQL database, so I’ll refer you there to get started: Create a PHP-MySQL Windows Azure web site and deploy using Git. If you intend to use Git to publish your Drupal site, then go ahead and follow the instructions for setting up a Git repository. Make sure to follow the instructions in the Get remote MySQL connection information section as you will need that information later. You can ignore the remainder of the tutorial for the purposes of deploying your Drupal site, but if you are new to Windows Azure Web Sites (and to Git), you might find the additional reading informative. Ok, now you have a new website with a MySQL database, your have your MySQL database connection information, and you have (optionally) created a remote Git repository and made note of the Git deployment instructions. Now you are ready to copy your database to MySQL in Windows Azure Web Sites. 2. Copy database to MySQL in Windows Azure Web Sites I’m sure there is more than one way to copy your Drupal database, but I found the mysqldump tool to be effective and easy to use. To copy from a local machine to Windows Azure Web Sites, here’s the command I used: mysqldump -u local_username --password=local_password drupal | mysql -h remote_host -u remote_username --password=remote_password remote_db_name You will, of course, have to provide the username and password for your existing Drupal database, and you will have to provide the hostname, username, password, and database name for the MySQL database you created in step 1. This information is available in the connection string information that you should have noted in step 1. i.e. You should have a connection string that looks something like this: Database=remote_db_name;Data Source=remote_host;User Id=remote_username;Password=remote_password Depending on the size of your database, the copying process could take several minutes. Now your Drupal database is live in Windows Azure Websites. Before you deploy your Drupal code, you need to modify it so it can connect to the new database. 3. Modify database connection info in settings.php Here, you will again need your new database connection information. Open the /drupal/sites/default/setting.php file in your favorite text editor, and replace the values of ‘database’, ‘username’, ‘password’, and ‘host’ in the $databases array with the correct values for your new database. When you are finished, you should have something similar to this: $databases = array ( 'default' => array ( 'default' => array ( 'database' => 'remote_db_name', 'username' => 'remote_username', 'password' => 'remote_password', 'host' => 'remote_host', 'port' => '', 'driver' => 'mysql', 'prefix' => '', ), ), ); Be sure to save the settings.phpfile, then you are ready to deploy. 4. Deploy Drupal code using Git or FTP The last step is to deploy your code to Windows Azure Web Sites using Git or FTP. If you are using FTP, you can get the FTP hostname and username from you website’s dashboard. Then, use your favorite FTP client to upload your Drupal files to the /site/wwwroot folder of the remote site. If you are using Git, you need to set up a Git repository in Windows Azure Web Sites (steps for this are in the tutorial mentioned earlier). And, you will need Git installed on your local machine. Then, just follow the instructions provided after you created the repository: One note about using Git here: depending on your Git settings, your .gitignore file (a hidden file and a sibling to the .git folder created in your local root directory after you executed git commit), some files in your Drupal application may be ignored. In my case, all the files in the sites directory were ignored. If this happens, you will want to edit the .gitignore file so that these files aren’t ignored and redeploy. After you have deployed Drupal to Windows Azure Web Sites, you can continue to deploy updates via Git or FTP. Related information If you are looking for more information about Windows Azure Web Sites, these posts might be helpful: Windows Azure Websites- A PHP Perspective Windows Azure Websites, Web Roles, and VMs- When to use which- Configuring PHP in Windows Azure Websites with .user.ini Files One last thing you might consider, depending on your site, is using the Windows Azure Integration Module to store and serve your site’s media files.

This is the second article of the series on Spring Integration. This article builds on top of the first article where we introduced Spring Integration. Context setting In the first article, we created a simple java application where A message was sent over a channel, It was intercepted by a service i.e. POJO and modified. It was then sent over a different channel The modified message was read from the channel and displayed. However, in doing this - keeping in mind that we were merely introducing the concepts there - we wrote some Spring specific code in our application i.e. the test classes. In this article we will take care of that and make our application code as insulated from Spring Integration api as possible. This is done by, what Spring Integration calls gateways. Gateways exist for the sole purpose of abstracting messaging related "plumbing" code away from "business" code. The business logic might really not care whether a functionality is being achieved be sending a message over a channel or by making a SOAP call. This abstraction - though logical and desirable - have not been very practical, till now. It is probably worth having a quick look at the Spring Integration Reference Manual at this point. However, if you are just getting started with Spring Integration, you are perhaps better off following this article for the moment. I would recommend you get your hands dirty before returning to reference manual, which is very good but also very exhaustive and hence could be overwhelming for a beginner. The gateway could be a POJO with annotations (which is convenient but in my mind beats the whole purpose) or with XML configurations (can very quickly turn into a nightmare in any decent sized application if unchecked). At the end of the day it is really your choice but I like to go the XML route. The configuration options for both styles are detailed out in this section of the reference implementation. Spring Integration with Gateways So, let's create another test with gateway throw in for our HelloWorld service (refer to the first article of this series for more context). Let's start with the Spring configuration for the test. File: src/test/resources/org/academy/integration/HelloWorld1Test-context.xml In this case, all that is different is that we have added a gateway. This is an interface called org.academy.integration.Greetings. It interacts with both "inputChannel" and "outputChannel", to send and read messages respectively. Let's write the interface. File: /src/main/java/org/academy/integration/Greetings.java package org.academy.integration; public interface Greetings { public void send(String message); public String receive(); } And then we add the implementation of this interface. Wait. There is no implementation. And we do not need any implementation. Spring uses something called GatewayProxyFactoryBean to inject some basic code to this gateway which allows it to read the simple string based message, without us needing to do anything at all. That's right. Nothing at all. Note - You will need to add more code for most of your production scenarios - assuming you are not using Spring Integration framework to just push around strings. So, don't get used to free lunches. But, while it is here, let's dig in. Now, lets write a new test class using the gateway (and not interact with the channels and messages at all). File: /src/test/java/org/academy/integration/HelloWorld1Test.java package org.academy.integration; import static org.junit.Assert.*; import org.junit.Test; import org.junit.runner.RunWith; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration public class HelloWorld1Test { private final static Logger logger = LoggerFactory .getLogger(HelloWorld1Test.class); @Autowired Greetings greetings; @Test public void test() { greetings.send("World"); assertEquals(greetings.receive(), "Hello World"); logger.debug("Spring Integration with gateways."); } } Our test class is much cleaner now. It does not know about channels, or messages or anything related to Spring Integration at all. It only knows about a greetings instance - to which it gave some data by .send() method - and got modified data back by .receive() method. Hence, the business logic is oblivious of the plumbing logic, making for a much cleaner code. Now, simply type "mvn -e clean install" (or use m2e plugin) and you should be able to run the unit test and confirm that given string "World" the HelloWorld service indeed returns "Hello World" over the entire arrangement of channels and messages. Again, something optional but I highly recommend, is to run "mvn -e clean install site". This - assuming you have correctly configured some code coverage tool (cobertura in my case) will give you a nice HTML report showing the code coverage. In this case it would be 100%. I have blogged a series on code quality which deals this subject in more detail, but to cut long story short, it is very important for me to ensure that whatever coding practice / framework I use and recommend use, complies to some basic code quality standards. Being able to unit test and measure that is one such fundamental check that I do. Needless to say, Spring in general (including Spring integration) passes that check with flying colours. Conclusion That's it for this article. Happy coding.

It has been a while since I wanted to have CDI running with JavaFX2. Some people already blogged on how to proceed by getting Guice injection [1] to work with JavaFX & FXML. Well, now it's my turn to provide a way to empower JavaFX with CDI, using Weld as the implementation. My goal was just to have CDI working, no matter how I was using JavaFX, by directly coding in plain Java or using FXML. Ready? Let's go!!! Bootstrap JavaFX & Weld/CDI The launcher class will be the only place where we will have Weld-specific code—all the rest will be totally CDI compliant. The only trick here is to make the application parameters available as a CDI-compliant object so we can reuse them afterwards. Notice also that we use the CDI event mechanism to start up our real application code. public class WeldJavaFXLauncher extends Application { /** * Nothing special, we just use the JavaFX Application methods to boostrap * JavaFX */ public static void main(String[] args) { Application.launch(WeldJavaFXLauncher.class, args); } @SuppressWarnings("serial") @Override public void start(final Stage primaryStage) throws Exception { // Let's initialize CDI/Weld. WeldContainer weldContainer = new Weld().initialize(); // Make the application parameters injectable with a standard CDI // annotation weldContainer.instance().select(ApplicationParametersProvider.class).get().setParameters(getParameters()); // Now that JavaFX thread is ready // let's inform whoever cares using standard CDI notification mechanism: // CDI events weldContainer.event().select(Stage.class, new AnnotationLiteral() {}).fire(primaryStage); } } Start our real JavaFX application Here we start our real application code. We're just listening to the previously fired event (containing the Scene object to render into) so we can start showing our application. In the following example, we load an FXML GUI, but it might have been any node created in any way. public class LoginApplicationStarter { // Let's have a FXMLLoader injected automatically @Inject FXMLLoader fxmlLoader; // Our CDI entry point, we just listen to an event providing the startup scene public void launchJavaFXApplication(@Observes @StartupScene Stage s) { InputStream is = null; try { is = getClass().getResourceAsStream("login.fxml"); // we just load our FXML form (including controler and so on) Parent root = (Parent) fxmlLoader.load(is); s.setScene(new Scene(root, 300, 275)); s.show(); // let's show the scene } catch (IOException e) { throw new IllegalStateException("cannot load FXML login screen", e); } finally { // omitted is cleanup } } } But what about the FXML controller? First let's have a look at the controller we want to use inside our application. It is a pure POJO class annotated with both JavaFX & CDI annotations. // Simple application controller that uses injected fields // to delegate login process and to get default values from the command line using: --user=SomeUser public class LoginController implements Initializable { // Standard FXML injected fields @FXML TextField loginField; @FXML PasswordField passwordField; @FXML Text feedback; // CDI Injected service @Inject LoginService loginService; // Default application parameters retrieved using CDI @Inject Parameters applicationParameters; @FXML protected void handleSubmitButtonAction(ActionEvent event) { feedback.setText(loginService.login(loginField.getText(), passwordField.getText())); } @Override public void initialize(URL location, ResourceBundle resources) { loginField.setText(applicationParameters.getNamed().get("user")); } } In order to have injection working inside the FXML controller, we need to set up JavaFX so that controller objects are created by CDI. As we are in a CDI environment we can also have the FXMLLoader classes injected (that's exactly what we did in the previous LoginApplicationStarter class). How can we achieve this? We just have to provide a Producer class whose responsibility will be to create FXMLLoader instances that are able to load FXML GUIs and instantiate controllers using CDI. The only part that's a little tricky there is that the controller instantiation depends on the required class or interface (using fx:controller in your fxml file). In order to have such a runtime injection/resolution available we use a CDI Instance Object. public class FXMLLoaderProducer { @Inject Instance, Object>() { @Override public Object call(Class param) { return instance.select(param).get(); } }); return loader; } } I hope you found the article interesting and you do not hesitate to comment if you see some errors or possible enhancements. Finally, if you are interested you can find the full source code here. [1] http://andrewtill.blogspot.be/2012/07/creating-javafx-controllers-using-guice.htm

Autoscaling your webserver tier is typically straightforward. Image your apache server with source code or without, then sync down files from S3 upon spinup. Roll that image into the autoscale configuration and you’re all set. With the database tier though, things can be a bit tricky. The typical configuration we see is to have a single master database where your application writes. But scaling out or horizontally on Amazon EC2 should be as easy as adding more slaves, right? Why not automate that process? Below we’ve set out to answer some of the questions you’re likely to face when setting up slaves against your master. We’ve included instructions on building an AMI that automatically spins up as a slave. Fancy! How can I autoscale my database tier? Build an auto-starting MySQL slave against your master. Configure those to spinup. Amazon’s autoscaling loadbalancer is one option, another is to use a roll-your-own solution, monitoring thresholds on servers, and spinning up or dropping off slaves as necessary. Does an AWS snapshot capture subvolume data or just the SIZE of the attached volume? In fact, if you have an attached EBS volume and you create an new AMI off of that, you will capture the entire root volume, plus your attached volume data. In fact we find this a great way to create an auto-building slave in the cloud. How do I freeze MySQL during AWS snapshot? mysql> flush tables with read lock;mysql> system xfs_freeze -f /data At this point you can use the Amazon web console, ylastic, or ec2-create-image API call to do so from the command line. When the server you are imaging off of above restarts – as it will do by default – it will start with /data partition unfrozen and mysql’s tables unlocked again. Voila! If you’re not using xfs for your /data filesystem, you should be. It’s fast! The xfsprogs docs seem to indicate this may also work with foreign filesystems. Check the docs for details. How do I build an AMI mysql slave that autoconnects to master? Install mysql_serverid script below. Configure mysql to use your /data EBS mount. Set all your my.cnf settings including server_id Configure the instance as a slave in the normal way. When using GRANT to create the ‘rep’ user on master, specify the host with a subnet wildcard. For example ’10.20.%’. That will subsequently allow any 10.20.x.y servers to connect and replicate. Point the slave at the master. When all is running properly, edit the my.cnf file and remove server_id. Don’t restart mysql. Freeze the filesystem as described above. Use the Amazon console, ylastic or API call to create your new image. Test it of course, to make sure it spins up, sets server_id and connects to master. Make a change in the test schema, and verify that it propagates to all slaves. How do I set server_id uniquely? As you hopefully already know, in MySQL replication environment each node requires a unique server_id setting. In my Amazon Machine Images, I want the server to startup and if it doesn’t find the server_id in the /etc/my.cnf file, to add it there, correctly! Is that so much to ask? Here’s what I did. Fire up your editor of choice and drop in this bit of code: #!/bin/shif grep -q “server_id” /etc/my.cnf then : # do nothing – it’s already set else # extract numeric component from hostname – should be internet IP in Amazon environment export server_id=`echo $HOSTNAME | sed ‘s/[^0-9]*//g’` echo “server_id=$server_id” >> /etc/my.cnf # restart mysql /etc/init.d/mysql restart fi Save that snippet at /root/mysql_serverid. Also be sure to make it executable: $ chmod +x /root/mysql_serverid Then just append it to your /etc/rc.local file with an editor or echo: $ echo "/root/mysql_serverid" >> /etc/rc.local Assuming your my.cnf file does *NOT* contain the server_id setting when you re-image, then it’ll set this automagically each time you spinup a new server off of that AMI. Nice! Can you easily slave off of a slave? How? It’s not terribly different from slaving off of a normal master. A. First enable slave updates. The setting is not dynamic, so if you don’t already have it set, you’ll have to restart your slave. log_slave_updates=true B. Get an initial snapshot of your slave data. You can do that the locking way: mysql> flush tables with read lock;mysql> show master status\G; mysql> system mysqldump -A > full_slave_dump.mysql mysql> unlock tables; You may also choose to use Percona’s excellent xtrabackup utility to create hotbackups without locking any tables. We are very lucky to have an open-source tool like this at our disposal. MySQL Enterprise Backup from Oracle Corp can also do this. C. On the slave, seed the database with your dump created above. $ mysql < full_slave_dump.mysql D. Now point your slave to the original slave. mysql> change master to master_user='rep', master_password='rep', master_host='192.168.0.1', master_log_file='server-bin-log.000004', master_log_pos=399;mysql> start slave; mysql> show slave status\G; Slave master is set as an IP address. Is there another way? It’s possible to use hostnames in MySQL replication, however it’s not recommended. Why? Because of the wacky world of DNS. Suffice it to say MySQL has to do a lot of work to resolve those names into IP addresses. A hickup in DNS can interrupt all MySQL services potentially as sessions will fail to authenticate. To avoid this problem do two things: A. Set this parameter in my.cnf skip_name_resolve = true Remove entries in mysql.user table where hostname is not an IP address. Those entries will be invalid for authentication after setting the above parameter. Doesn’t RDS take care of all of this for me? RDS is Amazon’s Relational Database Service which is built on MySQL. Amazon’s RDS solution presents MySQL as a service which brings certain benefits to administrators and startups: Simpler administration. Nuts and bolts are handled for you. Push-button replication. No more struggling with the nuances and issues of MySQL’s replication management. Simplicity of administration of course has it’s downsides. Depending on your environment, these may or may not be dealbreakers. No access to the slow query log. This is huge. The single best tool for troubleshooting slow database response is this log file. Queries are a large part of keeping a relational database server healthy and happy, and without this facility, you are severely limited. Locked in downtime window When you signup for RDS, you must define a thirty minute maintenance window. This is a weekly window during which your instance *COULD* be unavailable. When you host yourself, you may not require as much downtime at all, especially if you’re using master-master mysql and zero-downtime configuration. Can’t use Percona Server to host your MySQL data. You won’t be able to do this in RDS. Percona server is a high performance distribution of MySQL which typically rolls in serious performance tweaks and updates before they make it to community addition. Well worth the effort to consider it. No access to filesystem, server metrics & command line. Again for troubleshooting problems, these are crucial. Gathering data about what’s really happening on the server is how you begin to diagnose and troubleshoot a server stall or pileup. You are beholden to Amazon’s support services if things go awry. That’s because you won’t have access to the raw iron to diagnose and troubleshoot things yourself. Want to call in an outside consultant to help you debug or troubleshoot? You’ll have your hands tied without access to the underlying server. You can’t replicate to a non-RDS database. Have your own datacenter connected to Amazon via VPC? Want to replication to a cloud server? RDS won’t fit the bill. You’ll have to roll your own – as we’ve described above. And if you want to replicate to an alternate cloud provider, again RDS won’t work for you. Related posts: Deploying MySQL on Amazon EC2 – 8 Best Practices Review: Host Your Web Site In The Cloud, Amazon Web Services Made Easy 5 Ways to Boost MySQL Scalability Top MySQL DBA interview questions (Part 2) MySQL Cluster In The Cloud – Managers Guide

When defining a Zabbix (1.8.2) trigger e.g. to inform you that there are errors in a log file, how do you verify that it is correct? As somebody recommended in a forum, you can use a Calculated Item with a similar expression (the syntax is little different from triggers). Contrary to triggers, the value of a calculated item is easy to see and the historical values are stored so you can check how it evolved. If your trigger expression is complex the you can create multiple calculated items, one for each subexpression. Example If we have a log item that sends us data whenever the text “ERROR” appears in a log line and the corresponding trigger expected to fire if we have got any data from the item in the last 600 sec (nodata() returns 1 if there indeed was no data): {hive.example.com:log["/tmp/ada/hive.log","ERROR",,20].nodata(600)}=0 Then we could test it with a calculated item with the expression nodata("hive.example.com:log[\"/tmp/ada/hive.log\",\"ERROR\",,20]", 600) (Notice that the function comes first, taking the host:item as its 0th parameter and that this is enclosed with “”, escaping any nested ” with \.) The value of the calculated item will be re-checked every (independent on whether the source item changed or not) and stored, in this case it will either thave the value of 0 or 1. We can also construct more complex expressions with &, + etc. similarly to trigger expressions.

the question everybody always asks when they learn about activiti, is as old as software development itself: “how does it perform?”. up till now, when you would ask me that same question, i would tell you about how activiti minimizes database access in every way possible, how we break down the process structure into an ‘execution tree’ which allows for fast queries or how we leverage ten years of workflow framework development knowledge. you know, trying to get around the question without answering it. we knew it is fast, because of the theoretical foundation upon which we have built it. but now we have proof: real numbers …. yes, it’s going to be a lengthy post. but trust me, it’ll be worth your time! disclaimer: performance benchmarks are hard. really hard. different machines, slight different test setup … very small things can change the results seriously. the numbers here are only to prove that the activiti engine has a very minimal overhead, while also integrating very easily into the java eco-system and offering bpmn 2.0 process execution. the activiti benchmark project to test process execution overhead of the activiti engine, i created a little side project on github: https://github.com/jbarrez/activiti-benchmark the project contains currently 9 test processes, which we’ll analyse below. the logic in the project is pretty straightforward: a process engine is created for each test run each of the processes are sequentially executed on this process engine, using a threadpool from 1 up to 10 threads. all the processes are thrown into a bag, of which a number of random executions are drawn. all the results are collected and a html report with some nice charts are generated to run the benchmark, simply follow the instructions on the github page to build and execute the jar. benchmark results the test machine i used for the results is my (fairly old) desktop machine: amd phenom ii x4 940 3.0ghz, 8 gb 800mhz ram and an old-skool 7200 rpm hd running ubuntu 11.10. the database used for the test runs on the same machine on which the tests also run. so keep in mind that in a ‘real’ server environment the results could even be better! the benchmark project i mentioned above, was executed on a default ubuntu mysql 5 database. i just switched to the ‘large.cnf’ setting (which throws more ram at the db and stuff like that) instead the default config. each of the test processes ran for 2500 times, using a threadpool going from one to ten threads . in simpleton language: 2500 process executions using just one thread, 2500 threads using two threads, 2500 process executions using three … yeah, you get it. each benchmark run was done using a ‘default’ activiti process engine. this basically means a ‘regular’ standalone activiti engine, created in plain java. each benchmark run was also done in a ‘spring’ config. here, the process engine was constructed by wrapping it in the factory bean, the datasource is a spring datasource and also the transactions and connection pool is managed by spring (i’m actually using a tweaked bonecp threadpool) each benchmark run was executed with history on the default history level (ie. ‘audit’) and without history enabled (ie. history level ‘none’) . the processes are in detail analyzed in the sections below, but here are the integral results of the test runs already: activiti 5.9 – mysql – default – history enabled activiti 5.9 – mysql – default – history disabled activiti 5.9 – mysql – spring – history enabled activiti 5.9 – mysql – spring – history disabled i ran all the tests using the latest public release of activiti, being activiti 5.9. however, my test runs brought some potential performance fixes to the surface (i also ran the benchmark project through a profiler). it was quickly clear that most of the process execution time was done actually cleaning up when a process ended. basically, more than often queries were fired which were not necessary if we would save some more state in our execution tree. i sat together with daniel meyer from camunda and my colleague frederik heremans, and they’ve managed to commit fixes for this! as such, the current trunk of activiti, being activiti 5.10-snapshot at the moment, is significantly faster than 5.9 . activiti 5.10 – mysql – default – history enabled activiti 5.10 – mysql – default – history disabled activiti 5.10 – mysql – spring – history enabled activiti 5.10 – mysql – spring – history disabled from a high-level perspective (scroll down for detailed analysis), there are a few things to note: i had expected some difference between the default and spring config, due to the more ‘professional’ connection pool being used. however, the results for both environments are quite alike. sometimes the default is faster, sometimes spring. it’s hard to really find a pattern. as such, i omitted the spring results in the detailed analyses below. the best average timings are most of the times found when using four threads to execute the processes . this is probably due to having a quad-core machine. the best throughput numbers are most of the times found when using eight threads to execute the processes. i can only assume that is also has something to do with having a quad-core machine. when the number of threads in the threadpool go up, the throughput (processes executed / second) goes up, both it has a negative effect on the average time. certainly with more than six or seven threads, you see this effect very clear. this basically means that while the processes on itself take a little longer to execute, but due to the multiple threads you can execute more of these ‘slower’ processes in the same amount of time. enabling history does have an impact. often, enabling history will double execution time. this is logical, given that many extra records are inserted when history is on the default level (ie. ‘audit’). there was one last test i ran, just out of curiosity: running the best performing setting on an oracle xe 11.2 database. the oracle xe is a free version of the ‘real’ oracle database. no matter how hard, i tried, i couldn’t get it decently running on ubuntu. as such, i used an old windows xp install on that same machine. however, the os is 32 bit, wich means the system only has 3.2 of the 8gb of ram available. here are the results: activiti 5.10 – oracle on windows – default – history disabled the results speak for itself. oracle blows away any of the (single-threaded) results on mysql (and they are already very fast!). however, when going multi-threaded it is far worse than any of the mysql results. my guess is that these are due to the limitations of the xe version : only one cpu is used, only 1 gb of ram, etc. i would really like to run these test on a real oracle-managed-by-a-real-dba … feel free to contact me if you are interested ! in the next sections, we will take a detailed look into the performance numbers of each of the test processes. an excel sheet containing all the the numbers and charts below can be downloaded for yourself . process 1: the bare micromum (one transaction) the first process is not a very interesting one, business-wise at least. after starting the process, the end is immediately reached. not very useful on itself, but its numbers learn us one essential thing: the bare overhead of the activiti engine. here are the average timings: this process runs in a single transaction, which means that nothing is saved to the database when the history is disabled due to activiti’s optimizations. with history enabled, you’ll basically get the cost for inserting one row into the historical process instance table, which is around 4.44 ms here. it is also clear that our fix for activiti 5.10 has an enormous impact here. in the previous version, 99% of the time was spent in the cleanup check of the process. take a look at the best result here: 0.47 ms when using 4 threads to execute 2500 runs of this process. that’s only half a millisecond ! it’s fair to say that the activiti engine overhead is extremely small. the throughput numbers are equally impressive: in the best case here, 8741 processes are executed. per second. by the time you arrive here reading the post, you could have executed a few millions of this process . you can also see that there is little difference between 4 or 8 threads here. most of the execution time here is cpu time, and no potential collisions such as waiting for a database lock happens here. in these numbers, you can also easily see that the oracle xe doesn’t scale well with multiple threads (which is explained above). you will see the same behavior in the following results. process 2: the same, but a bit longer (one transaction) this process is pretty similar to the previous one. we have again only one transaction. after the process is started, we pass through seven no-op passthrough activities before reaching the end. some things to note here: the best result (again 4 threads, with history disabled) is actually better than the simpler previous process. but also note that the single threaded execution is a tad slower. this means that the process on itself is a bit slower, which is logical as is has more activities. but using more threads and having more activities in the process does allow for more potential interleaving. in the previous case, the thread was barely born before it was killed again. the difference between history enabled/disabled is bigger than the previous process. this is logical, as more history is written here (for each activity one record in the database). again, activiti 5.10 is far more superior to activiti 5.9. the throughput numbers follow these observations: there is more opportunity to use threading here. the best result lingers around 12000 process execution per second . again, it demonstrates the very lightweight execution of the activiti engine. process 3: parallelism in one transaction this process executes a parallel gateway that forks and one that joins in the same transaction. you would expect something along the lines of the previous results, but you’d be surprised: comparing these numbers with the previous process, you see that execution is slower. so why is this process slower, even if it has less activities? the reason lies with how the parallel gateway is implemented, especially the join behavior. the hard part, implementation-wise, is that you need to cope with the situation when multiple executions arrive at the join. to make sure that the behavior is atomic, we internally do some locking and fetch all child executions in the execution tree to find out whether the join activates or not. so it is quite a ‘costly’ operation, compared to the ‘regular’ activities. do mind, we’re talking here about only 5 ms single threaded and 3.59 ms in the best case for mysql . given the functionality that is required for implementing the parallel gateway functionality, this is peanuts if you’d ask me. the throughput numbers: this is the first process which actually contains some ‘logic’. in the best case above, it means 1112 processes can be executed in a second. pretty impressive, if you’d ask me! . process 4: now we’re getting somewhere (one transaction) this process already looks like something you’d see when modeling real business processes. we’re still running it in one database transaction though, as all the activities are automatic passthroughs. here we also have two forks and two joins. take a look at the lowest number: 6.88 ms on oracle when running with one thread. that’s freaking fast , taking in account all that is happening here. the history numbers are at least doubled here (activiti 5.10), which makes sense because there is quite a bit of activity audit logging going on here. you can also see that this causes to have a higher average time for four threads here, which is probably due to the implementation of the joining. if you know a bit about activiti internals, you’ll understand this means there are quite a bit of executions in the execution tree. we have one big concurrent root, but also multiple children which are sometimes also concurrent roots. but while the average time rises, the throughput definitely benefits: running this process with eight threads, allows you to do 411 runs of this process in a single second. there is also something peculiar here: the oracle database performs better with more thread concurrency. this is completely contrary with all other measurements, where oracle is always slower in that environment (see above for explanation). i assume it has something to do with the internal locking and forced update we are applying when forking/joining, which is better handled by oracle it seems. process 5: adding some java logic (single transaction) i added this process to see the influence of adding a java service task in a process. in this process, the first activity generates a random value, stores it as a process variable and then goes up or down in the process depending on the random value. the chance is about 50/50 to go up or down. the average timings are very very good. actually, the results are in the same range as those of process 1 and 2 above (which had no activities or only automatic passthroughs). this means that the overhead of integrating java logic into your process is nearly non-existant (nothing is of course for free). of course, you can still write slow code in that logic, but you can’t blame the activiti engine for that throughput numbers are comparable to those of process 1 and 2: very, very high. in the best case here, more than 9000 processes are executed per second . that indeed also means 9000 invocations of your own java logic. process 6, 7 and 8: adding wait states and transactions the previous processes demonstrated us the bare overhead of the activiti engine. here, we’ll take a look at how wait states and multiple transactions have influence on performance. for this, i added three test processes which contain user tasks. for each user task, the engine commits the current transaction and returns the thread to the client. since the results are pretty much compatible for these processes, we’re grouping them here. these are the processes: here are the average timings results, in order of the processes above. for the first process, containing just one user task: it is clear that having wait states and multiple transaction does have influence on the performance. this is also logical: before, the engine could optimize by not inserting the runtime state into the database, because the process was finished in one transaction. now, the whole state, meaning the pointers to where you are currently, need to be saved into the database. the process could be ‘sleeping’ like this for many days, months, years now …. the activiti engine doesn’t hold it into memory now anymore, and it is freed to give its full attention to other processes. if you check the results of the process with only one user task, you can see that in the best case (oracle, single thread – the 4 threads on mysql is pretty close) this is done in 6.27ms . this is really fast, if you take in account we have a few inserts (the execution tree, the task), a few updates (the execution tree) and deletes (cleaning up) going on here. the second process here, with 7 user tasks: the second chart learns us that logically, more transactions means more time. in the best case here the process is done in 32.12 ms . that is for seven transactions, which gives 4.6 ms for each transactions. so it is clear that average time scales in a linearly way when adding wait states. this makes of course sense, because transactions aren’t free. also note that enabling history does add quite some overhead here. this is due to having the history level set to ‘audit’, which stores all the user task information in the history tables. this is also noticeable from the difference between activiti 5.9 with history disabled and activiti 5.10 with history enabled: this is a rare case where activiti 5.10 with history enabled is slower than 5.9 with history disabled. but it is logical, given the volume of history stored here. and the third process learns us how user tasks and parallel gateways interact: the third chart learns us not much new. we have two user tasks now, and the more ‘expensive’ fork/join (see above). the average timings are how we expected them. the throughput charts are as you would expect given the average timings. between 70 and 250 processes per second. aw yeah! to save some space, you’ll need to click them to enlarge: process 9: so what about scopes? for the last process, we’ll take a look at ‘scopes’. a ‘scope’ is how we call it internally in the engine, and it has to do with variable visibility, relationships between the pointers indicating process state, event catching, etc. bpmn 2.0 has quite some cases for those scopes, for example with embedded subprocesses as shown in the process here. basically, every subprocess can have boundary events (catching an error, a message, etc) that only are applied on its internal activities when it’s scope is active. without going into too much technical details: to get scopes implemented in the correct way, you need some not so trivial logic. the example process here has 4 subprocesses, nested in each other. the inner process is using concurrency, which is a scope on itself again for the activiti engine. there are also two user tasks here, so that means two transactions. so let’s see how it performs: you can clearly see the big difference between activiti 5.9 and 5.10. scopes are indeed an area where the fixes around the ‘process cleanup’ at the end have a huge benefit, as many execution objects are created and persisted to represent the many different scopes. single threaded performance is not so good on activiti 5.9. luckily, as you can see from the gap between the blue and the red bars, those scopes do allow for high concurrency. the numbers of oracle, combined with the multi-threaded results of the 5.10 tests, do prove that scopes are now efficiently handled by the engine. the throughput charts prove that the process nicely scales with more threads, as you can see by the big gap between the red and green line in the second last block. in the best case, 64 processes of this more complex process are handled by the engine. random execution if you have already clicked on the full reports at the beginning of the post, you probably have noticed also random execution is tested for each environment. in this setting, 2500 process executions were done, both the process was randomly chosen. as shown in those reports this meant that over 2500 executions, each process was executed almost the same number of times (normal distribution). this last chart shows the best setting (activiti 5.10, history disabled) and how the throughput of those random process executions goes when adding more threads: as we’ve seen in many of the test above, once passed four threads things don’t change that much anymore. the numbers (167 processes/second) prove that in a realistic situation (ie. multiple processes executing at the same time), the activiti engine nicely scales up. conclusion the average timing charts show two things clearly: the activiti engine is fast and overhead is minimal ! the difference between history enabled or disabled is definitely noticeably. sometimes it comes even down to half the time needed. all history tests were done using the ‘audit’ level, but there is a simpler history level (‘activity’) which might be good enough for the use case. activiti is very flexible in history configuration, and you can tweak the history level for each process specifically. so do think about the level your process needs to have, if it needs to have history at all ! the throughput charts prove that the engine scales very well when more threads are available (ie. any modern application server). activiti is well designed to be used in high-throughput and availability (clustered) architectures . as i said in the introduction, the numbers are what they are: just numbers. my main point which i want to conclude here, is that the activiti engine is extremely lightweight. the overhead of using activiti for automating your business processes is small. in general, if you need to automate your business processes or workflows, you want top-notch integration with any java system and you like all of that fast and scalable … look no further!

Here are the most important subjects for software engineering, with brief explanations: 1.Object oriented analysis & design: For better maintainability, reusability and faster development, the most well accepted approach, shortly OOAD and its SOLID principals are very important for software engineering. 2.Software quality factors: Software engineering depends on some very important quality factors. Understanding and applying them is crucial. 3.Data structures & algorithms: Basic data structures like array, list, stack, tree, map, set etc. and useful algorithms are vital for software development. Their logical structure should be known. 4. Big-O notation: Big-O notation indicates the performance of an algorithm/code section. Understanding it is very important for comparing performances. 5.UML notation: UML is the universal and complete language for software design & analysis. If there is lack of UML in a development process, it feels there is no engineering. 6.Software processes and metrics: Software enginnering is not a random process. It requires a high level of systematic and some numbers to monitor those techniques. So, processes and metrics are essential. 7.Design patterns: Design patterns are standard and most effective solutions for specific problems. If you don't want to reinvent the wheel, you should learn them. 8.Operating systems basics: Learning OS basics is very important because all applications runs on it. By learning it, we can have better vision, viewpoints and performance for our applications. 9.Computer organization basics: All applications including OS requires a hardware for physical interaction. So, learning computer organization basics is vital again for better vision, viewpoints and performance. 10.Network basics: Network is related with computer organization, OS and the whole information transfer process. In any case we will face it while software development. So, it is important to learn network basics. 11.Requirement analysis: Requirement analysis is the starting point and one of the most important parts of software engineering. Performing it correctly and practically needs experience but it is very essential. 12.Software testing: Testing is another important part of software engineering. Unit testing, its best practices and techniques like black box, white box, mocking, TDD, integration testing etc. are subjects which must be known. 13.Dependency management: Library (JAR, DLL etc.) management, and widely known tools (Maven, Ant, Ivy etc.) are essential for large projects. Otherwise, antipatterns like Jar Hell are inevitable. 14.Continuous integration: Continuous integration brings easiness and automaticity for testing large modules, components and also performs auto-versioning. Its aim and tools (like Hudson etc.) should be known. 15.ORM (Object relational mapping): ORM and its widely known implementation Hibernate framework is an important technique for mapping objects into database tables. It reduces code length and maintenance time. 16.DI (Dependency Injection): DI or IoC (Inversion of Control) and its widely known implementation Spring framework makes life easy for object creation and lifetime management on big enterprise applications. 17.Version controlling systems: VCS tools (SVN, TFS, CVS etc.) are very important by saving so much time for collaborative works and versioning. Their logical viewpoint and standard cammands should be known. 18.Internationalization (i18n): i18n by extracting strings into external files is the best way of supporting multiple languages in our applications. Its practices on different IDEs and technologies must be known. 19.Architectural patterns: Understanding architectural design patterns (like MVC, MVP, MVVM etc.) is essential for producing a maintainable, clean, extendable and testable source code. 20.Writing clean code: Working code is not enough, it must be readable and maintainable also. So, code formatting and readable code development techniques are needed to be known and applied.

Envers is a Hibernate module that can be configured to automatically audit changes made to your entities. Each audited entity are thus associated with a list of revisions, each revision capturing the state of the entity when a change occurs. There is however an obstacle I came across while I was "unit testing" my DAO, and that's what I want to share to avoid others to fall in the same pit. First, let's have an overview of the couple of steps needed to use Envers: Annotate your entity with the @Audited annotation: @Entity @Audited public class Person { // Properties } Register the Envers AuditEventListener in your Hibernate SessionFactory through Spring: org.hibernate.dialect.H2Dialect Configure the Hibernate transaction manager as your transaction manager. Note auditing won't be triggered if you use another transaction manager (DataSourceTransactionManager comes to mind): Now is the time to create your test class: @ContextConfiguration("classpath:spring-persistence.xml") @TransactionConfiguration(defaultRollback = false) public class PersonDaoImplTest extends AbstractTransactionalTestNGSpringContextTests { @Autowired private PersonDao personDao; @BeforeMethod protected void setUp() { // Populate database } @Test public void personShouldBeAudited() { Person person = personDao.get(1L); person.setFirstName("Jane"); List history = personDao.getPersonHistory(1L); assertNotNull(history); assertFalse(history.isEmpty()); assertEquals(history.size(), 1); } } Strangely, when you execute the previous test class, the test method fails when checking the list is not empty: it is, meaning there's no revision associated with the entity. Morevoer, nothing shows up in the log. However, the revision shows up in the audited table at the end of the test (provide you didn't clear the table after its execution). Comes the dreaded question: why? Well, it seems Hibernate post-event listeners are only called when the transaction is commited. In our case, it matches: the transaction is commited by Spring after method completion, and our test trie to assert inside the method. In order for our test to pass, we have to manually manage a transaction inside our method, to commit the update to the database. @Test public void personShouldBeAuditedWhenUpdatedWithManualTransaction() { PlatformTransactionManager txMgr = applicationContext.getBean(PlatformTransactionManager.class); // A new transaction is required, the wrapping transaction is for Envers TransactionStatus status = txMgr.getTransaction(new DefaultTransactionDefinition(PROPAGATION_REQUIRES_NEW)); Person person = personDao.get(1L); person.setFirstName("Jane"); txMgr.commit(status); List history = personDao.getPersonHistory(1L); assertNotNull(history); assertFalse(history.isEmpty()); assertEquals(history.size(), 1); } On one hand, the test passes and the log shows the SQL commands accordingly. On the other hand, the cost is the additional boilerplate code needed to make it pass. Of course, one could (should?) question the need to test the feature in the first place. Since it's a functionality brought by a library, the reasoning behind could be that if you don't trust the library, don't use it at all. In my case, it was the first time I used Envers, so there's no denying I had to build the trust between me and the library. Yet, even with trusted libraries, I do test specific cases: for example, when using Hibernate, I create test classes to verify that complex queries get me the right results. As such, auditing qualifies as a complex use-case whose misbehaviors I want to be aware of as soon as possible. You'll find the sources for this article here, in Maven/Eclipse format.

One of the advantages of code generation is the ability to see how a specific language feature or framework is used. As I discussed in the post NetBeans 7.2 beta: Faster and More Helpful, NetBeans 7.2 beta provides TestNG integration. I did not elaborate further in that post other than a single reference to that feature because I wanted to devote this post to the subject. I use this post to demonstrate how NetBeans 7.2 can be used to help a developer new to TestNG start using this alternative (to JUnit) test framework. NetBeans 7.2's New File wizard makes it easier to create an empty TestNG test case. This is demonstrated in the following screen snapshots that are kicked off by using New File | Unit Tests (note that "New File" is available under the "File" drop-down menu or by right-clicking in the Projects window). Running the TestNG test case creation as shown above leads to the following generated test code. TestNGDemo.java (Generated by NetBeans 7.2) package dustin.examples; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeClass; import org.testng.annotations.Test; import org.testng.Assert; /** * * @author Dustin */ public class TestNGDemo { public TestNGDemo() { } @BeforeClass public void setUpClass() { } @AfterClass public void tearDownClass() { } @BeforeMethod public void setUp() { } @AfterMethod public void tearDown() { } // TODO add test methods here. // The methods must be annotated with annotation @Test. For example: // // @Test // public void hello() {} } The test generated by NetBeans 7.2 includes comments indicate how test methods are added and annotated (similar to modern versions of JUnit). The generated code also shows some annotations for overall test case set up and tear down and for per-test set up and tear down (annotations are similar to JUnit's). NetBeans identifies import statements that are not yet used at this point (import org.testng.annotations.Test; and import org.testng.Assert;), but are likely to be used and so have been included in the generated code. I can add a test method easily to this generated test case. The following code snippet is a test method using TestNG. testIntegerArithmeticMultiplyIntegers() @Test public void testIntegerArithmeticMultiplyIntegers() { final IntegerArithmetic instance = new IntegerArithmetic(); final int[] integers = {4, 5, 6}; final int expectedProduct = 2 * 3 * 4 * 5 * 6; final int product = instance.multiplyIntegers(2, 3, integers); assertEquals(product, expectedProduct); } This, of course, looks very similar to the JUnit equivalent I used against the same IntegerArithmetic class that I used for testing illustrations in the posts Improving On assertEquals with JUnit and Hamcrest and JUnit's Built-in Hamcrest Core Matcher Support. The following screen snapshot shows the output in NetBeans 7.2 beta from right-clicking on the test case class and selecting "Run File" (Shift+F6). The text output of the TestNG run provided in the NetBeans 7.2 beta is reproduced next. [TestNG] Running: Command line suite [VerboseTestNG] RUNNING: Suite: "Command line test" containing "1" Tests (config: null) [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @BeforeClass dustin.examples.TestNGDemo.setUpClass() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @BeforeClass dustin.examples.TestNGDemo.setUpClass() finished in 33 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @BeforeMethod dustin.examples.TestNGDemo.setUp() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @BeforeMethod dustin.examples.TestNGDemo.setUp() finished in 2 ms [VerboseTestNG] INVOKING: "Command line test" - dustin.examples.TestNGDemo.testIntegerArithmeticMultiplyIntegers() [VerboseTestNG] PASSED: "Command line test" - dustin.examples.TestNGDemo.testIntegerArithmeticMultiplyIntegers() finished in 12 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @AfterMethod dustin.examples.TestNGDemo.tearDown() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @AfterMethod dustin.examples.TestNGDemo.tearDown() finished in 1 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @AfterClass dustin.examples.TestNGDemo.tearDownClass() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @AfterClass dustin.examples.TestNGDemo.tearDownClass() finished in 1 ms [VerboseTestNG] [VerboseTestNG] =============================================== [VerboseTestNG] Command line test [VerboseTestNG] Tests run: 1, Failures: 0, Skips: 0 [VerboseTestNG] =============================================== =============================================== Command line suite Total tests run: 1, Failures: 0, Skips: 0 =============================================== Deleting directory C:\Users\Dustin\AppData\Local\Temp\dustin.examples.TestNGDemo test: BUILD SUCCESSFUL (total time: 2 seconds) The above example shows how easy it is to start using TestNG, especially if one is moving to TestNG from JUnit and is using NetBeans 7.2 beta. Of course, there is much more to TestNG than this, but learning a new framework is typically most difficult at the very beginning and NetBeans 7.2 gets one off to a fast start.

I.T. is full of complex things that should (and sometimes could) be simple. Getting the JQPL/SQL String representation for a JPA 2.0 CriteriaQuery is one of them. By now you all know the JPA 2.0 Criteria API : a type safe way to write a JQPL query. This API is clever in the way that you don’t use Strings to build your query, but is quite verbose… and sometimes you get lost in dozens of lines of Java code, just to write a simple query. You get lost in your CriteriaQuery, you don’t know why your query doesn’t work, and you would love to debug it. But how do you debug it ? Well, one way would be by just displaying the JPQL and/or SQL representation. Simple, isn’t it ? Yes, but JPA 2.0 javax.persistence.Query doesn’t have an API to do this. You then need to rely on the implementation… meaning, the code is different if you use EclipseLink, Hibernate or OpenJPA. The CriteriaQuery we want to debug Let’s say you have a simple Book entity and you want to retrieve all the books sorted by their id. Something like SELECT b FROM Book b ORDER BY b.id DESC. How would you write this with the CriteriaQuery ? Well, something like these 5 lines of Java code : CriteriaBuilder cb = em.getCriteriaBuilder(); CriteriaQuery q = cb.createQuery(Book.class); Root b = q.from(Book.class); q.select(b).orderBy(cb.desc(b.get("id"))); TypedQuery findAllBooks = em.createQuery(q); So imagine when you have more complex ones. Sometimes, you just get lost, it gets buggy and you would appreciate to have the JPQL and/or SQL String representation to find out what’s happening. You could then even unit test it. Getting the JPQL/SQL String Representations for a Criteria Query So let’s use an API to get the JPQL/SQL String representations of a CriteriaQuery (to be more precise, the TypedQuery created from a CriteriaQuery). The bad news is that there is no standard JPA 2.0 API to do this. You need to use the implementation API hoping the implementation allows it (thank god that’s (nearly) the case for the 3 main JPA ORM frameworks). The good news is that the Query interface (and therefore TypedQuery) has an unwrap method. This method returns the provider’s query API implementation. Let’s see how you can use it with EclipseLink, Hibernate and OpenJPA. EclipseLink EclipseLink‘s Query representation is the org.eclipse.persistence.jpa.JpaQuery interface and the org.eclipse.persistence.internal.jpa.EJBQueryImpl implementation. This interface gives you the wrapped native query (org.eclipse.persistence.queries.DatabaseQuery) with two very handy methods : getJPQLString() and getSQLString(). Unfortunatelly the getJPQLString() method will not translate a CriteriaQuery into JPQL, it only works for queries originally written in JPQL (dynamic or named query). The getSQLString() method relies on the query being “prepared”, meaning you have to run the query once before getting the SQL String representation. findAllBooks.unwrap(JpaQuery.class).getDatabaseQuery().getJPQLString(); // doesn't work for CriteriaQuery findAllBooks.unwrap(JpaQuery.class).getDatabaseQuery().getSQLString(); Hibernate Hibernate‘s Query representation is org.hibernate.Query. This interface has several implementations and the very useful method that returns the SQL query string : getQueryString(). I couldn’t find a method that returns the JPQL representation, if I’ve missed something, please let me know. findAllBooks.unwrap(org.hibernate.Query.class).getQueryString() OpenJPA OpenJPA‘s Query representation is org.apache.openjpa.persistence.QueryImpl and also has a getQueryString() method that returns the SQL (not the JPQL). It delegates the call to the internal org.apache.openjpa.kernel.Query interface. I couldn’t find a method that returns the JPQL representation, if I’ve missed something, please let me know. findAllBooks.unwrap(org.apache.openjpa.persistence.QueryImpl.class).getQueryString() Unit testing Once you get your SQL String, why not unit test it ? Hey, but I don’t want to test my ORM, why would I do that ? Well, it happens that I’ve discovered a but in the new releases of OpenJPA by unit testing a query… so, there is a use case for that. Anyway, this is how you could do it : assertEquals("SELECT b FROM Book b ORDER BY b.id DESC", findAllBooksCriteriaQuery.unwrap(org.apache.openjpa.persistence.QueryImpl.class).getQueryString()); Conclusion As you can see, it’s not that simple to get a String representation for a TypedQuery. Here is a digest of the three main ORMs : ORM Framework Query implementation How to get the JPQL String How to get the SPQL String EclipseLink JpaQuery getDatabaseQuery().getJPQLString()* getDatabaseQuery().getSQLString()** Hibernate Query N/A getQueryString() OpenJPA QueryImpl getQueryString() N/A (*) Only possible on a dynamic or named query. Not possible on a CriteriaQuery (**) You need to execute the query first, if not, the value is null To illustrate all that I’ve written simple test cases using EclipseLink, Hibernate and OpenJPA that you can download from GitHub. Give it a try and let me know. And what about having an API in JPA 2.1 ? For a developers’ point of view it would be great to have two methods in the javax.persistence.Query (and therefore javax.persistence.TypedQuery) interface that would be able to easily return the JPQL and SQL String representations, e.g : Query.getJPQLString() and Query.getSQLString(). Hey, that would be the perfect time to have it in JPA 2.1 that will be shipped in less than a year. Now, as an implementer, this might be tricky to do, I would love to ear your point of view on this. Anyway, I’m going to post an email to the JPA 2.1 Expert Group… just in case we can have this in the next version of JPA ;o) References http://efreedom.com/Question/1-6412774/Get-SQL-String-JPQLQuery http://old.nabble.com/Cannot-get-the-JPQL—SQL-String-of-a-CriteriaQuery-td33882629.html http://paddyweblog.blogspot.fr/2010/04/some-examples-of-criteria-api-jpa-20.html http://www.altuure.com/2010/09/23/jpa-criteria-api-by-samples-part-i/ http://www.altuure.com/2010/09/23/jpa-criteria-api-by-samples-%E2%80%93-part-ii/ http://www.jumpingbean.co.za/blogs/jpa2-criteria-api http://wiki.eclipse.org/EclipseLink/FAQ/JPA#How_to_get_the_SQL_for_a_Query.3F

I really like Spring, so I tend to use its features to the fullest. However, in some dark corners of its philosophy, I tend to disagree with some of its assumptions. One such assumption is the way database testing should work. In this article, I will explain how to configure your projects to make Spring Test and DBUnit play nice together in a multi-developers environment. Context My basic need is to be able to test some complex queries: before integration tests, I've to validate those queries get me the right results. These are not unit tests per se but let's assilimate them as such. In order to achieve this, I use since a while a framework named DBUnit. Although not maintained since late 2010, I haven't found yet a replacement (be my guest for proposals). I also have some constraints: I want to use TestNG for all my test classes, so that new developers wouldn't think about which test framework to use I want to be able to use Spring Test, so that I can inject my test dependencies directly into the test class I want to be able to see for myself the database state at the end of any of my test, so that if something goes wrong, I can execute my own queries to discover why I want every developer to have its own isolated database instance/schema Considering the last point, our organization let us benefit from a single Oracle schema per developer for those "unit-tests". Basic set up Spring provides the AbstractTestNGSpringContextTests class out-of-the-box. In turn, this means we can apply TestNG annotations as well as @Autowired on children classes. It also means we have access to the underlying applicationContext, but I prefer not to (and don't need to in any case). The structure of such a test would look like this: @ContextConfiguration(location = "classpath:persistence-beans.xml") public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Test public void whenXYZThenTUV() { ... } } Readers familiar with Spring and TestNG shouldn't be surprised here. Bringing in DBunit DbUnit is a JUnit extension targeted at database-driven projects that, among other things, puts your database into a known state between test runs. [...] DbUnit has the ability to export and import your database data to and from XML datasets. Since version 2.0, DbUnit can also work with very large datasets when used in streaming mode. DbUnit can also help you to verify that your database data match an expected set of values. DBunit being a JUnit extension, it's expected to extend the provided parent class org.dbunit.DBTestCase. In my context, I have to redefine some setup and teardown operation to use Spring inheritance hierarchy. Luckily, DBUnit developers thought about that and offer relevant documentation. Among the different strategies available, my tastes tend toward the CLEAN_INSERT and NONE operations respectively on setup and teardown. This way, I can check the database state directly if my test fails. This updates my test class like so: @ContextConfiguration(locations = {"classpath:persistence-beans.xml", "classpath:test-beans.xml"}) public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Autowired private IDatabaseTester databaseTester; @BeforeMethod protected void setUp() throws Exception { // Get the XML and set it on the databaseTester // Optional: get the DTD and set it on the databaseTester databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT); databaseTester.setTearDownOperation(DatabaseOperation.NONE); databaseTester.onSetup(); } @Test public void whenXYZThenTUV() { ... } } Per-user configuration with Spring Of course, we need to have a specific Spring configuration file to inject the databaseTester. As an example, here is one: However, there's more than meets the eye. Notice the databaseTester has to be fed a datasource. Since a requirement is to have a database per developer, there are basically two options: either use a in-memory database or use the same database as in production and provide one such database schema per developer. I tend toward the latter solution (when possible) since it tends to decrease differences between the testing environment and the production environment. Thus, in order for each developer to use its own schema, I use Spring's ability to replace Java system properties at runtime: each developer is characterized by a different user.name. Then, I configure a PlaceholderConfigurer that looks for {user.name}.database.properties file, that will look like so: db.username=myusername1 db.password=mypassword1 db.schema=myschema1 This let me achieve my goal of each developer using its own instance of Oracle. If you want to use this strategy, do not forget to provide a specific database.properties for the Continuous Integration server. Huh oh? Finally, the whole testing chain is configured up to the database tier. Yet, when the previous test is run, everything is fine (or not), but when checking the database, it looks untouched. Strangely enough, if you did load some XML dataset and assert it during the test, it does behaves accordingly: this bears all symptoms of a transaction issue. In fact, when you closely look at Spring's documentation, everything becomes clear. Spring's vision is that the database should be left untouched by running tests, in complete contradiction to DBUnit's. It's achieved by simply rollbacking all changes at the end of the test by default. In order to change this behavior, the only thing to do is annotate the test class with @TransactionConfiguration(defaultRollback=false). Note this doesn't prevent us from specifying specific methods that shouldn't affect the database state on a case-by-case basis with the @Rollback annotation. The test class becomes: @ContextConfiguration(locations = {classpath:persistence-beans.xml", "classpath:test-beans.xml"}) @TransactionConfiguration(defaultRollback=false) public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Autowired private IDatabaseTester databaseTester; @BeforeMethod protected void setUp() throws Exception { // Get the XML and set it on the databaseTester // Optional: get the DTD and set it on the databaseTester databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT); databaseTester.setTearDownOperation(DatabaseOperation.NONE); databaseTester.onSetup(); } @Test public void whenXYZThenTUV() { ... } } Conclusion Though Spring and DBUnit views on database testing are opposed, Spring's configuration versatility let us make it fit our needs (and benefits from DI). Of course, other improvements are possible: pushing up common code in a parent test class, etc. To go further: Spring Test documentation DBUnit site Database data verification Database testing best practices Generating DTD from your database schema

Insanity: doing the same thing over and over again and expecting different results. -- attributed to Albert Einstein Global state is essentially the same thing as insanity in this definition: a way to affect the execution of code hidden from sight, so that two apparently identical lines actually produce a different result depending on some external factor. For example: new SomeClass()->printId(); new SomeClass()->printId(); // output: 1, 2 has some global state (a static counter) affecting a field inside SomeClass instances. Therefore, it may not be easy to replicate scenarios (like in tests) multiple times. Examples Global and environmental variables, along with constants are simple examples of global state. The same goes for configuration directives and files which code silently depends upon, as long as they are global for each instance of the affected objects. Speaking about objects singletons and static classes containing fields are another example of global state. More subtle cases are hidden localizations like translations of output and of symbols (LC_ALL?) Parameterization is made difficult by global state, either because the seam for collaborators is hidden (config files and enviromental/global variables) or not accessible (singletons). Testability When there is some global state in an application, the affected unit tests won'tbe isolated from each other, and may change their results when run alone or in a different order with respect to being executed inside the whole test suite. Global state is one of the most common problems while working with legacy code which was written with little concern for testability (and separation of concerns). A typical annoying example is a test that passes when alone, but fails in the full suite due to some state left lingering from previous tests. Usually, it is then debugged by executing the exact same test twice or multiple times in a single process and verifying that it passes consistently. Actually global state cannot be always removed, even in a test environment: what this move would achieve would be a fully parameterized system, too general to be useful; imagine configuring every class name in your application, even in Factories. It may be simpler to test with some global state in, like in the case of a default locale defined in place of stubbing the Translator object; or in the case of a Fake database connection instead of a Stub or a Mock. Taking this approach to the extreme, we notice that global state is often hidden from our view because it's taken for granted. Base classes offered by the language (e.g. String) are not mocked or substituted by test doubles, even when they have quite some logic in them; all the classes and functions contained in our applications are global state as their implementation cannot be substituted, yet we don't consider them a trouble as singletons. Constant There is a reason why not all global state is necessarily bad: constant global state to allow context-free reasoning about code, and simplify testing and reuse of code considerably. In fact, the very definition of state (for example from hardware logic networks) is that of a component that can change its behavior in time, keeping information about previous inputs. In short, any computation that is frequently accessed but does not have the capability to change its result or to produce side-effects is not state (it is global). ROM is instead considered a purely combinatorial network, not being real "memory" but merely a function translating addresses to words. A singleton changing its responses after some calls is global state that makes testing difficult; a static class containing only pure functions may make tests long winded and infringe the object-oriented paradigm, but it's not as dangerous as the former. However, that's why I see monkey patching in dynamic languages as problematic. Monkey patching commonly consists of open classes where you can add methods at any time after their initial definition. class Array def sum inject {|sum, x| sum + x } end end When you see a call to this method, you have to ask some questions: where it was added in the code base? Which sourcefile should I look at? When it was added to the code base? Am I sure that definitions can only be included at startup and I am not calling the method before it is defined? Are there multiple redefinitions of the method? Maybe from other libraries or code to integrate? The same issues happened for prototype.js, which modifies the prototype object of base JavaScript objects like Array, effectively redefining and adding methods. The result is little interoperability with other libraries. But even monkey patching should be fine as long as the modifications are really constant and definitive. If you use a single framework (like Ext Js), and you always use it in all pages of the application and in tests, then it can monkeypatch the base classes of the language safely, without making you debug a method that works in one environment but not in the other. Conclusion Global state is not only a global parameter for the internals of your application, but also the product of stateful interaction that changes the output and side-effects of code in different invocations. Making global things constant is the first step towards simplifying reasoning about a design and raising testability. If you are able to run a unit test twice in the same method, you are officially free from global state in that scenario. Pay attention when embracing open classes, editable prototypes, embedded calls to registries and files, and so on: they can add several dimensions to the variables that can affect the result of a piece of code. They will hide a dependency but not making it go away.

Learn how Apache Camel implements the EIPs and offers a standardized, internal domain-specific language (DSL) to integrate applications.

An Alpha Animation is animation that controls the alpha level of an object, i.e. fading it in and out. In Android, you can apply that fading effect to almost anything, from simple text, to images, buttons, check boxes, etc... Android has a few classes that can help you add that special effect to your programs, like AlphaAnimation and AnimationUtils. Here's an example on how to apply fading on any Android component subclass of View. First, the XML resource. In the resources folder, we will create a tiny XML configuration file with the characteristics of the fading effect we want in an "anim" subfolder. So, under res/anim, here's our alpha.xml: We are choosing to have a very basic full fade in effect (alpha from 0 to 1) that lasts one second. The above can also be done directly in Java code: Animation animation = new AlphaAnimation(0.0f, 1.0f); animation.setDuration(1000); Configuring the animation in resources or in code is ultimately a matter of preference. We will use the XML in this example. This is our class that does the above fading to any View (TextView, Button, etc..): package com.ts.fx.utils; import android.app.Activity; import android.view.View; import android.view.animation.Animation; import android.view.animation.AnimationUtils; public class Fader { /** * handles all subclasses of View : TextView, Button, ImageView etc.. * given the component's id in their layout file * */ public static void runAlphaAnimation(Activity act, int viewId) { // load animation XML resource under res/anim Animation animation = AnimationUtils.loadAnimation(act, R.anim.alpha); if(animation == null){ return; // here, we don't care } // reset initialization state animation.reset(); // find View by its id attribute in the XML View v = act.findViewById(viewId); // cancel any pending animation and start this one if (v != null){ v.clearAnimation(); v.startAnimation(animation); } } } The runAlphaAnimation() method takes an Activity reference and a View id attribute (as set up in the View's layout XML). We're basically done. all we have to do now is call it from any one of our Activitites: // inside an Activity with text, checkbox and button Fader.runAlphaAnimation(this, a_text.getId()); Fader.runAlphaAnimation(this, a_checkbox.getId()); Fader.runAlphaAnimation(this, a_button.getId()); //etc... That's all there is to it. The same basic technique seen here applies to all other special effects like translating, scaling or rotating components. The Animation classes have of course lots of other cool stuff, like controlling the z-ordering of the animated components, acceleration and repeat effects. Here's a thirty-second video (by yours truly) demonstrating various Android special effects (fading, translation and rotation) used together in a concrete application: From Tony's Blog.

A few weeks ago, I co-authored an article (with my colleague Rama Ramani) about how the Screen Actors Guild Awards website migrated its Drupal deployment from LAMP to Windows Azure: Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. Since then, Rama and another colleague, Jason Roth, have been working on writing up how the SAG Awards website was managed and monitored in Windows Azure. The article below is the fruit of their work…a very interesting/educational read. Overview Drupal is an open source content management system that runs on PHP. Windows Azure offers a flexible platform for hosting, managing, and scaling Drupal deployments. This paper focuses on an approach to host Drupal sites on Windows Azure, based on learning from a BPD Customer Programs Design Win engagement with the Screen Actors Guild Awards Drupal website. This paper covers guidelines and best practices for managing an existing Drupal web site in Windows Azure. For more information on how to migrate Drupal applications to Windows Azure, see Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. The target audience for this paper is Drupal administrators who have some exposure to Windows Azure. More detailed pointers to Windows Azure content is provided throughout the paper as links. Drupal Application Architecture on Windows Azure Before reviewing the management and monitoring guidelines, it is important to understand the architecture of a typical Drupal deployment on Windows Azure. First, the following diagram displays the basic architecture of Drupal running on Windows and IIS7. In the Windows Server scenario, you could have one or more machines hosting the web site in a farm. Those machines would either persist the site content to the file system or point to other network shares. For Windows Azure, the basic architecture is the same, but there are some differences. In Windows Azure the site is hosted on a web role. A web role instance is hosted on a Windows Server 2008 virtual machine within the Windows Azure datacenter. Like the web farm, you can have multiple instances running the site. But there is no persistence guarantee for the data on the file system. Because of this, much of the shared site content should be stored in Windows Azure Blob storage. This allows them to be highly available and durable. Usually, a large portion of the site caters to static content which lends well to caching. And caching can be applied in a set of places – browser level caching, CDN to cache content in the edge closer to the browser clients, caching in Azure to reduce the load on backend, etc. Finally, the database can be located in SQL Azure. The following diagram shows these differences. For monitoring and management, we will look at Drupal on Windows Azure from three perspectives: Availability: Ensure the web site does not go down and that all tiers are setup correctly. Apply best practices to ensure that the site is deployed across data centers and perform backup operations regularly. Scalability: Correctly handle changes in user load. Understand the performance characteristics of the site. Manageability: Correctly handle updates. Make code and site changes with no downtime when possible. Although some management tasks span one or more of these categories, it is still helpful to discuss Drupal management on Windows Azure within these focus areas. Availability One main goal is that the Drupal site remains running and accessible to all end-users. This involves monitoring both the site and the SQL Azure database that the site depends on. In this section, we will briefly look at monitoring and backup tasks. Other crossover areas that affect availability will be discussed in the next section on scalability. Monitoring With any application, monitoring plays an important role with managing availability. Monitoring data can reveal whether users are successfully using the site or whether computing resources are meeting the demand. Other data reveals error counts and possibly points to issues in a specific tier of the deployment. There are several monitoring tools that can be used. The Windows Azure Management Portal. Windows Azure diagnostic data. Custom monitoring scripts. System Center Operations Manager. Third party tools such as Azure Diagnostics Manager and Azure Storage Explorer. The Windows Azure Management Portal can be used to ensure that your deployments are successful and running. You can also use the portal to manage features such as Remote Desktop so that you can directly connect to machines that are running the Drupal site. Windows Azure diagnostics allows you to collect performance counters and logs off of the web role instances that are running the Drupal site. Although there are many options for configuring diagnostics in Azure, the best solution with Drupal is to use a diagnostics configuration file. The following configuration file demonstrates some basic performance counters that can monitor resources such as memory, processor utilization, and network bandwidth. For more information about setting up diagnostic configuration files, see How to Use the Windows Azure Diagnostics Configuration File. This information is stored locally on each role instance and then transferred to Windows Azure storage per a defined schedule or on-demand. See Getting Started with Storing and Viewing Diagnostic Data in Windows Azure Storage. Various monitoring tools, such as Azure Diagnostics Manager, help you to more easily analyze diagnostic data. Monitoring the performance of the machines hosting the Drupal site is only part of the story. In order to plan properly for both availability and scalability, you should also monitor site traffic, including user load patterns and trends. Standard and custom diagnostic data could contribute to this, but there are also third-party tools that monitor web traffic. For example, if you know that spikes occur in your application during certain days of the week, you could make changes to the application to handle the additional load and increase the availability of the Drupal solution. Backup Tasks To remain highly available, it is important to backup your data as a defense-in-depth strategy for disaster recovery. This is true even though SQL Azure and Windows Azure Storage both implement redundancy to prevent data loss. One obvious reason is that these services cannot prevent administrator error if data is accidentally deleted or incorrectly changed. SQL Azure does not currently have a formal backup technology, although there are many third-party tools and solutions that provide this capability. Usually the database size for a Drupal site is relatively small. In the case of SAG Awards, it was only ~100-150 MB. So performing an entire backup using any strategy was relatively fast. If your database is much larger, you might have to test various backup strategies to find the one that works best. Apart from third-party SQL Azure backup solutions, there are several strategies for obtaining a backup of your data: · Use the Drush tool and the portabledb-export command. · Periodically copy the database using the CREATE DATABASE Transact-SQL command. · Use Data-tier applications (DAC) to assist with backup and restore of the database. SQL Azure backup and data security techniques are described in more detail in the topic, Business Continuity in SQL Azure. Note that bandwidth costs accrue with any backup operation that transfers information outside of the Windows Azure datacenter. To reduce costs, you can copy the database to a database within the same datacenter. Or you can export the data-tier applications to blob storage in the same datacenter. Another potential backup task involves the files in Blob storage. If you keep a master copy of all media files uploaded to Blob storage, then you already have an on-premises backup of those files. However, if multiple administrators are loading files into Blob storage for use on the Drupal site, it is a good idea to enumerate the storage account and to download any new files to a central location. The following PHP script demonstrates how this can be done by backing up all files in Blob storage after a specified modification date. setProxy(true, 'YOUR_PROXY_IF_NEEDED', 80); $blobs = (array)$blobObj->listBlobs(AZURE_STORAGE_CONTAINER, '', '', 35000); backupBlobs($blobs, $blobObj); function backupBlobs($blobs, $blobObj) { foreach ($blobs as $blob) { if (strtotime($blob->lastmodified) >= DEFAULT_BACKUP_FROM_DATE && strtotime($blob->lastmodified) <= DEFAULT_BACKUP_TO_DATE) { $path = pathinfo($blob->name); if ($path['basename'] != '$$$.$$$') { $dir = $path['dirname']; $oldDir = getcwd(); if (handleDirectory($dir)) { chdir($dir); $blobObj->getBlob( AZURE_STORAGE_CONTAINER, $blob->name, $path['basename'] ); chdir($oldDir); } } } } } function handleDirectory($dir) { if (!checkDirExists($dir)) { return mkdir($dir, 0755, true); } return true; } function checkDirExists($dir) { if(file_exists($dir) && is_dir($dir)) { return true; } return false; } ?> This script has a dependency on the Windows Azure SDK for PHP. Also note there are several parameters that you must modify such as the storage account, secret, and backup location. As with SQL Azure, bandwidth and transaction charges apply to a backup script like this. Scalability Drupal sites on Windows Azure can scale as load increased through typical strategies of scale-up, scale-out, and caching. The following sections describe the specifics of how these strategies are implemented in Windows Azure. Typically you make scalability decisions based on monitoring and capacity planning. Monitoring can be done in staging during testing or in production with real-time load. Capacity planning factors in projections for changes in user demand. Scale Up When you configure your web role prior to deployment, you have the option of specifying the Virtual Machine (VM) size, such as Small or ExtraLarge. Each size tier adds additional memory, processing power, and network bandwidth to each instance of your web role. For cost efficiency and smaller units of scale, you can test your application under expected load to find the smallest virtual machine size that meets your requirements. The workload usually in most popular Drupal websites can be separated out into a limited set of Drupal admins making content changes and a large user base who perform mostly read-only workload. End users can be allowed to make ‘writes’, such as uploading blogs or posting in forums, but those changes are not ‘content changes’. Drupal admins are setup to operate without caching so that the writes are made directly to SQL Azure or the corresponding backend database. This workload performs well with Large or ExtraLarge VM sizes. Also, note that the VM size is closely tied to all hardware resources, so if there are many content-rich pages that are streaming content, then the VM size requirements are higher. To make changes to the Virtual Machine size setting, you must change the vmsize attribute of the WebRole element in the service definition file, ServiceDefinition.csdef. A virtual machine size change requires existing applications to be redeployed. Scale Out In addition to the size of each web role instance, you can increase or decrease the number of instances that are running the Drupal site. This spreads the web requests across more servers, enabling the site to handle more users. To change the number of running instances of your web role, see How to Scale Applications by Increasing or Decreasing the Number of Role Instances. Note that some configuration changes can cause your existing web role instances to recycle. You can choose to handle this situation by applying the configuration change and continue running. This is done by handling the RoleEnvironment.Changing event. For more information see, How to Use the RoleEnvironment.Changing Event. A common question for any Windows Azure solution is whether there is some type of built-in automatic scaling. Windows Azure does not provide a service that provides auto-scaling. However, it is possible to create a custom solution that scales Azure services using the Service Management API. For an example of this approach, see An Auto-Scaling Module for PHP Applications in Windows Azure. Caching Caching is an important strategy for scaling Drupal applications on Windows Azure. One reason for this is that SQL Azure implements throttling mechanisms to regulate the load on any one database in the cloud. Code that uses SQL Azure should have robust error handling and retry logic to account for this. For more information, see Error Messages (SQL Azure Database). Because of the potential for load-related throttling as well as for general performance improvement, it is strongly recommended to use caching. Although Windows Azure provides a Caching service, this service does not currently have interoperability with PHP. Because of this, the best solution for caching in Drupal is to use a module that uses an open-source caching technology, such as Memcached. Outside of a specific Drupal module, you can also configure Memcached to work in PHP for Windows Azure. For more information, see Running Memcached on Windows Azure for PHP. Here is also an example of how to get Memcached working in Windows Azure using a plugin: Windows Azure Memcached plugin. In a future paper, we hope to cover this architecture in more detail. For now, here are several design and management considerations related to caching. Area Consideration Design and Implementation For a technology like Memcached, will the cache be collocated (spread across all web role instances)? Or will you attempt to setup a dedicated cache ring with worker roles that only run Memcached? Configuration What memory is required and how will items in the cache be invalidated? Performance and Monitoring What mechanisms will be used to detect the performance and overall health of the cache? For ease of use and cost savings, collocation of the cache across the web role instances of the Drupal site works best. However, this assumes that there is available reserve memory on each instance to apply toward caching. It is possible to increase the virtual machine size setting to increase the amount of available memory on each machine. It is also possible to add additional web role instances to add to the overall memory of the cache while at the same time improving the ability of the web site to respond to load. It is possible to create a dedicated cache cluster in the cloud, but the steps for this are beyond the scope of this paper[RR1] . For Windows Azure Blob storage, there is also a caching feature built into the service called the Content Delivery Network (CDN). CDN provides high-bandwidth access to files in Blob storage by caching copies of the files in edge nodes around the world. Even within a single geographic region, you could see performance improvements as there are many more edge nodes than Windows Azure datacenters. For more information, see Delivering High-Bandwidth Content with the Windows Azure CDN. Manageability It is important to note that each hosted service has a Staging environment and a Production environment. This can be used to manage deployments, because you can load and test and application in staging before performing a VIP swap with production. From a manageability standpoint, Drupal has an advantage on Windows Azure in the way that site content is stored. Because the data necessary to serve pages is stored in the database and blob storage, there is no need to redeploy the application to change the content of the site. Another best practice is to use a separate storage account for diagnostic data than the one that is used for the application itself. This can improve performance and also helps to separate the cost of diagnostic monitoring from the cost of the running application. As mentioned previously, there are several tools that can assist with managing Windows Azure applications. The following table summarizes a few of these choices. Tool Description Windows Azure Management Portal The web interface of the Windows Azure management portal shows deployments, instance counts and properties, and supports many different common management and monitoring tasks. Azure Diagnostics Managerq[RR2] [JR3] A Red Gate Software product that provides advanced monitoring and management of diagnostic data. This tool can be very useful for easily analyzing the performance of the Drupal site to determine appropriate scaling decisions. Azure Storage Explorer A tool created by Neudesic for viewing Windows Azure storage account. This can be useful for viewing both diagnostic data and the files in Blob storage.

See original post at https://muhammadkhojaye.blogspot.com/2012/04/how-to-run-amazon-elastic-mapreduce-job.html Introduction Amazon EMR is a web service which can be used to easily and efficiently process enormous amounts of data. It uses a hosted Hadoop framework running on the web-scale infrastructure of Amazon EC2 and Amazon S3. Amazon EMR removes most of the cumbersome details of Hadoop while taking care of provisioning of Hadoop, running the job flow, terminating the job flow, moving the data between Amazon EC2 and Amazon S3, and optimizing Hadoop. In this tutorial, we will use a developed WordCount Java example using Hadoop and thereafter, we execute our program on Amazon Elastic MapReduce. Prerequisites You must have valid AWS account credentials. You should also have a general familiarity with using the Eclipse IDE before you begin. The reader can also use any other IDE of their choice. Step 1 – Develop MapReduce WordCount Java Program In this section, we are first going to develop a WordCount application. A WordCount program will determine how many times different words appear in a set of files. In Eclipse (or whatever the IDE you are using), Create simple Java Project with the name "WordCount". Create a java class name Map and override the map method as follow, public class Map extends Mapper { private final static IntWritable one = new IntWritable(1); private Text word = new Text(); @Override public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { String line = value.toString(); StringTokenizer tokenizer = new StringTokenizer(line); while (tokenizer.hasMoreTokens()) { word.set(tokenizer.nextToken()); context.write(word, one); } } } Create a java class named Reduce and override the reduce method as shown below, public class Reduce extends Reducer { @Override protected void reduce(Text key, java.lang.Iterable values, org.apache.hadoop.mapreduce.Reducer.Context context) throws IOException, InterruptedException { int sum = 0; for (IntWritable value : values) { sum += value.get(); } context.write(key, new IntWritable(sum)); } } Create a java class named WordCount and defined the main method as below, public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); Job job = new Job(conf, "wordcount"); job.setJarByClass(WordCount.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); job.setMapperClass(Map.class); job.setReducerClass(Reduce.class); job.setInputFormatClass(TextInputFormat.class); job.setOutputFormatClass(TextOutputFormat.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); } Export the WordCount program in a jar using eclipse and save it to some location on disk. Make sure that you have provided the Main Class (WordCount.jar) during extraction ofu8u the jar file as shown below. Our jar is ready!!! Step 2 – Upload the WordCount JAR and Input Files to Amazon S3 Now we are going to upload the WordCount jar to Amazon S3. First, go to the following URL: https://console.aws.amazon.com/s3/home Next, click “Create Bucket”, give your bucket a name, and click the “Create” button. Select your new S3 bucket in the left-hand pane. Upload the WordCount JAR and sample input file for counting the words. Step 3 – Running an Elastic MapReduce job Now that the JAR is uploaded into S3, all we need to do is to create a new Job flow. let's execute the steps below. (I encourage readers to check out the following link for details regarding each step, How to Create a Job Flow Using a Custom JAR ) Sign in to the AWS Management Console and open the Amazon Elastic MapReduce console at https://console.aws.amazon.com/elasticmapreduce/ Click Create New Job Flow. In the DEFINE JOB FLOW page, enter the following details, a) Job Flow Name = WordCountJob b) Select Run your own applications) Select Custom JAR in the drop-down list) Click Continue In the SPECIFY PARAMETERS page, enter values in the boxes using the following table as a guide, and then click Continue.JAR Location = bucketName/jarFileLocationJAR Arguments =s3n://bucketName/inputFileLocations3n://bucketName/outputpath Please note that the output path must be unique each time we execute the job. The Hadoop always create a folder with the same name specified here. After executing the job, just wait and monitor your job that runs through the Hadoop flow. You can also look for errors by using the Debug button. The job should be complete within 10 to 15 minutes (can also depend on the size of the input). After completing the job, You can view results in the S3 Browser panel. You can also download the files from S3 and can analyze the outcome of the job. Amazon Elastic MapReduce Resources Amazon Elastic MapReduce Documentation,http://aws.amazon.com/documentation/elasticmapreduce/ Amazon Elastic MapReduce Getting Started Guide,http://docs.amazonwebservices.com/ElasticMapReduce/latest/GettingStartedGuide/ Amazon Elastic MapReduce Developer Guide,http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/ Apache Hadoop,http://hadoop.apache.org/ See more at https://muhammadkhojaye.blogspot.com/2012/04/how-to-run-amazon-elastic-mapreduce-job.html

Let's take a closer look at how to play sounds on an Android device with SoundPool and MediaPlayer.

Since the very early days of Java (and C-like languages overall), the canonical way to start your program has been something like this: public class A { public static void main(String[] args) { new A().run(args); } public void run(String[] args) { // Your application starts here } } If you are still doing this, I’m here to tell you it’s time to stop. Letting go of ‘new’ First, install Guice in your project: com.google.inject guice 3.0 and then, modify your main method as follows: public class A { public static void main(String[] args) { Injector.getInstance(A.class).run(args); } } So, what does this buy you exactly? You will find a lot of articles explaining the various benefits of Guice, such as being able to substitute different environments on the fly, but I’m going to use a different angle in this article. Let’s start by assuming the existence of a Config class that contains various configuration parameters. I’ll just hardcode them for now and use fields to make the class smaller: public class Config { String host = "com.example.com"; int port = 1234; } This class is a singleton, it is instantiated somewhere in your main class and not used anywhere else at the moment. One day, you realize you need this instance in another class which happens to be deep in your runtime hierarchy, which we will call Deep. For example, if you put a break point in the method where you need this config object, your debugger would show you stack frames similar to this: com.example.A.main() com.example.B.f(int, String) com.example.C.g(String) com.example.Deep.h(Foo, int) The easy and wrong way to solve this problem is to make the Config instance static on some class (probably A) and access it directly from Deep. I’m hoping I don’t need to explain why this is a bad idea: not only do you want to avoid using statics, but you also want to make sure that each object is exposed only to objects that need them, and making the Config object static would make your instance visible to your entire code base. Not a good thing. The second thought is to pass the object down the stack, so you modify all the signatures as follows: com.example.A.main() com.example.B.f(int, String, Config) com.example.C.g(String, Config) com.example.Deep.h(Foo, int, Config) This is a bit better since you have severely restricted the exposure of the Config object, but note that you are still making it available to more methods than really need to: B#f and C#g have really nothing to do with this object, and a little sting of discomfort hits you when you start writing the Javadoc: public class C { ... /** * @param config This method doesn't really use this parameter, * it just passes it down so Deep#h can use it. */ public void g(String s, Config config) { Unnecessary exposure is actually not the worst part of this approach, the problem is that it changes all these signatures along the way, which is certainly undesirable in a private API and absolutely devastating in a public API. And of course, it’s absolutely not scalable: if you keep adding a parameter to your method whenever you need access to a certain object, you will soon be dealing with methods that take ten parameters, most of which they just pass down the chain. Here is how we solve this problem with dependency injection (performed by Guice in this example, but this is applicable to any library that implements JSR 330, obviously): public class Deep { @Inject private Config config; and we’re done. That’s it. You don’t need to modify the Config class in any way, nor do you need to make any change in any of the classes that separate Deep from your main class. With this, you have also minimized the exposure of the Config object to just the class that needs it. Injecting right There are various ways you can inject object into your class but I’ll just mention the two that, I think, are the most important. I just showed “field injection” in the previous paragraph, but be aware that you can also prefer to use “constructor injection”: public class Deep { private final Config config; @Inject public Deep(Config config) { this.config = config; } This time, you are adding a parameter to the constructor of your Deep class (which shouldn’t worry you too much since you will never invoke it directly, Guice will) and you assign the parameter to the field in the constructor. The benefit is that you can declare your field final. The downside, obviously, is that this approach is much more verbose. Personally, I see little point in final fields since I have hardly ever encountered a bug that was due to accidentally reassigning a field, so I tend to use field injection whenever I can. Taking it to the next level Obviously, the kind of configuration object I used as an example if not very realistic. Typically, a configuration will not hardcode values like I did and will, instead, read them from some external source. Similarly, you will want to inject objects that can’t necessarily be instantiated so early in the lifecycle of your application, such as servlet contexts, database connections, or implementations of your own interfaces. This topic itself would probably cover several chapters of a book dedicated to dependency injection, so I’ll just summarize it: not all objects can be injected this way, and one benefit of using a dependency injection framework in your code is that it will force you to think about what life cycle category your objects belong to. Having said that, if you want to find out how Guice can inject objects that get created at a later time in your application life cycle, look up the Javadoc for the Provider class. Wrapping up I hope this quick introduction to dependency injection piqued your interest and that you will consider using it in your project since it has so much more to offer than what I described in this post. If you want to learn more, I suggest starting with the excellent Guice documentation.

today i have prepared another really great tutorial for you. recently i started development of my own radio software (as a module for dolphin cms) and got some interesting results i would like to share with you. it will be nice looking (css3) radio script that consists of three main elements: the header (with animated search bar and integrated radio player), the left side (with a list of categories and subcategories) and the right side (which will contain a list of recent or filtered stations). here's the final result of our player: here's our live demo and downloadable package: ok, download our source files and lets start coding! step 1. html markup this is the markup of one of the template files. this is a template of our main (index) page: templates/main_page.html stream radio script back to original tutorial on script tutorials alternative classic alternativeindustrialnew wavepunk classical modernoperapianoromanticsymphony electronic breakbeatdanceelectrohousetechnotrance metal classic metalheavy metalmetalcorepower metal pop dance popoldiestop 40world pop __stations__ powered by script tutorials first, pay attention to how the script loads the jquery library from google. this can be pretty useful if you don’t like to keep this file directly on your host. our header element contains a nice search bar with an embedded jasl player ( i used a great ffmp3 live stream player ), which allows us to play audio streams without any problems. next, on the left-hand side (beneath the header) we have a ul-li based list of categories and subcategories. the right-hand side will contain a list of the most recent stations and, when we search or select a category, the right-hand side will be filtered by ajaxy. for now – it contains __stations__ key (template key) and we will replace the actual value with php. on to our next template file, the radio player: templates/radio.html of course, it contains its own template keys (__title__ and __stream__) which we will use after. step 2. css here are our stylesheets files: css/main.css the first one contains the styles of our test page (this file is always available in our package) css/radio.css /* header area */ .header { height:62px; } .header input { background:#aaa url(../images/search.png) no-repeat 5px center; border:1px solid #888; border-radius:10px; float:right; margin:14px 10px 0 0; outline:none; padding-left:20px; width:200px; -webkit-transition: 0.5s; -moz-transition: 0.5s; -o-transition: 0.5s; transition: 0.5s; } .header input:focus { background-color:#eee; width:300px; } .header > span { display:block; float:left; line-height:40px; padding:7px; -webkit-transition: 0.5s; -moz-transition: 0.5s; -o-transition: 0.5s; transition: 0.5s; } /* stations list */ .stlist { float:right; margin-right:1%; width:71%; } .stlist ul { list-style:none outside none; margin:0; padding:0; } .stlist ul li { border-bottom:1px dotted #444; overflow:hidden; padding:10px; } .stlist ul li > a > img { border:1px solid #ccc; float:left; height:85px; margin-right:15px; padding:1px; width:85px; } .stlist ul li > div { float:right; margin-left:15px; margin-top:-5px; } .stlist ul li > p.label,.stlist ul li > p.track { font-size:11px; font-weight:700; } .stlist ul li > p.label { color:#888; } .stlist ul li > p.channel { font-size:14px; font-weight:700; margin-bottom:17px; } /* genres list */ .genres_par { border-right:1px solid #ccc; float:left; width:26%; } ul.genres,ul.genres ul { list-style-type:none; margin:0; padding:0; } ul.genres ul { display:none; overflow:hidden; padding:0 15px; } ul.genres ul li { margin:3px; } ul.genres a { color:#333; display:block; font-size:18px; padding:4px 0; text-align:center; text-decoration:none; } ul.genres ul a { font-size:12px; text-align:left; } ul.genres li { border-bottom:1px solid #ccc; margin:0; } ul.genres li ul li a { background:none repeat scroll 0 0 #5bb951; border-radius:2px; color:#fff; font-size:12px; padding:6px; } ul.genres li ul li a:hover { background-color:#53854e; } step 3. js js/script.js $(document).ready(function(){ $('#search').blur(function() { if ('' == $('#search').val()) $('#search').val('search'); }); $('#search').focus(function() { if ('search' == $('#search').val()) $('#search').val(''); }); $('ul.genres li a').click( // category slider function() { var checkelement = $(this).next(); if((checkelement.is('ul')) && (!checkelement.is(':visible'))) { $('.genres li ul').slideup(150); $(this).next().slidetoggle(150); } } ); $('ul.genres ul li a').click( // get stations by category function() { $.ajax({ type: 'get', url: 'index.php', data: 'action=get_genre_stations&id=' + $(this).parent().attr('id') + '&name=' + $(this).parent().attr('val'), success: function(data){ $('.stlist').fadeout(400, function () { $('.stlist').html(data); $('.stlist').fadein(400); }); } }); } ); }); function play(id) { // play function $('#rplayer').load('index.php?action=play&id=' + id, function() {}); return false; } function get_stations_by_keyword() { // get stations by keyword var keyword = $('#search').val().replace(/ /g,"+"); $.ajax({ type: 'get', url: 'index.php', data: 'action=get_keyword_stations&key=' + keyword, success: function(data){ $('.stlist').fadeout(400, function () { $('.stlist').html(data); $('.stlist').fadein(400); }); } }); } as you see – there's nothing difficult there. just several event handlers, and two new functions (to play radio station and to search for stations by keyword). step 4. php index.php =') == 1) error_reporting(e_all & ~e_notice & ~e_deprecated); else error_reporting(e_all & ~e_notice); $astations = array( 0 => array( 'category' => 31, 'name' => 'eurodance', 'desc' => 'the newest and best of eurodance hits', 'url' => 'http://www.di.fm/eurodance', 'br' => 96, 'stream' => 'http://scfire-mtc-aa06.stream.aol.com:80/stream/1024' ), 1 => array ( 'category' => 34, 'name' => 'house', 'desc' => 'silky sexy deep house music direct from new york city!', 'url' => 'http://www.di.fm/house', 'br' => 96, 'stream' => 'http://scfire-ntc-aa04.stream.aol.com:80/stream/1007' ), 2 => array ( 'category' => 13, 'name' => 'trance', 'desc' => 'the hottest, freshest trance music from around the globe!', 'url' => 'http://www.di.fm/trance', 'br' => 96, 'stream' => 'http://scfire-ntc-aa04.stream.aol.com:80/stream/1003' ), 3 => array ( 'category' => 51, 'name' => 'electro house', 'desc' => 'an eclectic mix of electro and dirty house', 'url' => 'http://www.di.fm/electro', 'br' => 96, 'stream' => 'http://scfire-ntc-aa04.stream.aol.com:80/stream/1025' ) ); function searchbycat($icat, $astations) { $ares = array(); foreach ($astations as $i => $ainfo) { if ($ainfo['category'] == $icat) { $ares[$i] = $ainfo; } } return $ares; } function searchbykeyword($skey, $astations) { $ares = array(); foreach ($astations as $i => $ainfo) { if (false !== strpos($ainfo['name'], $skey) || false !== strpos($ainfo['desc'], $skey)) { $ares[$i] = $ainfo; } } return $ares; } function parsestationlist($adata) { $sstations = ''; if (is_array($adata) && count($adata) > 0) { foreach ($adata as $i => $a) { $sstationid = $i; $sstationbr = (int)$a['br']; $sstationname = $a['name']; $sstationdesc = $a['desc']; $sstationurl = $a['url']; $sthumb = 'media/'.($sstationid+1).'.png'; $sstations .= << bitrate: {$sstationbr} {$sstationname} {$sstationdesc} {$sstationurl} eof; } } $sstations = ($sstations == '') ? 'nothing found' : $sstations; return '' . $sstations . ''; } switch ($_get['action']) { case 'play': $i = (int)$_get['id']; $ainfo = $astations[$i]; $avars = array ( '__stream__' => $ainfo['stream'], '__title__' => $ainfo['name'] ); echo strtr(file_get_contents('templates/radio.html'), $avars); exit; break; case 'get_genre_stations': $i = (int)$_get['id']; $asearch = searchbycat($i, $astations); $sstations = parsestationlist($asearch); header('content-type: text/html; charset=utf-8'); echo $sstations; exit; break; case 'get_keyword_stations': $skey = $_get['key']; $asearch = searchbykeyword($skey, $astations); $sstations = parsestationlist($asearch); header('content-type: text/html; charset=utf-8'); echo $sstations; exit; break; } $slaststations = parsestationlist($astations); echo strtr(file_get_contents('templates/main_page.html'), array('__stations__' => $slaststations)); at the beginning, i have prepared a list of our radio stations (4 stations total). then, two search functions: ‘searchbycat’ and ‘searchbykeyword’. next, the special function ‘parsestationlist’ which will transform array with filtered stations into its html representation. finally, a little switch case to manage with our inner ajax commands. conclusion you are always welcome to enhance our script and share your ideas. i will be glad to see your thanks and comments. good luck!

There’s still a lot of confusion over the difference types of managed beans provided in Java EE 6 with EJBs, CDI beans and JSF managed beans all being available. This article aims to clear up some of the differences between the them and define when to use them. A number of people assume that there is some meaning to all these different types of beans that they just don’t understand. However, the problem is down to the different APIs overlapping which is unfortunate. JSF Managed Beans, CDI Beans and EJBs JSF was initially developed with its own managed bean and dependency injection mechanism which was enhanced for JSF 2.0 to include annotation based beans. When CDI was released with Java EE 6, it was regarded as the managed bean framework for that platform and of course, EJBs outdated them all having been around for well over a decade. The problem of course is knowing which one to use and when, but they all involve the same process. Typically a class has to be identified as a managed bean, and where necessary, will need a scope,qualifiers and a name if it is to be used in JSF. What follows is a brief description of the different types of managed beans and how and when to use them. Let’s start with the simplest, JSF Managed beans. JSF Managed Beans In short, don’t use them if you are developing for Java EE 6 and using CDI. They provide a simple mechanism for dependency injection and defining backing beans for web pages, but they are far less powerful than CDI beans. They can be defined using the @javax.faces.bean.ManagedBean annotation which takes an optional name parameter. This name can be used to reference the bean from JSF pages. Scope can be applied to the bean using one of the different scopes defined in the javax.faces.bean package which include the request, session, applicaion, view and custom scopes. @ManagedBean(name="someBean") @RequestScoped public class SomeBean { .... .... } JSF beans cannot be mixed with other kinds of beans without some kind of manual coding. CDI Beans CDI is the bean management and dependency injection framework that was released as part of Java EE 6 and it includes a complete, comprehensive managed bean facility. CDI beans are far more advanced and flexible than simple JSF managed beans. They can make use of interceptors, conversation scope, Events, type safe injection, decorators, stereotypes and producer methods. To deploy CDI beans, you must place a file called beans.xml in a META-INF folder on the classpath. Once you do this, then every bean in the package becomes a CDI bean. There are a lot of features in CDI, too many to cover here, but as a quick reference for JSF-like features, you can define the scope of the CDI bean using one of the scopes defined in the javax.enterprise.context package (namely, request, conversation, session and application scopes). If you want to use the CDI bean from a JSF page, you can give it a name using the javax.inject.Named annotation. To inject a bean into another bean, you annotate the field with javax.inject.Inject annotation. @Named("someBean") @RequestScoped public class SomeBean { @Inject private SomeService someService; } Automatic injection like that defined above can be controlled through the use of Qualifiers that can help match the specific class that you want injected. If you have multiple payment types, you might add a qualifier for whether it is asynchronous or not. While you can use the @Named annotation as a qualifier, you shouldn’t as it is provided for exposing the beans in EL. CDI handles the injection of beans with mismatched scopes through the use of proxies. Because of this you can inject a request scoped bean into a session scoped bean and the reference will still be valid on each request because for each request, the proxy re-connects to a live instance of the request scoped bean. CDI also has support for interceptors, events, the new conversation scope and many other features which makes it a much better choice over JSF managed beans. EJB EJBs predate CDI beans and are in someways similar to CDI beans and in other ways very different. Primarily, the differences between CDI beans and EJBs is that EJBs are : Transactional Remote or local Able to passivate stateful beans freeing up resources Able to make use of timers Can be asynchronous The two types of EJBs are called stateless and stateful. Stateless EJBs can be thought of as thread safe single-use beans that don’t maintain any state between two web requests. Stateful EJBs do hold state and can be created and sit around for as long as they are needed until they are disposed of. Defining an EJB is simple, you just add either a javax.ejb.Stateless or javax.ejb.Stateful annotation to the class. @Stateless public class BookingService { public String makeReservation(Item Item,Customer customer) { ... ... } } Stateless beans must have a dependent scope while a stateful session bean can have any scope. By default they are transactional, but you can use the transaction attribute annotation. While EJBs and CDI beans are very different in terms of feaures, writing the code to integrate them is very similar since CDI beans can be injected into EJBs and EJBs can be injected into CDI beans. There is no need to make any distinction when injecting one into the other. Again, the different scopes are handled by CDI through the use of proxying. One exception to this is that CDI does not support the injection of remote EJBs but that can be implemented by writing a simple producer method for it. The javax.inject.Named annotation as well as any Qualifiers can be used on an EJB to match it to an injection point. When to use which bean How do you know when to use which bean? Simple. Never use JSF managed beans unless you are working in a servlet container and don’t want to try and get CDI working in Tomcat (although I have a Maven archetype for that so there’s no excuse). In general, you should use CDI beans unless you need the advanced functionality available in the EJBs such as transactional functions. You can write your own interceptor to make CDI beans transactional, but for now, its simpler to use an EJB until CDI gets transactional CDI beans which is just around the corner. If you are stuck in a servlet container and are using CDI, then either hand written transactions or your own transaction interceptor is the only option without EJBs. From http://www.andygibson.net/blog/article/comparing-jsf-beans-cdi-beans-and-ejbs/