I.T. is full of complex things that should (and sometimes could) be simple. Getting the JQPL/SQL String representation for a JPA 2.0 CriteriaQuery is one of them. By now you all know the JPA 2.0 Criteria API : a type safe way to write a JQPL query. This API is clever in the way that you don’t use Strings to build your query, but is quite verbose… and sometimes you get lost in dozens of lines of Java code, just to write a simple query. You get lost in your CriteriaQuery, you don’t know why your query doesn’t work, and you would love to debug it. But how do you debug it ? Well, one way would be by just displaying the JPQL and/or SQL representation. Simple, isn’t it ? Yes, but JPA 2.0 javax.persistence.Query doesn’t have an API to do this. You then need to rely on the implementation… meaning, the code is different if you use EclipseLink, Hibernate or OpenJPA. The CriteriaQuery we want to debug Let’s say you have a simple Book entity and you want to retrieve all the books sorted by their id. Something like SELECT b FROM Book b ORDER BY b.id DESC. How would you write this with the CriteriaQuery ? Well, something like these 5 lines of Java code : CriteriaBuilder cb = em.getCriteriaBuilder(); CriteriaQuery q = cb.createQuery(Book.class); Root b = q.from(Book.class); q.select(b).orderBy(cb.desc(b.get("id"))); TypedQuery findAllBooks = em.createQuery(q); So imagine when you have more complex ones. Sometimes, you just get lost, it gets buggy and you would appreciate to have the JPQL and/or SQL String representation to find out what’s happening. You could then even unit test it. Getting the JPQL/SQL String Representations for a Criteria Query So let’s use an API to get the JPQL/SQL String representations of a CriteriaQuery (to be more precise, the TypedQuery created from a CriteriaQuery). The bad news is that there is no standard JPA 2.0 API to do this. You need to use the implementation API hoping the implementation allows it (thank god that’s (nearly) the case for the 3 main JPA ORM frameworks). The good news is that the Query interface (and therefore TypedQuery) has an unwrap method. This method returns the provider’s query API implementation. Let’s see how you can use it with EclipseLink, Hibernate and OpenJPA. EclipseLink EclipseLink‘s Query representation is the org.eclipse.persistence.jpa.JpaQuery interface and the org.eclipse.persistence.internal.jpa.EJBQueryImpl implementation. This interface gives you the wrapped native query (org.eclipse.persistence.queries.DatabaseQuery) with two very handy methods : getJPQLString() and getSQLString(). Unfortunatelly the getJPQLString() method will not translate a CriteriaQuery into JPQL, it only works for queries originally written in JPQL (dynamic or named query). The getSQLString() method relies on the query being “prepared”, meaning you have to run the query once before getting the SQL String representation. findAllBooks.unwrap(JpaQuery.class).getDatabaseQuery().getJPQLString(); // doesn't work for CriteriaQuery findAllBooks.unwrap(JpaQuery.class).getDatabaseQuery().getSQLString(); Hibernate Hibernate‘s Query representation is org.hibernate.Query. This interface has several implementations and the very useful method that returns the SQL query string : getQueryString(). I couldn’t find a method that returns the JPQL representation, if I’ve missed something, please let me know. findAllBooks.unwrap(org.hibernate.Query.class).getQueryString() OpenJPA OpenJPA‘s Query representation is org.apache.openjpa.persistence.QueryImpl and also has a getQueryString() method that returns the SQL (not the JPQL). It delegates the call to the internal org.apache.openjpa.kernel.Query interface. I couldn’t find a method that returns the JPQL representation, if I’ve missed something, please let me know. findAllBooks.unwrap(org.apache.openjpa.persistence.QueryImpl.class).getQueryString() Unit testing Once you get your SQL String, why not unit test it ? Hey, but I don’t want to test my ORM, why would I do that ? Well, it happens that I’ve discovered a but in the new releases of OpenJPA by unit testing a query… so, there is a use case for that. Anyway, this is how you could do it : assertEquals("SELECT b FROM Book b ORDER BY b.id DESC", findAllBooksCriteriaQuery.unwrap(org.apache.openjpa.persistence.QueryImpl.class).getQueryString()); Conclusion As you can see, it’s not that simple to get a String representation for a TypedQuery. Here is a digest of the three main ORMs : ORM Framework Query implementation How to get the JPQL String How to get the SPQL String EclipseLink JpaQuery getDatabaseQuery().getJPQLString()* getDatabaseQuery().getSQLString()** Hibernate Query N/A getQueryString() OpenJPA QueryImpl getQueryString() N/A (*) Only possible on a dynamic or named query. Not possible on a CriteriaQuery (**) You need to execute the query first, if not, the value is null To illustrate all that I’ve written simple test cases using EclipseLink, Hibernate and OpenJPA that you can download from GitHub. Give it a try and let me know. And what about having an API in JPA 2.1 ? For a developers’ point of view it would be great to have two methods in the javax.persistence.Query (and therefore javax.persistence.TypedQuery) interface that would be able to easily return the JPQL and SQL String representations, e.g : Query.getJPQLString() and Query.getSQLString(). Hey, that would be the perfect time to have it in JPA 2.1 that will be shipped in less than a year. Now, as an implementer, this might be tricky to do, I would love to ear your point of view on this. Anyway, I’m going to post an email to the JPA 2.1 Expert Group… just in case we can have this in the next version of JPA ;o) References http://efreedom.com/Question/1-6412774/Get-SQL-String-JPQLQuery http://old.nabble.com/Cannot-get-the-JPQL—SQL-String-of-a-CriteriaQuery-td33882629.html http://paddyweblog.blogspot.fr/2010/04/some-examples-of-criteria-api-jpa-20.html http://www.altuure.com/2010/09/23/jpa-criteria-api-by-samples-part-i/ http://www.altuure.com/2010/09/23/jpa-criteria-api-by-samples-%E2%80%93-part-ii/ http://www.jumpingbean.co.za/blogs/jpa2-criteria-api http://wiki.eclipse.org/EclipseLink/FAQ/JPA#How_to_get_the_SQL_for_a_Query.3F

I really like Spring, so I tend to use its features to the fullest. However, in some dark corners of its philosophy, I tend to disagree with some of its assumptions. One such assumption is the way database testing should work. In this article, I will explain how to configure your projects to make Spring Test and DBUnit play nice together in a multi-developers environment. Context My basic need is to be able to test some complex queries: before integration tests, I've to validate those queries get me the right results. These are not unit tests per se but let's assilimate them as such. In order to achieve this, I use since a while a framework named DBUnit. Although not maintained since late 2010, I haven't found yet a replacement (be my guest for proposals). I also have some constraints: I want to use TestNG for all my test classes, so that new developers wouldn't think about which test framework to use I want to be able to use Spring Test, so that I can inject my test dependencies directly into the test class I want to be able to see for myself the database state at the end of any of my test, so that if something goes wrong, I can execute my own queries to discover why I want every developer to have its own isolated database instance/schema Considering the last point, our organization let us benefit from a single Oracle schema per developer for those "unit-tests". Basic set up Spring provides the AbstractTestNGSpringContextTests class out-of-the-box. In turn, this means we can apply TestNG annotations as well as @Autowired on children classes. It also means we have access to the underlying applicationContext, but I prefer not to (and don't need to in any case). The structure of such a test would look like this: @ContextConfiguration(location = "classpath:persistence-beans.xml") public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Test public void whenXYZThenTUV() { ... } } Readers familiar with Spring and TestNG shouldn't be surprised here. Bringing in DBunit DbUnit is a JUnit extension targeted at database-driven projects that, among other things, puts your database into a known state between test runs. [...] DbUnit has the ability to export and import your database data to and from XML datasets. Since version 2.0, DbUnit can also work with very large datasets when used in streaming mode. DbUnit can also help you to verify that your database data match an expected set of values. DBunit being a JUnit extension, it's expected to extend the provided parent class org.dbunit.DBTestCase. In my context, I have to redefine some setup and teardown operation to use Spring inheritance hierarchy. Luckily, DBUnit developers thought about that and offer relevant documentation. Among the different strategies available, my tastes tend toward the CLEAN_INSERT and NONE operations respectively on setup and teardown. This way, I can check the database state directly if my test fails. This updates my test class like so: @ContextConfiguration(locations = {"classpath:persistence-beans.xml", "classpath:test-beans.xml"}) public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Autowired private IDatabaseTester databaseTester; @BeforeMethod protected void setUp() throws Exception { // Get the XML and set it on the databaseTester // Optional: get the DTD and set it on the databaseTester databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT); databaseTester.setTearDownOperation(DatabaseOperation.NONE); databaseTester.onSetup(); } @Test public void whenXYZThenTUV() { ... } } Per-user configuration with Spring Of course, we need to have a specific Spring configuration file to inject the databaseTester. As an example, here is one: However, there's more than meets the eye. Notice the databaseTester has to be fed a datasource. Since a requirement is to have a database per developer, there are basically two options: either use a in-memory database or use the same database as in production and provide one such database schema per developer. I tend toward the latter solution (when possible) since it tends to decrease differences between the testing environment and the production environment. Thus, in order for each developer to use its own schema, I use Spring's ability to replace Java system properties at runtime: each developer is characterized by a different user.name. Then, I configure a PlaceholderConfigurer that looks for {user.name}.database.properties file, that will look like so: db.username=myusername1 db.password=mypassword1 db.schema=myschema1 This let me achieve my goal of each developer using its own instance of Oracle. If you want to use this strategy, do not forget to provide a specific database.properties for the Continuous Integration server. Huh oh? Finally, the whole testing chain is configured up to the database tier. Yet, when the previous test is run, everything is fine (or not), but when checking the database, it looks untouched. Strangely enough, if you did load some XML dataset and assert it during the test, it does behaves accordingly: this bears all symptoms of a transaction issue. In fact, when you closely look at Spring's documentation, everything becomes clear. Spring's vision is that the database should be left untouched by running tests, in complete contradiction to DBUnit's. It's achieved by simply rollbacking all changes at the end of the test by default. In order to change this behavior, the only thing to do is annotate the test class with @TransactionConfiguration(defaultRollback=false). Note this doesn't prevent us from specifying specific methods that shouldn't affect the database state on a case-by-case basis with the @Rollback annotation. The test class becomes: @ContextConfiguration(locations = {classpath:persistence-beans.xml", "classpath:test-beans.xml"}) @TransactionConfiguration(defaultRollback=false) public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Autowired private IDatabaseTester databaseTester; @BeforeMethod protected void setUp() throws Exception { // Get the XML and set it on the databaseTester // Optional: get the DTD and set it on the databaseTester databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT); databaseTester.setTearDownOperation(DatabaseOperation.NONE); databaseTester.onSetup(); } @Test public void whenXYZThenTUV() { ... } } Conclusion Though Spring and DBUnit views on database testing are opposed, Spring's configuration versatility let us make it fit our needs (and benefits from DI). Of course, other improvements are possible: pushing up common code in a parent test class, etc. To go further: Spring Test documentation DBUnit site Database data verification Database testing best practices Generating DTD from your database schema

Insanity: doing the same thing over and over again and expecting different results. -- attributed to Albert Einstein Global state is essentially the same thing as insanity in this definition: a way to affect the execution of code hidden from sight, so that two apparently identical lines actually produce a different result depending on some external factor. For example: new SomeClass()->printId(); new SomeClass()->printId(); // output: 1, 2 has some global state (a static counter) affecting a field inside SomeClass instances. Therefore, it may not be easy to replicate scenarios (like in tests) multiple times. Examples Global and environmental variables, along with constants are simple examples of global state. The same goes for configuration directives and files which code silently depends upon, as long as they are global for each instance of the affected objects. Speaking about objects singletons and static classes containing fields are another example of global state. More subtle cases are hidden localizations like translations of output and of symbols (LC_ALL?) Parameterization is made difficult by global state, either because the seam for collaborators is hidden (config files and enviromental/global variables) or not accessible (singletons). Testability When there is some global state in an application, the affected unit tests won'tbe isolated from each other, and may change their results when run alone or in a different order with respect to being executed inside the whole test suite. Global state is one of the most common problems while working with legacy code which was written with little concern for testability (and separation of concerns). A typical annoying example is a test that passes when alone, but fails in the full suite due to some state left lingering from previous tests. Usually, it is then debugged by executing the exact same test twice or multiple times in a single process and verifying that it passes consistently. Actually global state cannot be always removed, even in a test environment: what this move would achieve would be a fully parameterized system, too general to be useful; imagine configuring every class name in your application, even in Factories. It may be simpler to test with some global state in, like in the case of a default locale defined in place of stubbing the Translator object; or in the case of a Fake database connection instead of a Stub or a Mock. Taking this approach to the extreme, we notice that global state is often hidden from our view because it's taken for granted. Base classes offered by the language (e.g. String) are not mocked or substituted by test doubles, even when they have quite some logic in them; all the classes and functions contained in our applications are global state as their implementation cannot be substituted, yet we don't consider them a trouble as singletons. Constant There is a reason why not all global state is necessarily bad: constant global state to allow context-free reasoning about code, and simplify testing and reuse of code considerably. In fact, the very definition of state (for example from hardware logic networks) is that of a component that can change its behavior in time, keeping information about previous inputs. In short, any computation that is frequently accessed but does not have the capability to change its result or to produce side-effects is not state (it is global). ROM is instead considered a purely combinatorial network, not being real "memory" but merely a function translating addresses to words. A singleton changing its responses after some calls is global state that makes testing difficult; a static class containing only pure functions may make tests long winded and infringe the object-oriented paradigm, but it's not as dangerous as the former. However, that's why I see monkey patching in dynamic languages as problematic. Monkey patching commonly consists of open classes where you can add methods at any time after their initial definition. class Array def sum inject {|sum, x| sum + x } end end When you see a call to this method, you have to ask some questions: where it was added in the code base? Which sourcefile should I look at? When it was added to the code base? Am I sure that definitions can only be included at startup and I am not calling the method before it is defined? Are there multiple redefinitions of the method? Maybe from other libraries or code to integrate? The same issues happened for prototype.js, which modifies the prototype object of base JavaScript objects like Array, effectively redefining and adding methods. The result is little interoperability with other libraries. But even monkey patching should be fine as long as the modifications are really constant and definitive. If you use a single framework (like Ext Js), and you always use it in all pages of the application and in tests, then it can monkeypatch the base classes of the language safely, without making you debug a method that works in one environment but not in the other. Conclusion Global state is not only a global parameter for the internals of your application, but also the product of stateful interaction that changes the output and side-effects of code in different invocations. Making global things constant is the first step towards simplifying reasoning about a design and raising testability. If you are able to run a unit test twice in the same method, you are officially free from global state in that scenario. Pay attention when embracing open classes, editable prototypes, embedded calls to registries and files, and so on: they can add several dimensions to the variables that can affect the result of a piece of code. They will hide a dependency but not making it go away.

Learn how Apache Camel implements the EIPs and offers a standardized, internal domain-specific language (DSL) to integrate applications.

An Alpha Animation is animation that controls the alpha level of an object, i.e. fading it in and out. In Android, you can apply that fading effect to almost anything, from simple text, to images, buttons, check boxes, etc... Android has a few classes that can help you add that special effect to your programs, like AlphaAnimation and AnimationUtils. Here's an example on how to apply fading on any Android component subclass of View. First, the XML resource. In the resources folder, we will create a tiny XML configuration file with the characteristics of the fading effect we want in an "anim" subfolder. So, under res/anim, here's our alpha.xml: We are choosing to have a very basic full fade in effect (alpha from 0 to 1) that lasts one second. The above can also be done directly in Java code: Animation animation = new AlphaAnimation(0.0f, 1.0f); animation.setDuration(1000); Configuring the animation in resources or in code is ultimately a matter of preference. We will use the XML in this example. This is our class that does the above fading to any View (TextView, Button, etc..): package com.ts.fx.utils; import android.app.Activity; import android.view.View; import android.view.animation.Animation; import android.view.animation.AnimationUtils; public class Fader { /** * handles all subclasses of View : TextView, Button, ImageView etc.. * given the component's id in their layout file * */ public static void runAlphaAnimation(Activity act, int viewId) { // load animation XML resource under res/anim Animation animation = AnimationUtils.loadAnimation(act, R.anim.alpha); if(animation == null){ return; // here, we don't care } // reset initialization state animation.reset(); // find View by its id attribute in the XML View v = act.findViewById(viewId); // cancel any pending animation and start this one if (v != null){ v.clearAnimation(); v.startAnimation(animation); } } } The runAlphaAnimation() method takes an Activity reference and a View id attribute (as set up in the View's layout XML). We're basically done. all we have to do now is call it from any one of our Activitites: // inside an Activity with text, checkbox and button Fader.runAlphaAnimation(this, a_text.getId()); Fader.runAlphaAnimation(this, a_checkbox.getId()); Fader.runAlphaAnimation(this, a_button.getId()); //etc... That's all there is to it. The same basic technique seen here applies to all other special effects like translating, scaling or rotating components. The Animation classes have of course lots of other cool stuff, like controlling the z-ordering of the animated components, acceleration and repeat effects. Here's a thirty-second video (by yours truly) demonstrating various Android special effects (fading, translation and rotation) used together in a concrete application: From Tony's Blog.

A few weeks ago, I co-authored an article (with my colleague Rama Ramani) about how the Screen Actors Guild Awards website migrated its Drupal deployment from LAMP to Windows Azure: Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. Since then, Rama and another colleague, Jason Roth, have been working on writing up how the SAG Awards website was managed and monitored in Windows Azure. The article below is the fruit of their work…a very interesting/educational read. Overview Drupal is an open source content management system that runs on PHP. Windows Azure offers a flexible platform for hosting, managing, and scaling Drupal deployments. This paper focuses on an approach to host Drupal sites on Windows Azure, based on learning from a BPD Customer Programs Design Win engagement with the Screen Actors Guild Awards Drupal website. This paper covers guidelines and best practices for managing an existing Drupal web site in Windows Azure. For more information on how to migrate Drupal applications to Windows Azure, see Azure Real World: Migrating a Drupal Site from LAMP to Windows Azure. The target audience for this paper is Drupal administrators who have some exposure to Windows Azure. More detailed pointers to Windows Azure content is provided throughout the paper as links. Drupal Application Architecture on Windows Azure Before reviewing the management and monitoring guidelines, it is important to understand the architecture of a typical Drupal deployment on Windows Azure. First, the following diagram displays the basic architecture of Drupal running on Windows and IIS7. In the Windows Server scenario, you could have one or more machines hosting the web site in a farm. Those machines would either persist the site content to the file system or point to other network shares. For Windows Azure, the basic architecture is the same, but there are some differences. In Windows Azure the site is hosted on a web role. A web role instance is hosted on a Windows Server 2008 virtual machine within the Windows Azure datacenter. Like the web farm, you can have multiple instances running the site. But there is no persistence guarantee for the data on the file system. Because of this, much of the shared site content should be stored in Windows Azure Blob storage. This allows them to be highly available and durable. Usually, a large portion of the site caters to static content which lends well to caching. And caching can be applied in a set of places – browser level caching, CDN to cache content in the edge closer to the browser clients, caching in Azure to reduce the load on backend, etc. Finally, the database can be located in SQL Azure. The following diagram shows these differences. For monitoring and management, we will look at Drupal on Windows Azure from three perspectives: Availability: Ensure the web site does not go down and that all tiers are setup correctly. Apply best practices to ensure that the site is deployed across data centers and perform backup operations regularly. Scalability: Correctly handle changes in user load. Understand the performance characteristics of the site. Manageability: Correctly handle updates. Make code and site changes with no downtime when possible. Although some management tasks span one or more of these categories, it is still helpful to discuss Drupal management on Windows Azure within these focus areas. Availability One main goal is that the Drupal site remains running and accessible to all end-users. This involves monitoring both the site and the SQL Azure database that the site depends on. In this section, we will briefly look at monitoring and backup tasks. Other crossover areas that affect availability will be discussed in the next section on scalability. Monitoring With any application, monitoring plays an important role with managing availability. Monitoring data can reveal whether users are successfully using the site or whether computing resources are meeting the demand. Other data reveals error counts and possibly points to issues in a specific tier of the deployment. There are several monitoring tools that can be used. The Windows Azure Management Portal. Windows Azure diagnostic data. Custom monitoring scripts. System Center Operations Manager. Third party tools such as Azure Diagnostics Manager and Azure Storage Explorer. The Windows Azure Management Portal can be used to ensure that your deployments are successful and running. You can also use the portal to manage features such as Remote Desktop so that you can directly connect to machines that are running the Drupal site. Windows Azure diagnostics allows you to collect performance counters and logs off of the web role instances that are running the Drupal site. Although there are many options for configuring diagnostics in Azure, the best solution with Drupal is to use a diagnostics configuration file. The following configuration file demonstrates some basic performance counters that can monitor resources such as memory, processor utilization, and network bandwidth. For more information about setting up diagnostic configuration files, see How to Use the Windows Azure Diagnostics Configuration File. This information is stored locally on each role instance and then transferred to Windows Azure storage per a defined schedule or on-demand. See Getting Started with Storing and Viewing Diagnostic Data in Windows Azure Storage. Various monitoring tools, such as Azure Diagnostics Manager, help you to more easily analyze diagnostic data. Monitoring the performance of the machines hosting the Drupal site is only part of the story. In order to plan properly for both availability and scalability, you should also monitor site traffic, including user load patterns and trends. Standard and custom diagnostic data could contribute to this, but there are also third-party tools that monitor web traffic. For example, if you know that spikes occur in your application during certain days of the week, you could make changes to the application to handle the additional load and increase the availability of the Drupal solution. Backup Tasks To remain highly available, it is important to backup your data as a defense-in-depth strategy for disaster recovery. This is true even though SQL Azure and Windows Azure Storage both implement redundancy to prevent data loss. One obvious reason is that these services cannot prevent administrator error if data is accidentally deleted or incorrectly changed. SQL Azure does not currently have a formal backup technology, although there are many third-party tools and solutions that provide this capability. Usually the database size for a Drupal site is relatively small. In the case of SAG Awards, it was only ~100-150 MB. So performing an entire backup using any strategy was relatively fast. If your database is much larger, you might have to test various backup strategies to find the one that works best. Apart from third-party SQL Azure backup solutions, there are several strategies for obtaining a backup of your data: · Use the Drush tool and the portabledb-export command. · Periodically copy the database using the CREATE DATABASE Transact-SQL command. · Use Data-tier applications (DAC) to assist with backup and restore of the database. SQL Azure backup and data security techniques are described in more detail in the topic, Business Continuity in SQL Azure. Note that bandwidth costs accrue with any backup operation that transfers information outside of the Windows Azure datacenter. To reduce costs, you can copy the database to a database within the same datacenter. Or you can export the data-tier applications to blob storage in the same datacenter. Another potential backup task involves the files in Blob storage. If you keep a master copy of all media files uploaded to Blob storage, then you already have an on-premises backup of those files. However, if multiple administrators are loading files into Blob storage for use on the Drupal site, it is a good idea to enumerate the storage account and to download any new files to a central location. The following PHP script demonstrates how this can be done by backing up all files in Blob storage after a specified modification date. setProxy(true, 'YOUR_PROXY_IF_NEEDED', 80); $blobs = (array)$blobObj->listBlobs(AZURE_STORAGE_CONTAINER, '', '', 35000); backupBlobs($blobs, $blobObj); function backupBlobs($blobs, $blobObj) { foreach ($blobs as $blob) { if (strtotime($blob->lastmodified) >= DEFAULT_BACKUP_FROM_DATE && strtotime($blob->lastmodified) <= DEFAULT_BACKUP_TO_DATE) { $path = pathinfo($blob->name); if ($path['basename'] != '$$$.$$$') { $dir = $path['dirname']; $oldDir = getcwd(); if (handleDirectory($dir)) { chdir($dir); $blobObj->getBlob( AZURE_STORAGE_CONTAINER, $blob->name, $path['basename'] ); chdir($oldDir); } } } } } function handleDirectory($dir) { if (!checkDirExists($dir)) { return mkdir($dir, 0755, true); } return true; } function checkDirExists($dir) { if(file_exists($dir) && is_dir($dir)) { return true; } return false; } ?> This script has a dependency on the Windows Azure SDK for PHP. Also note there are several parameters that you must modify such as the storage account, secret, and backup location. As with SQL Azure, bandwidth and transaction charges apply to a backup script like this. Scalability Drupal sites on Windows Azure can scale as load increased through typical strategies of scale-up, scale-out, and caching. The following sections describe the specifics of how these strategies are implemented in Windows Azure. Typically you make scalability decisions based on monitoring and capacity planning. Monitoring can be done in staging during testing or in production with real-time load. Capacity planning factors in projections for changes in user demand. Scale Up When you configure your web role prior to deployment, you have the option of specifying the Virtual Machine (VM) size, such as Small or ExtraLarge. Each size tier adds additional memory, processing power, and network bandwidth to each instance of your web role. For cost efficiency and smaller units of scale, you can test your application under expected load to find the smallest virtual machine size that meets your requirements. The workload usually in most popular Drupal websites can be separated out into a limited set of Drupal admins making content changes and a large user base who perform mostly read-only workload. End users can be allowed to make ‘writes’, such as uploading blogs or posting in forums, but those changes are not ‘content changes’. Drupal admins are setup to operate without caching so that the writes are made directly to SQL Azure or the corresponding backend database. This workload performs well with Large or ExtraLarge VM sizes. Also, note that the VM size is closely tied to all hardware resources, so if there are many content-rich pages that are streaming content, then the VM size requirements are higher. To make changes to the Virtual Machine size setting, you must change the vmsize attribute of the WebRole element in the service definition file, ServiceDefinition.csdef. A virtual machine size change requires existing applications to be redeployed. Scale Out In addition to the size of each web role instance, you can increase or decrease the number of instances that are running the Drupal site. This spreads the web requests across more servers, enabling the site to handle more users. To change the number of running instances of your web role, see How to Scale Applications by Increasing or Decreasing the Number of Role Instances. Note that some configuration changes can cause your existing web role instances to recycle. You can choose to handle this situation by applying the configuration change and continue running. This is done by handling the RoleEnvironment.Changing event. For more information see, How to Use the RoleEnvironment.Changing Event. A common question for any Windows Azure solution is whether there is some type of built-in automatic scaling. Windows Azure does not provide a service that provides auto-scaling. However, it is possible to create a custom solution that scales Azure services using the Service Management API. For an example of this approach, see An Auto-Scaling Module for PHP Applications in Windows Azure. Caching Caching is an important strategy for scaling Drupal applications on Windows Azure. One reason for this is that SQL Azure implements throttling mechanisms to regulate the load on any one database in the cloud. Code that uses SQL Azure should have robust error handling and retry logic to account for this. For more information, see Error Messages (SQL Azure Database). Because of the potential for load-related throttling as well as for general performance improvement, it is strongly recommended to use caching. Although Windows Azure provides a Caching service, this service does not currently have interoperability with PHP. Because of this, the best solution for caching in Drupal is to use a module that uses an open-source caching technology, such as Memcached. Outside of a specific Drupal module, you can also configure Memcached to work in PHP for Windows Azure. For more information, see Running Memcached on Windows Azure for PHP. Here is also an example of how to get Memcached working in Windows Azure using a plugin: Windows Azure Memcached plugin. In a future paper, we hope to cover this architecture in more detail. For now, here are several design and management considerations related to caching. Area Consideration Design and Implementation For a technology like Memcached, will the cache be collocated (spread across all web role instances)? Or will you attempt to setup a dedicated cache ring with worker roles that only run Memcached? Configuration What memory is required and how will items in the cache be invalidated? Performance and Monitoring What mechanisms will be used to detect the performance and overall health of the cache? For ease of use and cost savings, collocation of the cache across the web role instances of the Drupal site works best. However, this assumes that there is available reserve memory on each instance to apply toward caching. It is possible to increase the virtual machine size setting to increase the amount of available memory on each machine. It is also possible to add additional web role instances to add to the overall memory of the cache while at the same time improving the ability of the web site to respond to load. It is possible to create a dedicated cache cluster in the cloud, but the steps for this are beyond the scope of this paper[RR1] . For Windows Azure Blob storage, there is also a caching feature built into the service called the Content Delivery Network (CDN). CDN provides high-bandwidth access to files in Blob storage by caching copies of the files in edge nodes around the world. Even within a single geographic region, you could see performance improvements as there are many more edge nodes than Windows Azure datacenters. For more information, see Delivering High-Bandwidth Content with the Windows Azure CDN. Manageability It is important to note that each hosted service has a Staging environment and a Production environment. This can be used to manage deployments, because you can load and test and application in staging before performing a VIP swap with production. From a manageability standpoint, Drupal has an advantage on Windows Azure in the way that site content is stored. Because the data necessary to serve pages is stored in the database and blob storage, there is no need to redeploy the application to change the content of the site. Another best practice is to use a separate storage account for diagnostic data than the one that is used for the application itself. This can improve performance and also helps to separate the cost of diagnostic monitoring from the cost of the running application. As mentioned previously, there are several tools that can assist with managing Windows Azure applications. The following table summarizes a few of these choices. Tool Description Windows Azure Management Portal The web interface of the Windows Azure management portal shows deployments, instance counts and properties, and supports many different common management and monitoring tasks. Azure Diagnostics Managerq[RR2] [JR3] A Red Gate Software product that provides advanced monitoring and management of diagnostic data. This tool can be very useful for easily analyzing the performance of the Drupal site to determine appropriate scaling decisions. Azure Storage Explorer A tool created by Neudesic for viewing Windows Azure storage account. This can be useful for viewing both diagnostic data and the files in Blob storage.

See original post at https://muhammadkhojaye.blogspot.com/2012/04/how-to-run-amazon-elastic-mapreduce-job.html Introduction Amazon EMR is a web service which can be used to easily and efficiently process enormous amounts of data. It uses a hosted Hadoop framework running on the web-scale infrastructure of Amazon EC2 and Amazon S3. Amazon EMR removes most of the cumbersome details of Hadoop while taking care of provisioning of Hadoop, running the job flow, terminating the job flow, moving the data between Amazon EC2 and Amazon S3, and optimizing Hadoop. In this tutorial, we will use a developed WordCount Java example using Hadoop and thereafter, we execute our program on Amazon Elastic MapReduce. Prerequisites You must have valid AWS account credentials. You should also have a general familiarity with using the Eclipse IDE before you begin. The reader can also use any other IDE of their choice. Step 1 – Develop MapReduce WordCount Java Program In this section, we are first going to develop a WordCount application. A WordCount program will determine how many times different words appear in a set of files. In Eclipse (or whatever the IDE you are using), Create simple Java Project with the name "WordCount". Create a java class name Map and override the map method as follow, public class Map extends Mapper { private final static IntWritable one = new IntWritable(1); private Text word = new Text(); @Override public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { String line = value.toString(); StringTokenizer tokenizer = new StringTokenizer(line); while (tokenizer.hasMoreTokens()) { word.set(tokenizer.nextToken()); context.write(word, one); } } } Create a java class named Reduce and override the reduce method as shown below, public class Reduce extends Reducer { @Override protected void reduce(Text key, java.lang.Iterable values, org.apache.hadoop.mapreduce.Reducer.Context context) throws IOException, InterruptedException { int sum = 0; for (IntWritable value : values) { sum += value.get(); } context.write(key, new IntWritable(sum)); } } Create a java class named WordCount and defined the main method as below, public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); Job job = new Job(conf, "wordcount"); job.setJarByClass(WordCount.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); job.setMapperClass(Map.class); job.setReducerClass(Reduce.class); job.setInputFormatClass(TextInputFormat.class); job.setOutputFormatClass(TextOutputFormat.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); } Export the WordCount program in a jar using eclipse and save it to some location on disk. Make sure that you have provided the Main Class (WordCount.jar) during extraction ofu8u the jar file as shown below. Our jar is ready!!! Step 2 – Upload the WordCount JAR and Input Files to Amazon S3 Now we are going to upload the WordCount jar to Amazon S3. First, go to the following URL: https://console.aws.amazon.com/s3/home Next, click “Create Bucket”, give your bucket a name, and click the “Create” button. Select your new S3 bucket in the left-hand pane. Upload the WordCount JAR and sample input file for counting the words. Step 3 – Running an Elastic MapReduce job Now that the JAR is uploaded into S3, all we need to do is to create a new Job flow. let's execute the steps below. (I encourage readers to check out the following link for details regarding each step, How to Create a Job Flow Using a Custom JAR ) Sign in to the AWS Management Console and open the Amazon Elastic MapReduce console at https://console.aws.amazon.com/elasticmapreduce/ Click Create New Job Flow. In the DEFINE JOB FLOW page, enter the following details, a) Job Flow Name = WordCountJob b) Select Run your own applications) Select Custom JAR in the drop-down list) Click Continue In the SPECIFY PARAMETERS page, enter values in the boxes using the following table as a guide, and then click Continue.JAR Location = bucketName/jarFileLocationJAR Arguments =s3n://bucketName/inputFileLocations3n://bucketName/outputpath Please note that the output path must be unique each time we execute the job. The Hadoop always create a folder with the same name specified here. After executing the job, just wait and monitor your job that runs through the Hadoop flow. You can also look for errors by using the Debug button. The job should be complete within 10 to 15 minutes (can also depend on the size of the input). After completing the job, You can view results in the S3 Browser panel. You can also download the files from S3 and can analyze the outcome of the job. Amazon Elastic MapReduce Resources Amazon Elastic MapReduce Documentation,http://aws.amazon.com/documentation/elasticmapreduce/ Amazon Elastic MapReduce Getting Started Guide,http://docs.amazonwebservices.com/ElasticMapReduce/latest/GettingStartedGuide/ Amazon Elastic MapReduce Developer Guide,http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/ Apache Hadoop,http://hadoop.apache.org/ See more at https://muhammadkhojaye.blogspot.com/2012/04/how-to-run-amazon-elastic-mapreduce-job.html

Let's take a closer look at how to play sounds on an Android device with SoundPool and MediaPlayer.

Since the very early days of Java (and C-like languages overall), the canonical way to start your program has been something like this: public class A { public static void main(String[] args) { new A().run(args); } public void run(String[] args) { // Your application starts here } } If you are still doing this, I’m here to tell you it’s time to stop. Letting go of ‘new’ First, install Guice in your project: com.google.inject guice 3.0 and then, modify your main method as follows: public class A { public static void main(String[] args) { Injector.getInstance(A.class).run(args); } } So, what does this buy you exactly? You will find a lot of articles explaining the various benefits of Guice, such as being able to substitute different environments on the fly, but I’m going to use a different angle in this article. Let’s start by assuming the existence of a Config class that contains various configuration parameters. I’ll just hardcode them for now and use fields to make the class smaller: public class Config { String host = "com.example.com"; int port = 1234; } This class is a singleton, it is instantiated somewhere in your main class and not used anywhere else at the moment. One day, you realize you need this instance in another class which happens to be deep in your runtime hierarchy, which we will call Deep. For example, if you put a break point in the method where you need this config object, your debugger would show you stack frames similar to this: com.example.A.main() com.example.B.f(int, String) com.example.C.g(String) com.example.Deep.h(Foo, int) The easy and wrong way to solve this problem is to make the Config instance static on some class (probably A) and access it directly from Deep. I’m hoping I don’t need to explain why this is a bad idea: not only do you want to avoid using statics, but you also want to make sure that each object is exposed only to objects that need them, and making the Config object static would make your instance visible to your entire code base. Not a good thing. The second thought is to pass the object down the stack, so you modify all the signatures as follows: com.example.A.main() com.example.B.f(int, String, Config) com.example.C.g(String, Config) com.example.Deep.h(Foo, int, Config) This is a bit better since you have severely restricted the exposure of the Config object, but note that you are still making it available to more methods than really need to: B#f and C#g have really nothing to do with this object, and a little sting of discomfort hits you when you start writing the Javadoc: public class C { ... /** * @param config This method doesn't really use this parameter, * it just passes it down so Deep#h can use it. */ public void g(String s, Config config) { Unnecessary exposure is actually not the worst part of this approach, the problem is that it changes all these signatures along the way, which is certainly undesirable in a private API and absolutely devastating in a public API. And of course, it’s absolutely not scalable: if you keep adding a parameter to your method whenever you need access to a certain object, you will soon be dealing with methods that take ten parameters, most of which they just pass down the chain. Here is how we solve this problem with dependency injection (performed by Guice in this example, but this is applicable to any library that implements JSR 330, obviously): public class Deep { @Inject private Config config; and we’re done. That’s it. You don’t need to modify the Config class in any way, nor do you need to make any change in any of the classes that separate Deep from your main class. With this, you have also minimized the exposure of the Config object to just the class that needs it. Injecting right There are various ways you can inject object into your class but I’ll just mention the two that, I think, are the most important. I just showed “field injection” in the previous paragraph, but be aware that you can also prefer to use “constructor injection”: public class Deep { private final Config config; @Inject public Deep(Config config) { this.config = config; } This time, you are adding a parameter to the constructor of your Deep class (which shouldn’t worry you too much since you will never invoke it directly, Guice will) and you assign the parameter to the field in the constructor. The benefit is that you can declare your field final. The downside, obviously, is that this approach is much more verbose. Personally, I see little point in final fields since I have hardly ever encountered a bug that was due to accidentally reassigning a field, so I tend to use field injection whenever I can. Taking it to the next level Obviously, the kind of configuration object I used as an example if not very realistic. Typically, a configuration will not hardcode values like I did and will, instead, read them from some external source. Similarly, you will want to inject objects that can’t necessarily be instantiated so early in the lifecycle of your application, such as servlet contexts, database connections, or implementations of your own interfaces. This topic itself would probably cover several chapters of a book dedicated to dependency injection, so I’ll just summarize it: not all objects can be injected this way, and one benefit of using a dependency injection framework in your code is that it will force you to think about what life cycle category your objects belong to. Having said that, if you want to find out how Guice can inject objects that get created at a later time in your application life cycle, look up the Javadoc for the Provider class. Wrapping up I hope this quick introduction to dependency injection piqued your interest and that you will consider using it in your project since it has so much more to offer than what I described in this post. If you want to learn more, I suggest starting with the excellent Guice documentation.

today i have prepared another really great tutorial for you. recently i started development of my own radio software (as a module for dolphin cms) and got some interesting results i would like to share with you. it will be nice looking (css3) radio script that consists of three main elements: the header (with animated search bar and integrated radio player), the left side (with a list of categories and subcategories) and the right side (which will contain a list of recent or filtered stations). here's the final result of our player: here's our live demo and downloadable package: ok, download our source files and lets start coding! step 1. html markup this is the markup of one of the template files. this is a template of our main (index) page: templates/main_page.html stream radio script back to original tutorial on script tutorials alternative classic alternativeindustrialnew wavepunk classical modernoperapianoromanticsymphony electronic breakbeatdanceelectrohousetechnotrance metal classic metalheavy metalmetalcorepower metal pop dance popoldiestop 40world pop __stations__ powered by script tutorials first, pay attention to how the script loads the jquery library from google. this can be pretty useful if you don’t like to keep this file directly on your host. our header element contains a nice search bar with an embedded jasl player ( i used a great ffmp3 live stream player ), which allows us to play audio streams without any problems. next, on the left-hand side (beneath the header) we have a ul-li based list of categories and subcategories. the right-hand side will contain a list of the most recent stations and, when we search or select a category, the right-hand side will be filtered by ajaxy. for now – it contains __stations__ key (template key) and we will replace the actual value with php. on to our next template file, the radio player: templates/radio.html of course, it contains its own template keys (__title__ and __stream__) which we will use after. step 2. css here are our stylesheets files: css/main.css the first one contains the styles of our test page (this file is always available in our package) css/radio.css /* header area */ .header { height:62px; } .header input { background:#aaa url(../images/search.png) no-repeat 5px center; border:1px solid #888; border-radius:10px; float:right; margin:14px 10px 0 0; outline:none; padding-left:20px; width:200px; -webkit-transition: 0.5s; -moz-transition: 0.5s; -o-transition: 0.5s; transition: 0.5s; } .header input:focus { background-color:#eee; width:300px; } .header > span { display:block; float:left; line-height:40px; padding:7px; -webkit-transition: 0.5s; -moz-transition: 0.5s; -o-transition: 0.5s; transition: 0.5s; } /* stations list */ .stlist { float:right; margin-right:1%; width:71%; } .stlist ul { list-style:none outside none; margin:0; padding:0; } .stlist ul li { border-bottom:1px dotted #444; overflow:hidden; padding:10px; } .stlist ul li > a > img { border:1px solid #ccc; float:left; height:85px; margin-right:15px; padding:1px; width:85px; } .stlist ul li > div { float:right; margin-left:15px; margin-top:-5px; } .stlist ul li > p.label,.stlist ul li > p.track { font-size:11px; font-weight:700; } .stlist ul li > p.label { color:#888; } .stlist ul li > p.channel { font-size:14px; font-weight:700; margin-bottom:17px; } /* genres list */ .genres_par { border-right:1px solid #ccc; float:left; width:26%; } ul.genres,ul.genres ul { list-style-type:none; margin:0; padding:0; } ul.genres ul { display:none; overflow:hidden; padding:0 15px; } ul.genres ul li { margin:3px; } ul.genres a { color:#333; display:block; font-size:18px; padding:4px 0; text-align:center; text-decoration:none; } ul.genres ul a { font-size:12px; text-align:left; } ul.genres li { border-bottom:1px solid #ccc; margin:0; } ul.genres li ul li a { background:none repeat scroll 0 0 #5bb951; border-radius:2px; color:#fff; font-size:12px; padding:6px; } ul.genres li ul li a:hover { background-color:#53854e; } step 3. js js/script.js $(document).ready(function(){ $('#search').blur(function() { if ('' == $('#search').val()) $('#search').val('search'); }); $('#search').focus(function() { if ('search' == $('#search').val()) $('#search').val(''); }); $('ul.genres li a').click( // category slider function() { var checkelement = $(this).next(); if((checkelement.is('ul')) && (!checkelement.is(':visible'))) { $('.genres li ul').slideup(150); $(this).next().slidetoggle(150); } } ); $('ul.genres ul li a').click( // get stations by category function() { $.ajax({ type: 'get', url: 'index.php', data: 'action=get_genre_stations&id=' + $(this).parent().attr('id') + '&name=' + $(this).parent().attr('val'), success: function(data){ $('.stlist').fadeout(400, function () { $('.stlist').html(data); $('.stlist').fadein(400); }); } }); } ); }); function play(id) { // play function $('#rplayer').load('index.php?action=play&id=' + id, function() {}); return false; } function get_stations_by_keyword() { // get stations by keyword var keyword = $('#search').val().replace(/ /g,"+"); $.ajax({ type: 'get', url: 'index.php', data: 'action=get_keyword_stations&key=' + keyword, success: function(data){ $('.stlist').fadeout(400, function () { $('.stlist').html(data); $('.stlist').fadein(400); }); } }); } as you see – there's nothing difficult there. just several event handlers, and two new functions (to play radio station and to search for stations by keyword). step 4. php index.php =') == 1) error_reporting(e_all & ~e_notice & ~e_deprecated); else error_reporting(e_all & ~e_notice); $astations = array( 0 => array( 'category' => 31, 'name' => 'eurodance', 'desc' => 'the newest and best of eurodance hits', 'url' => 'http://www.di.fm/eurodance', 'br' => 96, 'stream' => 'http://scfire-mtc-aa06.stream.aol.com:80/stream/1024' ), 1 => array ( 'category' => 34, 'name' => 'house', 'desc' => 'silky sexy deep house music direct from new york city!', 'url' => 'http://www.di.fm/house', 'br' => 96, 'stream' => 'http://scfire-ntc-aa04.stream.aol.com:80/stream/1007' ), 2 => array ( 'category' => 13, 'name' => 'trance', 'desc' => 'the hottest, freshest trance music from around the globe!', 'url' => 'http://www.di.fm/trance', 'br' => 96, 'stream' => 'http://scfire-ntc-aa04.stream.aol.com:80/stream/1003' ), 3 => array ( 'category' => 51, 'name' => 'electro house', 'desc' => 'an eclectic mix of electro and dirty house', 'url' => 'http://www.di.fm/electro', 'br' => 96, 'stream' => 'http://scfire-ntc-aa04.stream.aol.com:80/stream/1025' ) ); function searchbycat($icat, $astations) { $ares = array(); foreach ($astations as $i => $ainfo) { if ($ainfo['category'] == $icat) { $ares[$i] = $ainfo; } } return $ares; } function searchbykeyword($skey, $astations) { $ares = array(); foreach ($astations as $i => $ainfo) { if (false !== strpos($ainfo['name'], $skey) || false !== strpos($ainfo['desc'], $skey)) { $ares[$i] = $ainfo; } } return $ares; } function parsestationlist($adata) { $sstations = ''; if (is_array($adata) && count($adata) > 0) { foreach ($adata as $i => $a) { $sstationid = $i; $sstationbr = (int)$a['br']; $sstationname = $a['name']; $sstationdesc = $a['desc']; $sstationurl = $a['url']; $sthumb = 'media/'.($sstationid+1).'.png'; $sstations .= << bitrate: {$sstationbr} {$sstationname} {$sstationdesc} {$sstationurl} eof; } } $sstations = ($sstations == '') ? 'nothing found' : $sstations; return '' . $sstations . ''; } switch ($_get['action']) { case 'play': $i = (int)$_get['id']; $ainfo = $astations[$i]; $avars = array ( '__stream__' => $ainfo['stream'], '__title__' => $ainfo['name'] ); echo strtr(file_get_contents('templates/radio.html'), $avars); exit; break; case 'get_genre_stations': $i = (int)$_get['id']; $asearch = searchbycat($i, $astations); $sstations = parsestationlist($asearch); header('content-type: text/html; charset=utf-8'); echo $sstations; exit; break; case 'get_keyword_stations': $skey = $_get['key']; $asearch = searchbykeyword($skey, $astations); $sstations = parsestationlist($asearch); header('content-type: text/html; charset=utf-8'); echo $sstations; exit; break; } $slaststations = parsestationlist($astations); echo strtr(file_get_contents('templates/main_page.html'), array('__stations__' => $slaststations)); at the beginning, i have prepared a list of our radio stations (4 stations total). then, two search functions: ‘searchbycat’ and ‘searchbykeyword’. next, the special function ‘parsestationlist’ which will transform array with filtered stations into its html representation. finally, a little switch case to manage with our inner ajax commands. conclusion you are always welcome to enhance our script and share your ideas. i will be glad to see your thanks and comments. good luck!

There’s still a lot of confusion over the difference types of managed beans provided in Java EE 6 with EJBs, CDI beans and JSF managed beans all being available. This article aims to clear up some of the differences between the them and define when to use them. A number of people assume that there is some meaning to all these different types of beans that they just don’t understand. However, the problem is down to the different APIs overlapping which is unfortunate. JSF Managed Beans, CDI Beans and EJBs JSF was initially developed with its own managed bean and dependency injection mechanism which was enhanced for JSF 2.0 to include annotation based beans. When CDI was released with Java EE 6, it was regarded as the managed bean framework for that platform and of course, EJBs outdated them all having been around for well over a decade. The problem of course is knowing which one to use and when, but they all involve the same process. Typically a class has to be identified as a managed bean, and where necessary, will need a scope,qualifiers and a name if it is to be used in JSF. What follows is a brief description of the different types of managed beans and how and when to use them. Let’s start with the simplest, JSF Managed beans. JSF Managed Beans In short, don’t use them if you are developing for Java EE 6 and using CDI. They provide a simple mechanism for dependency injection and defining backing beans for web pages, but they are far less powerful than CDI beans. They can be defined using the @javax.faces.bean.ManagedBean annotation which takes an optional name parameter. This name can be used to reference the bean from JSF pages. Scope can be applied to the bean using one of the different scopes defined in the javax.faces.bean package which include the request, session, applicaion, view and custom scopes. @ManagedBean(name="someBean") @RequestScoped public class SomeBean { .... .... } JSF beans cannot be mixed with other kinds of beans without some kind of manual coding. CDI Beans CDI is the bean management and dependency injection framework that was released as part of Java EE 6 and it includes a complete, comprehensive managed bean facility. CDI beans are far more advanced and flexible than simple JSF managed beans. They can make use of interceptors, conversation scope, Events, type safe injection, decorators, stereotypes and producer methods. To deploy CDI beans, you must place a file called beans.xml in a META-INF folder on the classpath. Once you do this, then every bean in the package becomes a CDI bean. There are a lot of features in CDI, too many to cover here, but as a quick reference for JSF-like features, you can define the scope of the CDI bean using one of the scopes defined in the javax.enterprise.context package (namely, request, conversation, session and application scopes). If you want to use the CDI bean from a JSF page, you can give it a name using the javax.inject.Named annotation. To inject a bean into another bean, you annotate the field with javax.inject.Inject annotation. @Named("someBean") @RequestScoped public class SomeBean { @Inject private SomeService someService; } Automatic injection like that defined above can be controlled through the use of Qualifiers that can help match the specific class that you want injected. If you have multiple payment types, you might add a qualifier for whether it is asynchronous or not. While you can use the @Named annotation as a qualifier, you shouldn’t as it is provided for exposing the beans in EL. CDI handles the injection of beans with mismatched scopes through the use of proxies. Because of this you can inject a request scoped bean into a session scoped bean and the reference will still be valid on each request because for each request, the proxy re-connects to a live instance of the request scoped bean. CDI also has support for interceptors, events, the new conversation scope and many other features which makes it a much better choice over JSF managed beans. EJB EJBs predate CDI beans and are in someways similar to CDI beans and in other ways very different. Primarily, the differences between CDI beans and EJBs is that EJBs are : Transactional Remote or local Able to passivate stateful beans freeing up resources Able to make use of timers Can be asynchronous The two types of EJBs are called stateless and stateful. Stateless EJBs can be thought of as thread safe single-use beans that don’t maintain any state between two web requests. Stateful EJBs do hold state and can be created and sit around for as long as they are needed until they are disposed of. Defining an EJB is simple, you just add either a javax.ejb.Stateless or javax.ejb.Stateful annotation to the class. @Stateless public class BookingService { public String makeReservation(Item Item,Customer customer) { ... ... } } Stateless beans must have a dependent scope while a stateful session bean can have any scope. By default they are transactional, but you can use the transaction attribute annotation. While EJBs and CDI beans are very different in terms of feaures, writing the code to integrate them is very similar since CDI beans can be injected into EJBs and EJBs can be injected into CDI beans. There is no need to make any distinction when injecting one into the other. Again, the different scopes are handled by CDI through the use of proxying. One exception to this is that CDI does not support the injection of remote EJBs but that can be implemented by writing a simple producer method for it. The javax.inject.Named annotation as well as any Qualifiers can be used on an EJB to match it to an injection point. When to use which bean How do you know when to use which bean? Simple. Never use JSF managed beans unless you are working in a servlet container and don’t want to try and get CDI working in Tomcat (although I have a Maven archetype for that so there’s no excuse). In general, you should use CDI beans unless you need the advanced functionality available in the EJBs such as transactional functions. You can write your own interceptor to make CDI beans transactional, but for now, its simpler to use an EJB until CDI gets transactional CDI beans which is just around the corner. If you are stuck in a servlet container and are using CDI, then either hand written transactions or your own transaction interceptor is the only option without EJBs. From http://www.andygibson.net/blog/article/comparing-jsf-beans-cdi-beans-and-ejbs/

Execute the slow integration tests separately from unit tests and show as much information about them as possible in Sonar.

A short definition of an Android Parcel would be that of a message container for lightweight, high-performance Inter-process communication (IPC). On Android, a "process" is a standard Linux one, and one process cannot normally access the memory of another process, so with Parcels, the Android system decomposes objects into primitives that can be marshaled/unmarshaled across process boundaries. But Parcels can also be used within the same process, to pass data across different components of a same application. As an example, a typical Android application has several screens, called "Activities" , and needs to communicate data or action from one Activity to the next. To write an object than can be passed through, we can implement the Parcelable interface. Android itself provides a built-in Parcelable object called an Intent which is used to pass information from one component to another. Using an Intent is pretty straightforward. Let's say we're collecting user data from our initial screen called CollectDataActivity. // inside CollectDataActivity, construct intent to pass along the next Activity, i.e. screen Intent in = new Intent(this, ProcessDataActivity.class); in.putExtra("userid", id); // (key,value) pairs in.putExtra("age", age); in.putExtra("phone", phone); in.putExtra("is_registered", true); // call next Activity --> next screen comes up startActivity(in); We need to collect that information from our data collection screen to process it. So all we do is the following: // inside ProcessDataActivity, get the info needed from previous Activity Intent in = this.getIntent(); in.getLongExtra("userid", 0L); in.getIntExtra("age", 0); in.getStringExtra("phone"); in.getBooleanExtra("is_registered", false); // false = default value overridden by user input Again, pretty straightforward. We retrieve the data using the same keys used to send it, and using our Intent's corresponding methods for each data type. But even when communicating with Intents, we can still use Parcels to pass data within the intent. For instance, we can do the above in a more elegant way using a custom, Parcelable User class: In the first Activity: // in CollectDataActivity, populate the Parcelable User object using its setter methods User usr = new User(); usr.setId(id); // collected from user input// etc.. // pass it to another component Intent in = new Intent(this, ProcessDataActivity.class); in.putExtra("user", usr); startActivity(in); In the second Activity: // in ProcessDataActivity retrieve User Intent intent = getIntent(); User usr = (User) intent.getParcelableExtra("user"); And this is what a Parcelable User class looks like: import android.os.Parcel; import android.os.Parcelable; public class User implements Parcelable { private long id; private int age; private String phone; private boolean registered; // No-arg Ctor public User(){} // all getters and setters go here //... /** Used to give additional hints on how to process the received parcel.*/ @Override public int describeContents() { // ignore for now return 0; } @Override public void writeToParcel(Parcel pc, int flags) { pc.writeLong(id); pc.writeInt(age); pc.writeString(phone); pc.writeInt( registered ? 1 :0 ); } /** Static field used to regenerate object, individually or as arrays */ public static final Parcelable.Creator CREATOR = new Parcelable.Creator() { public User createFromParcel(Parcel pc) { return new User(pc); } public User[] newArray(int size) { return new User[size]; } }; /**Ctor from Parcel, reads back fields IN THE ORDER they were written */ public User(Parcel pc){ id = pc.readLong(); age = pc.readInt(); phone = pc.readString(); registered = ( pc.readInt() == 1 ); } } What we did was: Make our User class implement the Parcelable interface. Parcelable is not a marker interface, hence what follows: Implement its describeContents method, which in this case does nothing. Implement its abstract method writeToParcel, which takes the current state of the object and writes it to a Parcel Add a static field called CREATOR to our class, which is an object implementing the Parcelable.Creator interface Add a Constructor that takes a Parcel as parameter. The CREATOR calls that constructor to rebuild our object. This looks like a lot of extra code at first, but bear in mind that, as in most cases, our application might evolve into incorporating more data from the user... Sometimes we need to pass complex objects from one component to another, and passing an object yields a cleaner design. The same logic applies for communicating between an Activity (foreground UI) and a background Service. We would just call the startService method instead of startActivity and pass it our Parcelable User object. Note that a Service is not running in a separate process by default. At this point, there are a couple of questions that may be raised: Isn't using an IPC-friendly, custom object for in-process communication simply overkill? Why would we want to use Parcelable, when we already have built-in Java serialization? The answer to the first concern is...maybe. But communicating through a custom object than through a list of key-value pairs is more OO, and it has no noticeable negative performance impact. As for the second question, why not simply have User implement Serializable, a theoretically simpler, marker interface? In one word, performance. Using Parcels is more efficient than serializing, at the price of some added complexity. That extra efficiency has in turn its limits: passing an image ( Bitmap) using Parcelable is generally not a good idea (although Bitmap does in fact implement Parcelable). A much more memory-efficient way would be to pass only its URI or Resource ID, so that other Android components in your application can have access to it. Another limitation of Parcelable is that it must not be used for general-purpose serialization to storage, since the underlying implementation may vary with different versions of the Android OS. So yes, Parcels are faster by design, but as high-performance transport, not as a replacement for general-purpose serialization mechanism. Having said all that, since our User object is Parcelable, it can now be sent from this application to another one running in another process, in particular through an interface implementing a remote service. In an upcoming post, we'll look at IPC and Android's Interface Definition Language (AIDL). from Tony's Blog

If you’re testing a web application, you can’t go far wrong with Selenium WebDriver. But in this web 2.0 world of ajax-y goodness, it can be a pain dealing with the asynchronous nature of modern sites. Back when all we had was web 1.0 you clicked a button and eventually you got a new page, or if you were unlucky: an error message. But now when you click links all sorts of funky things happen – some of which happen faster than others. From the user’s perspective this creates a great UI. But if you’re trying to automate testing this you can get all sorts of horrible race conditions. Thread.sleep The naive approach is to write your tests the same way you did before: you click buttons and assert that what you expected to happen actually happened. For the most part, this works. Sites are normally fast enough, even in a continuous integration environment, that by the time the test harness looks for a change it’s already happened. But then… things slow down a little and you start getting flickers - tests that sometimes pass and sometimes fail. So you add a little delay. Just 500 milliseconds should do it, while you wait for the server to respond and update the page. Then a month later it’s flickering again, so you make it 1 second. Then two… then twenty. The trouble is, each test runs at the pace that it runs at its slowest. If login normally takes 0.1 seconds, but sometimes takes 10 seconds when the environment’s overloaded – the test has to wait for 10 seconds so as not to flicker. This means even though the app often runs faster, the test has to wait just in case. Before you know it, your tests are crawling and take hours to run – you’ve lost your fast feedback loop and developers no longer trust the tests. An Example Thankfully WebDriver has a solution to this. It allows you to wait for some condition to pass, so you can use it to control the pace of your tests. To demonstrate this, I’ve created a simple web application with a login form – the source is available on github. The login takes a stupid amount of time, so the tests need to react to this so as not to introduce arbitrary waits. The application is very simple – a username and password field with an authenticate button that makes an ajax request to log the user in. If the login is successful, we update the screen to let the user know. The first thing is to write our test (obviously in the real world we’d have written the test before our production code, but its the test that’s interesting here not what we’re testing – so we’ll do it in the wrong order just this once): @Test public void authenticatesUser() { driver.get("http://localhost:8080/"); LoginPage loginPage = LoginPage.open(driver); loginPage.setUsername("admin"); loginPage.setPassword("password"); loginPage.clickAuthenticate(); Assert.assertEquals("Logged in as admin", loginPage.welcomeMessage()); } We have a page object that encapsulates the login functionality. We provide the username & password then click authenticate. Finally we check that the page has updated with the user message. But how have we dealt with the asynchronous nature of this application? WebDriverWait Through the magic of WebDriverWait we can wait for a function to return true before we continue: public void clickAuthenticate() { this.authenticateButton.click(); new WebDriverWait(driver, 30).until(accountPanelIsVisible()); } private Predicate accountPanelIsVisible() { return new Predicate() { @Override public boolean apply(WebDriver driver) { return isAccountPanelVisible(); } }; } private boolean isAccountPanelVisible() { return accountPanel.isDisplayed(); } Our clickAuthenticate method clicks the button then instructs WebDriver to wait for our condition to pass. The condition is defined via a predicate (c’mon Java where’s the closures?). The predicate is simply a method that will run to determine whether or not the condition is true yet. In this case, we delegate to the isAccountPanelVisible method on the page object. This does exactly what it says on the tin, it uses the page element to check whether it’s visible yet. Simple, no? In this way we can define a condition we want to be true before we continue. In this case, the exit condition of the clickAuthenticate method is that the asynchronous authentication process has completed. This means that tests don’t need to worry about the internal mechanics of the page – about whether the operation is asynchronous or not. The test merely specifies what to test, the page object encapsulates how to do it. Javascript It’s all well and good waiting for elements to be visible or certain text to be present, but sometimes we might want more subtle control. A good approach is to update Javascript state when an action has finished. This means that tests can inspect javascript variables to determine whether something has completed or not – allowing very clear and simple coordination between production code and test. Continuing with our login example, instead of relying on a becoming visible, we could instead have set a Javascript variable. The code in fact does both, so we can have two tests. The second looks as follows: public void authenticate() { this.authenticateButton.click(); new WebDriverWait(driver, 30).until(authenticated()); } private Predicate authenticated() { return new Predicate() { @Override public boolean apply(WebDriver driver) { return isAuthenticated(); } }; } private boolean isAuthenticated() { return (Boolean) executor().executeScript("return authenticated;"); } private JavascriptExecutor executor() { return (JavascriptExecutor) driver; } This example follows the same basic pattern as the test before, but we use a different predicate. Instead of checking whether an element is visible or not, we instead get the status of a Javascript variable. We can do this because each WebDriver also implements the JavascriptExecutor allowing us to run Javascript inside the browser within the context of the test. I.e. the script “return authenticated” runs within the browser, but the result is returned to our test. We simply inspect the state of a variable, which is false initially and set to true once the authentication process has finished. This allows us to closely coordinate our production and test code without the risk of flickering tests because of race conditions. From http://blog.activelylazy.co.uk/2012/01/29/testing-asynchronous-applications-with-webdriverwait/

Tests developed during TDD can be classified into several levels, depending on the size of the object graph they need to work with. End-to-end tests span the whole application graph, while unit tests usually target a single public class at a time. In the middle we find functional tests, which exercise a group of objects. A recurring problem is that of nearby classes C creeping into unit tests of unrelated classes; this situation transform what would be a unit test of the original class O into a functional tests of O and C together (possibly with multiple C classes involved). Functional tests are handy for specifying behavior at an higher level of abstraction than that of a single object, and sometimes for checking the wiring of a component of the application. However, if they are introduced involuntarily in place of unit tests they are prone to raise maintenance problems, since they will need to change every time the C class is updated. Moreover, they will fail along with the unit test of C, pointing to a problem into either O or C, which are not able to localize immediately. Consider this test, where the original class is DocumentsDeclarationNodeCommand and the collaborating one is InMemoryDocumentCopy: @Test public void shouldSendTheListOfDocumentsAndWaitForAcknowledgement() throws ConnectionClosedException { UpstreamConnection upstream = mock(UpstreamConnection.class); DownstreamConnection downstream = mock(DownstreamConnection.class); InMemoryDocumentCopy first = new InMemoryDocumentCopy("1.txt", "hello"); InMemoryDocumentCopy second = new InMemoryDocumentCopy("2.txt", "hello2"); DocumentsDeclarationNodeCommand command = DocumentsDeclarationNodeCommand.fromDocumentCopies( Arrays.asList(first, second), 10001); command.execute(upstream, downstream); InOrder inOrder = inOrder(upstream, downstream); inOrder.verify(upstream).command("DOCUMENTS|PORT=10001"); inOrder.verify(upstream).command("1.txt|5"); inOrder.verify(upstream).command("2.txt|6"); inOrder.verify(upstream).endCommandSection(); inOrder.verify(downstream, times(1)).readResponse(); } The two expectations on command() make this test a functional one: a change in the textual serialization format of InMemoryDocumentCopy (such as "1.txt|sha1_hash|5") will break this checks, even if DocumentsDeclarationNodeCommand still works. Yet we cannot avoid to verify that the documents are really sent to the server by this object. Functional tests can be transformed again into unit tests by testing O with a Test Double instead of C (a Stub, or a Mock.) The only remaining dependency will be the one of the interface of C, which can be even extracted into an independent entity (a first-class interface in language that support them such as Java, C# and PHP.) Pure functions What happens when you can't easily inject a Test Double to maintain the tests at the unit level? This issues exists in functional languages where functions call a tree of other functions. A analogue approach to dependency injection is to inject the function as a parameter, but doesn't probably scale to the level of injection we perform on objects: every function signature would have to receive all the collaborating ones as additional parameters. There are even mocking frameworks for functional languages like Marick's one which are able to isolate a function from its collaborators. Uncle Bob uses the Derived Expectation pattern instead: testing "update-all" (let [ o1 (make-object ...) o2 (make-object ...) o3 (make-object ...) os [o1 o2 o3] us (update-all os) ] (is (= (nth us 0) (reposition (accelerate (accumulate-forces os o1) (is (= (nth us 1) (reposition (accelerate (accumulate-forces os o2) (is (= (nth us 2) (reposition (accelerate (accumulate-forces os o3) ) ) The update-all function calls internally reposition, accelerate and accumulate-forces (or it calls other functions which in turn call these three). Instead of specifying unreadable literal expectations in the tests like (1.096, 4.128), this approach let the test specify update-all link to the other functions without introducing magic numbers. It is therefore a unit test for update-all, while the same test containing numbers would be a functional test. Note that this approach is safe for functional languages because the collaborating functions have no state, being pure; you can call reposition and accelerate how many times you want, and their result won't change. This is not necessarily true for collaborators in object-oriented languages: in principle, a method can return a different value for each call. Tests with derived expectations As long as the composed methods do not change their result, this approach would build real unit tests, whose success does not depend on the correctness of classes other than the one under test. Apart from corner cases like the composed methods throwing exceptions, a change in the collaborator's behavior would change only the collaborator's test. Value Objects are the ideal collaborator to stub out with derived expectations: they are immutable, so their methods always return the same result. Their code is usually self-contained and simple, so it's difficult for a method to throw an exception or to break internally once the Value Object has been correctly built. Being simple, final classes they do not implement an explicit interface; and they are not commonly substituted by Test Doubles. Their behavior is mixed in with the objects using them. The test becomes: @Test public void shouldSendTheListOfDocumentsAndWaitForAcknowledgement() throws ConnectionClosedException { UpstreamConnection upstream = mock(UpstreamConnection.class); DownstreamConnection downstream = mock(DownstreamConnection.class); InMemoryDocumentCopy first = new InMemoryDocumentCopy("1.txt", "hello"); InMemoryDocumentCopy second = new InMemoryDocumentCopy("2.txt", "hello2"); DocumentsDeclarationNodeCommand command = DocumentsDeclarationNodeCommand.fromDocumentCopies( Arrays.asList(first, second), 10001); command.execute(upstream, downstream); InOrder inOrder = inOrder(upstream, downstream); inOrder.verify(upstream).command("DOCUMENTS|PORT=10001"); inOrder.verify(upstream).command(first.toString()); inOrder.verify(upstream).command(second.toString()); inOrder.verify(upstream).endCommandSection(); inOrder.verify(downstream, times(1)).readResponse(); } Conclusion We saw that Test Doubles like Mocks and Stubs are not the only way to achieve isolated tests, which fail only where the class under test fail and not when a collaborator changes its implementation. In the Example, DocumentsDeclarationNodeCommand is tested by involving the real collaborator, but setting up Derived Expectation from it instead of literal ones. The result is this test is only tied to the method signatures of the collaborator instead of to the real behavior (the output format of toString()). This technique doesn't need to be used often: its purpose is to isolate from an immutable object, without introducing a Test Double.

To test mobile versions of websites, it is useful to be able to connect to a web server on your local machine from a web browser on an Android emulator without having to expose the web server to the Internet. You can’t use the normal loop-back IP address of 127.0.0.1 because that refers to the emulated Android device itself. Instead you have to use 10.0.2.2 to connect to the host machine. That’s fine if your local web server is serving a single site, but if you are using name-based virtual hosting to serve different sites depending on the host name of the request (with aliases for localhost defined in your machine’s hosts file), then you need to be making requests from the browser using the correct host name, not the IP address. The Android emulator does not use the host machine’s hosts file for name resolution so attempting to access http://myvirtualhost in the emulator’s browser will not work. This is because the emulated Android device has it’s own hosts file, so you have to update this to map the virtual host names to the local machine. The first step is to start the AVD with an increased partition size otherwise you may get an out of memory error when you try to save the modified hosts file: emulator -avd MyAVD -partition-size 128 You then have to remount the system partition so that it is writeable: adb remount Then copy the hosts file from the emulated device to the host machine: adb pull /etc/hosts Edit the hosts file so that it includes mappings for all relevant virtual host names: 127.0.0.1 localhost 10.0.2.2 myvirtualhost1 myvirtualhost2 Then copy the updated file back to the emulated device: adb push hosts /etc/hosts You should then be able to visit http://myvirtualhost1 in the emulator’s browser and see the correct site. From http://blog.uncommons.org/2012/01/12/accessing-local-name-based-virtual-hosts-from-the-android-emulator/

clover is a great tool for generating code coverage reports from your unit tests. it can be executed as a plugin in eclipse, maven or ant. however, not everyone knows that it can also be used to collect coverage data of integration tests. this post explains how to collect coverage data with clover at runtime. this post assumes that you already know what are unit and integration tests. this post assumes that you know what clover is, and already used it either with eclipse, ant or maven. * let me assured you that even though the directions bellow seems complicated and clumsy at first, after doing them once or twice it is really easy to repeat them. motivation the default action of clover is to gather code coverage information during build time or compile time. therefore, this information includes just the coverage data created by unit tests. if you are developing web applications, you probably use more technologies to test your applications beside unit tests. these technologies may include httpunit/ htmlunit or automation technologies (like selenium). these technologies do not work at build time, they can only work during run time, where a web server is up and running and http calls are made. as a result, the code coverage made during build time is not reflecting the actual code coverage. we should be able to test the coverage while a server is running. the idea the idea is that we will first run clover regularly during build time. we will than take the clover artifacts, put them in our server and then run the integration tests. while running the integration tests, the clover database will be updated and we would be able to generate reports from it which will reflect both unit and integration tests. step 1 – preparation make sure that you have a web/application server (tomcat/jboss/weblogic…) with your web application already deployed. execute clover on your application as you would normally do (either by compiling the code on eclipse or by building with maven or ant, it doesn’t matter). the result of this action would be: clover db files. one of the outcome of executing the clover on your code are the db files. the db files hold all the information about your code and the coverage itself. the location of those files may change depending on the way you use clover and according to the way you configured clover in your environment. this is how the files looks like the .db file holds the information regarding your code (classes, methods and so on). all the other files hold all the coverage data. it is important that you will locate those files because we are going to use them in the next step. an instrumented code. another outcome of clover is that it instruments your code. a clover call is injected into each method so it would be reported in the coverage calculation. we will need this code. we will use this instrumented code in runtime to update the coverage data. if you use eclipse than the generated classes would be instrumented. if you use maven or ant than most chances are that a jar with all the instrumented code would be generated separately. search the instrument code jar. again, i can’t tell you exactly where it is located, but usually it generates a jar with a ‘clover’ postfix. example: if your jar name is my_app.jar, than the generated instrumented code jar will probably be something like my_app-clover.jar. so you will need to do some detective work here to find the instrumented classes/jar. if you are not sure whether the classes are instrumented or not, just decompile one of them with jad and search for the word clover inside of it. a code coverage report. this is a report with the unit test coverage. we don’t really need this, but it would be good so that we would be able to compare it with the report we will generate at the end. step 2 – updating the server the next steps are very important, please make sure you do them properly. replacing the existing application jar/classes with the instrumented jar/classes. take the instrumented jar/classes that were created by the clover and add it to your server’s classpath instead of the original jar/classes. the instrumented code will cause the clover db to be updated with the runtime data. adding clover jars and license. since we will use clover on runtime we will need also the clover jars in our server’s classpath. so add the clover own jars to your server’s classpath. if you are using clover with eclipse than these jars are located in the plugin folder of eclipse. if you are using maven than they will be loacted in your repository. * make sure you are using the same version of clover in your server as you used to generate the db files and instrumented code, otherwise it will not work and you will get error messages. also add the clover license file to the same location as the jars. adding the clover java argument. add the following java arg -dclover.initstring.basedir={ location of the db files that were created by the clover } . * notice – the path you have entered above is the path of the folder which contains the db files. example: -dclover.initstring.basedir=c:/workspace/mywebapp/target/clover. this java argument is used by the clover to locate the db files and update them. step 3 – restarting the server and running the tests now that hopefully all is set properly all that you need to do now is to restart your server and than running your integration tests. the tests should trigger the instrumented code which will call the clover api’s and will update the clover db. while running the tests: look at your log/console and search for error messages from clover. look at the folder which holds the clover db files. if everything is going as it should, new files will be created in this folder while running the tests. if not everything is going well the first time, don’t discourage, just go over each of the steps again. step 4 – generating an updated report if everything went well and new files were created in the db folder than that means you just need to generate a new report. if you are using clover with eclipse than you can simply push the reload button to reload the coverage data. if you are using maven or ant you can execute just the task which generates the report. another way is to use the clover htmlreporter to generate a report easily. now compare the new report to the old report. you should see that the new report coverage is much bigger than the old one since it contains also the integration tests coverage. * notice that not all the data is updated, even though the percentages are being updated, for some reason the calls counter does not. to summarize. as mentioned; yes, these instructions seems a bit complicated but after you succeed the first time, it is very easy to repeat it. in the company i work for we even made this whole process automatic and we are able to generate a full coverage report with unit and integrated tests combined. source: http://www.aviyehuda.com/2011/12/use-clover-to-generate-code-coverage-reports-of-your-integrationautomation-tests/

Introduction A common thing I come across is that teams using a mocking framework assume they are mocking. They are not aware that Mocks are just one of a number of 'Test Doubles' which Gerard Meszaros has categorised at xunitpatterns.com. It’s important to realise that each type of test double has a different role to play in testing. In the same way that you need to learn different patterns or refactoring’s, you need to understand the primitive roles of each type of test double. These can then be combined to achieve your testing needs. I'll cover a very brief history of how this classification came about, and how each of the types differs. I'll do this using some short, simple examples in Mockito. A Very Brief History For years people have been writing lightweight versions of system components to help with testing. In general it was called stubbing. In 2000' the article 'Endo-Testing: Unit Testing with Mock Objects' introduced the concept of a Mock Object. Since then Stubs, Mocks and a number of other types of test objects have been classified by Meszaros as Test Doubles. This terminology has been referenced by Martin Fowler in "Mocks Aren't Stubs" and is being adopted within the Microsoft community as shown in "Exploring The Continuum of Test Doubles" A link to each of these important papers are shown in the reference section. Categories of test doubles The diagram above shows the commonly used types of test double. The following URL gives a good cross reference to each of the patterns and their features as well as alternative terminology. http://xunitpatterns.com/Test%20Double.html Mockito Mockito is a test spy framework and it is very simple to learn. Notable with Mockito is that expectations of any mock objects are not defined before the test as they sometimes are in other mocking frameworks. This leads to a more natural style(IMHO) when beginning mocking. The following examples are here purely to give a simple demonstration of using Mockito to implement the different types of test doubles. There are a much larger number of specific examples of how to use Mockito on the website. http://docs.mockito.googlecode.com/hg/latest/org/mockito/Mockito.html Test Doubles with Mockito Below are some basic examples using Mockito to show the role of each test double as defined by Meszaros. I’ve included a link to the main definition for each so you can get more examples and a complete definition. Dummy Object http://xunitpatterns.com/Dummy%20Object.html This is the simplest of all of the test doubles. This is an object that has no implementation which is used purely to populate arguments of method calls which are irrelevant to your test. For example, the code below uses a lot of code to create the customer which is not important to the test. The test couldn't care less which customer is added, as long as the customer count comes back as one. public Customer createDummyCustomer() { County county = new County("Essex"); City city = new City("Romford", county); Address address = new Address("1234 Bank Street", city); Customer customer = new Customer("john", "dobie", address); return customer; } @Test public void addCustomerTest() { Customer dummy = createDummyCustomer(); AddressBook addressBook = new AddressBook(); addressBook.addCustomer(dummy); assertEquals(1, addressBook.getNumberOfCustomers()); } We actually don't care about the contents of customer object - but it is required. We can try a null value, but if the code is correct you would expect some kind of exception to be thrown. @Test(expected=Exception.class) public void addNullCustomerTest() { Customer dummy = null; AddressBook addressBook = new AddressBook(); addressBook.addCustomer(dummy); } To avoid this we can use a simple Mockito dummy to get the desired behaviour. @Test public void addCustomerWithDummyTest() { Customer dummy = mock(Customer.class); AddressBook addressBook = new AddressBook(); addressBook.addCustomer(dummy); Assert.assertEquals(1, addressBook.getNumberOfCustomers()); } It is this simple code which creates a dummy object to be passed into the call. Customer dummy = mock(Customer.class); Don't be fooled by the mock syntax - the role being played here is that of a dummy, not a mock. It's the role of the test double that sets it apart, not the syntax used to create one. This class works as a simple substitute for the customer class and makes the test very easy to read. Test stub http://xunitpatterns.com/Test%20Stub.html The role of the test stub is to return controlled values to the object being tested. These are described as indirect inputs to the test. Hopefully an example will clarify what this means. Take the following code public class SimplePricingService implements PricingService { PricingRepository repository; public SimplePricingService(PricingRepository pricingRepository) { this.repository = pricingRepository; } @Override public Price priceTrade(Trade trade) { return repository.getPriceForTrade(trade); } @Override public Price getTotalPriceForTrades(Collection trades) { Price totalPrice = new Price(); for (Trade trade : trades) { Price tradePrice = repository.getPriceForTrade(trade); totalPrice = totalPrice.add(tradePrice); } return totalPrice; } The SimplePricingService has one collaborating object which is the trade repository. The trade repository provides trade prices to the pricing service through the getPriceForTrade method. For us to test the business logic in the SimplePricingService, we need to control these indirect inputs i.e. inputs we never passed into the test. This is shown below. In the following example we stub the PricingRepository to return known values which can be used to test the business logic of the SimpleTradeService. @Test public void testGetHighestPricedTrade() throws Exception { Price price1 = new Price(10); Price price2 = new Price(15); Price price3 = new Price(25); PricingRepository pricingRepository = mock(PricingRepository.class); when(pricingRepository.getPriceForTrade(any(Trade.class))) .thenReturn(price1, price2, price3); PricingService service = new SimplePricingService(pricingRepository); Price highestPrice = service.getHighestPricedTrade(getTrades()); assertEquals(price3.getAmount(), highestPrice.getAmount()); } Saboteur Example There are 2 common variants of Test Stubs: Responder’s and Saboteur's. Responder's are used to test the happy path as in the previous example. A saboteur is used to test exceptional behaviour as below. @Test(expected=TradeNotFoundException.class) public void testInvalidTrade() throws Exception { Trade trade = new FixtureHelper().getTrade(); TradeRepository tradeRepository = mock(TradeRepository.class); when(tradeRepository.getTradeById(anyLong())) .thenThrow(new TradeNotFoundException()); TradingService tradingService = new SimpleTradingService(tradeRepository); tradingService.getTradeById(trade.getId()); } Mock Object http://xunitpatterns.com/Mock%20Object.html Mock objects are used to verify object behaviour during a test. By object behaviour I mean we check that the correct methods and paths are excercised on the object when the test is run. This is very different to the supporting role of a stub which is used to provide results to whatever you are testing. In a stub we use the pattern of defining a return value for a method. when(customer.getSurname()).thenReturn(surname); In a mock we check the behaviour of the object using the following form. verify(listMock).add(s); Here is a simple example where we want to test that a new trade is audited correctly. Here is the main code. public class SimpleTradingService implements TradingService{ TradeRepository tradeRepository; AuditService auditService; public SimpleTradingService(TradeRepository tradeRepository, AuditService auditService) { this.tradeRepository = tradeRepository; this.auditService = auditService; } public Long createTrade(Trade trade) throws CreateTradeException { Long id = tradeRepository.createTrade(trade); auditService.logNewTrade(trade); return id; } The test below creates a stub for the trade repository and mock for the AuditService We then call verify on the mocked AuditService to make sure that the TradeService calls it's logNewTrade method correctly @Mock TradeRepository tradeRepository; @Mock AuditService auditService; @Test public void testAuditLogEntryMadeForNewTrade() throws Exception { Trade trade = new Trade("Ref 1", "Description 1"); when(tradeRepository.createTrade(trade)).thenReturn(anyLong()); TradingService tradingService = new SimpleTradingService(tradeRepository, auditService); tradingService.createTrade(trade); verify(auditService).logNewTrade(trade); } The following line does the checking on the mocked AuditService. verify(auditService).logNewTrade(trade); This test allows us to show that the audit service behaves correctly when creating a trade. Test Spy http://xunitpatterns.com/Test%20Spy.html It's worth having a look at the above link for the strict definition of a Test Spy. However in Mockito I like to use it to allow you to wrap a real object and then verify or modify it's behaviour to support your testing. Here is an example were we check the standard behaviour of a List. Note that we can both verify that the add method is called and also assert that the item was added to the list. @Spy List listSpy = new ArrayList(); @Test public void testSpyReturnsRealValues() throws Exception { String s = "dobie"; listSpy.add(new String(s)); verify(listSpy).add(s); assertEquals(1, listSpy.size()); } Compare this with using a mock object where only the method call can be validated. Because we only mock the behaviour of the list, it does not record that the item has been added and returns the default value of zero when we call the size() method. @Mock List listMock = new ArrayList(); @Test public void testMockReturnsZero() throws Exception { String s = "dobie"; listMock.add(new String(s)); verify(listMock).add(s); assertEquals(0, listMock.size()); } Another useful feature of the testSpy is the ability to stub return calls. When this is done the object will behave as normal until the stubbed method is called. In this example we stub the get method to always throw a RuntimeException. The rest of the behaviour remains the same. @Test(expected=RuntimeException.class) public void testSpyReturnsStubbedValues() throws Exception { listSpy.add(new String("dobie")); assertEquals(1, listSpy.size()); when(listSpy.get(anyInt())).thenThrow(new RuntimeException()); listSpy.get(0); } In this example we again keep the core behaviour but change the size() method to return 1 initially and 5 for all subsequent calls. public void testSpyReturnsStubbedValues2() throws Exception { int size = 5; when(listSpy.size()).thenReturn(1, size); int mockedListSize = listSpy.size(); assertEquals(1, mockedListSize); mockedListSize = listSpy.size(); assertEquals(5, mockedListSize); mockedListSize = listSpy.size(); assertEquals(5, mockedListSize); } This is pretty Magic! Fake Object http://xunitpatterns.com/Fake%20Object.html Fake objects are usually hand crafted or light weight objects only used for testing and not suitable for production. A good example would be an in-memory database or fake service layer. They tend to provide much more functionality than standard test doubles and as such are probably not usually candidates for implementation using Mockito. That’s not to say that they couldn’t be constructed as such, just that its probably not worth implementing this way. References Test Double Patterns Endo-Testing: Unit Testing with Mock Objects Mock Roles, Not Objects Mocks Aren't Stubs http://msdn.microsoft.com/en-us/magazine/cc163358.aspx Original Article. The original article and others can be found here http://johndobie.blogspot.com/2011/11/test-doubles-with-mockito.html

have been working on a fairly simple JMeter load script that I can run a series of 4 sequential pages against an Apache server, but the goal was to have the server support 2,000 concurrent requests for 5 minutes without error. Most of my issues in this exercise have been with JMeter and the client machine used to test the Apache server. To begin, I must state I was originally configuring Apache with a prefork MPM: StartServers 100 MinSpareServers 75 MaxSpareServers 100 ServerLimit 2000 MaxClients 2000 MaxRequestsPerChild 0 At approximately line 72 of Jmeter.bat, there are several entries the manage the JVM for running Jmeter. set HEAP=-Xms512m -Xmx512m set NEW=-XX:NewSize=128m -XX:MaxNewSize=128m set SURVIVOR=-XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=50% set TENURING=-XX:MaxTenuringThreshold=2 set RMIGC=-Dsun.rmi.dgc.client.gcInterval=600000 -Dsun.rmi.dgc.server.gcInterval=600000 set PERM=-XX:PermSize=64m -XX:MaxPermSize=64m I decided to start with 1,000 concurrent requests for 5 minutes just to see how the test would fair. With the above settings I started getting OOM errors almost immediately so I decided to increase the HEAP and NEW memory to eliminate the issue and wanted to add more GC settings to increase the JVM’s ability to clean up: set HEAP=-Xms1024m -Xmx1024m -Xss128k set NEW=-XX:NewSize=256m -XX:MaxNewSize=256m set SURVIVOR=-XX:SurvivorRatio=14 -XX:TargetSurvivorRatio=50% set "TENURING=-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:+DisableExplicitGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxTenuringThreshold=4" set "EVACUATION=-XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:+UseCompressedStrings -XX:+OptimizeStringConcat" set RMIGC=-Dsun.rmi.dgc.client.gcInterval=600000 -Dsun.rmi.dgc.server.gcInterval=600000 set PERM=-XX:PermSize=64m -XX:MaxPermSize=64m This did resolve the JMeter OOM issues, but now started getting Apache errors. During the ramp-up phase, I started getting connection refused errors: Response code: Non HTTP response code: org.apache.http.conn.HttpHostConnectException Response message: Non HTTP response message: Connection to http://pasundtastgprt2:8001 refused I started looking at the Apache server and noticed that the number of httpd threads was at 1,000 and it appeared that JMeter was running out of memory because the requests where starting to back up. This is why we load test right! So I decided to run a worker MQM and recompiled Apache to support the new MPM ServerLimit 80 StartServers 25 MaxClients 2000 MinSpareThreads 75 MaxSpareThreads 125 ThreadsPerChild 5 MaxRequestsPerChild 0 I started testing this configuration and while monitoring the server running 1,000 concurrent requests and the server looked like Apache was handling 1,000 requests just fine. I was running a simple command to output the sockets and httpd processes on the server during the load test: while true do echo -----`date '+%r'` -----: netstat -ant | awk '{print $6}' | sort | uniq -c | sort -n echo httpd processes: [`ps aux | grep httpd | wc -l`] echo . sleep 30 done Then when I was monitoring the load test, I was concerned about seeing 82 httpd processes running which was the ServerLimit I had set. -----08:02:37 AM -----: 1 established) 1 Foreign 4 CLOSE_WAIT 17 LISTEN 32 FIN_WAIT2 41 ESTABLISHED 69 FIN_WAIT1 630 SYN_RECV 45386 TIME_WAIT [82] httpd processes . I now increased the load to my target of 2,000 concurrent requests and restarted the JMeter test and was able to get to around 1,800 concurrent request and started getting connection refused errors again. I suspected that my Servers where maxed out and was not able to create anymore threads for those servers where having: 80 server * 5 threads each server == 400 requests processed concurrently So I increased the number of threads to 25 80 server * 25 threads each server == 2,000 requests processed concurrently To end up with this worker setting: ServerLimit 80 StartServers 25 MaxClients 2000 MinSpareThreads 75 MaxSpareThreads 125 ThreadsPerChild 25 MaxRequestsPerChild 0 I was then able to turn the load up to 2,000 concurrent requests. -----08:53:27 AM -----: 1 established) 1 FIN_WAIT2 1 Foreign 4 CLOSE_WAIT 12 CLOSING 17 LISTEN 129 ESTABLISHED 621 FIN_WAIT1 1203 SYN_RECV 55556 TIME_WAIT [53] httpd processes At this point we are only using 53 Servers and 2,000 clients. The tests sustained zero errors for 5 minutes during the test. As a test I increased the ThreadsPerChild to 40 and run the load test against 3,000 concurrent requests. I was able to get to around 2,800 concurrent requests then I started getting connection refused errors: Error Count: 1 Response code: Non HTTP response code: org.apache.http.conn.HttpHostConnectException Response message: Non HTTP response message: Connection to http://pasundtastgprt2:8001 refused and I also started getting JMeter errors: Response code: Non HTTP response code: java.net.BindException Response message: Non HTTP response message: Address already in use: connect So in the furure I would like to see how much further I can push the server and I think I can get to 3,000 concurrent and most likely far more than that on my current installation. From http://www.baselogic.com/blog/development/adding-memory-jmeter/

This article describes some practices for test-driving multithreaded and distributed applications written in Java. The example I worked on and we will use is a peer-to-peer application composed of many Nodes (clients) and of a few Supernodes (servers). The ultimate goal it to build an application composed of all these entities, but the first tests target a Supernode serving one or more Nodes. The walking skeleton TDD is mostly iterative, but needs a starting point. The simplest story we can think of is that of a Node connecting to a Supernode. Client and servers usually run in their own threads (in the case of the server, multiple ones), but initially the Node object can just be a POJO and run in the test's thread because we do not need to manage multiple Nodes yet. The Supernode object instead is a Thread (or a Runnable) and so we already face a simplified version of the synchronization problem: how to make sure the Supernode is ready to answer to connections once we have started its thread? The JUnit test is the following: @Test public void aNodeCanConnectToASupernode() throws Exception { Supernode supernode = new Supernode(8888); supernode.start(); supernode.ensureStartupIsFinished(); Node n = new Node(); n.connect("127.0.0.1", 8888); assertEquals(1, supernode.getNodes()); } supernode.start() runs the new thread, while the call to ensureStartupIsFinished() will have to block until the other thread is ready. Then, we create a Node object and tell it to connect; after it has finished this operation, we count how many nodes have connected to the Supernode. To satisfy this test, the Supernode can be a single-threaded server: public class Supernode extends Thread { private int port; private boolean startupCompleted; private int nodes = 0; public Supernode(int port) { this.port = port; this.startupCompleted = false; } public void run() { ServerSocket sock; try { sock = new ServerSocket(this.port); // ...networking setup... synchronized (this) { this.startupCompleted = true; notify(); } while (true) { // ...accepting new connections on sock and other stuff } } public int getNodes() { return nodes; } synchronized public void ensureStartupIsFinished() throws InterruptedException { while (!this.startupCompleted) { wait(); } } } What's in this first example? Thread objects are manageable as POJOs from a single JVM: as long as we write them with this API it will be simple to instantiate and terminate them, and to add primitives for synchronization. The startupCompleted field, which is an example of this synchronization behavior added to the production code. Adding production code just for end-to-end testing purposes is not uncommon. The test thread blocks inside ensureStartupIsFinished() until it is woken up via notification. Even then, startupCompleted must be true or it will wait more. This is Plain Old Java Synchronization: note the synchronized blocks around this.wait() and this.notify(). The problem with frameworks and containers is you have to hope they provide the synchronization facilities to test your code once it's inside them: have you ever tried to wait for Tomcat to start? There are some noticeable missing parts in this code: the threads for each node. The current test does not require them as only one Node is connecting for now. Thread.sleep() calls: at least for the happy paths I have covered until now, I never need to introduce them and considered them a smell. Configuration files: if we had to read configuration, the tests would take really long to write and would refer continuously to external resources. This is the case when testing with external tools which are not embeddable (Tomcat requiring configuration files while Jetty allowing configuration to be passed in Java test code). You can always add file-based configuration later, but for now it will slow us down. Evolution By adding one test at the time with a larger scope, we can try to evolve the code and add the difficult networking, multithreading part one bit at a time. After some iterations, the test becomes: public class FileSharingNetworkTest { Supernode supernode; @Before public void setUp() throws Exception { supernode = new Supernode(8888); supernode.start(); supernode.ensureStartupIsFinished(); } @After public void tearDown() throws Exception { supernode.ensureStop(); } @Test public void aNodeCanConnectToASupernode() throws Exception { Node n = newNode(Arrays.asList("1.txt", "2.txt")); n.ensureConnectionIsFinished(); assertEquals(1, supernode.getNodes()); assertEquals(2, supernode.getDocuments()); } @Test public void multipleNodesCanConnectToASupernodeSimultaneously() throws Exception { Node n1 = newNode(); Node n2 = newNode(); n1.ensureConnectionIsFinished(); n2.ensureConnectionIsFinished(); assertEquals(2, supernode.getNodes()); } private Node newNode() { Node n = new Node("127.0.0.1", 8888); n.start(); n.setDocumentList(Arrays.asList("1.txt", "2.txt")); return n; } private Node newNode(List documentList) { Node n = new Node("127.0.0.1", 8888); n.start(); n.setDocumentList(documentList); return n; } } The server-side code doesn't have multiple threads yet. What is the test case that will call for them? You have to find it and write it. This workflow will ensure that there is a test that targets this case. In my case, it was the first test requiring interaction between the two clients, where one had to see the documents listed by the other after both had connected. Even if you know where you will end up, you can test-drive the implementation: the advantage is that you understand better a standard design and ensure its test coverage. After a few more tests, I have reached a multithreaded server with a main thread and chidren for managing the connections; and Node objects implemented as independent threads. Conclusions When working with TDD at a system scale that includes asynchronous behavior, we should strive for a test suite that is: fast; even with multiple threads to wait for, a single end to end test should take less than a second to complete. Comprehensive; TDD makes us only write tested code instead of copying down snippets from the web. Robust: totally deterministic, as every run will either pass or fail, even when repeated dozens of times. There should be no sleeping calls for all the happy paths; there should be synchronization and stopping facilities built into the system. Featuring unit tests: along with the end to end tests we should write unit tests for the objects we need to extract (and that will be single threaded). It was easy for me to get caught up into covering more and more cases with a full scale test, but unit tests are better at pointing out where a bug resides. We also have to keep in mind how to design our objects and interfaces: not starting with N threads but with at most 1 more than the test's one (the server or a remote peer). Evolving them: adding a few lines of verbose Java networking code each time. My example has evolved to N client threads, a server main thread and N server children threads talking with each client. I will now have to evolve it to a network of supernodes, being this about a file sharing network; to introduce secure channels and certificates. The difficult part is to constantly refactor to support new stories without having to rework the whole system for a single one. Not only extracting methods (an automated operation), but also to extract interfaces and most importantly objects; targeting the longest and complex classes and chopping them down into basic responsibilities.