Introduction I had an interesting conversation the other day about custom domain-specific languages and we happened to talk about a feature of Spring that I’ve used before but doesn’t seem to be widely known: the static application context. This post illustrates a basic example I wrote that introduces the static application context and shows how it might be useful. It’s also an interesting topic as it shows some of the well-architected internals of the Spring framework. Most uses of Spring start with XML or annotations and wind up with an application context instance. Behind the scenes, Spring has been working hard to instantiate objects, inject properties, invoke context aware listeners, and so forth. There are a set of classes internal to Spring to help this process along, as Spring needs to hold all of the configuration data about beans before any beans are instantiated. (This is because the beans may be defined in any order, and Spring doesn’t have the exhaustive set of dependencies until all beans are defined.) Spring Static Application Context Spring offers a class called StaticApplicationContext that gives programmatic access from Java to this whole configuration and registration process. This means we can define an entire application context from pure Java code, without using XML or Java annotations or any other tricks. The Javadoc for StaticApplicationContext is here, but an example is coming. Why might we use this? As the Javadoc says, it’s mainly useful for testing. Spring uses it for its own testing, but I’ve found it useful for testing applications that use Spring or other dependency management frameworks. Often, for unit testing, we want to inject different objects into a class from those used in production (e.g. mock objects, or objects that simulate remote invocation, database, or messaging). Of course, we can just keep a separate Spring XML configuration file for testing, but it’s very nice to have our whole configuration right there in the Java unit test class as it makes it easier to maintain. Example I’ve added an example to my intro-to-java repository on GitHub. I created aStaticContext class that provides a very basic Java domain-specific language (DSL) for Spring beans. This is just to make it easier to use from the unit test. The DSL only includes the most basic Spring capabilities: register a bean, set properties, and wire dependencies. package org.anvard.introtojava.spring; import org.springframework.beans.MutablePropertyValues; import org.springframework.beans.factory.config.ConstructorArgumentValues; import org.springframework.beans.factory.config.RuntimeBeanReference; import org.springframework.beans.factory.support.RootBeanDefinition; import org.springframework.context.ApplicationContext; import org.springframework.context.support.StaticApplicationContext; public class StaticContext { public class BeanContext { private String name; private Class beanClass; private ConstructorArgumentValues args; private MutablePropertyValues props; private BeanContext(String name, Class beanClass) { this.name = name; this.beanClass = beanClass; this.args = new ConstructorArgumentValues(); this.props = new MutablePropertyValues(); } public BeanContext arg(Object arg) { args.addGenericArgumentValue(arg); return this; } public BeanContext arg(int index, Object arg) { args.addIndexedArgumentValue(index, arg); return this; } public BeanContext prop(String name, Object value) { props.add(name, value); return this; } public BeanContext ref(String name, String beanRef) { props.add(name, new RuntimeBeanReference(beanRef)); return this; } public void build() { RootBeanDefinition def = new RootBeanDefinition(beanClass, args, props); ctx.registerBeanDefinition(name, def); } } private StaticApplicationContext ctx; private StaticContext() { this.ctx = new StaticApplicationContext(); } public static StaticContext create() { return new StaticContext(); } public ApplicationContext build() { ctx.refresh(); return ctx; } public BeanContext bean(String name, Class beanClass) { return new BeanContext(name, beanClass); } } This class uses several classes that are normally internal to Spring: StaticApplicationContext: Holds bean definitions and provides regular Java methods for registering beans. ConstructorArgumentValues: A smart list for a bean’s constructor arguments. Can hold both wire-by-type and indexed constructor arguments. MutablePropertyValues: A smart list for a bean’s properties. Can hold regular objects and references to other Spring beans. RuntimeBeanReference: A reference by name to a bean in the context. Used for wiring beans together because it allows Spring to delay resolution of a dependency until it’s been instantiated. The StaticContext class uses the builder pattern and provides for method chaining. This makes for cleaner use from our unit test code. Here’s the simplest example: @Test public void basicBean() { StaticContext sc = create(); sc.bean("basic", InnerBean.class).prop("prop1", "abc"). prop("prop2", "def").build(); ApplicationContext ctx = sc.build(); assertNotNull(ctx); InnerBean bean = (InnerBean) ctx.getBean("basic"); assertNotNull(bean); assertEquals("abc", bean.getProp1()); assertEquals("def", bean.getProp2()); } A slightly more realistic example that includes wiring beans together is not much more complicated: @Test public void innerBean() { StaticContext sc = create(); sc.bean("outer", OuterBean.class).prop("prop1", "xyz"). ref("inner", "inner").build(); sc.bean("inner", InnerBean.class).prop("prop1", "ghi"). prop("prop2", "jkl").build(); ApplicationContext ctx = sc.build(); assertNotNull(ctx); InnerBean inner = (InnerBean) ctx.getBean("inner"); assertNotNull(inner); assertEquals("ghi", inner.getProp1()); assertEquals("jkl", inner.getProp2()); OuterBean outer = (OuterBean) ctx.getBean(OuterBean.class); assertNotNull(outer); assertEquals("xyz", outer.getProp1()); assertEquals(inner, outer.getInner()); } Note that once we build the context, we can use it like any other Spring application context, including fetching beans by name or type. Also note that the two contexts we created here are completely separate, which is important for unit testing. Conclusion Much like my post on custom Spring XML, the static application context is a specialty feature that isn’t intended for everyday users of Spring. But I’ve found it convenient when unit testing and it provides an interesting peek into how Spring works.

we all know junit test-classes can be parameterized , which means that for a given set of test-elements, the test class is instantiated a few times, but using constructors for that isn’t always what you want. i’ve taken the stringsorttest from this blog as an example. @runwith(parameterized.class) public class stringsorttest { @parameters public static collection data() { return arrays.aslist(new object[][] { { "abc", "abc" }, { "cba", "abc" }, }); } private final string input; private final string expected; public stringsorttest(final string input, final string expected) { this.input = input; this.expected = expected; } @test public void testsort() { assertequals(expected, mysortmethod(input)); } } this is pretty darn obnoxious sometimes if you have multiple sets of data for various tests, which all go through the constructor, which would force you to write multiple test classes. testng solves this better by allowing you to provide separate data sets to individual test methods using the @dataprovider annotation . but don’t worry, now you can achieve the same with the junit-dataprovider , available on github. pull in the dependency with e.g. maven. com.tngtech.java junit-dataprovider 1.5.0 test the above example now could be rewritten as: @runwith(dataproviderrunner.class) public class stringsorttest { @dataprovider public static object[][] data() { return new object[][] { { "abc", "abc" }, { "cba", "abc" }, }; } @test @usedataprovider("data") public void testsort(final string input, final string expected) { assertequals(expected, mysortmethod(input)); } } you’ll see: no constructor. in this example it doesn’t have many benefits, except maybe for less boiler-plate, but now you can create as many @dataprovider-annotated methods which are fed directly to your @usedataprovider-annotated testmethod(s). sidenote for eclipse: if you’re using that ide, the junit plugin is unable to map the names of the passed/failed testmethods to the ones in the testclass correctly. if a method fails, you’ll have to find it back manually in the testclass (or vote for the patch andreas smidt created to get this fixed in eclipse). in the mean time, if you’re stuck with junit and you’d love to use this feature you’re so accustomed to using with testng, go ahead and try the junit-dataprovider now.

This blog is more of a tutorial where we describe the development of a simple data access module, more for fun and learning than anything else. All code can be found here for those who don’t want to type along: https://github.com/ricston-git/tododb As a heads-up, we will be covering the following: Using Groovy in a Maven project within Eclipse Using Groovy to interact with our database Testing our code using the Spock framework We include Spring in our tests with ContextConfiguration A good place to start is to write a pom file as shown here. The only dependencies we want packaged with this artifact are groovy-all and commons-lang. The others are either going to be provided by Tomcat or are only used during testing (hence the scope tags in the pom). For example, we would put the jar with PostgreSQL driver in Tomcat’s lib, and tomcat-jdbc and tomcat-dbcp are already there. (Note: regarding the postgre jar, we would also have to do some minor configuration in Tomcat to define a DataSource which we can get in our app through JNDI – but that’s beyond the scope of this blog. See here for more info). Testing-wise, I’m depending on spring-test, spock-core, and spock-spring (the latter is to get spock to work with spring-test). Another significant addition in the pom is the maven-compiler-plugin. I have tried to get gmaven to work with Groovy in Eclipse, but I have found the maven-compiler-plugin to be a lot easier to work with. With your pom in an empty directory, go ahead and mkdir -p src/main/groovy src/main/java src/test/groovy src/test/java src/main/resources src/test/resources. This gives us a directory structure according to the Maven convention. Now you can go ahead and import the project as a Maven project in Eclipse (install the m2e plugin if you don’t already have it). It is important that you do not mvn eclipse:eclipse in your project. The .classpath it generates will conflict with your m2e plugin and (at least in my case), when you update your pom.xml the plugin will not update your dependencies inside Eclipse. So just import as a maven project once you have your pom.xml and directory structure set up. Okay, so our tests are going to be integration tests, actually using a PostgreSQL database. Since that’s the case, lets set up our database with some data. First go ahead and create a tododbtest database which will only be used for testing purposes. Next, put the following files in your src/test/resources: Note, fill in your username/password: DROP TABLE IF EXISTS todouser CASCADE; CREATE TABLE todouser ( id SERIAL, email varchar(80) UNIQUE NOT NULL, password varchar(80), registered boolean DEFAULT FALSE, confirmationCode varchar(280), CONSTRAINT todouser_pkey PRIMARY KEY (id) ); insert into todouser (email, password, registered, confirmationCode) values ('[email protected]', 'abc123', FALSE, 'abcdefg') insert into todouser (email, password, registered, confirmationCode) values ('[email protected]', 'pass1516', FALSE, '123456') insert into todouser (email, password, registered, confirmationCode) values ('[email protected]', 'anon', FALSE, 'codeA') insert into todouser (email, password, registered, confirmationCode) values ('[email protected]', 'anon2', FALSE, 'codeB') Basically, testContext.xml is what we’ll be configuring our test’s context with. The sub-division into datasource.xml and initdb.xml may be a little too much for this example… but changes are usually easier that way. The gist is that we configure our data source in datasource.xml (this is what we will be injecting in our tests), and the initdb.xml will run the schema.sql and test-data.sql to create our table and populate it with data. So lets create our test, or should I say, our specification. Spock is specification framework that allows us to write more descriptive tests. In general, it makes our tests easier to read and understand, and since we’ll be using Groovy, we might as well make use of the extra readability Spock gives us. package com.ricston.blog.sample.model.spec; import javax.sql.DataSource import org.springframework.beans.factory.annotation.Autowired import org.springframework.test.annotation.DirtiesContext import org.springframework.test.annotation.DirtiesContext.ClassMode import org.springframework.test.context.ContextConfiguration import spock.lang.Specification import com.ricston.blog.sample.model.data.TodoUser import com.ricston.blog.sample.model.dao.postgre.PostgreTodoUserDAO // because it supplies a new application context after each test, the initialize-database in initdb.xml is // executed for each test/specification @DirtiesContext(classMode=ClassMode.AFTER_EACH_TEST_METHOD) @ContextConfiguration('classpath:testContext.xml') class PostgreTodoUserDAOSpec extends Specification { @Autowired DataSource dataSource PostgreTodoUserDAO postgreTodoUserDAO def setup() { postgreTodoUserDAO = new PostgreTodoUserDAO(dataSource) } def "findTodoUserByEmail when user exists in db"() { given: "a db populated with a TodoUser with email [email protected] and the password given below" String email = '[email protected]' String password = 'anon' when: "searching for a TodoUser with that email" TodoUser user = postgreTodoUserDAO.findTodoUserByEmail email then: "the row is found such that the user returned by findTodoUserByEmail has the correct password" user.password == password } } One specification is enough for now, just to make sure that all the moving parts are working nicely together. The specification itself is easy enough to understand. We’re just exercising the findTodoUserByEmail method of PostgreTodoUserDAO – which we will be writing soon. Using the ContextConfiguration from Spring Test we are able to inject beans defined in our context (the dataSource in our case) through the use of annotations. This keeps our tests short and makes them easier to modify later on. Additionally, note the use of DirtiesContext. Basically, after each specification is executed, we cannot rely on the state of the database remaining intact. I am using DirtiesContext to get a new Spring context for each specification run. That way, the table creation and test data insertions happen all over again for each specification we run. Before we can run our specification, we need to create at least the following two classes used in the spec: TodoUser and PostgreTodoUserDAO package com.sample.data import org.apache.commons.lang.builder.ToStringBuilder class TodoUser { long id; String email; String password; String confirmationCode; boolean registered; @Override public String toString() { ToStringBuilder.reflectionToString(this); } } package com.ricston.blog.sample.model.dao.postgre import groovy.sql.Sql import javax.sql.DataSource import com.ricston.blog.sample.model.dao.TodoUserDAO import com.ricston.blog.sample.model.data.TodoUser class PostgreTodoUserDAO implements TodoUserDAO { private Sql sql public PostgreTodoUserDAO(DataSource dataSource) { sql = new Sql(dataSource) } /** * * @param email * @return the TodoUser with the given email */ public TodoUser findTodoUserByEmail(String email) { sql.firstRow """SELECT * FROM todouser WHERE email = $email""" } } package com.ricston.blog.sample.model.dao; import com.ricston.blog.sample.model.data.TodoUser; public interface TodoUserDAO { /** * * @param email * @return the TodoUser with the given email */ public TodoUser findTodoUserByEmail(String email); } We’re just creating a POGO in TodoUser, implementing its toString using common’s ToStringBuilder. In PostgreTodoUserDAO we’re using Groovy’s SQL to access the database, for now, only implementing the findTodoUserByEmail method. PostgreTodoUserDAO implements TodoUserDAO, an interface which specifies the required methods a TodoUserDAO must have. Okay, so now we have all we need to run our specification. Go ahead and run it as a JUnit test from Eclipse. You should get back the following error message: org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object '{id=3, [email protected], password=anon, registered=false, confirmationcode=codeA}' with class 'groovy.sql.GroovyRowResult' to class 'com.ricston.blog.sample.model.data.TodoUser' due to: org.codehaus.groovy.runtime.metaclass.MissingPropertyExceptionNoStack: No such property: confirmationcode for class: com.ricston.blog.sample.model.data.TodoUser Possible solutions: confirmationCode at com.ricston.blog.sample.model.dao.postgre.PostgreTodoUserDAO.findTodoUserByEmail(PostgreTodoUserDAO.groovy:23) at com.ricston.blog.sample.model.spec.PostgreTodoUserDAOSpec.findTodoUserByEmail when user exists in db(PostgreTodoUserDAOSpec.groovy:37) Go ahead and connect to your tododbtest database and select * from todouser; As you can see, our confirmationCode varchar(280), ended up as the column confirmationcode with a lower case ‘c’. In PostgreTodoUserDAO’s findTodoUserByEmail, we are getting back GroovyRowResult from our firstRow invocation. GroovyRowResult implements Map and Groovy is able to create a POGO (in our case TodoUser) from a Map. However, in order for Groovy to be able to automatically coerce the GroovyRowResult into a TodoUser, the keys in the Map (or GroovyRowResult) must match the property names in our POGO. We are using confirmationCode in our TodoUser, and we would like to stick to the camel case convention. What can we do to get around this? Well, first of all, lets change our schema to use confirmation_code. That’s a little more readable. Of course, we still have the same problem as before since confirmation_code will not map to confirmationCode by itself. (Note: remember to change the insert statements in test-data.sql too). One way to get around this is to use Groovy’s propertyMissing methods as show below: def propertyMissing(String name, value) { if(isConfirmationCode(name)) { this.confirmationCode = value } else { unknownProperty(name) } } def propertyMissing(String name) { if(isConfirmationCode(name)) { return confirmationCode } else { unknownProperty(name) } } private boolean isConfirmationCode(String name) { 'confirmation_code'.equals(name) } def unknownProperty(String name) { throw new MissingPropertyException(name, this.class) } By adding this to our TodoUser.groovy we are effectively tapping in on how Groovy resolves property access. When we do something like user.confirmationCode, Groovy automatically calls getConfirmationCode(), a method which we got for free when declared the property confirmationCode in our TodoUser. Now, when user.confirmation_code is invoked, Groovy doesn’t find any getters to invoke since we never declared the property confirmation_code, however, since we have now implemented the propertyMissing methods, before throwing any exceptions it will use those methods as a last resort when resolving properties. In our case we are effectively checking whether a get or set on confirmation_code is being made and mapping the respective operations to our confirmationCode property. It’s as simple as that. Now we can keep the auto coercion in our data access object and the property name we choose to have in our TodoUser. Assuming you’ve made the changes to the schema and test-data.sql to use confirmation_code, go ahead and run the spec file and this time it should pass. That’s it for this tutorial. In conclusion, I would like to discuss some finer points which someone who’s never used Groovy’s SQL before might not know. As you can see in PostgreTodoUserDAO.groovy, our database interaction is pretty much a one-liner. What about resource handling (e.g. properly closing the connection when we’re done), error logging, and prepared statements? Resource handling and error logging are done automatically, you just have to worry about writing your SQL. When you do write your SQL, try to stick to using triple quotes as used in the PostgreTodoUserDAO.groovy example. This produces prepared statements, therefore protecting against SQL injection and avoids us having to put ‘?’ all over the place and properly lining up the arguments to pass in to the SQL statement. Note that transaction management is something which the code using our artifact will have to take care of. Finally, note that a bunch of other operations (apart from findTodoUserByEmail) are implemented in the project on GitHub: https://github.com/ricston-git/tododb. Additionally, there is also a specification test for TodoUser, making sure that the property mapping works correctly. Also, in the pom.xml, there is some maven-surefire-plugin configuration in order to get the surefire-plugin to pick up our Spock specifications as well as any JUnit tests which we might have in our project. This allows us to run our specifications when we, for example, mvn clean package. After implementing all the operations you require in PostgreTodoUserDAO.groovy, you can go ahead and compile the jar or include in a Maven multi-module project to get a data access module you can use in other applications.

Android 4.4 has made a big change in the OS’ internals for HTML5 development: it has replaced its original WebKit-based WebView with modern Chromium. The new Android Browser is also powered by Chromium, but it’s not clear yet its future. Besides the good news, not everything looks exciting in these changes: let’s see why. Every web developer that has played with native webapps, PhoneGap and the Android’s WebView knows how terrible it was in terms of performance and HTML5 compatibility. These are the same problems that most web developers suffer right now with the Android Browser, which is reported to be 32% of the mobile web browsing market share, compared with just 5% of the modern Chrome for Android according to Akamai. I’ve been talking about this problem in a recent post this year: Android Browser: the eternal mobile browser. Therefore I’m the first one to celebrate the beginning of the end for this dying web platform and the Chrome team now in charge of Android’s web runtimes. Chroming Android From Android 4.4, Chromium 30 is the web engine for the WebView native widget, including the V8 JavaScript engine. Let’s start with good news: Support for remote debugging Support for new HTML5 features Better performance Now why we should take this change with moderated excitement: We will still deal with the old WebView for a couple of years. It won’t be upgraded without an OS upgrade There might be some compatibility issues Where is My Browser? Everybody at Android and Chrome team is talking about the new WebView but nobody is even mentioning what will happen to the browser. We all want Chrome as the default browser, but it seems it’s not there yet (licenses issues, I guess). I’ve even seen a couple of members of the Chrome team saying that the stock Android Browser didn’t exist in the latest previous versions, which is not true. From Google’s perspective, Android Browser sounds much like IE6 and nobody wants to talk about it. They give us the idea that Chrome has been powering web browsing in Android for a while, but that is only true for some particular Android devices - Nexuses and devices from top manufactures. However, as I’ve mentioned before, the relationship between users browsing with Android Browser and Chrome is still 7 to 1. Besides what some people believe, the previous version of Android, 4.3, included minor upgrades to the Browser, so it is there for sure. The question is: what will happen on 4.4 with the stock browser? We know that the Nexus 5 has Google Chrome by default; the question here is what will happen with other devices having in mind that average users don’t download browsers from the store and use what the devices offers for browsing. Based on the emulator, the Android Browser is still there on the emulator and it’s using the classic browser UI with the Chromium 30 engine (it can coexist with Chrome but they will be radically different) Unfortunately, there is no mention of this on docs and blogs on Android 4.4. I hope we can get a real answer from the Android team soon about the future of the browser itself. The Good News Remote debugging Finally we have the ability to debug remotely Android native webviews, including PhoneGap apps, and the Android Browser works smoothly both from real devices and from the emulator. When we have an Android app opened with a web view or the Android Browser, the Chrome remote debugger tools will recognize it as a “Chrome 30” session and we have the full package of excellent tools for debug, profile and test our webapps. HTML5 new features Compared with the classic web view and the Android Browser until 4.3, we now have support for: Server Sent events Web Sockets Web Workers Advanced form input selectors, such as date and time FileSystem API IndexedDB MediaCapture Stream ??? test Animation Timing API Page Visibility API Canvas Blend modes CSS3 Flexbox (latest version) CSS3 Filters Even matching Chrome 30 for Android, the Web View (and potentially the Android Browser) will not have support (no reasons given) for: WebGL WebRTC WebAudio FullScreen Form validation Compared with the classic Web View, the new one doesn’t have Network Information API Performance difference Having V8 as the JavaScript engine for the new web view, the JavaScript performance if much better, besides general performance on CSS thanks to hardware acceleration. The Not so Good News The Classic Web View is still alive Don’t get so excited. We will deal with the old Web View (known as “classic”) for a couple of years. In fact, some devices such as Galaxy Nexus that are today on 4.3 will not get the update. And remember that still today 30% of Android users are on 2.x after 2 years of being replaced by 4.0, so it’s fair to guess that at the beginning of 2016 we will still have around a third of the users on the “classic” WebView that we hate today. The migration on the market will be slow based on Android’s fragmentation. WebView upgrade The KitKat WebView is based on Chromium 30 and it won’t be updated. That means you are stuck with it unless to get an upgrade in the future of the whole OS to next version. Even Google has announced OS delta updates without vendors’ intervention, but it seems the WebView will not get that deal yet. Therefore and based on Chrome's release cycle, in one year we will have Chrome 40 and the WebView will still be in 30. In a couple of years we might be complaining about an “old and outdated” webview again Compatibility issues Because there are changes between the old WebKit-based rendering engine and the modern Chromium engine, you should test your native webapp on KitKat to make sure it’s still working great. To reduce problems, if our app was packaged before KitKat the WebView will enter a “quirks mode” (any similarity with IE6 is pure coincidence) that will reduce the risk of incompatibilities while still getting the new APIs. In fact, this compatibility mode will get in action if the configuration file of your app has a target SDK lower than 19 (the API number for KitKat). To get more detailed information on migration and compatibility issues you can try the new Guides at Android and Chrome websites: http://developer.android.com/guide/webapps/migrating.html http://developers.google.com/chrome/mobile/docs/webview Looking Forward I’m really looking forward to remove the old WebKit and Android Browser from the market. The Chrome team is doing a great job empowering the mobile web (just remember homescreen webapps from Chrome 31), but sometimes the Android ecosystem is slowing down HTML5 penetration and helping promoting companies to avoid using web technologies. I hope this is the beginning of a change.

Apache Shiro can be used in a wide range of applications as part of the Java Security Framework.

Publisher confirms are a RabbitMQ addition to AMQP to guarantee message delivery. You can read all about them here and here. In short they provide a asynchronous confirmation that a publish has successfully reached all the queues that it was routed to. To turn on publisher confirms with EasyNetQ set the publisherConfirms connection string parameter like this: var bus = RabbitHutch.CreateBus("host=localhost;publisherConfirms=true"); When you set this flag, EasyNetQ will wait for the confirmation, or a timeout, before returning from the Publish method: bus.Publish(new MyMessage { Text = "Hello World!" }); // here the publish has been confirmed. Nice and easy. There’s a problem though. If I run the above code in a while loop without publisher confirms, I can publish around 4000 messages per second, but with publisher confirms switched on that drops to around 140 per second. Not so good. With EasyNetQ 0.15 we introduced a new PublishAsync method that returns a Task. The Task completes when the publish is confirmed: bus.PublishAsync(message).ContinueWith(task => { if (task.IsCompleted) { Console.WriteLine("Publish completed fine."); } if (task.IsFaulted) { Console.WriteLine(task.Exception); } }); Using this code in a while loop gets us back to 4000 messages per second with publisher confirms on. Happy confirms!

One piece to Docker that is interesting AMAZING is the Remote API that can be used to programatically interact with docker. I recently had a situation where I wanted to run many containers on a host with a single container managing the other containers through the API. But the problem I soon discovered is that at the moment when you turn networking on it is an all or nothing type of thing… you can’t turn networking off selectively on a container by container basis. You can disable IPv4 forwarding, but you can still reach the docker remote API on the machine if you can guess the IP address of it. One solution I came up with for this is to use nginx to expose the unix socket for docker over HTTPS and utilize client-side ssl certificates to only allow trusted containers to have access. I liked this setup a lot so I thought I would share how it’s done. Disclaimer: assumes some knowledge of docker! Generate The SSL Certificates We’ll use openssl to generate and self-sign the certs. Since this is for an internal service we’ll just sign it ourselves. We also remove the password from the keys so that we aren’t prompted for it each time we start nginx. # Create the CA Key and Certificate for signing Client Certs openssl genrsa -des3 -out ca.key 4096 openssl rsa -in ca.key -out ca.key # remove password! openssl req -new -x509 -days 365 -key ca.key -out ca.crt # Create the Server Key, CSR, and Certificate openssl genrsa -des3 -out server.key 1024 openssl rsa -in server.key -out server.key # remove password! openssl req -new -key server.key -out server.csr # We're self signing our own server cert here. This is a no-no in production. openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt # Create the Client Key and CSR openssl genrsa -des3 -out client.key 1024 openssl rsa -in client.key -out client.key # no password! openssl req -new -key client.key -out client.csr # Sign the client certificate with our CA cert. Unlike signing our own server cert, this is what we want to do. openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt Another option may be to leave the passphrase in and provide it as an environment variable when running a docker container or through some other means as an extra layer of security. We’ll move ca.crt, server.key and server.crt to /etc/nginx/certs. Setup Nginx The nginx setup for this is pretty straightforward. We just listen for traffic on localhost on port 4242. We require client-side ssl certificate validation and reference the certificates we generated in the previous step. And most important of all, set up an upstream proxy to the docker unix socket. I simply overwrote what was already in /etc/nginx/sites-enabled/default. upstream docker { server unix:/var/run/docker.sock fail_timeout=0; } server { listen 4242; server localhost; ssl on; ssl_certificate /etc/nginx/certs/server.crt; ssl_certificate_key /etc/nginx/certs/server.key; ssl_client_certificate /etc/nginx/certs/ca.crt; ssl_verify_client on; access_log on; error_log /dev/null; location / { proxy_pass http://docker; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 90; proxy_send_timeout 120; proxy_read_timeout 120; proxy_buffer_size 4k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k; } } One important piece to make this work is you should add the user nginx runs as to the docker group so that it can read from the socket. This could be www-data, nginx, or something else! Hack It Up! With this setup and nginx restarted, let’s first run a curl command to make sure that this setup correctly. First we’ll make a call without the client cert to double check that we get denied access then a proper one. # Is normal http traffic denied? curl -v http://localhost:4242/info # How about https, sans client cert and key? curl -v -s -k https://localhost:4242/info # And the final good request! curl -v -s -k --key client.key --cert client.crt https://localhost:4242/info For the first two we should get some run of the mill 400 http response codes before we get a proper JSON response from the final command! Woot! But wait there’s more… let’s build a container that can call the service to launch other containers! For this example we’ll simply build two containers: one that has the client certificate and key and one that doesn’t. The code for these examples are pretty straightforward and to save space I’ll leave the untrusted container out. You can view the untrusted container on github (although it is nothing exciting). First, the node.js application that will connect and display information: https = require 'https' fs = require 'fs' options = host: 172.42.1.62 port: 4242 method: 'GET' path: '/containers/json' key: fs.readFileSync('ssl/client.key') cert: fs.readFileSync('ssl/client.crt') headers: { 'Accept': 'application/json'} # not required, but being semantic here! req = https.request options, (res) -> console.log res req.end() And the Dockerfile used to build the container. Notice we add the client.crt and client.key as part of building it! FROM shykes/nodejs MAINTAINER James R. Carr ADD ssl/client* /srv/app/ssl ADD package.json /srv/app/package.json ADD app.coffee /srv/app/app.coffee RUN cd /srv/app && npm install . CMD cd /srv/app && npm start That’s about it. Run docker build . and docker run -n >IMAGE ID< and we should see a json dump to the console of the actively running containers. Doing the same in the untrusted directory should present us with some 400 error about not providing a client ssl certificate. I’ve shared a project with all this code plus a vagrant file on github for your own prusual. Enjoy!

Since git hooks can be any executable script with an appropriate #! line, Python is more than suitable for writing your git hooks. Simply stated, git hooks are scripts which are called at different points of time in the life cycle of working with your git repository. Let’s start by creating a new git repository: ~/work> git init git-hooks-exp Initialized empty Git repository in /home/gene/work/git-hooks-exp/.git/ ~/work> cd git-hooks-exp/ ~/work/git-hooks-exp (master)> tree -al .git/ .git/ ├── branches ├── config ├── description ├── HEAD ├── hooks │ ├── applypatch-msg.sample │ ├── commit-msg.sample │ ├── post-update.sample │ ├── pre-applypatch.sample │ ├── pre-commit.sample │ ├── prepare-commit-msg.sample │ ├── pre-rebase.sample │ └── update.sample ├── info │ └── exclude ├── objects │ ├── info │ └── pack └── refs ├── heads └── tags 9 directories, 12 files Inside the .git are a number of directories and files, one of them being hooks/ which is where the hooks live. By default, you will have a number of hooks with the file names ending in .sample. They may be useful as starting points for your own scripts. However, since they all have an extension .sample, none of the hooks are actually activated. For a hook to be activated, it must have the right file name and it should be executable. Let’s see how we can write a hook using Python. We will write a post-commit hook. This hook is called immediately after you have made a commit. We are going to do something fairly useless, but quite interesting in this hook. We will take the commit SHA1 of this commit, and print how it may look like in a more human form. I do the latter using the humanhash module. You will need to have it installed. Here is how the hook looks like: #!/usr/bin/python import subprocess import humanhash # get the last commit SHA and print it after humanizing it # https://github.com/zacharyvoase/humanhash print humanhash.humanize( subprocess.check_output( ['git','rev-parse','HEAD'])) I use the subprocess.check_output() function to execute the command git rev-parse HEAD so that I can get the commit SHA1 and then call the humanhash.humanize() function with it. Save the hook as a file, post-commit in your hooks/ directory and make it executable using chmod +x .git/hooks/post-commit. Let’s see the hook in action: ~/work/git-hooks-exp (master)> touch file ~/work/git-hooks-exp (master)> git add file ~/work/git-hooks-exp (master)> git commit -m "Added a file" carbon-network-connecticut-equal [master (root-commit) 2d7880b] Added a file 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 file The commit SHA1 for the commit turned out to be 2d7880be746a1c1e75844fc1aa161e2b8d955427. Let’s check it with the humanize function and check if we get the same message as above: >>> humanhash.humanize('2d7880be746a1c1e75844fc1aa161e2b8d955427') 'carbon-network-connecticut-equal' And you can see the same message above as well. For some of the hooks, you will see that they are called with some parameters. In Python you can access them using the sys.argv attribute from the sys module, with the first member being the name of the hook of course and the others will be the parameters that the hook is called with.

This blog post demonstrates how easy it is to use Apache Camel and its new json-path component along with the camel-sqs component to produce and consume messages on Amazon SQS. Amazon Web Services SQS is a message queuing “software as a service” (SaaS) in the cloud. To be able to use it, you need to sign up for AWS. It’s primary access mechanism is XML over HTTP through various AWS SDK clients provided by Amazon. Please check out the SQS documentation for more. And as “luck” would have it, one of the users in the Apache Camel community created a component to be able to integrate with SQS. This makes it trivial to add a producer or consumer to an SQS queue and plugs in nicely with the Camel DSL. SQS, however, is not a “one-size fits all” queueing service; you must be aware of your use case and make sure it fits (current requirements as well as somewhat into the future…). There are limitations that, if not studied and accounted for ahead of time, could come back to sink your project. An example of a viable alternative, and one that more closely fits the profile of a high performance and full featured message queue is Apache ActiveMQ. For example, one limitation to keep in mind is that unlike traditional JMS consumers, you cannot create a subscription to a queue that filters messages based on some predicate (at least not using the AWS-SQS API — you’d have to build that into your solution). Some other things to keep in mind when using SQS: The queue does not preserve FIFO messaging That is, message order is not preserved. They can arrive out of order from when they were sent. Apache Camel can help with its resequencer pattern. Bilgin Ibryam, now a colleague of mine at Red Hat, has written a great blog post about how to restore message order using the resequencer pattern. Message size is limited to 256K This is probably sufficient, but if your message sizes are variable, or contain more data that 256K, you will have to chunk them and send in smaller chunks. No selector or selective consumption If you’re familiar with JMS, you know that you can specify consumers to use a “selector” or a predicate expression that is evaluated on the broker side to determine whether or not a specific message should be dispatched to a specific consumer. For example, Durability constraints Some use cases call for the message broker to store messages until consumers return. SQS allows a limit of up to 14 days. This is most likely sufficient, but something to keep in mind. Binary payloads not allowed SQS only allows text-based messages, e.g., XML, JSON, fixed format text, etc. Binary such as Avro, Protocol Buffers, or Thrift are not allowed. For some of these limitations, you can work around them by building out the functionality yourself. I would always recommend taking a look at how an integration library like Apache Camel can help — which has out-of-the-box support for doing some of these things. Doing JMS-style selectors So the basic problem is we want to subscribe to a SQS queue, but we want to filter which messages we process. For those messages that we do not process, those should be left in the queue. To do this, we will make use of Apache Camel’s Filter EIP as well as the visibility timeouts available on the SQS queue. By default, SQS will dispatch all messages in its queue when it’s queried. We cannot change this, and thus not avoid the message being dispatched to us — we’ll have to do the filtering on our side (this is different than how a full-featured broker like ActiveMQ does it, i.e., filtering is done on the broker side so the consumer doesn’t even see the message it does not want to see). Once SQS dispatches a message, it does not remove it from the queue unless the consumer has acknowledged that it has it and is finished with it. The consumer does this by sending a DeleteMessage command. Until the DeleteMessage command is sent, the message is always in the queue, however visibility comes in to play here. When a message is dispatched to a consumer, there is a period of time which it will not be visible to other consumers. So if you browsed the queue, you would not see it (it should appear in the stats as “in-flight”). However, there is a configurable period of time you can specify for how long this “visibility timeout” should be active. So if you set the visibility to a lower time period (default is 30 seconds), you can more quickly get messages re-dispatched to consumers that would be able to handle the message. Take a look at the following Camel route which does just that: @Override public void configure() throws Exception { // every two seconds, send a message to the "demo" queue in SQS from("timer:kickoff?period=5000") .setBody().method(this, "generateJsonString") .to("aws-sqs://demo?amazonSQSClient=#sqsClient&defaultVisibilityTimeout=2"); } In the above Camel Route, we create a new message every 5 seconds and send it to an SQS queue named demo — note we set the defaultVisibilityTimeout to 2 seconds. This means that after a message gets dispatched to a consumer, SQS will wait about 2 seconds before considering it eligible to be dispatched to another consumer if it has not been deleted. On the consumer side, we take advantage of a couple Apache Camel conveniences Using JSON Path + Filter EIP Camel has an excellent new component named JSON-Path. Claus Ibsen tweeted about it when he hacked it up. This allows you to do Content-Based Routing on a JSON payload very easily by using XPath-style expressions to pick out and evaluate attributes in a JSON encoded object. So in the following example, we can test an attribute named ‘type’ to be equal to ‘LOGIN’ and use Camel’s Filter EIP to allow only those messages that match to go through and continue processing: public class ConsumerRouteBuilder extends RouteBuilder { @Override public void configure() throws Exception { from("aws-sqs://demo?amazonSQSClient=#sqsClient&deleteIfFiltered=false") .setHeader("identity").jsonpath("$['type']") .filter(simple("${header.identity} == 'login'")) .log("We have a message! ${body}") .to("file:target/output?fileName=login-message-${date:now:MMDDyy-HHmmss}.json"); } } To complete the functionality, we have to pay attention to a new configuration option added for the Camel-SQS component: deleteIfFiltered — Whether or not to send the DeleteMessage to the SQS queue if an exchange fails to get through a filter. If ‘false’ and exchange does not make it through a Camel filter upstream in the route, then don’t send DeleteMessage. By default, Camel will send the “DeleteMessage” command to SQS after a route has completed successfully (without an exception). However, in this case, we are specifying to not send the DeleteMessage command if the message had been previously filtered by Camel. This example demonstrates how easy it is to use Apache Camel and its new json-path component along with the camel-sqs component to produce and consume messages on Amazon SQS. Please take a look at the source code on my github repo to play with the live code and try it out yourself.

The examples in this post were updated in September to work with the current version of the Windows Azure Storage REST API. In the Windows Azure MSDN Azure Forum there are occasional questions about the Windows Azure Storage Services REST API. I have occasionally responded to these with some code examples showing how to use the API. I thought it would be useful to provide some examples of using the REST API for tables, blobs and queues – if only so I don’t have to dredge up examples when people ask how to use it. This post is not intended to provide a complete description of the REST API. The REST API is comprehensively documented (other than the lack of working examples). Since the REST API is the definitive way to address Windows Azure Storage Services I think people using the higher level Storage Client API should have a passing understanding of the REST API to the level of being able to understand the documentation. Understanding the REST API can provide a deeper understanding of why the Storage Client API behaves the way it does. Fiddler The Fiddler Web Debugging Proxy is an essential tool when developing using the REST (or Storage Client) API since it captures precisely what is sent over the wire to the Windows Azure Storage Services. Authorization Nearly every request to the Windows Azure Storage Services must be authenticated. The exception is access to blobs with public read access. The supported authentication schemes for blobs, queues and tables and these are described here. The requests must be accompanied by an Authorization header constructed by making a hash-based message authentication code using the SHA-256 hash. The following is an example of performing the SHA-256 hash for the Authorization header: public static String CreateAuthorizationHeader(String canonicalizedString) { String signature = String.Empty; using (HMACSHA256 hmacSha256 = new HMACSHA256( Convert.FromBase64String(storageAccountKey) )) { Byte[] dataToHmac = System.Text.Encoding.UTF8.GetBytes(canonicalizedString); signature = Convert.ToBase64String(hmacSha256.ComputeHash(dataToHmac)); } String authorizationHeader = String.Format( CultureInfo.InvariantCulture, "{0} {1}:{2}", AzureStorageConstants.SharedKeyAuthorizationScheme, AzureStorageConstants.Account, signature ); return authorizationHeader; } This method is used in all the examples in this post. AzureStorageConstants is a helper class containing various constants. Key is a secret key for Windows Azure Storage Services account specified by Account. In the examples given here, SharedKeyAuthorizationScheme is SharedKey. The trickiest part in using the REST API successfully is getting the correct string to sign. Fortunately, in the event of an authentication failure the Blob Service and Queue Service responds with the authorization string they used and this can be compared with the authorization string used in generating the Authorization header. This has greatly simplified the us of the REST API. Table Service API The Table Service API supports the following table-level operations: Create Table Delete Table Query Tables The Table Service API supports the following entity-level operations: Delete Entity Insert Entity Merge Entity Update Entity Query Entities These operations are implemented using the appropriate HTTP VERB: DELETE – delete GET – query MERGE – merge POST – insert PUT – update This section provides examples of the Insert Entity and Query Entities operations. Insert Entity The InsertEntity() method listed in this section inserts an entity with two String properties, Artist and Title, into a table. The entity is submitted as an ATOM entry in the body of a request POSTed to the Table Service. In this example, the ATOM entry is generated by the GetRequestContentInsertXml() method. The date must be in RFC 1123 format in the x-ms-date header supplied to the canonicalized resource used to create the Authorization string. Note that the storage service version is set to “2012-02-12″ which requires the DataServiceVersion and MaxDataServiceVersion to be set appropriately. public void InsertEntity(String tableName, String artist, String title) { String requestMethod = "POST"; String urlPath = tableName; String storageServiceVersion = "2012-02-12"; String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture); String contentMD5 = String.Empty; String contentType = "application/atom+xml"; String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath); String stringToSign = String.Format( "{0}\n{1}\n{2}\n{3}\n{4}", requestMethod, contentMD5, contentType, dateInRfc1123Format, canonicalizedResource); String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign); UTF8Encoding utf8Encoding = new UTF8Encoding(); Byte[] content = utf8Encoding.GetBytes(GetRequestContentInsertXml(artist, title)); Uri uri = new Uri(AzureStorageConstants.TableEndPoint + urlPath); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Accept = "application/atom+xml,application/xml"; request.ContentLength = content.Length; request.ContentType = contentType; request.Method = requestMethod; request.Headers.Add("x-ms-date", dateInRfc1123Format); request.Headers.Add("x-ms-version", storageServiceVersion); request.Headers.Add("Authorization", authorizationHeader); request.Headers.Add("Accept-Charset", "UTF-8"); request.Headers.Add("DataServiceVersion", "2.0;NetFx"); request.Headers.Add("MaxDataServiceVersion", "2.0;NetFx"); using (Stream requestStream = request.GetRequestStream()) { requestStream.Write(content, 0, content.Length); } using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { Stream dataStream = response.GetResponseStream(); using (StreamReader reader = new StreamReader(dataStream)) { String responseFromServer = reader.ReadToEnd(); } } } private String GetRequestContentInsertXml(String artist, String title) { String defaultNameSpace = "http://www.w3.org/2005/Atom"; String dataservicesNameSpace = "http://schemas.microsoft.com/ado/2007/08/dataservices"; String metadataNameSpace = "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"; XmlWriterSettings xmlWriterSettings = new XmlWriterSettings(); xmlWriterSettings.OmitXmlDeclaration = false; xmlWriterSettings.Encoding = Encoding.UTF8; StringBuilder entry = new StringBuilder(); using (XmlWriter xmlWriter = XmlWriter.Create(entry)) { xmlWriter.WriteProcessingInstruction("xml", "version=\"1.0\" encoding=\"UTF-8\""); xmlWriter.WriteWhitespace("\n"); xmlWriter.WriteStartElement("entry", defaultNameSpace); xmlWriter.WriteAttributeString("xmlns", "d", null, dataservicesNameSpace); xmlWriter.WriteAttributeString("xmlns", "m", null, metadataNameSpace); xmlWriter.WriteElementString("title", null); xmlWriter.WriteElementString("updated", String.Format("{0:o}", DateTime.UtcNow)); xmlWriter.WriteStartElement("author"); xmlWriter.WriteElementString("name", null); xmlWriter.WriteEndElement(); xmlWriter.WriteElementString("id", null); xmlWriter.WriteStartElement("content"); xmlWriter.WriteAttributeString("type", "application/xml"); xmlWriter.WriteStartElement("properties", metadataNameSpace); xmlWriter.WriteElementString("PartitionKey", dataservicesNameSpace, artist); xmlWriter.WriteElementString("RowKey", dataservicesNameSpace, title); xmlWriter.WriteElementString("Artist", dataservicesNameSpace, artist); xmlWriter.WriteElementString("Title", dataservicesNameSpace, title + "\n" + title); xmlWriter.WriteEndElement(); xmlWriter.WriteEndElement(); xmlWriter.WriteEndElement(); xmlWriter.Close(); } String requestContent = entry.ToString(); return requestContent; } This generates the following request (as captured by Fiddler): POST https://STORAGE_ACCOUNT.table.core.windows.net/authors HTTP/1.1 Accept: application/atom+xml,application/xml Content-Type: application/atom+xml x-ms-date: Sun, 08 Sep 2013 06:31:12 GMT x-ms-version: 2012-02-12 Authorization: SharedKey STORAGE_ACCOUNT:w7Uu4wHZx4fFwa2bsxd/TJVZZ1AqMPwxvW+pYtoWHd0= Accept-Charset: UTF-8 DataServiceVersion: 2.0;NetFx MaxDataServiceVersion: 2.0;NetFx Host: STORAGE_ACCOUNT.table.core.windows.net Content-Length: 514 Expect: 100-continue Connection: Keep-Alive The body of the request is: 2013-09-08T07:19:07Z Beckett Molloy 2013-09-08T07:19:07.2189243Z Beckett Molloy Molloy Note that I should have URLEncoded the PartitionKey and RowKey but did not do so for simplicity. There are, in fact, some issues with the URL encoding of spaces and other symbols. Get Entity The GetEntity() method described in this section retrieves the single entity inserted in the previous section. The particular entity to be retrieved is identified directly in the URL. public void GetEntity(String tableName, String partitionKey, String rowKey) { String requestMethod = "GET"; String urlPath = String.Format("{0}(PartitionKey='{1}',RowKey='{2}')", tableName, partitionKey, rowKey); String storageServiceVersion = "2012-02-12"; String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture); String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath); String stringToSign = String.Format( "{0}\n\n\n{1}\n{2}", requestMethod, dateInRfc1123Format, canonicalizedResource); String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign); Uri uri = new Uri(AzureStorageConstants.TableEndPoint + urlPath); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = requestMethod; request.Headers.Add("x-ms-date", dateInRfc1123Format); request.Headers.Add("x-ms-version", storageServiceVersion); request.Headers.Add("Authorization", authorizationHeader); request.Headers.Add("Accept-Charset", "UTF-8"); request.Accept = "application/atom+xml,application/xml"; request.Headers.Add("DataServiceVersion", "2.0;NetFx"); request.Headers.Add("MaxDataServiceVersion", "2.0;NetFx"); using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { Stream dataStream = response.GetResponseStream(); using (StreamReader reader = new StreamReader(dataStream)) { String responseFromServer = reader.ReadToEnd(); } } } This generates the following request (as captured by Fiddler): GET https://STORAGE_ACCOUNT.table.core.windows.net/authors(PartitionKey='Beckett',RowKey='Molloy') HTTP/1.1 x-ms-date: Sun, 08 Sep 2013 06:31:14 GMT x-ms-version: 2012-02-12 Authorization: SharedKey STORAGE_ACCOUNT:1hWbr4aNq4JWCpNJY3rsLH1SkIyeFTJflbqyKMPQ1Gk= Accept-Charset: UTF-8 Accept: application/atom+xml,application/xml DataServiceVersion: 2.0;NetFx MaxDataServiceVersion: 2.0;NetFx Host: STORAGE_ACCOUNT.table.core.windows.net The Table Service generates the following response: HTTP/1.1 200 OK Cache-Control: no-cache Content-Type: application/atom+xml;charset=utf-8 ETag: W/"datetime'2013-09-08T06%3A31%3A14.1579056Z'" Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0 x-ms-request-id: f4bd4c77-6fb6-42a8-8dff-81ea8d28fa2e x-ms-version: 2012-02-12 Date: Sun, 08 Sep 2013 06:31:15 GMT Content-Length: 1108 The returned entities, in this case a single entity, are returned in ATOM entry format in the response body: https://STORAGE_ACCOUNT.table.core.windows.net/authors(PartitionKey='Beckett',RowKey='Molloy') 2013-09-08T06:31:15Z Beckett Molloy 2013-09-08T06:31:14.1579056Z Beckett Molloy Molloy Blob Service API The Blob Service API supports the following account-level operation: List Containers The Blob Service API supports the following container-level operation: Create Container Delete Container Get Container ACL Get Container Properties Get Container Metadata List Blobs Set Container ACL Set Container Metadata The Blob Service API supports the following blob-level operation: Copy Blob Delete Blob Get Blob Get Blob Metadata Get Blob Properties Lease Blob Put Blob Set Blob Metadata Set Blob Properties Snapshot Blob The Blob Service API supports the following operations on block blobs: Get Block List Put Block Put Block List The Blob Service API supports the following operations on page blobs: Get Page Regions Put Page This section provides examples of the Put Blob and Lease Blob operations. Put Blob The Blob Service and Queue Service use a different form of shared-key authentication from the Table Service so care should be taken in creating the string to be signed for authorization. The blob type, BlockBlob or PageBlob, must be specified as a request header and consequently appears in the authorization string. public void PutBlob(String containerName, String blobName) { String requestMethod = "PUT"; String urlPath = String.Format("{0}/{1}", containerName, blobName); String storageServiceVersion = "2012-02-12"; String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture); String content = "Andrew Carnegie was born in Dunfermline"; UTF8Encoding utf8Encoding = new UTF8Encoding(); Byte[] blobContent = utf8Encoding.GetBytes(content); Int32 blobLength = blobContent.Length; const String blobType = "BlockBlob"; String canonicalizedHeaders = String.Format( "x-ms-blob-type:{0}\nx-ms-date:{1}\nx-ms-version:{2}", blobType, dateInRfc1123Format, storageServiceVersion); String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath); String stringToSign = String.Format( "{0}\n\n\n{1}\n\n\n\n\n\n\n\n\n{2}\n{3}", requestMethod, blobLength, canonicalizedHeaders, canonicalizedResource); String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign); Uri uri = new Uri(AzureStorageConstants.BlobEndPoint + urlPath); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = requestMethod; request.Headers.Add("x-ms-blob-type", blobType); request.Headers.Add("x-ms-date", dateInRfc1123Format); request.Headers.Add("x-ms-version", storageServiceVersion); request.Headers.Add("Authorization", authorizationHeader); request.ContentLength = blobLength; using (Stream requestStream = request.GetRequestStream()) { requestStream.Write(blobContent, 0, blobLength); } using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { String ETag = response.Headers["ETag"]; } } This generates the following request: PUT https://STORAGE_ACCOUNT.blob.core.windows.net/fife/dunfermline HTTP/1.1 x-ms-blob-type: BlockBlob x-ms-date: Sun, 08 Sep 2013 06:28:29 GMT x-ms-version: 2012-02-12 Authorization: SharedKey STORAGE_ACCOUNT:ntvh/lamVmikvwHhy6vRVBIh87kibkPlEOiHyLDia6g= Host: STORAGE_ACCOUNT.blob.core.windows.net Content-Length: 39 Expect: 100-continue Connection: Keep-Alive The body of the request is: Andrew Carnegie was born in Dunfermline The Blob Service generates the following response: HTTP/1.1 201 Created Transfer-Encoding: chunked Content-MD5: RYJnWGXLyt94l5jG82LjBw== Last-Modified: Sun, 08 Sep 2013 06:28:31 GMT ETag: "0x8D07A73C5704A86" Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0 x-ms-request-id: b74ef0a2-294d-4581-b8f1-6cda724bbdbf x-ms-version: 2012-02-12 Date: Sun, 08 Sep 2013 06:28:30 GMT Lease Blob The Blob Service allows a user to lease a blob for a minute at a time and so acquire a write lock on it. The use case for this is the locking of a page blob used to store the VHD backing an writeable Azure Drive. The LeaseBlob() example in this section demonstrates a subtle issue with the creation of authorization strings. The URL has a query string, comp=lease. Rather than using this directly in creating the authorization string it must be converted into comp:lease with a colon replacing the equal symbol – see modifiedURL in the example. Furthermore, the Lease Blob operation requires the use of an x-ms-lease-action to indicate whether the lease is being acquired, renewed, released or broken. public void LeaseBlob(String containerName, String blobName) { String requestMethod = "PUT"; String urlPath = String.Format("{0}/{1}?comp=lease", containerName, blobName); String modifiedUrlPath = String.Format("{0}/{1}\ncomp:lease", containerName, blobName); const Int32 contentLength = 0; String storageServiceVersion = "2012-02-12"; String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture); String leaseAction = "acquire"; String leaseDuration = "60"; String canonicalizedHeaders = String.Format( "x-ms-date:{0}\nx-ms-lease-action:{1}\nx-ms-lease-duration:{2}\nx-ms-version:{3}", dateInRfc1123Format, leaseAction, leaseDuration, storageServiceVersion); String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, modifiedUrlPath); String stringToSign = String.Format( "{0}\n\n\n{1}\n\n\n\n\n\n\n\n\n{2}\n{3}", requestMethod, contentLength, canonicalizedHeaders, canonicalizedResource); String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign); Uri uri = new Uri(AzureStorageConstants.BlobEndPoint + urlPath); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = requestMethod; request.Headers.Add("x-ms-date", dateInRfc1123Format); request.Headers.Add("x-ms-lease-action", leaseAction); request.Headers.Add("x-ms-lease-duration", leaseDuration); request.Headers.Add("x-ms-version", storageServiceVersion); request.Headers.Add("Authorization", authorizationHeader); request.ContentLength = contentLength; using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { String leaseId = response.Headers["x-ms-lease-id"]; } } This generates the following request: PUT https://STORAGE_ACCOUNT.blob.core.windows.net/fife/dunfermline?comp=lease HTTP/1.1 x-ms-date: Sun, 08 Sep 2013 06:28:31 GMT x-ms-lease-action: acquire x-ms-lease-duration: 60 x-ms-version: 2012-02-12 Authorization: SharedKey rebus:+SQ5+RFZg3hUaws5XCRHxsDgXb1ycdRIz5EKyHJWP7s= Host: rebus.blob.core.windows.net Content-Length: 0 The Blob Service generates the following response: HTTP/1.1 201 Created Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0 x-ms-request-id: 4b6ff77f-f885-4f74-803a-c92920d225c3 x-ms-version: 2012-02-12 x-ms-lease-id: b1320c2c-65ad-41d6-a7bd-85a4242c0ac5 Date: Sun, 08 Sep 2013 06:28:31 GMT Content-Length: 0 Queue Service API The Queue Service API supports the following queue-level operation: List Queues The Queue Service API supports the following queue-level operation: Create Queue Delete Queue Get Queue Metadata Set Queue Metadata The Queue Service API supports the following message-level operations: Clear Messages Delete Message Get Messages Peek Messages Put Message This section provides examples of the Put Message and Get Message operations. Put Message The most obvious curiosity about Put Message is that it uses the HTTP verb POST rather than PUT. The issue is presumably the interaction of the English language and the HTTP standard which states that PUT should be idempotent and that the Put Message operation is clearly not since each invocation merely adds another message to the queue. Regardless, it did catch me out when I failed to read the documentation well enough – so take that as a warning. The content of a message posted to the queue must be formatted in a specified XML schema and must then be UTF8 encoded. public void PutMessage(String queueName, String message) { String requestMethod = "POST"; String urlPath = String.Format("{0}/messages", queueName); String storageServiceVersion = "2012-02-12"; String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture); String messageText = String.Format( "{0}", message); UTF8Encoding utf8Encoding = new UTF8Encoding(); Byte[] messageContent = utf8Encoding.GetBytes(messageText); Int32 messageLength = messageContent.Length; String canonicalizedHeaders = String.Format( "x-ms-date:{0}\nx-ms-version:{1}", dateInRfc1123Format, storageServiceVersion); String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath); String stringToSign = String.Format( "{0}\n\n\n{1}\n\n\n\n\n\n\n\n\n{2}\n{3}", requestMethod, messageLength, canonicalizedHeaders, canonicalizedResource); String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign); Uri uri = new Uri(AzureStorageConstants.QueueEndPoint + urlPath); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = requestMethod; request.Headers.Add("x-ms-date", dateInRfc1123Format); request.Headers.Add("x-ms-version", storageServiceVersion); request.Headers.Add("Authorization", authorizationHeader); request.ContentLength = messageLength; using (Stream requestStream = request.GetRequestStream()) { requestStream.Write(messageContent, 0, messageLength); } using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { String requestId = response.Headers["x-ms-request-id"]; } } This generates the following request: POST https://rebus.queue.core.windows.net/revolution/messages HTTP/1.1 x-ms-date: Sun, 08 Sep 2013 06:34:08 GMT x-ms-version: 2012-02-12 Authorization: SharedKey rebus:nyASTVWifnxHKnj2wXwuzzzXz5CxUBZj58SToV5QFK8= Host: rebus.queue.core.windows.net Content-Length: 76 Expect: 100-continue Connection: Keep-Alive The body of the request is: Saturday in the cafe The Queue Service generates the following response: HTTP/1.1 201 Created Server: Windows-Azure-Queue/1.0 Microsoft-HTTPAPI/2.0 x-ms-request-id: 14c6e73b-15d9-480c-b251-c4c01b48e529 x-ms-version: 2012-02-12 Date: Sun, 08 Sep 2013 06:34:09 GMT Content-Length: 0 Get Messages The Get Messages operation described in this section retrieves a single message with the default message visibility timeout of 30 seconds. public void GetMessage(String queueName) { string requestMethod = "GET"; String urlPath = String.Format("{0}/messages", queueName); String storageServiceVersion = "2012-02-12"; String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture); String canonicalizedHeaders = String.Format( "x-ms-date:{0}\nx-ms-version:{1}", dateInRfc1123Format, storageServiceVersion); String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath); String stringToSign = String.Format( "{0}\n\n\n\n\n\n\n\n\n\n\n\n{1}\n{2}", requestMethod, canonicalizedHeaders, canonicalizedResource); String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign); Uri uri = new Uri(AzureStorageConstants.QueueEndPoint + urlPath); HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri); request.Method = requestMethod; request.Headers.Add("x-ms-date", dateInRfc1123Format); request.Headers.Add("x-ms-version", storageServiceVersion); request.Headers.Add("Authorization", authorizationHeader); request.Accept = "application/atom+xml,application/xml"; using (HttpWebResponse response = (HttpWebResponse)request.GetResponse()) { Stream dataStream = response.GetResponseStream(); using (StreamReader reader = new StreamReader(dataStream)) { String responseFromServer = reader.ReadToEnd(); } } } This generates the following request: GET https://rebus.queue.core.windows.net/revolution/messages HTTP/1.1 x-ms-date: Sun, 08 Sep 2013 06:34:11 GMT x-ms-version: 2012-02-12 Authorization: SharedKey rebus:K67XooYhokw0i0AlCzYQ4GeLLrJih1r1vSqiO9DBo0c= Accept: application/atom+xml,application/xml Host: rebus.queue.core.windows.net The Queue Service generates the following response: HTTP/1.1 200 OK Content-Type: application/xml Server: Windows-Azure-Queue/1.0 Microsoft-HTTPAPI/2.0 x-ms-request-id: efb21a86-7d66-47fd-b13d-7aa74fce0568 x-ms-version: 2012-02-12 Date: Sun, 08 Sep 2013 06:34:12 GMT Content-Length: 484 The message is returned in the response body as follows: 05fd902f-6031-4ef4-8298-ef3844ec3bc6Sun, 08 Sep 2013 06:34:11 GMTSun, 15 Sep 2013 06:34:11 GMT1AgAAAAMAAAAAAAAAAL+zgF2szgE=Sun, 08 Sep 2013 06:34:43 GMTSaturday in the cafe I noticed that some newline specifiers in strings (\n) were lost when the blog was auto-ported from Windows Live Spaces to WordPress. I have put them back in but it is possible I missed some. Consequently, in the event of a problem you should check the newlines in canonicalizedHeaders and stringToSign.

Before getting into the action bar and Contextual Action bar concept and the coding for it, let me take you through the concept of two ways to show contextual actions: 1. Floating Context Menu 2. Contextual Action Mode 1. Floating Context Menu In earlier versions of Android, we were used to seeing almost all the apps having context menus ready for showing options (menu items) whenever the user performs a long press on any element. We can say long press gesture was universally used to display contextual actions in a context menu. “Long press gesture – That is, a touch that’s held in the same position for a moment.” Now, since Android 3.0, the purpose of Long press gesture has changed. It's now used to handle multi-select and contextual actions. 2. Contextual Action Mode The contextual action mode is a system implementation of ActionMode that focuses user interaction toward performing contextual actions. When a user enables this mode by selecting an item, a contextual action bar appears at the top of the screen to present actions the user can perform on the currently selected item(s). ActionMode Represents a contextual mode of the user interface. Action modes can be used to provide alternative interaction modes and replace parts of the normal UI until finished. Examples of good action modes include text selection and contextual actions. Contextual Action Bar (CAB) A Contextual action bar (CAB) is a temporary action bar that overlays the app’s action bar for the duration of a particular sub-task. As I have mentioned earlier, CABs are used for tasks that involve acting on selected data or text. For example: Cut, Copy, Paste, Delete, or any other operations can be performed on single or batches of selected data. As shown in snap-1 (left) above, the Contextual Action bar (Selection CAB) appears at the top bar as soon as the user performs the long press gesture. From here the user can: Select more items or deselect items by just touching them Select and trigger any actions displayed in the bar; the selected action triggers all the selected items. Then the action bar automatically dismiss itself. You can dismiss CAB in 3 ways: Deselect all the selected items Press the Back key from navigation bar Select the Check mark button (left) from the CAB. It doesn’t dismiss only the CAB, but also removes the selection on data which you have done. When to Use Which? (Context Menu or CAB) Now, I am sure there is no doubt regarding when to use Context Menu and when to CAB. As I have mentioned, if you are developing an app for android 3.0 or higher, you should use Contextual Action bar instead of displaying menu items in floating context menu. And if you are providing compatibility to a lower Android version, you should fall back to a floating context menu on those devices. Using Contextual Action Bar (CAB): There are 2 designs by which you can implement Contextual Action bar: Enable CAB when the user selects a particular view Enable CAB whenever the user performs a long press gesture on particular view 1: Enable CAB When User Selects a Particular View If you want to invoke the contextual action mode only when the user selects particular views, then follow the below steps: Implement the ActionMode.Callback interface. In its callback methods, you can specify the actions for the contextual action bar, respond to click events on action items, and handle other lifecycle events for the action mode. Call startActionMode() when you want to show the bar (such as when the user long-clicks the view). Implement the ActionMode.Callback interface: class ActionBarCallBack implements ActionMode.Callback { @Override public boolean onActionItemClicked(ActionMode mode, MenuItem item) { // TODO Auto-generated method stub return false; } @Override public boolean onCreateActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub mode.getMenuInflater().inflate(R.menu.contextual_menu, menu); return true; } @Override public void onDestroyActionMode(ActionMode mode) { // TODO Auto-generated method stub } @Override public boolean onPrepareActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub mode.setTitle("CheckBox is Checked"); return false; } } Call startActionMode() MainActivity.this.startActionMode(new ActionBarCallBack()); For example: Let’s build an example to enable Contextual action mode on the CheckBox selection. package com.technotalkative.contextualactionbarsingle; import android.app.Activity; import android.os.Bundle; import android.view.ActionMode; import android.view.Menu; import android.view.MenuItem; import android.widget.CheckBox; import android.widget.CompoundButton; import android.widget.CompoundButton.OnCheckedChangeListener; public class MainActivity extends Activity { private ActionMode mActionMode; private CheckBox checkBox1; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); getActionBar().setTitle("CAB demo - Individual view"); checkBox1 = (CheckBox) findViewById(R.id.checkBox1); checkBox1.setOnCheckedChangeListener(new OnCheckedChangeListener() { @Override public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) { // TODO Auto-generated method stub if(isChecked) mActionMode = MainActivity.this.startActionMode(new ActionBarCallBack()); else mActionMode.finish(); } }); } class ActionBarCallBack implements ActionMode.Callback { @Override public boolean onActionItemClicked(ActionMode mode, MenuItem item) { // TODO Auto-generated method stub return false; } @Override public boolean onCreateActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub mode.getMenuInflater().inflate(R.menu.contextual_menu, menu); return true; } @Override public void onDestroyActionMode(ActionMode mode) { // TODO Auto-generated method stub } @Override public boolean onPrepareActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub mode.setTitle("CheckBox is Checked"); return false; } } } 2: Enable CAB When User Performs a Long Press Gesture on Particular View If you want to invoke the contextual action mode only when the user performs a long press gesture on a view like ListView or GridView, and want to perform batch actions on multiple selected items, then you can implement this by following the below steps: Implement the AbsListView.MultiChoiceModeListener and set it to your ViewGroup (e.g. ListView). In its callback methods, you can specify the actions for the contextual action bar, respond to click events on action items, and handle its callback events (Which are actually inherited from ActionMode.Callback interface). Call setChoiceMode() with the CHOICE_MODE_MULTIPLE_MODAL argument. AbsListView.MultiChoiceModeListener: A MultiChoiceModeListener receives events for CHOICE_MODE_MULTIPLE_MODAL. It acts as the ActionMode.Callback for the selection mode and also receives onItemCheckedStateChanged(ActionMode, int, long, boolean) events when the user selects and deselects list items. Implement the AbsListView.MultiChoiceModeListener: getListView().setMultiChoiceModeListener(new MultiChoiceModeListener() { @Override public boolean onPrepareActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub return false; } @Override public void onDestroyActionMode(ActionMode mode) { // TODO Auto-generated method stub } @Override public boolean onCreateActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub MenuInflater inflater = getMenuInflater(); inflater.inflate(R.menu.contextual_menu, menu); return true; } @Override public boolean onActionItemClicked(ActionMode mode, MenuItem item) { // TODO Auto-generated method stub return false; } @Override public void onItemCheckedStateChanged(ActionMode mode, int position, long id, boolean checked) { // TODO Auto-generated method stub } }); Call setChoiceMode() with the CHOICE_MODE_MULTIPLE_MODAL argument: getListView().setChoiceMode(ListView.CHOICE_MODE_MULTIPLE_MODAL); Full example: Have you used Gmail Android app? (stupid question ), but if you have used it, then I am sure you have tried to perform long gesture on mails to delete messages, so let’s develop a similar example. Here we will enable contextual action mode whenever the user performs long press gestures, and we will display number of items selected. Step 1: Take ListView in activity_main.xml layout Step 2: Define row layout (row_list_item.xml) for ListView Step 3: Create a contextual menu (contextual_menu.xml) in menu folder, this menu gets displayed as contextual action bar whenever user performs long press gesture Step 4: Implement MultiChoiceModeListener and call setChoiceMode() inside MainActivity Inside onCreateActionMode() – We will enable contextual action mode with menu we have defined. Inside onActionItemClicked() – We can perform contextual actions on the selected items. Inside onItemCheckedStateChanged() – we can decide which items are selected and which are not. Here we will prepare title for the action bar with particular no. of items are selected. package com.technotalkative.contextualactionmultiple; import java.util.HashMap; import java.util.Set; import android.app.ListActivity; import android.content.Context; import android.os.Bundle; import android.view.ActionMode; import android.view.Menu; import android.view.MenuInflater; import android.view.MenuItem; import android.view.View; import android.view.ViewGroup; import android.widget.AbsListView.MultiChoiceModeListener; import android.widget.AdapterView; import android.widget.AdapterView.OnItemLongClickListener; import android.widget.ArrayAdapter; import android.widget.ListView; public class MainActivity extends ListActivity { private String[] data = {"One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine","Ten"}; private SelectionAdapter mAdapter; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); mAdapter = new SelectionAdapter(this, R.layout.row_list_item, R.id.textView1, data); setListAdapter(mAdapter); getListView().setChoiceMode(ListView.CHOICE_MODE_MULTIPLE_MODAL); getListView().setMultiChoiceModeListener(new MultiChoiceModeListener() { private int nr = 0; @Override public boolean onPrepareActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub return false; } @Override public void onDestroyActionMode(ActionMode mode) { // TODO Auto-generated method stub mAdapter.clearSelection(); } @Override public boolean onCreateActionMode(ActionMode mode, Menu menu) { // TODO Auto-generated method stub nr = 0; MenuInflater inflater = getMenuInflater(); inflater.inflate(R.menu.contextual_menu, menu); return true; } @Override public boolean onActionItemClicked(ActionMode mode, MenuItem item) { // TODO Auto-generated method stub switch (item.getItemId()) { case R.id.item_delete: nr = 0; mAdapter.clearSelection(); mode.finish(); } } @Override public void onItemCheckedStateChanged(ActionMode mode, int position, long id, boolean checked) { // TODO Auto-generated method stub if (checked) { nr++; mAdapter.setNewSelection(position, checked); } else { nr--; mAdapter.removeSelection(position); } mode.setTitle(nr + " selected"); } }); getListView().setOnItemLongClickListener(new OnItemLongClickListener() { @Override public boolean onItemLongClick(AdapterView arg0, View arg1, int position, long arg3) { // TODO Auto-generated method stub getListView().setItemChecked(position, !mAdapter.isPositionChecked(position)); return false; } }); } private class SelectionAdapter extends ArrayAdapter { private HashMap mSelection = new HashMap(); public SelectionAdapter(Context context, int resource, int textViewResourceId, String[] objects) { super(context, resource, textViewResourceId, objects); } public void setNewSelection(int position, boolean value) { mSelection.put(position, value); notifyDataSetChanged(); } public boolean isPositionChecked(int position) { Boolean result = mSelection.get(position); return result == null ? false : result; } public Set getCurrentCheckedPosition() { return mSelection.keySet(); } public void removeSelection(int position) { mSelection.remove(position); notifyDataSetChanged(); } public void clearSelection() { mSelection = new HashMap(); notifyDataSetChanged(); } @Override public View getView(int position, View convertView, ViewGroup parent) { View v = super.getView(position, convertView, parent);//let the adapter handle setting up the row views v.setBackgroundColor(getResources().getColor(android.R.color.background_light)); //default color if (mSelection.get(position) != null) { v.setBackgroundColor(getResources().getColor(android.R.color.holo_blue_light));// this is a selected position so make it red } return v; } } } Download Example: https://github.com/PareshMayani/Contextual-Action-Bar

Learn more about Android Package Manager and Installer, including where APK files are stored in Android, where the manager stores data, and more.

These days, no community member writes or speaks about using DataTables and DataSets for data operations. But, there are a number of real projects built using them, and many developers still feel happy when they use them in their projects. Sometimes it is not easy to completely replace DataTables with typed generic lists, particularly in bulky projects. But now is the right time to move, as future developers may not even learn about DataTables :). Generic collections have a number of advantages over DataTables. One cannot imagine a day without generic collections once he/she gets to know how beneficial they are. The following is a list of the reasons to move from DataTables to collections that I could think of now: DataTable stores boxed objects, and one needs to unbox values when needed. This adds overhead on the runtime environment. However, values in generic collections are strongly typed, so no boxing involved. Unboxing happens at runtime, as does the type checking. If there is a mismatch between types of source and target, it leads to a runtime exception. This may lead to a number of issues while using DataTables. In case of collections, as the types are checked at the compile time, such type mismatches are caught during compilation. .NET languages got very nice support for creating collections, like object initializer and collection initializer. We don’t have such features for DataTables. LINQ queries can be used on both DataTables and collections. But the experience of writing the queries on generic collections is better because of IntelliSense support provided by Visual Studio. DataTables are framework specific; we often see issues with serializing and de-serializing them in web services. Generic collections are easier to serialize and de-serialize, so they can be easily used in any service and consumed from a client written in any language. ORMs are becoming increasingly popular, and they use generic collections for all data operations. Mocking DataTables in unit tests is a pain, as it involves creating the structure of the table wherever needed. But a generic collection needs a class defined just once. These are my opinions on preferring collections over DataTables. Any feedback is welcome. Happy coding!

Some months ago I wrote an article on how to transition between Scrum and a Lean Kanban operation. It's an important capability for an organization to have, because when a Scrum project finishes it is likely to enter a "leaner" BAU (Business As Usual) support phase. There are consequences arising from such a move which experienced Scrum hands may find surprising, and perhaps even a little off-putting. In this article we'll look at the shift in mindset that is required to do this. "Whoa! Something screwy has happened to our task board, it looks different" Kanban boards are subtly different to the task boards commonly used in Scrum. At first blush they might look similar. Both have columns showing the progress of user story "tickets" from a backlog through states such as in progress, peer review, in test, and done. In either case there might also be a blocked column, although it is equally acceptable to add a "blocked" sticker, or to simply invert the ticket on the board. As the name suggests, a task board will show the progress of the tasks that are needed to complete user stories. Often these tasks will be kept within horizontal swim lanes - one lane per user story. When all of the tasks are done, the user story will also move into done. Each user story therefore "chases" its tasks across the board. A Kanban board on the other hand - which is meant to deal with smaller and finer-grained pieces of work - will typically track the progress of user stories themselves across the board. The requirements should be well understood and there should be little appreciable depth to the solutioning; there will be few if any explicit tasks associated with the user stories. There is therefore no need for horizontal swim lanes to keep tasks and user stories aligned. You might also notice that Work in Progress limits are given particular emphasis in Lean Kanban. This is because scope is not timeboxed into sprints. The only way to throttle the rate of ticket throughput, and to keep it to manageable levels, is therefore by making sure that WIP limits are rigorously enforced. These are often annotated to the column headers on a Kanban board. For example, if there are 3 developers and 1 tester, the WIP for in progress would be 3, and 1 for in test. "Hey…there's just one backlog" That's right. Since there are no sprints in Lean Kanban, there can be no meaningful separation between a "sprint backlog" and a "product backlog". Instead there's just a single backlog of enqueued work items being brought into progress. This has repercussions for product ownership because you no longer have a clear separation between the prioritization that a team does for itself on a sprint backlog, and the prioritization done by a Product Owner on the product backlog. In effect you've just got a product backlog. In this situation clear product ownership can become more important then ever…or it can become a complete non-issue. "The Product Owner has too much power, he keeps jerking our chain" Since there is only one backlog, the Product Owner (or customer representative) must constantly reprioritize the user stories within it. The Product Owner needs to have more operational control in Lean Kanban than in Scrum. Developers can action tickets from the backlog on a daily or even hourly basis. There is no notion of getting a product backlog in shape before "the next sprint starts". Product Owners are therefore much more closely involved in day-to-day delivery than they would be in Scrum, and their involvement in daily standups becomes much more important. Note that the extent of a Product Owner's decision making should not extend beyond the backlog, and a good Kanban Leader will protect the team and its work in progress just like a good ScrumMaster would. "Now the Product Owner has disappeared altogether" Business as Usual work often boils down to the maintenance of existing systems post-delivery. Depending upon the level of demand, it's quite plausible to have one Lean-Kanban team responsible for the maintenance of multiple systems. In this situation there is no product being delivered as such, and consequently there is no clear product ownership. Instead, work items are raised as change requests and triaged by the team who then manage and prioritize their own backlog. This means that the team needs a strong and shared sense of direction and purpose. "There's no vision for this project" That's because a Lean Kanban operation typically isn't a project at all. A defined end point is likely to be missing… remember that it's covering "Business as Usual work". These are small, repeatable changes that may affect diverse systems and without any sort of narrative to bind them together. There'll certainly be a purpose and a rationale for operating a Lean Kanban… but don't expect a project vision. "We don't even seem to have decent sprint goals any more" Yep, they've gone too. Since there is no project vision and no sprints on a Lean Kanban, we won't have any "sprint goals" either. What we might get is a grouping of work requests that fall within a larger epic of changes…but if we do, it could well be a cause for concern. We must ask: are those related changes really representative of "Business as Usual" work, or are they too high risk? Do they constitute a project? "Lean Kanban work seems very bitty. I can't get a decent chunk to chew on" The diet of a Lean Kanban should consist of small, "digestible" pieces of work that do not require much breaking down in order to action them. By definition they must be well-understood and low-risk. A team must know how to handle them without the need for impact analysis or de-scoping. You're unlikely to get a meaty piece of work; you're more likely to be sucking these things up through a straw. Velocity and lead times are particularly significant metrics in Lean Kanban. Having said that, substantial and time consuming pieces of work can be taken on board if they satisfy the criteria of low risk and clear scope. An example would be the sort of work that conforms to a templated change. Of course, this sort of work might not appeal to an agile developer. So let's be clear: it takes a different temperament to do Lean Kanban BAU work than project work in Scrum. They are different skill sets. Agile developers who are happy doing one can find it unsettling, or even unrewarding, if they are switched to the other. "Why aren't we doing planning poker any more?" Without a sprint backlog there is no budget of story points to be brought into a sprint. This in turn means that estimation exercises such as planning poker lose much of their significance. In a Lean Kanban operation velocity can be measured not in terms of story points - either estimated or actual - but simply as the number of tickets actioned over a set period. This also provides an indication of the lead time before a ticket is handled. If tickets are of too variable a size - for example, if they include small ones as well as larger templated changes - then they can be awarded points for how long, or how much effort, they took. T-Shirt sizes is one approach. Remember that these points should represent the actuals, not estimates, so there's still no need for planning poker. Velocity can be averaged for each size. Alternatively the sizes can be mapped to points (e.g. small = 1, medium = 3, large = 7) and an aggregate velocity calculated. "Some of the BAU work that's been coming through looks like project work to me" You could well be right. It's important that you raise your suspicions with your team lead. There's often politics involved, but here's the lowdown. In many organizations "Business as Usual" work is classed - you could almost say "written off" - as an operational expenditure (OpEx), and is not drawn from the capital expenditure (CapEx) assigned to projects. Internal customers often have an incentive to sneak through initiatives as BAU work so as not to incur capital expense on their departmental budgets. This is indeed a political issue. But be on your guard otherwise your team could be hobbled with project work being slipped in on the sly. Be particularly wary of significant numbers of related changes, large changes, a seemingly high level of risk with any work items, or changes of uncertain scope. These suggest, but do not prove, that a fast one might be being pulled. Your team lead (who is analagous to a ScrumMaster) should try and defend against this, so if you as a team member have your suspicions, it's important to bring them to your lead's attention. Conclusion, and what's next In this post we've looked at the important differences between Lean Kanban and Scrum, and what that means for a team. We've also reviewed how a reasonably informed choice can be made between them. In my next post we'll look at a hybrid approach known as ScrumBan which can potentially address both project and BAU work. ScrumBan is becoming increasingly popular and has significant ramifications for project scalability.

this post comes from ernie souhrada at the mysql performance blog. note: this is part 1 of what will be a two-part series on the performance implications of using in-flight data encryption. some of you may recall my security webinar from back in mid-august; one of the follow-up questions that i was asked was about the performance impact of enabling ssl connections. my answer was 25%, based on some 2011 data that i had seen over on yassl’s website, but i included the caveat that it is workload-dependent, because the most expensive part of using ssl is establishing the connection. not long thereafter, i received a request to conduct some more specific benchmarks surrounding ssl usage in mysql, and today i’m going to show the results. first, the testing environment. all tests were performed on an intel core i7-2600k 3.4ghz cpu (8 cores, ht included) with 32gb of ram and centos 6.4. the disk subsystem is a 2-disk raid-0 of samsung 830 ssds, although since we’re only concerned with measuring the overhead added by using ssl connections, we’ll only be conducting read-only tests with a dataset that fits completely in the buffer pool. the version of mysql used for this experiment is community edition 5.6.13, and the testing tools are sysbench 0.5 and perl. we conduct two tests, each one designed to simulate one of the most common mysql usage patterns. first, we examine connection pooling, often seen in the java world, where some small set of connections are established by, for example, the servlet container and then just passed around to the application as needed, and one-request-per-connection, typical in the lamp world, where the script that displays a given page might connect to the database, run a couple of queries, and then disconnect. test 1: connection pool for the first test, i ran sysbench in read-only mode at concurrency levels of 1, 2, 4, 8, 16, and 32 threads, first with no encryption and then with ssl enabled and key lengths of 1024, 2048, and 4096 bits. 8 sysbench tables were prepared, each containing 100,000 rows, resulting in a total data size of approximately 256mb. the size of my innodb buffer pool was 4gb, and before conducting each official measurement run, i ran a warm-up run to prime the buffer pool. each official test run lasted 10 minutes; this might seem short, but unlike, say, a pcie flash storage device, i would not expect the variable under observation to really change that much over time or need time to stabilize. the basic sysbench syntax used is shown below. #!/bin/bash for ssl in on off ; do for threads in 1 2 4 8 16 32 ; do sysbench --test=/usr/share/sysbench/oltp.lua --mysql-user=msandbox$ssl --mysql-password=msandbox \ --mysql-host=127.0.0.1 --mysql-port=5613 --mysql-db=sbtest --mysql-ssl=$ssl \ --oltp-tables-count=8 --num-threads=$threads --oltp-dist-type=uniform --oltp-read-only=on \ --report-interval=10 --max-time=600 --max-requests=0 run > sb-ssl_${ssl}-threads-${threads}.out done done if you’re not familiar with sysbench, the important thing to know about it for our purposes is that it does not connect and disconnect after each query or after each transaction. it establishes n connections to the database (where n is the number of threads) and runs queries though them until the test is over. this behavior provides our connection-pool simulation. the assumption, given what we know about where ssl is the slowest, is that the performance penalty here should be the lowest. first, let’s look at raw throughput, measured in queries per second: the average throughput and standard deviation (both measured in queries per second) for each test configuration is shown below in tabular format: # of threads ssl key size 1 2 4 8 16 32 ssl off 9250.18 (1005.82) 18297.61 (689.22) 33910.31 (446.02) 50077.60 (1525.37) 49844.49 (934.86) 49651.09 (498.68) 1024-bit 2406.53 (288.53) 4650.56 (558.58) 9183.33 (1565.41) 26007.11 (345.79) 25959.61 (343.55) 25913.69 (192.90) 2048-bit 2448.43 (290.02) 4641.61 (510.91) 8951.67 (1043.99) 26143.25 (360.84) 25872.10 (324.48) 25764.48 (370.33) 4096-bit 2427.95 (289.00) 4641.32 (547.57) 8991.37 (1005.89) 26058.09 (432.86) 25990.13 (439.53) 26041.27 (780.71) so, given that this is an 8-core machine and io isn’t a factor, we would expect throughput to max out at 8 threads, so the levelling-off of performance is expected. what we also see is that it doesn’t seem to make much difference what key length is used, which is also largely expected. however, i definitely didn’t think the encryption overhead would be so high. the next graph here is 95th-percentile latency from the same test: and in tabular format, the raw numbers (average and standard deviation): # of threads ssl key size 1 2 4 8 16 32 ssl off 1.882 (0.522) 1.728 (0.167) 1.764 (0.145) 2.459 (0.523) 6.616 (0.251) 27.307 (0.817) 1024-bit 6.151 (0.241) 6.442 (0.180) 6.677 (0.289) 4.535 (0.507) 11.481 (1.403) 37.152 (0.393) 2048-bit 6.083 (0.277) 6.510 (0.081) 6.693 (0.043) 4.498 (0.503) 11.222 (1.502) 37.387 (0.393) 4096-bit 6.120 (0.268) 6.454 (0.119) 6.690 (0.043) 4.571 (0.727) 11.194 (1.395) 37.26 (0.307) with the exception of 8 and 32 threads, the latency introduced by the use of ssl is constant at right around 5ms, regardless of the key length or the number of threads. i’m not surprised that there’s a large jump in latency at 32 threads, but i don’t have an immediate explanation for the improvement in the ssl latency numbers at 8 threads. test 2: connection time for the second test, i wrote a simple perl script to just connect and disconnect from the database as fast as possible. we know that it’s the connection setup which is the slowest part of ssl, and the previous test already shows us roughly what we can expect for ssl encryption overhead for sending data once the connection has been established, so let’s see just how much overhead ssl adds to connection time. the basic script to do this is quite simple (non-ssl version shown): #!/usr/bin/perl use dbi; use time::hires qw(time); $start = time; for (my $i=0; $i<100; $i++) { my $dbh = dbi->connect("dbi:mysql:host=127.0.0.1;port=5613", "msandbox","msandbox",undef); $dbh->disconnect; undef $dbh; } printf "%.6f\n", time - $start; as with test #1, i ran test #2 with no encryption and ssl encryption of 1024, 2048, and 4098 bits, and i conducted 10 trials of each configuration. then i took the elapsed time for each test and converted it to connections per second. the graph below shows the results from each run: here are the averages and standard deviations: encryption average connections per second standard deviation none 2701.75 165.54 1024-bit 77.04 6.14 2048-bit 28.183 1.713 4096-bit 5.45 0.015 yes, that’s right, 4096-bit ssl connections are 3 orders of magnitude slower to establish than unencrypted connections. really, the connection overhead for any level of ssl usage is quite high when compared to the unencrypted test, and it’s certainly much higher than my original quoted number of 25%. analysis and parting thoughts so, what do we take away from this? the first thing is, of course, is that ssl overhead is a lot higher than 25%, particularly if your application uses anything close to the one-connection-per-request pattern. for a system which establishes and maintains long-running connections, the initial connection overhead becomes a non-factor, regardless of the encryption strength, but there’s still a rather large performance penalty compared to the unencrypted connection. this leads directly into the second point, which is that connection pooling is by far a more efficient method of using ssl if your application can support it. but what if connection pooling isn’t an option, mysql’s ssl performance is insufficient, and you still need full encryption of data in-flight? run the encryption component of your system at a lower layer – a vpn with hardware crypto would be the fastest approach, but even something as simple as an ssh tunnel or openvpn *might* be faster than ssl within mysql. i’ll be exploring some of these solutions in a follow-up post. and finally… when in doubt, run your own benchmarks. i don’t have an explanation for why the yassl numbers are so different from these (maybe yassl is a faster ssl library than openssl, or maybe they used a different cipher – if you’re curious, the original 25% number came from slides 56-58 of this presentation ), but in any event, this does illustrate why it’s important to run tests on your own hardware and with your own workload when you’re interested in finding out how well something will perform rather than taking someone else’s word for it.

qunit , used by projects like jquery and jquery mobile , is a rather popular javascript testing framework. for tests written using qunit, how do we measure its code coverage ? a possible solution which is quite easy to setup is to leverage the deadly combination of karma and istanbul . just like our previous adventure with jasmine code coverage , let's take a look at some simple code we need to test. this function my.sqrt is a reimplementation of math.sqrt which may throw an exception if the input is invalid. var my = { sqrt: function(x) { if (x < 0) throw new error("sqrt can't work on negative number"); return math.exp(math.log(x)/2); } }; a very simple qunit-based test for the above code is as follows. test("sqrt", function() { deepequal(my.sqrt(4), 2, "square root of 4 is 2"); }); manually running the test is easy as opening the test runner in a web browser: for a smoothed development workflow, an automated way to run the tests will be much preferred. this is where karma becomes very useful. karma also has the ability to launch a predetermined collection of browsers, or even to use phantomjs for a pure headless execution (suitable for smoke testing and/or continuous delivery). before we can use karma, installation is necessary: npm install karma karma-qunit karma-coverage karma requires a configuration file. for this purpose, the config file is very simple. as an illustration, the execution is done by phantomjs but it is easy to include other browsers as well. module.exports = function(config) { config.set({ basepath: '', frameworks: ['qunit'], files: [ '*.js', 'test/spec/*.js' ], browsers: ['phantomjs'], singlerun: true, reporters: ['progress', 'coverage'], preprocessors: { '*.js': ['coverage'] } }); }; now you can start karma with the above configuration, it would say that the test passes just fine. should you encounter some problems, you can look at an example repository i have setup github.com/ariya/coverage-qunit-istanbul-karma , it may be useful as a starting point or a reference for your own project. as a convenience, the test in that repository can be executed via npm test . what is more interesting here is that karma runs its coverage processor, as indicated by preprocessors in the above configuration. karma will run istanbul , a full-featured instrumenter and coverage tracker. essentially, istanbul grabs the original javascript source and injects extra instrumentation code so that it can gather the execution metrics once the process finishes (read also my previous blog post on javascript code coverage with istanbul ). in this karma and istanbul combo, the generated coverage report is available in the under the subdirectory coverage . the above report indicates that the single test for my.sqrt is still missing the test for an invalid input, thanks to branch coverage feature of istanbul. the i indicator next to the conditional statement tells us that the if branch was never taken. of course, once the issue is known, adding another test which will cover that branch is easy (left as an exercise for the reader). now that code coverage is tracker, perhaps you are ready for the next level? it is about setting the hard threshold so that future coverage regression will never happen. protect yourself and your team from carelessness, overconfidence, or honest mistakes!

The following question will again test your knowledge of the Oracle Weblogic threading model. I’m looking forward for your comments and experience on the same. If you are a Weblogic administrator, I’m certain that you heard of this common problem: stuck threads. This is one of the most common problems you will face when supporting a Weblogic production environment. A Weblogic stuck thread simply means a thread performing the same request for a very long time and more than the configurable Stuck Thread Max Time. Question: How can you detect the presence of STUCK threads during and following a production incident? Answer: As we saw from our last article “Weblogic Thread Monitoring Tips”, Weblogic provides functionalities allowing us to closely monitor its internal self-tuning thread pool. It will also highlight you the presence of any stuck thread. This monitoring view is very useful when you do a live analysis but what about after a production incident? The good news is that Oracle Weblogic will also log any detected stuck thread to the server log. Such information includes details on the request and more importantly, the thread stack trace. This data is crucial and will allow you to potentially better understand the root cause of any slowdown condition that occurred at a certain time. < ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)'> <[STUCK] ExecuteThread: '35' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "608" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 608213 ms POST /App1/jsp/test.jsp HTTP/1.1 Accept: application/x-ms-application... Referer: http://.. Accept-Language: en-US User-Agent: Mozilla/4.0 .. Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate Content-Length: 539 Connection: Keep-Alive Cache-Control: no-cache Cookie: JSESSIONID= ]", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace: ................................... javax.servlet.http.HttpServlet.service(HttpServlet.java:727) javax.servlet.http.HttpServlet.service(HttpServlet.java:820) weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227) weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125) weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301) weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:184) weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.... weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run() weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120) weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2281) weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2180) weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1491) weblogic.work.ExecuteThread.execute(ExecuteThread.java:256) weblogic.work.ExecuteThread.run(ExecuteThread.java:221) Here is one more tip: the generation and analysis of a JVM thread dump will also highlight you stuck threads. As we can see from the snapshot below, the Weblogic thread state is now updated to STUCK, which means that this particular request is being executed since at least 600 seconds or 10 minutes. This is very useful information since the native thread state will typically remain to RUNNABLE. The native thread state will only get updated when dealing with BLOCKED threads etc. You have to keep in mind that RUNNABLE simply means that this thread is healthy from a JVM perspective. However, it does not mean that it truly is from a middleware or Java EE container perspective. This is why Oracle Weblogic has its own internal ExecuteThread state. Finally, if your organization or client is using any commercial monitoring tool, I recommend that you enable some alerting around both hogging thread and stuck thread. This will allow your support team to take some pro-active actions before the affected Weblogic managed server(s) become fully unresponsive.

for modern web application development, having dozens of unit tests is not enough anymore. the actual code coverage of those tests would reveal if the application is thoroughly stressed or not. for tests written using the famous jasmine test library, an easy way to have the coverage report is via istanbul and karma . for this example, let’s assume that we have a simple library sqrt.js which contains an alternative implementation of math.sqrt . note also how it will throw an exception instead of returning nan for an invalid input. var my = { sqrt: function(x) { if (x < 0) throw new error("sqrt can't work on negative number"); return math.exp(math.log(x)/2); } }; using jasmine placed under test/lib/jasmine-1.3.1 , we can craft a test runner that includes the following spec: describe("sqrt", function() { it("should compute the square root of 4 as 2", function() { expect(my.sqrt(4)).toequal(2); }); }); opening the spec runner in a web browser will give the expected outcome: so far so good. now let's see how the code coverage of our test setup can be measured. the first order of business is to install karma . if you are not familiar with karma, it is basically a test runner which can launch and connect to a specific set of web browsers, run your tests, and then gather the report. using node.js, what we need to do is: npm install karma karma-coverage before launching karma, we need to specify its configuration . it could be as simple as the following my.conf.js (most entries are self-explained). note that the tests are executed using phantomjs for simplicity, it is however quite trivial to add other web browsers such as chrome and firefox. module.exports = function(config) { config.set({ basepath: '', frameworks: ['jasmine'], files: [ '*.js', 'test/spec/*.js' ], browsers: ['phantomjs'], singlerun: true, reporters: ['progress', 'coverage'], preprocessors: { '*.js': ['coverage'] } }); }; running the tests, as well as performing code coverage at the same time, can be triggered via: node_modules/.bin/karma start my.conf.js which will dump the output like: info [karma]: karma v0.10.2 server started at http://localhost:9876/ info [launcher]: starting browser phantomjs info [phantomjs 1.9.2 (linux)]: connected on socket n9ndnhj0np92ntspgx-x phantomjs 1.9.2 (linux): executed 1 of 1 success (0.029 secs / 0.003 secs) as expected (from the previous manual invocation of the spec runner), the test passed just fine. however, the most particular interesting piece here is the code coverage report, it is stored (in the default location) under the subdirectory coverage . open the report in your favorite browser and there you'll find the coverage analysis report. behind the scene, karma is using istanbul , a comprehensive javascript code coverage tool (read also my previous blog post on javascript code coverage with istanbul ). istanbul parses the source file, in this example sqrt.js , using esprima and then adds some extra instrumentation which will be used to gather the execution statistics. the above report that you see is one of the possible outputs, istanbul can also generate lcov report which is suitable for many continuous integration systems (jenkins, teamcity, etc). an extensive analysis of the coverage data should also prevent any future coverage regression, check out my other post hard thresholds on javascript code coverage . one important thing about code coverage is branch coverage . if you pay attention carefully, our test above is still not exercising the situation where the input to my.sqrt is negative. there is a big "i" marking in the third-line of the code, this is istanbul telling us that the if branch is not taken at all (for the else branch, it will be an "e" marker). once this missing branch is noticed, improving the situation is as easy as adding one more test to the spec: it("should throw an exception if given a negative number", function() { expect(function(){ my.sqrt(-1); }). tothrow(new error("sqrt can't work on negative number")); }); once the test is executed again, the code coverage report looks way better and everyone is happy. if you have some difficulties following the above step-by-step instructions, take a look at a git repository i have prepared: github.com/ariya/coverage-jasmine-istanbul-karma . feel free to play with it and customize it to suit your workflow!

Feeling good to be back at the blog . Actually, I have been managing GDG Ahmedabad, delivering android talks, and managing workshops locally and outside my region. Last month, I was quite busy in organizing the “DevFest” event for GDG Ahmedabad, and then for the preparation of my two talks for the GDG Kathmandu DevFest. I was invited to deliver two talks at DevFest, which was organized by GDG Kathmandu. I have already published slides on my Speakerdeck. I am not sure whether you have already checked and learned from my speaker deck, but still give me a chance to write about Introduction to Android studio here. What is Android Studio? It’s an Android focused IDE, designed specially for Android development. It was launched on 16th May 2013, during Google's I/O 2013 event. Android studio contains all the Android SDK tools to design, test, debug and profile your app. By looking at the development tools and environment, we can see its similar to Eclipse with the ADT plug-in, but as I have mentioned above, it's an Android focused IDE, and there are many cool features available in Android Studio that can foster and increase your development productivity. One great thing is that it depends on the IntelliJ Idea IDE, which has proved itself to be a great IDE and has been in use by many Android engineers. What is the Difference Between IntelliJ Idea and Android Studio? Nothing, in regards to Android. If you use IntelliJ… Keep using it IntelliJ 13 will have the same stuff EAP of IntelliJ Idea 13 includes all the new stuff If Not… Give Android Studio a try You may have some questions in mind regarding IntelliJ and Android Studio. If so, check the FAQ section: IntelliJ IDEA and Android Studio FAQ. Let’s Download Android Studio You can download Android Studio from the android developer site: http://developer.android.com/sdk/installing/studio.html. Cool Features of Android Studio As I have mentioned, it's similar to Eclipse with the ADT plug-in, but Android Studio has many cool features that can help you to increase development productivity. Here are the cool features: Powerful code editing (smart editing, code re-factoring) Rich layout Editor (As soon as you drag and drop views on the layout, it shows you a preview in all the screens including Nexus 4, Nexus 7, Nexus 10 and many other resolutions. Layout designing can be done much faster way as compared to eclipse.) Gradle-based build support Maven Support Template-based wizards Lint tool analysis (The Android lint tool is a static code analysis tool that checks your Android project source files for potential bugs and optimization improvements for correctness, security, performance, usability, accessibility, and internationalization). You can experience all the cool features by using Android Studio yourself Awesome Stuff Inside Darcula Theme It's actually a black-based theme. While using Android Studio, I enjoy working in Darcula theme environment. By the way, Its Darcula theme, not Dracula. I am correcting this just because I have seen many people on Stackoverflow and Google+ saying Dracula. You can set the Darcula theme in Android Studio by: File > Settings > IDE Settings > Appearance > Theme: Darcula. Preview All the Screens We can consider this is as part of the Rich layout editor feature. With this privilege, users can design layouts and can check layouts by previewing in all the possible screens, such as Nexus 4, Nexus 7, Nexus and many other devices. It helps the user to improve layout designs while providing compatibility to various resolutions available. Device Framed Screen Capture It provides ability to directly generate a screenshot of your application. Yes, it was already included in the SDK, but Android Studio provides something more: Device frame (As frames for many Nexus devices are available, you can capture screenshot in whichever frame you like most) Drop shadow Screen glare Color Preview I like this feature very much and I have found this feature helpful while working on big projects. While using Eclipse, we have to have 3rd party color chooser and picker but this feature gives privilege to select color from in-build color chooser and can also have preview in Colors.xml file. Color Preview – Activity class While using Eclipse, it’s difficult to check which color we have used. Yes, we can imagine the color by its name, but an actual preview is much better. This feature was recently introduced in Android Studio, so you must have latest version installed. Hard Coded Strings Here is another feature I like and have found useful: Whenever you use any string resources from Strings.xml, it displays actual value instead of variable name. This setting comes by default, but in case you aren’t able to get hard coded strings in your activity class, then try any of the below ways. Settings > Editor > Code Folding > Android String References OR Select String and right click on it and then go to Folding > Collapse OR CTRL + Numpad ‘-’ Create Layout Variation This provides the ability to create layout variation directly. For example: layout for the large screen, layout for Xlarge screen, etc. The great thing is that the created variant layout gets stored in particular folders like layout-xlarge, layout-large-land, etc. Should I Use Android Studio? You might have explored all the cool features, or you are ready to explore right now. But questions might have arisen in your mind: “Should I use Android Studio,” or “should we start using Android Studio right now,” or “should I continue with IntelliJ or Eclipse?” My answer is a big NO to use Android Studio as your main IDE for Android development, because currently its EARLY ACCESS PREVIEW and it's maturing over days. Engineers have been working hard to improve this IDE. So, you should wait until the BETA comes out. I agree with Carlos Vega (commented over G+) on this point: “You should at least migrate to Intellij Idea 12 so that you get familiar with the IDE’s workflow and keyboard shortcuts. That way when Android Studio reach a more stable level, you can switch without a major learning curve.” Thanks, Carlos Vega, for the input. By the way, here is the presentation I delivered at the GDG Kathmandu DevFest.

In the previous post, we have seen an example where dataProvider attribute has been used3 to test methods with different sets of input data for the same test method. TestNG provides another attribute dataProviderClass in conjunction with dataProvider to fetch the input data for the test methods from an external class. The actual class that holds input data is set to the dataProviderClass attribute and datProvider by itself holds the method name where the input data is actually fetched. Here is a quick example to show how to use dataProviderClass and dataProvide attribute Code Service Class ? view source print? 01.package com.skilledmonster.example; 02./** 03.* Simple calculator service to demonstrate TestNG Framework 04.* 05.* @author Jagadeesh Motamarri 06.* @version 1.0 07.*/ 08.public interface CalculatorService { 09.int sum(int a, int b); 10.int multiply(int a, int b); 11.int div(int a, int b); 12.int sub(int a, int b); 13.} Service Implementation Class ? view source print? 01.package com.skilledmonster.example; 02./** 03.* Simple calculator service implementation to demonstrate TestNG Framework 04.* 05.* @author Jagadeesh Motamarri 06.* @version 1.0 07.*/ 08.public class SimpleCalculator implements CalculatorService { 09.public int sum(int a, int b) { 10.return a + b; 11.} 12.public int multiply(int a, int b) { 13.return a * b; 14.} 15.public int div(int a, int b) { 16.return a / b; 17.} 18.public int sub(int a, int b) { 19.return a - b; 20.} 21.} Data Provider Class ? view source print? 01.package com.skilledmonster.common; 02.import org.testng.annotations.DataProvider; 03./** 04.* Data Provider class for TestNG test cases 05.* 06.* @author Jagadeesh Motamarri 07.* @version 1.0 08.*/ 09.public class TestNGDataProvider { 10./** 11.* Data Provider for testing sum of 2 numbers 12.* 13.* @return 14.*/ 15.@DataProvider 16.public static Object[][] testSumInput() { 17.return new Object[][] { { 5, 5 }, { 10, 10 }, { 20, 20 } }; 18.} 19./** 20.* Data Provider for testing multiplication of 2 numbers 21.* 22.* @return 23.*/ 24.@DataProvider 25.public static Object[][] testMultipleInput() { 26.return new Object[][] { { 5, 5 }, { 10, 10 }, { 20, 20 } }; 27.} 28.} Finally, test class that uses dataProviderClass attribute to feed the input data for the test methods ? package com.skilledmonster.example; import org.testng.Assert; import org.testng.annotations.BeforeClass; import org.testng.annotations.Test; import com.skilledmonster.common.TestNGDataProvider; /** * Example to demonstrate use of dataProviderClass and dataProvide attributes of TestNG framework * * @author Jagadeesh Motamarri * @version 1.0 */ public class TestNGAnnotationTestDataProviderExample { public CalculatorService service; @BeforeClass public void init() { System.out.println("@BeforeClass: The annotated method will be run before the first test method in the current class is invoked."); System.out.println("init service"); service = new SimpleCalculator(); } @Test(dataProviderClass = TestNGDataProvider.class, dataProvider = "testSumInput") public void testSum(int a, int b) { System.out.println("@Test : testSum()"); int result = service.sum(a, b); Assert.assertEquals(result, a + b); } @Test(dataProviderClass = TestNGDataProvider.class, dataProvider = "testMultipleInput") public void testMultiple(int a, int b) { System.out.println("@Test : testMultiple()"); int result = service.multiply(a, b); Assert.assertEquals(result, a * b); } } Output As shown in the above console output, each of the testSum() and testMutiple() methods are invoked with different sets of input data using an external class with dataProviderClass attribute. Advantage More flexibility and re-usability of commonly used data across several test classes. Download Download TestNG DataProvider Example