Today I am going to discuss about creating a Grid in Android having image and associated checkbox. The idea is to have Grid with clickable images and checkbox. When an image is clicked the corresponding checkbox state is toggled. It is relatively easy to add custom items to the Grid. But its bit tricky to add clickable images and check/uncheck checkbox on click of the image. If you want to make the images clickable and check/uncheck the checkbox on image click, that is not that straightforward. In this article, I will take you through the creation of the Grid with clickable images and checkboxes. Creating a new project Open Eclipse and create a new android project. Choose a blank activity for the project. Create main Grid layout Open the layout file for the main activity and add a GridView component to it. Create Custom layout for grid items The next step is to create a custom layout that will be used to represent each item of the grid. Here we will define a custom layout using an ImageView and CheckBox android components. Create a new layout file named griditem.xml and add ImageView and CheckBox components to it as shown below: Adding Images to project For this sample project I am adding images to the project itself for use in the grid. But you can source those from database or any other content provider supported by android. In Android project there are few drawable-* folders under res folder. These are used for storing various resources used by the project. We can put images in any of these folders and Android SDK will copy them to other folders. If you have different images for different resolutions you can copy them to respective folders and correct image would be picked up based on the device resolution. Another option is to create a drawable folder under res and add images there. These will be copied to all drawable-* folders. We will use this approach here. Create custom adapter The next important step is to create a custom adapter which uses custom layout that we defined earlier to add items to the grid. The custom adapter should extend the BaseAdapter. The getView()method of the adapter is invoked for each item of the grid. In this method we assign values to the ImageView and checkbox. The listeners on the custom grid items are added here only. The code for getView() looks like the following: @Override public View getView(int position, View convertView, ViewGroup parent) { ViewHolder holder; if (convertView == null) { holder = new ViewHolder(); convertView = mInflater.inflate( R.layout.griditems, null); holder.imageview = (ImageView) convertView.findViewById(R.id.grid_item_image); holder.checkbox = (CheckBox) convertView.findViewById(R.id.grid_item_checkbox); convertView.setTag(holder); } else { holder = (ViewHolder) convertView.getTag(); } holder.checkbox.setId(position); holder.imageview.setId(position); holder.checkbox.setOnClickListener(new OnClickListener() { public void onClick(View v) { CheckBox cb = (CheckBox) v; int id = cb.getId(); if (thumbnailsselection[id]){ //cb.setChecked(false); thumbnailsselection[id] = false; } else { //cb.setChecked(true); thumbnailsselection[id] = true; } } }); holder.imageview.setOnClickListener(new ImageClickListener(context, holder,thumbnailsselection)); holder.imageview.setImageResource(thumbnails[position]); holder.checkbox.setChecked(thumbnailsselection[position]); holder.id = position; return convertView; } Here we are saving the state of checkboxes in an array, thumbnailsselection. Since checkboxes are being added dynamically and there is no unique identifier for them, this will help in identifying the current state of checkbox whose state has to be toggled when an image is clicked. For the same reason we need to create a separate ImageClickListener. As we cannot identify the checkbox on click of corresponding image, we have to create the separate listener which holds the reference to holder class. Using this holder instance we can check/uncheck the checkbox when a corresponding image is clicked. Sometimes when you click on the image, the ImageClickListener is not invoked. The reason for this is that the checkbox overrides the focus event of container grid. Hence checkbox takes the focus when any item of the grid is clicked. To solve this issue you need to add the following properties to the checkbox. android:focusable=“false” android:focusableInTouchMode=“false” References Image icons source: http://www.iconarchive.com http://developer.android.com/

This article follows on from my Omnipresent, Infallible, Omnipotent and Instantaneous Build Technologies one a couple of day ago. Specifically the last section: “Minimalist test execution, via hacks”, that addressed test times being very lengthy. I’ve made a proof of concept of that for Maven. The tests impacted by a change (a pending change specifically) can now be quickly determined and fed into the test runner to massively reduce test times. Massively reduced, this is, if you’ve managed to engineer hours of Selenium tests. Proof of concept I forked a Github project that Petri Kainulainen had made to discuss coverage for a Maven projects. Clone my fork, but cd into the “code-coverage-jacoco” directory. Running testimpact.sh there – it will rebuild a map of what tests cover what production-classes (sources). I’ve checked in the previous results from running this script, that’s what a team would do. It runs Maven against one test at a time, to calculate coverage then store that per test. Actually for Petri’s example, I’m only focussing on the integration tests (run by the “failsafe” plugin) rather than the unit tests (run by the “surefire” plugin). Even though Petri’s example project is not launching Selenium (or equivalent slow test), the integration test phase is where you would run that for a canonical Maven project. The testimpact.sh script uses python and ack (you’ll have to install those if you want to run this). I tried to use sqlite3’s CSV ingesting, but it was impossibly opaque, even with using StackOverflow’s best Questions/Answers, so I flipped to Python. Petri’s example uses JaCoCo for coverage, which spits out a handy CSV report (as well as HTML). There are some text files in src/integration-test/meta/ccexample that show the sources covered by each test. Yes, they are checked in. Those files end in .java but are actually plain text (sorry): There’s another file src/integration-test/impact-map.txt that contains a list of production sources and the tests that would exercise them. Actually its a map of sources vs tests: Experimenting with what I’ve done Change one of the two production classes in src/main/java/ccexample. Yes they are clones of each other – that’s just something I did after forking to increase the class count at the start. Don’t commit that pending change, just leave it there showing up as modified in git status. Run python tests.py and watch it run one or two tests in the same invocation. Now undo that change, and change the other source file, and run python tests.py again. Different tests ran, right? Undo that change, and do python tests.py once more. No tests ran, right? That would be the same for changed sources/classes that had no tests exercising them (covering them under test invocation) at all. Of course these few tests are really quick, but they could have been three subset from hundreds or thousands of tests with an elapsed time of many hours. Turning the idea into a solution It is also worth noting that scripts as I have them are not robust or optimized. There’s more source-control systems than just Git, of course. The script needs to be able to work for a commit too, not just a pending commit. The storage format for impact-map.txt will not scale, and you might want to excise certain categories of POJO if every test exercises them. To be correct in Maven-land, this should be a bunch of plugins that fit the Maven style. One plugin would be invoked from Jenkins and would probably run constantly if corresponding test times are up in the hours. That Jenkins job should check in changes to the meta-files as it sees changes. This is benign, and useful to team members who may want the shorter build. Would you check the impact map into source-control? It seems to me that the map data is related to the other source files. Perhaps if you took a historical view to things, the impact maps change with the source code at the same time. If you were bug-fixing something from the past (checking out a prior revision), you might be happy that the impact map also goes back in time. Of course there’s nothing here that couldn’t be stored in a key/value store, including the changing map over time. Or a service that could answer the question “what tests should I run if these source files are changed?” Except perhaps that uncommitted work (on your own workstation) isn’t going to be present in that store until after a commit, and the impact map data us updated to that into account. Taking the idea even further I’ve suggested that Selenium is the technology that would greatly lengthen test times. There are many other test technologies that are one, two or three orders slower than perfect unit tests. This idea is applicable to much more than just Selenium – the only requirement is to be able to measure coverage while individual tests are running. What I have done could also be extended to the “surefire” phase – almost identical to what I’ve done already – an opportunity for reuse of course. CI daemons like Jenkins could benefit from the same impact-driven test time reduction. At least the per-commit jobs that we do for the Continuous Delivery era of enterprise development. This idea could be extended to the test-method level. That would be harder still, but achievable in the same way. It comes with arguably negligible gains, though. You’d code it all, and work out that 5% wasn’t worth it (versus other ways of speeding up tests). We’d need Intellij and Eclipse plugins for this. The script needs to be able to work for range of commits as many teams batch them in in Jenkins-land. I wonder what is out there that already does this sort of thing already. Followups (Jan 13, 17, 2015) Markus Kohler notes: This is a great idea and I guess similar to what Google does (there was blog post about it, can’t find it ATM). But I think as it is it would not be completely correct. The reason is that as far as I can see, you just run the tests that use the modified class files. That is not enough, because in Java changing one source code file might result in other files needed to be recompiled. Gradle’s Java plugin (http://www.gradle.org/docs/current/userguide/java_plugin.html) computes that set of files for example for incremental compilation. As an example if a final static String is changed and that String is used in other classes, these classes has to be recompiled because the Java compiler inlines these kind of strings. Any Test that uses these classes would have to be re-run. He’s quite right there’s room for some edge case mistakes. To overcome those, you would need to check for changes that would lead to final static fields being inlined into other classes. Maybe a list of source files that, if changed, would cause all tests to be re-run (after a suitable warning).

My objective here is to recreate a similar set-up in a smaller unit test mode.

NEW EDIT: As of Spring Boot 1.4.0, faking of Spring Beans is supported natively via annotation @MockBean. Read Spring Boot docs for more info. OLD EDIT: Here is better example how to mock Spring bean. I've worked with Spring for several years. But I was always frustrated with how messy can XML configuration become. As various annotations and possibilities of Java configuration were popping up, I started to enjoy programming with Spring. That is why I strongly entourage using Java configuration. In my opinion, XML configuration is suitable only when you need to have visualized Spring Integration or Spring Batch flow. Hopefully Spring Tool Suite will be able to visualize Java configurations for these frameworks also. One of the nasty aspects of XML configuration is that it often leads to huge XML configuration files. Developers therefore often create test context configuration for integration testing. But what is the purpose of integration testing, when there isn’t production wiring tested? Such integration test has very little value. So I was always trying to design my production contexts in testable fashion. I except that when you are creating new project / module you would avoid XML configuration as much as possible. So with Java configuration you can create Spring configuration per module / package and scan them in main context (@Configuration is also candidate for component scanning). This way you can naturally create islands Spring beans. These islands can be easily tested in isolation. But I have to admit that it’s not always possible to test production Java configuration as is. Rarely you need to amend behavior or spy on certain beans. There is library for it called Springockito. To be honest I didn’t use it so far, because I always try to design Spring configuration to avoid need for mocking. Looking at Springockito pace of development and number of open issues, I would be little bit worried to introduce it into my test suite stack. Fact that last release was done before Spring 4 release brings up questions like “Is it possible to easily integrate it with Spring 4?”. I don’t know, because I didn’t try it. I prefer pure Spring approach if I need to mock Spring bean in integration test. Spring provides @Primary annotation for specifying which bean should be preferred in the case when two beans with same type are registered. This is handy because you can override production bean with fake bean in integration test. Let’s explore this approach and some pitfalls on examples. I chose this simplistic / dummy production code structure for demonstration: @Repository public class AddressDao { public String readAddress(String userName) { return "3 Dark Corner"; } } @Service public class AddressService { private AddressDao addressDao; @Autowired public AddressService(AddressDao addressDao) { this.addressDao = addressDao; } public String getAddressForUser(String userName){ return addressDao.readAddress(userName); } } @Service public class UserService { private AddressService addressService; @Autowired public UserService(AddressService addressService) { this.addressService = addressService; } public String getUserDetails(String userName){ String address = addressService.getAddressForUser(userName); return String.format("User %s, %s", userName, address); } } AddressDao singleton bean instance is injected into AddressService. AddressService is similarly used in UserService. I have to warn you at this stage. My approach is slightly invasive to production code. To be able to fake existing production beans, we have to register fake beans in integration test. But these fake beans are usually in the same package sub-tree as production beans (assuming you are using standard Maven files structure: “src/main/java” and “src/test/java”). So when they are in the same package sub-tree, they would be scanned during integration tests. But we don’t want to use all bean fakes in all integration tests. Fakes could break unrelated integration tests. So we need to have mechanism, how to tell the test to use only certain fake beans. This is done by excluding fake beans from component scanning completely. Integration test explicitly define which fake/s are being used (will show this later). Now let’s take a look at mechanism of excluding fake beans from component scanning. We define our own marker annotation: public @interface BeanMock { } And exclude @BeanMock annotation from component scanning in main Spring configuration. @Configuration @ComponentScan(excludeFilters = @Filter(BeanMock.class)) @EnableAutoConfiguration public class Application { } Root package of component scan is current package of Application class. So all above production beans needs to be in same package or sub-package. We are now need to create integration test forUserService. Let’s spy on address service bean. Of course such testing doesn’t make practical sense with this production code, but this is just example. So here is our spying bean: @Configuration @BeanMock public class AddressServiceSpy { @Bean @Primary public AddressService registerAddressServiceSpy(AddressService addressService) { return spy(addressService); } } Production AddressService bean is autowired from production context, wrapped into Mockito‘s spy and registered as primary bean for AddressService type. @Primary annotation makes sure that our fake bean will be used in integration test instead of production bean. @BeanMock annotation ensures that this bean can’t be scanned by Application component scanning. Let’s take a look at the integration test now: @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = { Application.class, AddressServiceSpy.class }) public class UserServiceITest { @Autowired private UserService userService; @Autowired private AddressService addressService; @Test public void testGetUserDetails() { // GIVEN - spring context defined by Application class // WHEN String actualUserDetails = userService.getUserDetails("john"); // THEN Assert.assertEquals("User john, 3 Dark Corner", actualUserDetails); verify(addressService, times(1)).getAddressForUser("john"); } } @SpringApplicationConfigration annotation has two parameters. First (Application.class) declares Spring configuration under test. Second parameter (AddressServiceSpy.class) specifies fake bean that will be loaded for our testing into Spring IoC container. It’s obvious that we can use as many bean fakes as needed, but you don’t want to have many bean fakes. This approach should be used rarely and if you observe yourself using such mocking often, you are probably having serious problem with tight coupling in your application or within your development team in general. TDD methodology should help you target this problem. Bear in mind: “Less mocking is always better!”. So consider production design changes that allow for lower usage of mocks. This applies also for unit testing. Within integration test we can autowire this spy bean and use it for various verifications. In this case we verified if testing method userService.getUserDetails called methodaddressService.getAddressForUser with parameter “john”. I have one more example. In this case we wouldn’t spy on production bean. We will mock it: @Configuration @BeanMock public class AddressDaoMock { @Bean @Primary public AddressDao registerAddressDaoMock() { return mock(AddressDao.class); } } Again we override production bean, but this time we replace it with Mockito’s mock. We can than record behavior for mock in our integration test: @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = { Application.class, AddressDaoMock.class }) public class AddressServiceITest { @Autowired private AddressService addressService; @Autowired private AddressDao addressDao; @Test public void testGetAddressForUser() { // GIVEN when(addressDao.readAddress("john")).thenReturn("5 Bright Corner"); // WHEN String actualAddress = addressService.getAddressForUser("john"); // THEN Assert.assertEquals("5 Bright Corner", actualAddress); } @After public void resetMock() { reset(addressDao); } } We load mocked bean via @SpringApplicationConfiguration‘s parameter. In test method, we stubaddressDao.readAddress method to return “5 Bright Corner” string when “john” is passed to it as parameter. But bear in mind that recorded behavior can be carried to different integration test via Spring context. We don’t want tests affecting each other. So you can avoid future problems in your test suite by reseting mocks after test. This is done in method resetMock. Source code is on Github.

Authors of PVS-Studio static code analyzers offer programmers to test their sight and to try finding errors in C/C++ code fragments. Code analyzers work tirelessly and are able to find many bugs that can be difficult to notice. We chose some code fragments in which we had founded some errors using PVS-Studio. Quiz is not intended to check C++ language knowledge. There are many quality and interesting tests. For instance, we would recommend this C++ Quiz then. In our case, we made our test just for fun. We quite frequently hear an opinion that code analyzers are pointless tools. It is possible to find misplaced parenthesis or comma in five seconds. However, analyzer would not find difficult logical errors. Therefore, this tool could be useful only for students. We decided to troll these people. There is a time limit in tests. We ask them to find an error in five seconds. Well, OK, not in five seconds, but in a minute. Fifteen randomly selected problems would be shown. Every solved problem worth one point, but only if user provided the answer in one minute. We want to stress that we are not talking about syntax errors. We found all these code fragments in open-source projects that compiles flawlessly. Let us explain on a pair of examples how to point out the correct answer. First example. For instance, you got this code: The bug here is highlighted with red color. Of course, there would be no such emphasizing in a quiz problem. Programmer accidently made a misprint and wrote index 3 instead of index 2. Mouse cursor movement would highlight fragments of code, such as words and numbers. You should point the cursor into number 3 and press left mouse button. This would be the correct answer. Second example. It is not always possible to point out the error exactly. Buffer size should be compared with number 48. An excess sizeof() operator was put there by accident. In result, buffer size is compared with size of int type. At my opinion, an error there is in sizeof operator, and it is required to point it out to score a correct answer. However, without knowledge about the whole text, it is possible to think this way. Sizeof operator should have evaluated the size of some buffer, but accidently evaluates the value of the macro. The error is in “SSL3_MASTER_SECRET_LENGTH” usage. In this case, the answer will be scored no matter what you choose: “sizeof” or “SSL3_MASTER_SECRET_LENGTH”. Good luck! You can start a game. Footnote. Test does not support mobile devices. It is very easy to miss with finger. We are working on new version of tests with better mobile devices support, new problems to solve etc. However, it is not implemented yet. We offer you to subscribe on twitter to read about our new and interesting news and to read about new things in a C++ world.

{editor's note: thanks to yegor bugayenko, a new mvb at dzone. among other things, yegor blogs about java and devops. we're pleased to have him on board as a most valuable blogger. check out his blog, yegor256.com .} tl;dr orm is a terrible anti-pattern that violates all principles of object-oriented programming, tearing objects apart and turning them into dumb and passive data bags. there is no excuse for orm existence in any application, be it a small web app or an enterprise-size system with thousands of tables and crud manipulations on them. what is the alternative? sql-speaking objects . vinni-pukh (1969) by fyodor khitruk how orm works object-relational mapping (orm) is a technique (a.k.a. design pattern) of accessing a relational database from an object-oriented language (java, for example). there are multiple implementations of orm in almost every language; for example: hibernate for java, activerecord for ruby on rails, doctrine for php, and sqlalchemy for python. in java, the orm design is even standardized as jpa . first, let's see how orm works, by example. let's use java, postgresql, and hibernate. let's say we have a single table in the database, called post : +-----+------------+--------------------------+ | id | date | title | +-----+------------+--------------------------+ | 9 | 10/24/2014 | how to cook a sandwich | | 13 | 11/03/2014 | my favorite movies | | 27 | 11/17/2014 | how much i love my job | +-----+------------+--------------------------+ now we want to crud-manipulate this table from our java app (crud stands for create, read, update, and delete). first, we should create a post class (i'm sorry it's so long, but that's the best i can do): @entity @table(name = "post") public class post { private int id; private date date; private string title; @id @generatedvalue public int getid() { return this.id; } @temporal(temporaltype.timestamp) public date getdate() { return this.date; } public title gettitle() { return this.title; } public void setdate(date when) { this.date = when; } public void settitle(string txt) { this.title = txt; } } before any operation with hibernate, we have to create a session factory: sessionfactory factory = new annotationconfiguration() .configure() .addannotatedclass(post.class) .buildsessionfactory(); this factory will give us "sessions" every time we want to manipulate with post objects. every manipulation with the session should be wrapped in this code block: session session = factory.opensession(); try { transaction txn = session.begintransaction(); // your manipulations with the orm, see below txn.commit(); } catch (hibernateexception ex) { txn.rollback(); } finally { session.close(); } when the session is ready, here is how we get a list of all posts from that database table: list posts = session.createquery("from post").list(); for (post post : (list) posts){ system.out.println("title: " + post.gettitle()); } i think it's clear what's going on here. hibernate is a big, powerful engine that makes a connection to the database, executes necessary sql select requests, and retrieves the data. then it makes instances of class post and stuffs them with the data. when the object comes to us, it is filled with data, and we should use getters to take them out, like we're using gettitle() above. when we want to do a reverse operation and send an object to the database, we do all of the same but in reverse order. we make an instance of class post , stuff it with the data, and ask hibernate to save it: post post = new post(); post.setdate(new date()); post.settitle("how to cook an omelette"); session.save(post); this is how almost every orm works. the basic principle is always the same — orm objects are anemic envelopes with data. we are talking with the orm framework, and the framework is talking to the database. objects only help us send our requests to the orm framework and understand its response. besides getters and setters, objects have no other methods. they don't even know which database they came from. this is how object-relational mapping works. what's wrong with it, you may ask? everything! what's wrong with orm? seriously, what is wrong? hibernate has been one of the most popular java libraries for more than 10 years already. almost every sql-intensive application in the world is using it. each java tutorial would mention hibernate (or maybe some other orm like toplink or openjpa) for a database-connected application. it's a standard de-facto and still i'm saying that it's wrong? yes. i'm claiming that the entire idea behind orm is wrong. its invention was maybe the second big mistake in oop after null reference . actually, i'm not the only one saying something like this, and definitely not the first. a lot about this subject has already been published by very respected authors, including ormhate by martin fowler, object-relational mapping is the vietnam of computer science by jeff atwood, the vietnam of computer science by ted neward, orm is an anti-pattern by laurie voss, and many others. however, my argument is different than what they're saying. even though their reasons are practical and valid, like "orm is slow" or "database upgrades are hard", they miss the main point. you can see a very good, practical answer to these practical arguments given by bozhidar bozhanov in his orm haters don’t get it blog post. the main point is that orm, instead of encapsulating database interaction inside an object, extracts it away, literally tearing a solid and cohesive living organism apart. one part of the object keeps the data while another one, implemented inside the orm engine (session factory), knows how to deal with this data and transfers it to the relational database. look at this picture; it illustrates what orm is doing. i, being a reader of posts, have to deal with two components: 1) the orm and 2) the "obtruncated" object returned to me. the behavior i'm interacting with is supposed to be provided through a single entry point, which is an object in oop. in the case of orm, i'm getting this behavior via two entry points — the orm and the "thing", which we can't even call an object. because of this terrible and offensive violation of the object-oriented paradigm, we have a lot of practical issues already mentioned in respected publications. i can only add a few more. sql is not hidden . users of orm should speak sql (or its dialect, like hql ). see the example above; we're calling session.createquery("from post") in order to get all posts. even though it's not sql, it is very similar to it. thus, the relational model is not encapsulated inside objects. instead, it is exposed to the entire application. everybody, with each object, inevitably has to deal with a relational model in order to get or save something. thus, orm doesn't hide and wrap the sql but pollutes the entire application with it. difficult to test . when some object is working a list of posts, it needs to deal with an instance of sessionfactory . how can we mock this dependency? we have to create a mock of it? how complex is this task? look at the code above, and you will realize how verbose and cumbersome that unit test will be. instead, we can write integration tests and connect the entire application to a test version of postgresql. in that case, there is no need to mock sessionfactory , but such tests will be rather slow, and even more important, our having-nothing-to-do-with-the-database objects will be tested against the database instance. a terrible design. again, let me reiterate. practical problems of orm are just consequences. the fundamental drawback is that orm tears objects apart, terribly and offensively violating the very idea of what an object is . sql-speaking objects what is the alternative? let me show it to you by example. let's try to design that class, post , my way. we'll have to break it down into two classes: post and posts , singular and plural. i already mentioned in one of my previous articles that a good object is always an abstraction of a real-life entity. here is how this principle works in practice. we have two entities: database table and table row. that's why we'll make two classes; posts will represent the table, and post will represent the row. as i also mentioned in that article , every object should work by contract and implement an interface. let's start our design with two interfaces. of course, our objects will be immutable. here is how posts would look: @immutable interface posts { iterable iterate(); post add(date date, string title); } this is how a single post would look: @immutable interface post { int id(); date date(); string title(); } here is how we will list all posts in the database table: posts posts = // we'll discuss this right now for (post post : posts.iterate()){ system.out.println("title: " + post.title()); } here is how we will create a new post: posts posts = // we'll discuss this right now posts.add(new date(), "how to cook an omelette"); as you see, we have true objects now. they are in charge of all operations, and they perfectly hide their implementation details. there are no transactions, sessions, or factories. we don't even know whether these objects are actually talking to the postgresql or if they keep all the data in text files. all we need from posts is an ability to list all posts for us and to create a new one. implementation details are perfectly hidden inside. now let's see how we can implement these two classes. i'm going to use jcabi-jdbc as a jdbc wrapper, but you can use something else or just plain jdbc if you like. it doesn't really matter. what matters is that your database interactions are hidden inside objects. let's start with posts and implement it in class pgposts ("pg" stands for postgresql): @immutable final class pgposts implements posts { private final source dbase; public pgposts(datasource data) { this.dbase = data; } public iterable iterate() { return new jdbcsession(this.dbase) .sql("select id from post") .select( new listoutcome( new listoutcome.mapping() { @override public post map(final resultset rset) { return new pgpost(rset.getinteger(1)); } } ) ); } public post add(date date, string title) { return new pgpost( this.dbase, new jdbcsession(this.dbase) .sql("insert into post (date, title) values (?, ?)") .set(new utc(date)) .set(title) .insert(new singleoutcome(integer.class)) ); } } next, let's implement the post interface in class pgpost : @immutable final class pgpost implements post { private final source dbase; private final int number; public pgpost(datasource data, int id) { this.dbase = data; this.number = id; } public int id() { return this.number; } public date date() { return new jdbcsession(this.dbase) .sql("select date from post where id = ?") .set(this.number) .select(new singleoutcome(utc.class)); } public string title() { return new jdbcsession(this.dbase) .sql("select title from post where id = ?") .set(this.number) .select(new singleoutcome(string.class)); } } this is how a full database interaction scenario would look like using the classes we just created: posts posts = new pgposts(dbase); for (post post : posts.iterate()){ system.out.println("title: " + post.title()); } post post = posts.add(new date(), "how to cook an omelette"); system.out.println("just added post #" + post.id()); you can see a full practical example here . it's an open source web app that works with postgresql using the exact approach explained above — sql-speaking objects. what about performance? i can hear you screaming, "what about performance?" in that script a few lines above, we're making many redundant round trips to the database. first, we retrieve post ids with select id and then, in order to get their titles, we make an extra select title call for each post. this is inefficient, or simply put, too slow. no worries; this is object-oriented programming, which means it is flexible! let's create a decorator of pgpost that will accept all data in its constructor and cache it internally, forever: @immutable final class constpost implements post { private final post origin; private final date dte; private final string ttl; public constpost(post post, date date, string title) { this.origin = post; this.dte = date; this.ttl = title; } public int id() { return this.origin.id(); } public date date() { return this.dte; } public string title() { return this.ttl; } } pay attention: this decorator doesn't know anything about postgresql or jdbc. it just decorates an object of type post and pre-caches the date and title. as usual, this decorator is also immutable. now let's create another implementation of posts that will return the "constant" objects: @immutable final class constpgposts implements posts { // ... public iterable iterate() { return new jdbcsession(this.dbase) .sql("select * from post") .select( new listoutcome( new listoutcome.mapping() { @override public post map(final resultset rset) { return new constpost( new pgpost(rset.getinteger(1)), utc.gettimestamp(rset, 2), rset.getstring(3) ); } } ) ); } } now all posts returned by iterate() of this new class are pre-equipped with dates and titles fetched in one round trip to the database. using decorators and multiple implementations of the same interface, you can compose any functionality you wish. what is the most important is that while functionality is being extended, the complexity of the design is not escalating, because classes don't grow in size. instead, we're introducing new classes that stay cohesive and solid, because they are small. what about transactions? every object should deal with its own transactions and encapsulate them the same way as select or insert queries. this will lead to nested transactions, which is perfectly fine provided the database server supports them. if there is no such support, create a session-wide transaction object that will accept a "callable" class. for example: final class txn { private final datasource dbase; public t call(callable callable) { jdbcsession session = new jdbcsession(this.dbase); try { session.sql("start transaction").exec(); t result = callable.call(); session.sql("commit").exec(); return result; } catch (exception ex) { session.sql("rollback").exec(); throw ex; } } } then, when you want to wrap a few object manipulations in one transaction, do it like this: new txn(dbase).call( new callable() { @override public integer call() { posts posts = new pgposts(dbase); post post = posts.add(new date(), "how to cook an omelette"); posts.comments().post("this is my first comment!"); return post.id(); } } ); this code will create a new post and post a comment to it. if one of the calls fail, the entire transaction will be rolled back. this approach looks object-oriented to me. i'm calling it "sql-speaking objects", because they know how to speak sql with the database server. it's their skill, perfectly encapsulated inside their borders. related posts you may also find these posts interesting: how much your objects encapsulate? how an immutable object can have state and behavior? seven virtues of a good object how immutability helps paired brackets

First and foremost, it is important to define what AWS Activate is and what it is used for before we can take a deeper look. Exactly one year ago, Amazon created a program specifically designed for a particular group of customers that often times is in need of as much help as they can get (AKA startups). This program supports startups in their initial phase of building their businesses. This includes providing AWS credits, taking part in startup contests, and receiving benefits from third party solutions on the AWS cloud. Activate allows AWS partners that want to create a presence within the Activate community offer perks to member startups. Some of which include discounts and extended free tiers. Some startups that have attained high levels of success with AWS include Spotify, Pinterest, and Dropbox. With the big shots maintaining their places in startup stardom, Amazon has opened its doors to the next generation of innovators. As such, Amazon offers two different Activate packages. The Self-Starter package is comprised of a limited amount of each of the offerings listed above, whereas the Portfolio package includes some added bonuses along the lines of more high-profile and technical support as well as more in-depth training. On his blog AWS’ CTO, Werner Vogel, reiterated the importance of startups, “Startups will forever be a very important customer segment of AWS. They were among our first customers and along the way some amazing businesses have been built by these startups, many of which running for 100% on AWS.” “We’re excited to be a part of this global momentum in the startup ecosystem. The challenge now is to support and assist an increasing number of startups across the world.” The fun doesn’t stop there. In April of this year, AWS expanded the Activate package to offer much more than generalupport. This entailed sponsoring solution architects to take startups through step by step consultations in the fields of security, architecture and performance. Consequently, though Amazon’s professional services teams were established for customers, it was natural to have them take part in Activate. By nurturing new startups and making them rely heavily on the AWS cloud. As we can see today, companies that started with AWS 4 years ago are now worth billions of dollars. Airbnb and Dropbox, for example, now thoroughly enjoy the flexibility Amazon offers, as well as the fact that they no longer have to maintain cumbersome IT operations. Why not from the get-go? So the question is, if Amazon essentially built AWS on startups, why hasn’t Activate been around from the get-go, 6 years ago? AWS owes a great deal of its success to scalable startups that wanted and needed servers to run their businesses, yet didn’t have the initial capital to build their own data centers. No one really knows why Amazon did not provide startups back then with the kind of support they do today. However, as the market matured, it became clear that Amazon realized that an increasing number of startups could use their help. As a result, Amazon discovered that marketing their support services through Venture Capitalists and incubators around the world would include them as partners in this program and aid in marketing the service to startups of all kinds. “AWS Activate requires a special registration that allows startup customers with a valid AWS account to apply for either a self-starter package or a portfolio package. If a startup is a member of one of the accelerators, seed funds, or startup organizations that Amazon already works with, they may apply for the more exclusive AWS Activate Portfolio Package.” Learn More Incubators and Accelerators It was a natural step for Amazon to partner with accelerators all over the world with the Activate package. In addition to supporting startups, as mentioned above, these accelerators act as channels in the startup scene.At the first AWS re:Invent, Bezos jokes to his fellow investors, saying that eventually some of the investments will return to him because of how heavily the startup scene relies on Amazon. Activate and the approximately 150 accelerators across the world, including White Accel, Techstars, Appwest, and Battery Ventures, genuinely support and understand the values of the AWS service. They are happy to be able to use the Activate platform to help their startups flourish within the AWS clouds. 3rd Party Partners Aside from the accelerators, as an Amazon partner, you can enroll special offers to Activate members. For example, members that are part of the Self-Starter package may receive a 3 month free trial for Chef, whereas Portfolio members may receive a 6 month trial. Most of the partners will provide an extended free trial or credits via Activate. For instance, Trend Micro, one of Amazon’s biggest partners in the security domain, provides $2500 credit for Activate members in the Portfolio package. While there are not many partners on the list, the ones that are mentioned are very helpful and provide nice benefits for Activate members. Reviews of the program from both the partners’ and startups’ side showed that Activate is ideal for startups that have resource constraints. While members within the Self-Starter package are able to use the AWS Free Usage Tier, Portfolio members can receive anywhere from $1,000 to $15,000 in AWS Promotional Credit. The credit is maybe the most important value for these startups. Bearing in mind that Google also has their own line of packages and credit for new companies, it makes sense for AWS to start giving more life to these companies, above the free tier. Everyone has access to the free tier, these startups simply get more of it. Seems that there is no downside to participating. There is no obligation and the worst thing that can happen is that you will find that the services are great, and simply continue using them, which may result in you being locked-in to the point where you need to eventually pay. On the other hand, seems that the last announcement in April, which is actually “meet our architects”. Meaning the knowledge that Amazon’s architects share with startups in their consultation sessions help them get a better grasp on the ecosystem, as well as understand that more resource utilization is ultimately the next logical step for growth. All in all, although Amazon didn’t offer with this program 4 years ago, the AWS cloud was still the natural choice for startups. It included all of the benefits a startup can get using and online and on-demand infinite amount of resources. As a result, it is the clear choice for web scale startups. There are many reasons why Amazon only recently decided to offer free benefits to their prized potential customers. While it could have stemmed from competition from Microsoft and Google, or Amazon may want to simply show their support for their potential customers, demonstrating their cloud’s benefits at an early stage. Aside from that, Amazon understands and is built on companies with long term goals and possibilities. Therefore Amazon sees startups as a long term investment, which starts off with little risk.

serenity bdd (previously known as thucydides ) is an open source reporting library that helps you write better structured, more maintainable automated acceptance criteria, and also produces rich meaningful test reports (or "living documentation") that not only report on the test results, but also what features have been tested. and for when your automated acceptance tests exercise a web interface, serenity comes with a host of features that make writing your automated web tests easier and faster. 1. bdd fundamentals but before we get into the nitty-gritty details, let’s talk about behaviour driven development, which is a core concept underlying many of serenity’s features. behaviour driven development, or bdd, is an approach where teams use conversations around concrete examples to build up a shared understanding of the features they are supposed to build. for example, suppose you are building a site where artists and craftspeople can sell their good online. one important feature for such a site would be the search feature. you might express this feature using a story-card format commonly used in agile projects like this: in order for buyers to find what they are looking for more efficiently as a seller i want buyers to be able to search for articles by keywords to build up a shared understanding of this requirement, you could talk through a few concrete examples. the converstaion might go something like this: "so give me an example of how a search might work." "well, if i search for wool , then i should see only woolen products." "sound’s simple enough. are there any other variations on the search feature that would produce different outcomes?" "well, i could also filter the search results; for example, i could look for only handmade woolen products." and so on. in practice, many of the examples that get discussed become "acceptance criteria" for the features. and many of these acceptance criteria become automated acceptance tests. automating acceptence tests provides valuable feedback to the whole team, as these tests, unlike unit and integrationt tests, are typically expressed in business terms, and can be easily understood by non-developers. and, as we will se later on in this article, the reports that are produced when these teste are executed give a clear picture of the state of the application. 2. serenity bdd and junit in this article, we will learn how to use serenity bdd using nothing more than junit, serenity bdd, and a little selenium webdriver. automated acceptance tests can use more specialized bdd tools such as cucumber or jbehave, but many teams like to keep it simple, and use more conventional unit testing tools like junit. this is fine: the essence of the bdd approach lies in the conversations that the teams have to discuss the requirements and discover the acceptance criteria. 2.1. writing the acceptance test let’s start off with a simple example. the first example that was discussed was searching for wool . the corresponding automated acceptance test for this example in junit looks like this: @runwith(serenityrunner.class) public class whensearchingbykeyword { @managed(driver="chrome", uniquesession = true) webdriver driver; @steps buyersteps buyer; @test public void should_see_a_list_of_items_related_to_the_specified_keyword() { // given buyer.opens_etsy_home_page(); // when buyer.searches_for_items_containing("wool"); // then. buyer.should_see_items_related_to("wool"); } } the serenity test runner sets up the test and records the test results this is a web test, and serenity will manage the webdriver driver for us we hide implementation details about how the test will be executed in a "step library" our test itself is reduced to the bare essential business logic that we want to demonstrate there are several things to point out here. when you use serenity with junit, you need to use the serenityrunner test runner. this instruments the junit class and instantiates the webdriver driver (if it is a web test), as well as any step libraries and page objects that you use in your test (more on these later). the @managed annotation tells serenity that this is a web test. serenity takes care of instantiating the webdriver instance, opening the browser, and shutting it down at the end of the test. you can also use this annotation to specify what browser you want to use, or if you want to keep the browser open during all of the tests in this test case. the @steps annotation tells serenity that this variable is a step library. in serenity, we use step libraries to add a layer of abstraction between the "what" and the "how" of our acceptance tests. at the top level, the step methods document "what" the acceptance test is doing, in fairly implementation-neutral, business-friendly terms. so we say "searches for items containing wool ", not "enters wool into the search field and clicks on the search button". this layered approach makes the tests both easier to understand and to maintain, and helps build up a great library of reusable business-level steps that we can use in other tests. 2.2. the step library the step library class is just an ordinary java class, with methods annotated with the @step annotation: public class buyersteps { homepage homepage; searchresultspage searchresultspage; @step public void opens_etsy_home_page() { homepage.open(); } @step public void searches_for_items_containing(string keywords) { homepage.searchfor(keywords); } @step public void should_see_items_related_to(string keywords) { list resulttitles = searchresultspage.getresulttitles(); resulttitles.stream().foreach(title -> assertthat(title.contains(keywords))); } } //end:tail step libraries often use page objects, which are automatically instantiated the @step annotation indicates a method that will appear as a step in the test reports for automated web tests, the step library methods do not call webdriver directly, but rather they typically interact with page objects . 2.3. the page objects page objects encapsulate how a test interacts with a particular web page. they hide the webdriver implementation details about how elements on a page are accessed and manipulated behind more business-friendly methods. like steps, page objects are reusable components that make the tests easier to understand and to maintain. serenity automatically instantiates page objects for you, and injects the current webdriver instance. all you need to worry about is the webdriver code that interacts with the page. and serenity provides a few shortcuts to make this easier as well. for example, here is the page object for the home page: @defaulturl("http://www.etsy.com") public class homepage extends pageobject { @findby(css = "button[value='search']") webelement searchbutton; public void searchfor(string keywords) { $("#search-query").sendkeys(keywords); searchbutton.click(); } } what url should be used by default when we call the open() method a serenity page object must extend the pageobject class you can use the $ method to access elements directly using css or xpath expressions or you may use a member variable annotated with the @findby annotation and here is the second page object we use: public class searchresultspage extends pageobject { @findby(css=".listing-card") list listingcards; public list getresulttitles() { return listingcards.stream() .map(element -> element.gettext()) .collect(collectors.tolist()); } } in both cases, we are hiding the webdriver implementation of how we access the page elements inside the page object methods. this makes the code both easier to read and reduces the places you need to change if a page is modified. this approach encourages a very high degree of reuse. for example, the second example mentioned at the start of this article involved filtering results by type. the corresponding automated acceptance criteria might look like this: @test public void should_be_able_to_filter_by_item_type() { // given buyer.opens_etsy_home_page(); // when buyer.searches_for_items_containing("wool"); int unfiltereditemcount = buyer.get_matching_item_count(); // and buyer.filters_results_by_type("handmade"); // then buyer.should_see_items_related_to("wool"); // and buyer.should_see_item_count(lessthan(unfiltereditemcount)); } @test public void should_be_able_to_view_details_about_a_searched_item() { // given buyer.opens_etsy_home_page(); // when buyer.searches_for_items_containing("wool"); buyer.selects_item_number(5); // then buyer.should_see_matching_details(); } notice how most of the methods here are reused from the previous steps: in fact, only two new methods are required. 3. reporting and living documentation reporting is one of serenity’s fortes. serenity not only reports on whether a test passes or fails, but documents what it did, in a step-by-step narrative format that inculdes test data and screenshots for web tests. for example, the following page illustrates the test results for our first acceptance criteria: figure 1. test results reported in serenity but test outcomes are only part of the picture. it is also important to know what work has been done, and what is work in progress. serenity provides the @pending annotation, that lets you indicate that a scenario is not yet completed, but has been scheduled for work, as illustrated here: @runwith(serenityrunner.class) public class whenputtingitemsintheshoppingcart { @pending @test public void shouldupdateshippingpricefordifferentdestinationcountries() { } } this test will appear in the reports as pending (blue in the graphs): figure 2. test result overview we can also organize our acceptance tests in terms of the features or requirements they are testing. one simple approach is to organize your requirements in suitably-named packages: |----net | |----serenity_bdd | | |----samples | | | |----etsy | | | | |----features | | | | | |----search | | | | | | |----whensearchingbykeyword.java | | | | | | |----whenviewingitemdetails.java | | | | | |----shopping_cart | | | | | | |----whenputtingitemsintheshoppingcart.java | | | | |----pages | | | | | |----homepage.java | | | | | |----itemdetailspage.java | | | | | |----registerpage.java | | | | | |----searchresultspage.java | | | | | |----shoppingcartpage.java | | | | |----steps | | | | | |----buyersteps.java all the test cases are organized under the features directory. test cass related to the search feature test cases related to the ‘shopping cart’ feature serenity can use this package structure to group and aggregate the test results for each feature. you need to tell serenity the root package that you are using, and what terms you use for your requirements. you do this in a special file called (for historical reasons) thucydides.properties , which lives in the root directory of your project: thucydides.test.root=net.serenity_bdd.samples.etsy.features thucydides.requirement.types=feature,story with this configured, serenity will report about how well each requirement has been tested, and will also tell you about the requirements that have not been tested: figure 3. serenity reports on requirements as well as tests 4. conclusion hopefully this will be enough to get you started with serenity. that said, we have barely scratched the surface of what serenity can do for your automated acceptance tests. you can read more about serenity, and the principles behind it, by reading the users manual , or by reading bdd in action , which devotes several chapters to these practices. and be sure to check out the online courses at parleys . you can get the source code for the project discussed in this article on github .

This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications. Authenticating iOS app users with Azure Active Directory How to Best handle AAD access tokens in native mobile apps Using Azure SSO tokens for Multiple AAD Resources From Native Mobile Apps(this post) Sharing Azure SSO access tokens across multiple native mobile apps. Brief Start In an enterprise context, it is highly likely that you would have multiple web services that your native mobile app needs to consume. I had exactly this scenario, where one of my clients had asked if they could maintain the same token in the background in the mobile app to use it for accessing multiple web services. I spent some time digging through the documentation and conducting some experiments to confirm some points. Therefore, this post is to share my findings on accessing multiple Azure AD resources from native mobile apps using ADAL. In the previous two posts, we looked at implementing Azure AD SSO login on native mobile apps, then we looked at how to best maintain these access tokens. This post discusses how to use Azure AD SSO tokens to manage access to multiple AAD resources. Let’s assume that we have 2 web services sitting in Azure (ie WebApi1, and WebApi2), both of which are set to use Azure AD authentication. Then, we have the native mobile app, which needs access to both web services (WebApi1, and WebApi2). Let’s look at what we can and cannot do. Cannot Use the Same Azure AD Access-Token for Multiple Resources The first thing that comes to mind is to use the same access token for multiple Azure AD resources, and that is what the client asked about. However, this is not allowed. Azure AD issues a token for certain resource (which is mapped to an Azure AD app). When we call AcquireToken(), we need to provide a resourceID, only ONE resourceID. The result would have a token that can only be used for the supplied resource (id). There are ways where you could use the same token (as we will see later in this post), but it is not recommended as it complicates operations logging, authentication process tracing, etc. Therefore it is better to look at the other options provided by Azure and the ADAL library. Use the Refresh-Token to Acquire Tokens for Multiple Resources The ADAL library supports acquiring multiple access-Tokens for multiple resources using a refresh token. This means once a user is authenticated, the ADAL’s authentication context, would be able to generate an access-token to multiple resources without authenticating the user again. This was mentioned briefly by the MSDN documentation here. The refresh token issued by Azure AD can be used to access multiple resources. For example, if you have a client application that has permission to call two web APIs, the refresh token can be used to get an access token to the other web API as well. (MSDN documentation) public async Task RefreshTokens() { var tokenEntry = await tokensRepository.GetTokens(); var authorizationParameters = new AuthorizationParameters (_controller); var result = "Refreshed an existing Token"; bool hasARefreshToken = true; if (tokenEntry == null) { var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId1, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); tokenEntry = new Tokens { WebApi1AccessToken = localAuthResult.AccessToken, RefreshToken = localAuthResult.RefreshToken, Email = localAuthResult.UserInfo.DisplayableId, ExpiresOn = localAuthResult.ExpiresOn }; hasARefreshToken = false; result = "Acquired a new Token"; } var refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync(tokenEntry.RefreshToken, clientId, resourceId2); tokenEntry.WebApi2AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; if (hasARefreshToken) { // this will only be called when we try refreshing the tokens (not when we are acquiring new tokens. refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync (refreshAuthResult.RefreshToken, clientId, resourceId1); tokenEntry.WebApi1AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; } await tokensRepository.InsertOrUpdateAsync (tokenEntry); return result; } As you can see from above, we check if we have an access-token from previous runs, and if we do, we refresh the access-tokens for both web services. Notice how the _authContext.AcquireTokenByRefreshTokenAsync() provides an overloading parameter that takes a resourceId. This enables us to get multiple access tokens for multiple resources without having to re-authenticate the user. The rest of the code is similar to what we have seen in the previous two posts. ADAL Library Can Produce New Tokens For Other Resources In the previous two posts, we looked at ADAL library and how it uses TokenCache. Although ADAL does not support persistent caching of tokens yet on mobile apps, it still uses the TokenCache for in-memory caching. This enables ADAL library to generate new access-tokens if the context (AuthenticationContext) still exists from previous authentications. Remember in the previous post we said it is recommended to keep a reference to the authentication-context? Here it comes in handy, as it enables us to generate new access-tokens for accessing multiple Azure AD resources. var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId2, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); Calling AcquireToken() (even with no refresh-token) would give us a new access-token to webApi2. This is due to ADAL great goodness where it checks if we have a refresh-token in-memory (managed by ADAL), then it uses that to generate a new access-token for webApi2. An alternative The third alternative option is the simplest, but not necessarily the best. In this option, we could use the same access token to consume multiple Azure AD resources. To do this, we need to use the same Azure AD app ID when setting the web application’s authentication. This requires some understanding of how the Azure AD authentication happens on our web apps. If you refer to Taiseer Joudeh’s tutorial, which we mentioned before, you will see that in our web app, we need to tell the authentication framework what’s our Authority and the Audience (Azure AD app Id). If we set up both of our web apps, to use the same Audience (Azure AD app Id), meaning that we link them both into the same Azure AD application, then we could use the same access-token to use both web services. // linking our web app authentication to an Azure AD application private void ConfigureAuth(IAppBuilder app) { app.UseWindowsAzureActiveDirectoryBearerAuthentication( new WindowsAzureActiveDirectoryBearerAuthenticationOptions { Audience = ConfigurationManager.AppSettings["Audience"], Tenant = ConfigurationManager.AppSettings["Tenant"] }); } As we said before, this is very simple and requires less code, but could cause complications in terms of security logging and maintenance. At the end of the day, it depends on your context and what you are trying to achieve. Therefore, I thought it would be worth mentioning and I will leave the judgement for you on which option you choose. Conclusions We looked at how we could use Azure AD SSO with ADAL to access multiple resources from native mobile apps. As we saw, there are three main options, and the choice could be made based on the context of your app. I hope you find this useful and if you have any questions or you need help with some development that you are doing, then just get in touch. This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications.

both high availability (ha) and disaster recovery (dr) have been essential it topics. fundamentally ha is about fault tolerance relevant to the availability of an examined subject like application, database, vms, etc. while dr roots on the ability to resume operations in the aftermath of a catastrophic event. a fundamental difference of these two is that ha expects no down time and no data loss, while dr does. they are different issues and should be addressed separately. background for many it shops, either ha or dr has been a high risk and high cost item. both are essential to business continuity, while traditionally tough technical problems to solve with very significant and long-term commitments on resources. not only they are technically challenging, but a continual cost-cutting which has become an it standard practice in the past two decades makes purchasing hardware/software and constructing either ha or dr solution on premises further distant from it’s financial and technical realties. sense of urgency too often, the technical challenges and resource commitments overwhelm it and turn ha and dr into academic discussions, or symbolic items on a project checklist. at the same time, information is rapidly exploding as internet, mobility and social-network are becoming integral in our daily lives and businesses. there are progressively more data to process and store. for many businesses, the needs for ha and dr is urgent for better managing risks. and continual availability and on-demand recoverability of it are becoming increasingly critical. this is the reality, now the good news is that the recent introduction of cloud computing has fundamentally changed how an ha or dr solution can be implemented. microsoft azure is a vivid example of ha and dr solutions with significantly reduced the required financial commitment and involved technical complexities. the traditional approach by establishing redundancy and acquiring a physical dr site with long-term resources and financial commitments is now largely replaced with consumable services which can be configured in minutes by mouse-clicking and with a manageable cost structure based on usage. ha and dr have become it solutions which are financially realistic and technically feasible for businesses in all sizes. ha, redundancy, and microsoft azure lrs ha is to eliminate a single point of failure of an examined component, an application for example. it denotes a strategy to employ redundancy such that a target application can and will continue being available without downtime while experiencing a failure of hosting hardware or software. there are various and well-developed ha solutions like a hyper-v host cluster using redundant hardware to eliminate a single point of failure of hosting os or hardware, and an application cluster for eliminating a single point of failure by running the application in multiple vm instances with a synchronous state. although ha implementations may vary, the fundamental principle nevertheless remains the same. ha expects neither downtime nor data loss while experiencing an outage of a target hardware or software. ha has become dramatically simple in microsoft azure. basically, all data written to disk in microsoft azure are kept at least in the so-called lrs, locally redundant storage. lrs replicates a transaction synchronously to three different storage nodes across fault domains and upgrade domains within the same region for durability. in layman’s terms, microsoft azure by default maintains at least three copies of user data to achieve ha. dr, replication, and microsoft azure grs dr is about having a plan and backups in place to resume operations in the aftermath of a catastrophic event. unplanned outage is assumed in a dr scenario, therefore some data loss is also expected. notice that ha and dr are different business problems and addressed differently. while both ha and dr are based on applying redundancy, i.e. a source and replicas, or multiple identical nodes of an examines component like application instance, databases, or vms, there are however differences between the two. a dr solution generally employs replicas or backups, are implemented with asynchronous processes, and expects an outage of a source and with some data loss in transit while the outage occurs. while ha requires a logical representation with a real-time integrity using synchronous processes across all participating nodes, expects neither downtime nor data loss while experiencing an outage of a participating node. for a critical workload, one approach of dr is to establish geo-replication to address an outage of an entire geographic area caused by a natural disaster, for example. the concern is that a catastrophic event may impact an entire geographic area causing a datacenter where a mission critical application is being hosted becomes unavailable for an extended period of time. in microsoft azure, geo redundant storage or grs is the default and an optional setting, as shown above, while configuring a storage account. grs will queue a transaction committed to lrs as an asynchronous replication to a secondary region, a few hundreds miles away from the primary region where a storage account is originated. at the secondary region, data is also stored in lrs, i.e. made durable by replicating it to three storage nodes. specifically, a microsoft azure storage account configured with grs essentially maintains three replicas locally for high availability, and replicates the content and maintains three replicas at a secondary datacenter a few hundreds miles away for dr. so all are six copies, three locally and three remotely. all these are configured by one, yes one mouse click from a dropdown list while creating a storage account. the above is a conceptual model illustrated a data flow of grs. grs replication has little performance impact on an application since application data are committed to lrs in real-time while replication to grs is queued, i.e. asynchronously. a write to lrs is synchronous and in real-time, once committed, the changes are expected within 15 minutes to be asynchronously replicated to the secondary site. for a ra-grs storage account, in addition to one primary endpoint for read/write operations as it is in a grs, there is also one secondary endpoint as read only becomes available as shown below. the cost implications of grs or ra-grs include the additional storage and the transmission costs for egress traffic, as applicable, of the secondary datacenter. ingress traffic is free . and microsoft azure storage sla offers 99.9% availability and a cost calculator is also available. microsoft azure recovery services so far, much is about backing up or replicating data. to successfully restore, a dr plan must be put in place and ensure its availability upon a dr scenario in progress. either placing a dr plan at a primary site where the source is or a secondary site where a replica stays has some issues and concerns. keeping a dr plan at the source site where all the resources are in place and on-the-job trainings seems logical. or does it? dr is assuming a catastrophic event over an extended geographic areas where the source site is experiencing an outage. in such case, keeping a dr plan in the source site defeats the purpose. maintaining a dr plan at the secondary site is the choice then. in a dr scenario, a recovery site is to be brought on line within a expected period of time according to a dr plan, and having the dr plan right there and then at a recovery site makes all the sense. or does it? this decision introduces a number of requirements including the physical readiness, the timeliness, and the financial implications on securing and maintaining a dr plan at a remote physical facility. for a vmm server running on system center 2012 sp1 or later, an idea, reliable and straightforward way is to use azure recovery services to maintain a dr plan as shown below. and for any backup needs, using cloud as a backup site makes backing up and restoring data an anytime anywhere operation. azure site recovery vault this service essentially acts as the director of a dr process. it orchestrates and manages the protection and failover of vms in clouds managed by virtual machine manager 2012 sp1 or later. a noticeable advantage is the ability to test a recovery configuration, exercise a proactive failover and recovery, and automate recovery in the event of a site outage. the sla of site recovery services is 99.9% availability to ensure a configured dr plan is always in place with expected updates. this is a dr solution that it can implement, simulate, verify, bring online and be absolutely confident with the readiness. azure backup vault this is a reliable, scalable and inexpensive data protection solution with zero capital investment and extremely low operational expense. like other secure communication with microsoft azure, you will first upload a public certificate to microsoft azure. then download the backup agent to register a target server with the backup vault. then select what to be backed up. both microsoft azure backup sla (99.9% availability) and cost calculator are available for better assessing the solution. closing thoughts form an application’s view, ha is an on-going event while dr is an anticipation. ha and dr are different business problems and should be addressed differently. nevertheless, microsoft azure provides a single platform to gracefully address ha with lrs, dr with grs, and dr orchestration with recovery services, and all with published sla s and a predictable cost structure . going forward, it pros can now include ha and dr as a reliable, scalable and relatively inexpensive proposition by employing microsoft azure as a solution platform. call to action register at microsoft virtual academy, http://aka.ms/mva1 , and train yourself on microsoft azure by taking the track of courses. go to http://aka.ms/azure200 and acquire a free trial subscription and assess microsoft azure for ha and dr solutions. review my recommended content at http://aka.ms/recommended .

When I needed to do prototyping, proof of concept or play with some new technology in free time, starting new project was always a little annoying barrier with Maven. Have to say that setting up Maven project is not hard and you can use Maven Archetypes. But Archetypes are often out of date. Who wants to play with old technologies? So I always end up wiring in dependencies I wanted to play with. Not very productive spent time. But than Spring Boot came to my way. I fell in love. In last few months I created at least 50 small playground projects, prototypes with Spring Boot. Also incorporated it at work. It’s just perfect for prototyping, learning, microservices, web, batch, enterprise, message flow or command line applications. You have to be dinosaur or be blind not to evaluate Spring Boot for your next Spring project. And when you finish evaluate it, you will go for it. I promise. I feel a need to highlight how easy is Black Box Testing of Spring Boot microservice. Black Box Testing refers to testing without any poking with application artifact. Such testing can be called also integration testing. You can also perform performance or stress testing way I am going to demonstrate. Spring Boot Microservice is usually web application with embedded Tomcat. So it is executed as JAR from command line. There is possibility to convert Spring Boot project into WAR artifact, that can be hosted on shared Servlet container. But we don’t want that now. It’s better when microservice has its own little embedded container. I used existing Spring’s REST service guide as testing target. Focus is mostly on testing project, so it is handy to use this “Hello World” REST application as example. I expect these two common tools are set up and installed on your machine: Maven 3 Git So we’ll need to download source code and install JAR artifact into our local repository. I am going to use command line to download and install the microservice. Let’s go to some directory where we download source code. Use these commands: git clone [email protected]:spring-guides/gs-rest-service.git cd gs-rest-service/complete mvn clean install If everything went OK, Spring Boot microservice JAR artifact is now installed in our local Maven repository. In serious Java development, it would be rather installed into shared repository (e.g. Artifactory, Nexus,… ). When our microservice is installed, we can focus on testing project. It is also Maven and Spring Boot based. Black box testing will be achieved by downloading the artifact from Maven repository (doesn’t matter if it is local or remote). Maven-dependency-plugin can help us this way: org.apache.maven.plugins maven-dependency-plugin copy-dependencies compile copy-dependencies gs-rest-service true It downloads microservice artifact into target/dependency directory by default. As you can see, it’s hooked to compile phase of Maven lifecycle, so that downloaded artifact is available during test phase. Artifact version is stripped from version information. We use latest version. It makes usage of JAR artifact easier during testing. Readers skilled with Maven may notice missing plugin version. Spring Boot driven project is inherited from parent Maven project called spring-boot-starter-parent. It contains versions of main Maven plugins. This is one of the Spring Boot’s opinionated aspects. I like it, because it provides stable dependencies matrix. You can change the version if you need. When we have artifact in our file system, we can start testing. We need to be able to execute JAR file from command line. I used standard JavaProcessBuilder this way: public class ProcessExecutor { public Process execute(String jarName) throws IOException { Process p = null; ProcessBuilder pb = new ProcessBuilder("java", "-jar", jarName); pb.directory(new File("target/dependency")); File log = new File("log"); pb.redirectErrorStream(true); pb.redirectOutput(Redirect.appendTo(log)); p = pb.start(); return p; } } This class executes given process JAR based on given file name. Location is hard-coded to target/dependency directory, where maven-dependency-plugin located our artifact. Standard and error outputs are redirected to file. Next class needed for testing is DTO (Data transfer object). It is simple POJO that will be used for deserialization from JSON. I use Lombok project to reduce boilerplate code needed for getters, setters, hashCode and equals. @Data @AllArgsConstructor @NoArgsConstructor public class Greeting { private long id; private String content; } Test itself looks like this: public class BlackBoxTest { private static final String RESOURCE_URL = "http://localhost:8080/greeting"; @Test public void contextLoads() throws InterruptedException, IOException { Process process = null; Greeting actualGreeting = null; try { process = new ProcessExecutor().execute("gs-rest-service.jar"); RestTemplate restTemplate = new RestTemplate(); waitForStart(restTemplate); actualGreeting = restTemplate.getForObject(RESOURCE_URL, Greeting.class); } finally { process.destroyForcibly(); } Assert.assertEquals(new Greeting(2L, "Hello, World!"), actualGreeting); } private void waitForStart(RestTemplate restTemplate) { while (true) { try { Thread.sleep(500); restTemplate.getForObject(RESOURCE_URL, String.class); return; } catch (Throwable throwable) { // ignoring errors } } } } It executes Spring Boot microservice process first and wait unit it starts. To verify if microservice is started, it sends HTTP request to URL where it’s expected. The service is ready for testing after first successful response. Microservice should send simple greeting JSON response for HTTP GET request. Deserialization from JSON into our Greeting DTO is verified at the end of the test. Source code is shared on Github.

Scrapy, being based on Twisted, introduces an incredible host of obstacles to easily and efficiently writing self-contained unit tests: 1. You can't call reactor.run() multiple times 2. You can't stop the reactor multiple times, so you can't blindly call "crawler.signals.connect(reactor.stop, signal=signals.spider_closed)" 3. Reactor runs in its own thread, so your failed assertions won't make it to the main unittest thread, so test failures will be thrown as assertion errors but unittest doesn't know about them To get around these hurdles, I created a BaseScrapyTestCase class that uses tl.testing's ThreadAwareTestCase and the following workarounds. class BaseScrapyTestCase(ThreadAwareTestCase): in_suite = False def setUp(self): self.last_crawler = None self.settings = get_project_settings() def run_reactor(self, called_from_suite=False): if not called_from_suite and BaseScrapyTestCase.in_suite: return log.start() self.last_crawler.signals.connect(reactor.stop, signal=signals.spider_closed) reactor.run() def queue_spider(self, spider, callback): crawler = Crawler(self.settings) self.last_crawler = crawler crawler.signals.connect(callback, signal=signals.spider_closed) crawler.configure() crawler.crawl(spider) crawler.start() return crawler def wrap_asserts(self, fn): with ThreadJoiner(1): self.run_in_thread(fn) You'll use it like so: class SimpleScrapyTestCase(BaseScrapyTestCase): def test_suite(self): BaseScrapyTestCase.in_suite = True self.do_test_simple() self.run_reactor(True) def do_test_simple(self): spider = Spider("site.com") def _fn(): def __fn(): self.assertTrue(False) self.wrap_asserts(__fn) self.queue_spider(spider, _fn) self.run_reactor() 1. Call run_reactor() at the end of test method. 2. You have to place your assertions in its own function which gets called in a ThreadJoiner so that unittest knows about assertion failures. 3. If you're testing multiple spiders, just call queue_spider() for each, and run_reactor() at the end. 4. BaseScrapyTestCase keeps track of the crawlers created, and makes sure to only attach a reactor.stop signal to the last one. Let me know if you come up with a better/more elegant way of testing scrapy spiders!

this article represents top 6 popular angularjs topics that has been used most by the angularjs developer community to date. the inference is derived based on number of tagged discussions happening on stackoverflow . clearly, “directive” is the winner and attracts most of them all. the article presents my thoughts on why these topics have been most popular. please feel free to comment/suggest if i missed to mention one or more important points. also, sorry for the typos. following is the list of top 6 popular topics: directives scope object ng-repeat angular ui & bootstrap routing service following plot demonstrates the popularity of different feature/topics in relation with angularjs. angularjs topics popularity inference : some of the following could as well be inferred from the above data/plot. three features which have been most used by the developers and therefore, should be key reasons why you would also want to use angular in next project are following: directives routing ng-repeat one of the pain point (or shortcoming) that have been talked most by the angular developers is the ui widgets related support by angular. this is where most of them have jumped to angular ui and bootstrap. the topic/concept that has intrigued most to several developers is scope object. thoughts on why these topics may be most popular following are top 6 popular topics in angularjs discussed on forums such as stackoverflow: directives : this is, no doubt, the most popular and powerful feature of angularjs directives as also indicated by count of discussion threads posted on stackoverflow as of today. the power of directives lies in the following and this is why it is the most popular topic of angularjs. re-usability : once created a directive as part of a module, all that one need to do to use the directive is include the module as a dependency when defining new module and define the directives wherever required on the page. usability : owing to the fact that one could give intuitive names to directives, directive enhances the readability and understandability of code by a notch. greater adherence to dry principle : the aspect of templating makes directive a very attractive feature. it does reduce the duplication of code as same html template code could be used at several places without the need to write the code in html file. scope object : this is second most popular topic found based on the discussion count. rightfully expected as well! the whole notion of scope object and how it is key to dependency injection makes it one of the most powerful as well as tricky concept of angularjs. also, this is one of the topic which raised the barrier to entry for angularjs and contributed in making steep learning curve for developers. that said, scope is going to r.i.p in angular 2.0 which could be seen as a good sign for those who always struggled with scope object. ng-repeat : the ng-repeat feature brings power to angularjs from the fact that it is one of the feature that removed the need of server-side code required to repeat the html code over multiple iterations. with ng-repeat, one could easily repeat html code multiple times. angular ui & bootstrap : one of the shortcoming of angularjs for good or bad is its inability to be one and all solution to create some great ui along with powerful eventing feature. for creating fancy or great looking ui, one would still have to go to ui frameworks such as bootstrap, kendo-ui etc. this is where people have been looking for angularui and bootstrap. angularui comes with attractive feature set for enhanced routing, grid util, angularjs code editor plugins, bootstrap module etc. routing : routing feature is key to creating single page application. one of key reason why angularjs is very popular is the ease with which one could create single-page application using it. and, routing feature makes it all happen. no doubt, this is why many developers have been looking for it. service : service feature helps one to create reusable components in an angular module. these services could then be injected in another modules using dependency injection feature. the service could be injected in one of the following components: controllers services doing a quick recap, one may recall that for creating a service, one could use factory recipe method and define service that way. you could know details about creating a custom service on our another page dedicated on this.

CompletableFuture from Java 8 is an advanced abstraction over a promise that value of type T will be available in the future. Observable is quite similar, but it promises arbitrary number of items in the future, from 0 to infinity. These two representations of asynchronous results are quite similar to the point where Observable with just one item can be used instead of CompletableFuture and vice-versa. On the other hand CompletableFuture is more specialized and because it's now part of JDK, should become prevalent quite soon. Let's celebrate RxJava 1.0 release with a short article showing how to convert between the two, without loosing asynchronous and event-driven nature of them. From CompletableFuture to Observable CompletableFuture represents one value in the future, so turning it into Observable is rather simple. When Futurecompletes with some value, Observable will emit that value as well immediately and close stream: class FuturesTest extends Specification { public static final String MSG = "Don't panic" def 'should convert completed Future to completed Observable'() { given: CompletableFuture future = CompletableFuture.completedFuture("Abc") when: Observable observable = Futures.toObservable(future) then: observable.toBlocking().toIterable().toList() == ["Abc"] } def 'should convert failed Future into Observable with failure'() { given: CompletableFuture future = failedFuture(new IllegalStateException(MSG)) when: Observable observable = Futures.toObservable(future) then: observable .onErrorReturn({ th -> th.message } as Func1) .toBlocking() .toIterable() .toList() == [MSG] } CompletableFuture failedFuture(Exception error) { CompletableFuture future = new CompletableFuture() future.completeExceptionally(error) return future } } First test of not-yet-implemented Futures.toObservable() converts Future into Observable and makes sure value is propagated correctly. Second test created failed Future, replaces failure with exception's message and makes sure exception was propagated. The implementation is much shorter: public static Observable toObservable(CompletableFuture future) { return Observable.create(subscriber -> future.whenComplete((result, error) -> { if (error != null) { subscriber.onError(error); } else { subscriber.onNext(result); subscriber.onCompleted(); } })); } NB: Observable.fromFuture() exists, however we want to take full advantage of ComplatableFuture's asynchronous operators. From Observable toCompletableFuture> There are actually two ways to convert Observable to Future - creating CompletableFuture> orCompletableFuture (if we assume Observable has just one item). Let's start from the former case, described with the following test cases: def 'should convert Observable with many items to Future of list'() { given: Observable observable = Observable.just(1, 2, 3) when: CompletableFuture> future = Futures.fromObservable(observable) then: future.get() == [1, 2, 3] } def 'should return failed Future when after few items exception was emitted'() { given: Observable observable = Observable.just(1, 2, 3) .concatWith(Observable.error(new IllegalStateException(MSG))) when: Futures.fromObservable(observable) then: def e = thrown(Exception) e.message == MSG } Obviously Future doesn't complete until source Observable signals end of stream. Thus Observable.never() would never complete wrapping Future, rather then completing it with empty list. The implementation is much shorter and sweeter: public static CompletableFuture> fromObservable(Observable observable) { final CompletableFuture> future = new CompletableFuture<>(); observable .doOnError(future::completeExceptionally) .toList() .forEach(future::complete); return future; } The key is Observable.toList() that conveniently converts from Observable and Observable>. The latter emits one item of List type when source Observable finishes. From Observable to CompletableFuture Special case of the previous transformation happens when we know that CompletableFuture will return exactly one item. In that case we can convert it directly to CompletableFuture, rather than CompletableFuture>with one item only. Tests first: def 'should convert Observable with single item to Future'() { given: Observable observable = Observable.just(1) when: CompletableFuture future = Futures.fromSingleObservable(observable) then: future.get() == 1 } def 'should create failed Future when Observable fails'() { given: Observable observable = Observable. error(new IllegalStateException(MSG)) when: Futures.fromSingleObservable(observable) then: def e = thrown(Exception) e.message == MSG } def 'should fail when single Observable produces too many items'() { given: Observable observable = Observable.just(1, 2) when: Futures.fromSingleObservable(observable) then: def e = thrown(Exception) e.message.contains("too many elements") } Again the implementation is quite straightforward and almost identical: public static CompletableFuture> fromObservable(Observable observable) { final CompletableFuture> future = new CompletableFuture<>(); observable .doOnError(future::completeExceptionally) .toList() .forEach(future::complete); return future; } Helpers methods above aren't fully robust yet, but if you ever need to convert between JDK 8 and RxJava style of asynchronous computing, this article should be enough to get you started.

A long while ago a optimisation was added to the JVM so that if the same exception is thrown again and again and again a single instance of the Exception is created without the stack trace filled in in order to increase performance. This is an excellent idea unless you are trying to diagnose a problem and you have missed the original error. If you forgot about this optimisation you send the afternoon looking at the following log output and weeping slightly. (In my defence I have a little one in the house hence the fuzzy brain and lack of blogging action this year) java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException ... java.lang.Exception: Uncaught exceptions during test at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/common-adf/CreateNewCustomAppAndProjectWithName.xml:25) at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/level0/testADFcMATS.xml:94) at oracle.abbot.JDevScriptFixture.runTest(JDevScriptFixture.java:555) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at junit.textui.TestRunner.doRun(TestRunner.java:116) at junit.textui.TestRunner.doRun(TestRunner.java:109) at oracle.abbot.AbbotRunner.run(AbbotRunner.java:614) at oracle.abbot.AbbotAddin$IdeAbbotRunner.run(AbbotAddin.java:634) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException [Crickets] It turns out that you can turn off this optimisation with a simple flag: java -XX:-OmitStackTraceInFastThrow .... In my particular case the actual exception causing this trouble was a NPE from the GlyphView code in JDK8. (One that is being caused by a glitch in hotspot it seems) But that in turn was causing the AIOOBE in some logging code clouding the issue even more. This in particular is a good flag to add by default when running your automated tests, particularly in combination with the stack trace length override I have talked about before.

today i'd like to introduce a new series of posts named untangling concepts . in the it business we are overloaded day by day with more and more information and sometimes it's difficult to handle all this knowledge without making mistakes. the main goal of these posts is to clarify the difference between concepts that sometimes are mixed together. in the first post i'll write about unit testing and acceptance testing. i'd like to emphasize that this post isn't about test driven development . tdd is a subject to other posts. unit tests when we talk about unit tests the first thing that should come to mind is the single responsibility principle . the name unit tests isn't a coincidence, you should test one small piece of code which does just one thing. i guarantee that if you code big classes with big methods which do a lot of stuff, you'll have trouble to write good and useful unit tests. high cohesion matters. so, what is the point in unit tests? imagine the design of a system. if you follow the object oriented software design good practices you will come up to a system where a lot of objects communicate with each other in order to achieve the program's goal. imagine the following situation: you have to implement an e-mail system. the program receives as input a list of e-mail addresses and have to send an e-mail to each one of the recipients. you might have something like this: in this case, someone will call the readrecipientlist method with an inputstream (e.g. a fileinputstream of a txt file with some e-mail addresses). after that, the method sendmails is called, returning the number of e-mails sent. but what should happen if the method sendmails is called before the method readrecipientlist is called? and what if i pass a fileinputstream of an empty file? or even a null inputstream object? how can i trust that the class mailsender is doing it's job right? those are the type of question that unit tests try to answer. when writing unit tests, our concern is to assert that our objects' internal structure is working as expected. it's like a puzzle... if i can assert that each piece is working as expected, i can trust that when the pieces are put together (in the right places... more about that in the next session), the puzzle will be completed correctly. and that's not everything. what if you have to change something inside one piece of code, or if you need to do some refactoring (you should do it a lot)? if you have tests to assert that this piece of code is working as expected you can run these tests after the changes and see if your modifications broke something. you can make changes more confidently. there are a lot to say about unit tests and how they can be beneficial to your code. here i just want that you understand the concept, but i'll put some links in the useful resources section so you can read a lot more about that. acceptance tests now let's talk about acceptance tests. put yourself in the place of a stakeholder. you know what you want the software to do but you know nothing about software development nor programming. how can you believe that the software is doing everything you want, in the way you want? unit tests are good to test small pieces of code and to assert that the code works as expected. but that isn't enough to be sure that the code is satisfying the stakeholders. acceptance tests are test cases written based on scenarios specified by the customer. usually each history will have at least one associated acceptance test. they work as black box tests , meaning that shouldn't be considered implementation details in those tests (one more reason to write them first...). it's like an input-output evaluation: " given some context, when something happen, then i expect the system to answer that (or to be in that state)." we use this given-when-then template to write good acceptance tests for a user history. let's try a little example. remember our e-mail system? which scenario do you think that our customer would like to satisfy? i believe that the most obvious is something like that: given that i am logged in the mail system, when i send an e-mail to [email protected] , then i expect that there is one new mail in someone's inbox. this is just one super basic example. it's not unusual to have acceptance tests with far more given and then clauses. the best way to write acceptance tests is using something that everyone can understand. but that's not easy. we can use, for example, selenium to capture tests based in user's input, but that depends in having at least a prototype interface where the user can navigate. in the last project i have been working on, we have successfully adopted acceptance tests written in gherkin , a language used by cucumber . for example, a more detailed version of our test of sending an e-mail would look like that in gherkin: feature: sending an e-mail to someone scenario: send an e-mail to an existing user given that i am authenticated in the mail system and that exists an account: [email protected] when i send an e-mail to [email protected] then someone's inbox must have one unread mail and my sent itens folder must have the sent mail as you can see, we are writing our tests in plain english. anyone can read them and understand what the system is supposed to do (the bold words have nothing to do with gherkin, it's a bug in the syntax highlight feature). i see two great benefits in writing acceptance tests before the development of a history. the first one is that we can be sure that the customer knows what he wants to be developed and that the developer knows what the customer wants . so we reduce the chance of misunderstandings, increasing customer satisfaction. the second is that we have an easy feedback when any of the user histories are broken during the development cycle. the more we run the acceptance tests, faster we can see if our system is not doing what it is supposed to do. and faster we fix it . conclusion in this article we learned that unit tests are the best tool that developers have to protect themselves from mistakes. the wider our test coverage is, more we can do refactoring with confidence, leading to better code. remember, code rots . acceptance tests is an excelent tool to help to make the gap between the customer desires and the developer understanding smaller. they help in the fast feedback loop, leading to software that makes our customer happier. remember that unit tests and acceptance tests complement each other. our goal is not just to build the right thing but make sure that we build the thing right . useful resources martin fowler's post: unit tests extremeprogramming.org site: page about unit tests extremeprogramming.org site: page about acceptance tests uncle bob's post: the truth about bdd

[This article was written by Barak Merimovich.] We have discussed OpenStack networking extensively in previous posts. In this post, I’d like to dive into a more advanced OpenStack networking scenario. Many cloud images are not configured to automatically bring up all network cards that are available. They will usually only have a single network card configured. To correctly set up a host in the cloud with multiple network cards, log on to the machine and bring up the additional interfaces. echo $'auto eth1\niface eth1 inet dhcp' | sudo tee /etc/network/interfaces.d/eth1.cfg > /dev/null sudo ifup eth1 Networks in the cloud A complex network architecture is a mainstay of modern IaaS clouds. Understanding how to configure your cloud-based networks, and hosts, is critical to getting your application working in the cloud. This is especially true with Cloudify, the open source cloud orchestration platform I work on. The cloud, like the world, used to be flat It was not that long a time ago that most IaaS providers only supported flat networks – all of your hosts were in one large network. Separation between services running in the cloud was enforced in software or with firewalls/security-groups. But technically, all of the hosts were connected to the same network and visible to each other. The flat network model is simple, and therefore easy to reason and understand. It was a good choice for the early days of the IaaS cloud and no doubt helped with getting applications into the cloud in the first place. It was one of the things that made EC2 so easy to use for anyone just starting out with the ‘cloud’. This model is in fact still available on Amazon Web Services under the title ‘EC2-Classic’. And for many applications, a flat network is good enough. But as cloud adoption increases, more complex applications are moving into the clouds, and issues like network separation, security, SLA and broadcast domains make more complex networks models a must. Software Defined Networks (SDN) fill that gap. They are now a staple of most major IaaS clouds. AWS has AWS-VPC, OpenStack has the Neutron project and there are many other implementations. Working with SDN requires knowing a bit more about how information moves around between your cloud resources. In this post I am going to discuss how to set up a host in the cloud so it will play nice with complex networks. I’ll be using OpenStack, but the concepts are similar for other cloud infrastructures. Openstack configuration I am going to start with an empty tenant, only the public network is available. First, lets set up out networks and router: neutron router-create demo-router neutron net-create demo-network-1 neutron net-create demo-network-2 neutron subnet-create --name demo-subnet-1 demo-network-1 10.0.0.0/24 neutron subnet-create --name demo-subnet-2 demo-network-2 10.0.1.0/24 neutron router-interface-add demo-router demo-subnet-1 neutron router-interface-add demo-router demo-subnet-2 neutron router-gateway-set demo-router public Note the network IDs: neutron net-list | id | name | subnets | | 2c33efe2-6204-4125-9716-3bc525630016 | demo-network-1 | 928dafa0-83ef-459c-b20d-71d8ea596fa2 10.0.0.0/24 | | aa30627e-c181-4a4b-89bf-5dd7c26c244e | demo-network-2 | 26d573f7-7953-4a54-825b-ed7bbc0661c7 10.0.1.0/24 | | e502de8d-929a-4ee0-bd18-efa297875cf6 | public | d40dab51-a729-452c-9ee6-b9ad08d10808 | We’ll start with a standard Ubuntu cloud image: glance image-create --name "Ubuntu 12.04 Standard" --location "http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img" --disk-format qcow2 --container-format bare Create the keypair and security group: nova keypair-add demo-keypair > demo-keypair.pem chmod 400 demo-keypair.pem nova secgroup-create demo-security-group "Security group for demo" nova secgroup-add-rule demo-security-group tcp 22 22 0.0.0.0/0 Let’s spin up an instance connected to both our networks: nova boot -flavor m1.small --image "Ubuntu 12.04 Standard" --nic net-id=2c33efe2-6204-4125-9716-3bc525630016 --nic net-id=aa30627e-c181-4a4b-89bf-5dd7c26c244e --security-groups demo-security-group --key-name demo-keypair demo-vm And set up floating IPs for the first network: nova list | ID | Name | Status | Task State | Power State | Networks | 2b17588b-8980-4489-9a04-6539a159dc3c | demo-vm | ACTIVE | None | Running | demo-network-1=10.0.0.2; demo-network-2=10.0.1.2 | neutron floatingip-create public neutron floatingip-list | id | fixed_ip_address | floating_ip_address | port_id | | 49c8b05e-bb8f-4b07-80ed-3155ab6ffc09 | | 192.168.15.42 | | neutron port-list | id | name | mac_address | fixed_ips | | 1ccfd334-7328-4b22-b93e-24a0888276ab | | fa:16:3e:14:39:39 | {"subnet_id": "94598487-c1fc-4f55-ac1f-ef2545d5cfeb", "ip_address": "10.0.1.3"} | | a482c4f6-fa74-476e-b1ce-cd8dd0c70815 | | fa:16:3e:18:92:79 | {"subnet_id": "94598487-c1fc-4f55-ac1f-ef2545d5cfeb", "ip_address": "10.0.1.2"} | | b23d7836-30c5-4bff-b873-15c87ba051f6 | | fa:16:3e:3a:28:40 | {"subnet_id": "dec6ec74-cfa9-4a08-8792-54900631b98e", "ip_address": "10.0.0.3"} | | d421b447-2adf-406f-876b-142238683344 | | fa:16:3e:9d:fc:7f | {"subnet_id": "dec6ec74-cfa9-4a08-8792-54900631b98e", "ip_address": "10.0.0.2"} | | dcf8696b-cc80-4b48-b09c-61c0f8ab02ac | | fa:16:3e:5b:39:fb | {"subnet_id": "94598487-c1fc-4f55-ac1f-ef2545d5cfeb", "ip_address": "10.0.1.1"} | | f6a1666e-495a-4d3f-afa3-754b3cb3cfc0 | | fa:16:3e:8a:1b:fb | {"subnet_id": "dec6ec74-cfa9-4a08-8792-54900631b98e", "ip_address": "10.0.0.1"} | neutron floatingip-associate 49c8b05e-bb8f-4b07-80ed-3155ab6ffc09 d421b447-2adf-406f-876b-142238683344 Note how we matched the VM’s IP to its port, and associated the floating IP to the port. I wish there was an easier way to do this from the CLI… If everything worked correctly, you should have the following setup: Let’s make sure ssh works correctly: ssh -i demo-keypair.pem [email protected] hostname demo-vm Cool, ssh works. Now, we should have two network cards, right? ssh -i demo-keypair.pem [email protected] hostname demo-vm Cool, ssh works. Now, we should have two network cards, right? ssh -i demo-keypair.pem [email protected] ifconfig eth0 Link encap:Ethernet HWaddr fa:16:3e:5f:a2:5f inet addr:10.0.0.4 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe5f:a25f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:230 errors:0 dropped:0 overruns:0 frame:0 TX packets:224 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:46297 (46.2 KB) TX bytes:31130 (31.1 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Huh?! The VM only has one working network interface! Where is my second NIC? Was there a configuration problem with the OpenStack network setup? The answer is here: ssh -i demo-keypair.pem [email protected] ifconfig -a eth0 Link encap:Ethernet HWaddr fa:16:3e:5f:a2:5f inet addr:10.0.0.4 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe5f:a25f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:324 errors:0 dropped:0 overruns:0 frame:0 TX packets:332 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:69973 (69.9 KB) TX bytes:47218 (47.2 KB) eth1 Link encap:Ethernet HWaddr fa:16:3e:29:6d:22 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) The second NIC exists, but is not running. The issue is not with the OpenStack network configuration – it’s with the image. The image itself should be configured to work correctly with multiple NICs. All we have to do is bring up the NIC. So we ssh into the instance: ssh -i demo-keypair.pem [email protected] And run the following commands: echo $'auto eth1\niface eth1 inet dhcp' | sudo tee /etc/network/interfaces.d/eth1.cfg > /dev/null sudo ifup eth1 The second NIC should now be running: ifconfig eth1 eth1 Link encap:Ethernet HWaddr fa:16:3e:18:92:79 inet addr:10.0.1.2 Bcast:10.0.1.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe18:9279/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:81 errors:0 dropped:0 overruns:0 frame:0 TX packets:45 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:15376 (15.3 KB) TX bytes:3960 (3.9 KB) And there you go – your VM can access both networks. This issue can make life complicated when setting up a complex, or even a not very complex, application. When will this issue hurt you? Well, imagine a scenario where you have a web server and a database server. The web server is connected to both Network1 and Network2, and the database server is only connected to Network2. Network1 is connected to the external world over a router, and Network 2 is completely internal, adding another layer of security to the critical database server. So what happens if the web server only has one network card? If only the NIC for Network1 is up, the web server can’t access the database. If only the NIC for Network2 is up, the web server can’t be reached from the external world. Even worse, if this web server is accessed via a floating IP, this IP will also not work, so you won’t be able to access the web server and fix the issue. Tricky. In conclusion The above commands will bring up your additional network card. You will of-course need to repeat this process for each additional network card, and for each VM. You can use a start-up script (a.k.a. user-data script) or system service to run these commands, but there are better ways. I’ll discuss how to automate the network setup in a follow-up post. This was originally posted at Barak's blog Head in the Clouds, find it here.

For a long time JSF for me was just another web framework I didn’t care too much about. This changed. After being forced to use it for a couple of months now, I consider it a major project risk in almost all cases. Here I present the reasons for this verdict. Bad entanglement of UI and processing Logic The official tutorial claims the following about the benefits of JSF: One of the greatest advantages of Java Server Faces technology is that it offers a clean separation between behavior and presentation for web applications. The opposite is the case. Facelets, the preferred presentation technology of JSF looks at first sight like an ordinary templating technology like the good old JSP or Thyme Leaf. But if you look closer the horror becomes obvious. In the same place where you structure your HTML, you also place the logic what parts of the UI should get updated on an action. A clear violation of the separation of concerns principle in my book. Even better is the immediate attribute which changes the server side life cycle! And if this isn’t enough it does it in different waysdepending on what tag you use it on. You can’t make stuff like this up. It tries to abstract what you can not abstract. Except some weird edge cases clients and server of web application are located on rather different computers, separated by some kind of network. From this follows a simple fact: communication between client and server is slow and unreliable. JSF tries to abstract away the separation of client and server. It processes everything on the backend wildly communicating between client and server in a hard to control way. The result are all kind of failure scenarios just popping into existence because you use JSF. For me the most annoying one is this one: If you open a JSF page, let’s say a simple search page, wait an hour, then hit the submit button you will get an exception because the server side state expired. WAT? Why is there server state of any relevance for a trivial search page? (Yes I know you can change that behavior with the latest versions of JSF, but it is still the way JSF is designed to work) I though everybody learned since EJBs: If you want to abstract over the fact, if two parts of an application run on the same machine or not, you have to assume they don’t. Everything else is just hiding problems until they grow so large that they can eat your project for breakfast. Making stuff complex and complicated that was easy to start with.The architecture of the World Wide Web is a simple one. Simple meaning: It consists of a small set of concepts with limited interaction. This is what made it so widely successful. It also makes it not obvious for beginners how to use it to implement certain features. I’m sure most of us remember the first time they tried to implement something like a shopping cart without having session state. But the solutions for almost all these problems are well known and understood by know. And all you need is a little reading and what you gain is a strong conceptual understanding how to solve this kind of issue. And again, the basics are extremely simple: You send a request to an URL, with some headers and content using a HTTP verb. And you reply with some resource containing links and some headers. And you don’t have state in the server session. Making load balancing and fail over rather simple. Making bookmarkable URLs trivial. Making your site searchable for zero costs. Making your site cachable. Allowing the user to use their back buttons, history and tabs as they wish. Making it trivial to have nice URLs Compare that to the live cycle model of JSF: The page from which a user submitted a request will get synchronized with a model on the server side, then submitted values validated, converted, events generated and processed. As mentioned above the order in which things happen, and if they happen at all are controlled by XML Tags hidden away in a document camouflaged as markup. Apart from hardly anybody properly understanding all this (BalusC seems to be the only one available in the interwebs) it has the following effect on your application: The URLs become ugly. You’ll see the URL of the resource you came from instead of the one you are looking at, thus making bookmarking URLs as useful as a doorknob on your knee. Same for caching, fail over, load balancing and so on. Sure you can fix it with some convention here, and an additional library there. Which of course makes perfect sense when you are in the business of breaking stuff so people have to pay you for fixing it. I personally prefer helping to solve real problems. Hindering testability: I can’t speak for most frameworks but I can compare Spring MVC with JSF. Let me tell you this: If anybody is telling you JSF is nicely testable he probably doesn’t know automatic testing. With JSF you can test your backend beans using unit tests. You can test the whole UI, by deploying the application to a server and hitting it with Selenium. That’s basically it. Just in case you are wondering what else one should be able to test: Load a static version of a page in a browser and testing it with selenium, in order test your Client side UI behavior. Test your generated markup without starting a full blown application server. Test the mapping of attributes/parameter to bean methods. Test your generated markup without bootstrapping a complete application. All this is perfectly possible with Spring MVC and probably with many other sane server side frameworks, but not with JSF … Again: I’m aware there are fixes for many issues, but the simplest fix is> Don’t use JSF.

This article explains how errors are handled when using the messaging system with Spring Integration and how to handle route and redirect to specific channel.

This technical tip shows how to read Excel worksheet cells values in multiple threads simultaneously inside android applications. Often you need to read worksheet cells values in multiple threads simultaneously. To do so, set Worksheet.getCells().setMultiThreadReading() to true. If you do not set this property you might get the wrong cell values. Setting it to true, you always get the correct values. To achieve this task first create a workbook and adds a worksheet. Populates the worksheet with some string values then create two threads that simultaneously read values from random cells. If the values read are correct, then nothing happens. But if the values read are incorrect, then a message box shows up in the LogCat window. If you comment this line: testWorkbook.getWorksheets().get(0).getCells().setMultiThreadReading(true); the following message will show up in LogCat window: if (s.equals("R" + row + "C" + col)!=true) { System.out.println("This message box will show up when cells read values are incorrect."); } Otherwise, the program run without showing any message which means all values read from cells are correct. public class ThreadProc implements Runnable { boolean isRunning = true; Workbook testWorkbook; Random r = new Random(); public ThreadProc(Workbook workbook) { this.testWorkbook = workbook; } public int randomNext(int Low, int High) { int R = r.nextInt(High-Low) + Low; return R; } public void kill() { this.isRunning = false; } public void run(){ while(this.isRunning) { int row = randomNext(0, 100); int col = randomNext(0, 10); String s = testWorkbook.getWorksheets().get(0).getCells().get(row, col).getStringValue(); if (s.equals("R" + row + "C" + col)!=true) { System.out.println("This message box will show up when cells read values are incorrect."); } } } } //........MainActivity.java........ //................................. import java.io.File; import java.util.Random; import android.app.Activity; import android.os.Bundle; import android.os.Environment; import android.view.Menu; import com.aspose.cells.CellsHelper; import com.aspose.cells.IWarningCallback; import com.aspose.cells.WarningInfo; import com.aspose.cells.WarningType; import com.aspose.cells.Workbook; public class MainActivity extends Activity { @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); try{ Workbook testWorkbook = new Workbook(); testWorkbook.getWorksheets().clear(); testWorkbook.getWorksheets().add("Sheet1"); for (int row = 0; row < 100; row++) for (int col = 0; col < 10; col++) testWorkbook.getWorksheets().get(0).getCells().get(row, col).setValue("R" + row + "C" + col); //Commenting this line will show a pop-up message testWorkbook.getWorksheets().get(0).getCells().setMultiThreadReading(true); ThreadProc tp = new ThreadProc(testWorkbook); Thread myThread1 = new Thread(tp); myThread1.start(); Thread myThread2 = new Thread(tp); myThread2.start(); Thread.currentThread().sleep(5*1000); tp.kill(); } catch (Exception e) { e.printStackTrace(); } } ......... }