This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications. Authenticating iOS app users with Azure Active Directory How to Best handle AAD access tokens in native mobile apps Using Azure SSO tokens for Multiple AAD Resources From Native Mobile Apps(this post) Sharing Azure SSO access tokens across multiple native mobile apps. Brief Start In an enterprise context, it is highly likely that you would have multiple web services that your native mobile app needs to consume. I had exactly this scenario, where one of my clients had asked if they could maintain the same token in the background in the mobile app to use it for accessing multiple web services. I spent some time digging through the documentation and conducting some experiments to confirm some points. Therefore, this post is to share my findings on accessing multiple Azure AD resources from native mobile apps using ADAL. In the previous two posts, we looked at implementing Azure AD SSO login on native mobile apps, then we looked at how to best maintain these access tokens. This post discusses how to use Azure AD SSO tokens to manage access to multiple AAD resources. Let’s assume that we have 2 web services sitting in Azure (ie WebApi1, and WebApi2), both of which are set to use Azure AD authentication. Then, we have the native mobile app, which needs access to both web services (WebApi1, and WebApi2). Let’s look at what we can and cannot do. Cannot Use the Same Azure AD Access-Token for Multiple Resources The first thing that comes to mind is to use the same access token for multiple Azure AD resources, and that is what the client asked about. However, this is not allowed. Azure AD issues a token for certain resource (which is mapped to an Azure AD app). When we call AcquireToken(), we need to provide a resourceID, only ONE resourceID. The result would have a token that can only be used for the supplied resource (id). There are ways where you could use the same token (as we will see later in this post), but it is not recommended as it complicates operations logging, authentication process tracing, etc. Therefore it is better to look at the other options provided by Azure and the ADAL library. Use the Refresh-Token to Acquire Tokens for Multiple Resources The ADAL library supports acquiring multiple access-Tokens for multiple resources using a refresh token. This means once a user is authenticated, the ADAL’s authentication context, would be able to generate an access-token to multiple resources without authenticating the user again. This was mentioned briefly by the MSDN documentation here. The refresh token issued by Azure AD can be used to access multiple resources. For example, if you have a client application that has permission to call two web APIs, the refresh token can be used to get an access token to the other web API as well. (MSDN documentation) public async Task RefreshTokens() { var tokenEntry = await tokensRepository.GetTokens(); var authorizationParameters = new AuthorizationParameters (_controller); var result = "Refreshed an existing Token"; bool hasARefreshToken = true; if (tokenEntry == null) { var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId1, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); tokenEntry = new Tokens { WebApi1AccessToken = localAuthResult.AccessToken, RefreshToken = localAuthResult.RefreshToken, Email = localAuthResult.UserInfo.DisplayableId, ExpiresOn = localAuthResult.ExpiresOn }; hasARefreshToken = false; result = "Acquired a new Token"; } var refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync(tokenEntry.RefreshToken, clientId, resourceId2); tokenEntry.WebApi2AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; if (hasARefreshToken) { // this will only be called when we try refreshing the tokens (not when we are acquiring new tokens. refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync (refreshAuthResult.RefreshToken, clientId, resourceId1); tokenEntry.WebApi1AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; } await tokensRepository.InsertOrUpdateAsync (tokenEntry); return result; } As you can see from above, we check if we have an access-token from previous runs, and if we do, we refresh the access-tokens for both web services. Notice how the _authContext.AcquireTokenByRefreshTokenAsync() provides an overloading parameter that takes a resourceId. This enables us to get multiple access tokens for multiple resources without having to re-authenticate the user. The rest of the code is similar to what we have seen in the previous two posts. ADAL Library Can Produce New Tokens For Other Resources In the previous two posts, we looked at ADAL library and how it uses TokenCache. Although ADAL does not support persistent caching of tokens yet on mobile apps, it still uses the TokenCache for in-memory caching. This enables ADAL library to generate new access-tokens if the context (AuthenticationContext) still exists from previous authentications. Remember in the previous post we said it is recommended to keep a reference to the authentication-context? Here it comes in handy, as it enables us to generate new access-tokens for accessing multiple Azure AD resources. var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId2, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); Calling AcquireToken() (even with no refresh-token) would give us a new access-token to webApi2. This is due to ADAL great goodness where it checks if we have a refresh-token in-memory (managed by ADAL), then it uses that to generate a new access-token for webApi2. An alternative The third alternative option is the simplest, but not necessarily the best. In this option, we could use the same access token to consume multiple Azure AD resources. To do this, we need to use the same Azure AD app ID when setting the web application’s authentication. This requires some understanding of how the Azure AD authentication happens on our web apps. If you refer to Taiseer Joudeh’s tutorial, which we mentioned before, you will see that in our web app, we need to tell the authentication framework what’s our Authority and the Audience (Azure AD app Id). If we set up both of our web apps, to use the same Audience (Azure AD app Id), meaning that we link them both into the same Azure AD application, then we could use the same access-token to use both web services. // linking our web app authentication to an Azure AD application private void ConfigureAuth(IAppBuilder app) { app.UseWindowsAzureActiveDirectoryBearerAuthentication( new WindowsAzureActiveDirectoryBearerAuthenticationOptions { Audience = ConfigurationManager.AppSettings["Audience"], Tenant = ConfigurationManager.AppSettings["Tenant"] }); } As we said before, this is very simple and requires less code, but could cause complications in terms of security logging and maintenance. At the end of the day, it depends on your context and what you are trying to achieve. Therefore, I thought it would be worth mentioning and I will leave the judgement for you on which option you choose. Conclusions We looked at how we could use Azure AD SSO with ADAL to access multiple resources from native mobile apps. As we saw, there are three main options, and the choice could be made based on the context of your app. I hope you find this useful and if you have any questions or you need help with some development that you are doing, then just get in touch. This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications.

both high availability (ha) and disaster recovery (dr) have been essential it topics. fundamentally ha is about fault tolerance relevant to the availability of an examined subject like application, database, vms, etc. while dr roots on the ability to resume operations in the aftermath of a catastrophic event. a fundamental difference of these two is that ha expects no down time and no data loss, while dr does. they are different issues and should be addressed separately. background for many it shops, either ha or dr has been a high risk and high cost item. both are essential to business continuity, while traditionally tough technical problems to solve with very significant and long-term commitments on resources. not only they are technically challenging, but a continual cost-cutting which has become an it standard practice in the past two decades makes purchasing hardware/software and constructing either ha or dr solution on premises further distant from it’s financial and technical realties. sense of urgency too often, the technical challenges and resource commitments overwhelm it and turn ha and dr into academic discussions, or symbolic items on a project checklist. at the same time, information is rapidly exploding as internet, mobility and social-network are becoming integral in our daily lives and businesses. there are progressively more data to process and store. for many businesses, the needs for ha and dr is urgent for better managing risks. and continual availability and on-demand recoverability of it are becoming increasingly critical. this is the reality, now the good news is that the recent introduction of cloud computing has fundamentally changed how an ha or dr solution can be implemented. microsoft azure is a vivid example of ha and dr solutions with significantly reduced the required financial commitment and involved technical complexities. the traditional approach by establishing redundancy and acquiring a physical dr site with long-term resources and financial commitments is now largely replaced with consumable services which can be configured in minutes by mouse-clicking and with a manageable cost structure based on usage. ha and dr have become it solutions which are financially realistic and technically feasible for businesses in all sizes. ha, redundancy, and microsoft azure lrs ha is to eliminate a single point of failure of an examined component, an application for example. it denotes a strategy to employ redundancy such that a target application can and will continue being available without downtime while experiencing a failure of hosting hardware or software. there are various and well-developed ha solutions like a hyper-v host cluster using redundant hardware to eliminate a single point of failure of hosting os or hardware, and an application cluster for eliminating a single point of failure by running the application in multiple vm instances with a synchronous state. although ha implementations may vary, the fundamental principle nevertheless remains the same. ha expects neither downtime nor data loss while experiencing an outage of a target hardware or software. ha has become dramatically simple in microsoft azure. basically, all data written to disk in microsoft azure are kept at least in the so-called lrs, locally redundant storage. lrs replicates a transaction synchronously to three different storage nodes across fault domains and upgrade domains within the same region for durability. in layman’s terms, microsoft azure by default maintains at least three copies of user data to achieve ha. dr, replication, and microsoft azure grs dr is about having a plan and backups in place to resume operations in the aftermath of a catastrophic event. unplanned outage is assumed in a dr scenario, therefore some data loss is also expected. notice that ha and dr are different business problems and addressed differently. while both ha and dr are based on applying redundancy, i.e. a source and replicas, or multiple identical nodes of an examines component like application instance, databases, or vms, there are however differences between the two. a dr solution generally employs replicas or backups, are implemented with asynchronous processes, and expects an outage of a source and with some data loss in transit while the outage occurs. while ha requires a logical representation with a real-time integrity using synchronous processes across all participating nodes, expects neither downtime nor data loss while experiencing an outage of a participating node. for a critical workload, one approach of dr is to establish geo-replication to address an outage of an entire geographic area caused by a natural disaster, for example. the concern is that a catastrophic event may impact an entire geographic area causing a datacenter where a mission critical application is being hosted becomes unavailable for an extended period of time. in microsoft azure, geo redundant storage or grs is the default and an optional setting, as shown above, while configuring a storage account. grs will queue a transaction committed to lrs as an asynchronous replication to a secondary region, a few hundreds miles away from the primary region where a storage account is originated. at the secondary region, data is also stored in lrs, i.e. made durable by replicating it to three storage nodes. specifically, a microsoft azure storage account configured with grs essentially maintains three replicas locally for high availability, and replicates the content and maintains three replicas at a secondary datacenter a few hundreds miles away for dr. so all are six copies, three locally and three remotely. all these are configured by one, yes one mouse click from a dropdown list while creating a storage account. the above is a conceptual model illustrated a data flow of grs. grs replication has little performance impact on an application since application data are committed to lrs in real-time while replication to grs is queued, i.e. asynchronously. a write to lrs is synchronous and in real-time, once committed, the changes are expected within 15 minutes to be asynchronously replicated to the secondary site. for a ra-grs storage account, in addition to one primary endpoint for read/write operations as it is in a grs, there is also one secondary endpoint as read only becomes available as shown below. the cost implications of grs or ra-grs include the additional storage and the transmission costs for egress traffic, as applicable, of the secondary datacenter. ingress traffic is free . and microsoft azure storage sla offers 99.9% availability and a cost calculator is also available. microsoft azure recovery services so far, much is about backing up or replicating data. to successfully restore, a dr plan must be put in place and ensure its availability upon a dr scenario in progress. either placing a dr plan at a primary site where the source is or a secondary site where a replica stays has some issues and concerns. keeping a dr plan at the source site where all the resources are in place and on-the-job trainings seems logical. or does it? dr is assuming a catastrophic event over an extended geographic areas where the source site is experiencing an outage. in such case, keeping a dr plan in the source site defeats the purpose. maintaining a dr plan at the secondary site is the choice then. in a dr scenario, a recovery site is to be brought on line within a expected period of time according to a dr plan, and having the dr plan right there and then at a recovery site makes all the sense. or does it? this decision introduces a number of requirements including the physical readiness, the timeliness, and the financial implications on securing and maintaining a dr plan at a remote physical facility. for a vmm server running on system center 2012 sp1 or later, an idea, reliable and straightforward way is to use azure recovery services to maintain a dr plan as shown below. and for any backup needs, using cloud as a backup site makes backing up and restoring data an anytime anywhere operation. azure site recovery vault this service essentially acts as the director of a dr process. it orchestrates and manages the protection and failover of vms in clouds managed by virtual machine manager 2012 sp1 or later. a noticeable advantage is the ability to test a recovery configuration, exercise a proactive failover and recovery, and automate recovery in the event of a site outage. the sla of site recovery services is 99.9% availability to ensure a configured dr plan is always in place with expected updates. this is a dr solution that it can implement, simulate, verify, bring online and be absolutely confident with the readiness. azure backup vault this is a reliable, scalable and inexpensive data protection solution with zero capital investment and extremely low operational expense. like other secure communication with microsoft azure, you will first upload a public certificate to microsoft azure. then download the backup agent to register a target server with the backup vault. then select what to be backed up. both microsoft azure backup sla (99.9% availability) and cost calculator are available for better assessing the solution. closing thoughts form an application’s view, ha is an on-going event while dr is an anticipation. ha and dr are different business problems and should be addressed differently. nevertheless, microsoft azure provides a single platform to gracefully address ha with lrs, dr with grs, and dr orchestration with recovery services, and all with published sla s and a predictable cost structure . going forward, it pros can now include ha and dr as a reliable, scalable and relatively inexpensive proposition by employing microsoft azure as a solution platform. call to action register at microsoft virtual academy, http://aka.ms/mva1 , and train yourself on microsoft azure by taking the track of courses. go to http://aka.ms/azure200 and acquire a free trial subscription and assess microsoft azure for ha and dr solutions. review my recommended content at http://aka.ms/recommended .

When I needed to do prototyping, proof of concept or play with some new technology in free time, starting new project was always a little annoying barrier with Maven. Have to say that setting up Maven project is not hard and you can use Maven Archetypes. But Archetypes are often out of date. Who wants to play with old technologies? So I always end up wiring in dependencies I wanted to play with. Not very productive spent time. But than Spring Boot came to my way. I fell in love. In last few months I created at least 50 small playground projects, prototypes with Spring Boot. Also incorporated it at work. It’s just perfect for prototyping, learning, microservices, web, batch, enterprise, message flow or command line applications. You have to be dinosaur or be blind not to evaluate Spring Boot for your next Spring project. And when you finish evaluate it, you will go for it. I promise. I feel a need to highlight how easy is Black Box Testing of Spring Boot microservice. Black Box Testing refers to testing without any poking with application artifact. Such testing can be called also integration testing. You can also perform performance or stress testing way I am going to demonstrate. Spring Boot Microservice is usually web application with embedded Tomcat. So it is executed as JAR from command line. There is possibility to convert Spring Boot project into WAR artifact, that can be hosted on shared Servlet container. But we don’t want that now. It’s better when microservice has its own little embedded container. I used existing Spring’s REST service guide as testing target. Focus is mostly on testing project, so it is handy to use this “Hello World” REST application as example. I expect these two common tools are set up and installed on your machine: Maven 3 Git So we’ll need to download source code and install JAR artifact into our local repository. I am going to use command line to download and install the microservice. Let’s go to some directory where we download source code. Use these commands: git clone [email protected]:spring-guides/gs-rest-service.git cd gs-rest-service/complete mvn clean install If everything went OK, Spring Boot microservice JAR artifact is now installed in our local Maven repository. In serious Java development, it would be rather installed into shared repository (e.g. Artifactory, Nexus,… ). When our microservice is installed, we can focus on testing project. It is also Maven and Spring Boot based. Black box testing will be achieved by downloading the artifact from Maven repository (doesn’t matter if it is local or remote). Maven-dependency-plugin can help us this way: org.apache.maven.plugins maven-dependency-plugin copy-dependencies compile copy-dependencies gs-rest-service true It downloads microservice artifact into target/dependency directory by default. As you can see, it’s hooked to compile phase of Maven lifecycle, so that downloaded artifact is available during test phase. Artifact version is stripped from version information. We use latest version. It makes usage of JAR artifact easier during testing. Readers skilled with Maven may notice missing plugin version. Spring Boot driven project is inherited from parent Maven project called spring-boot-starter-parent. It contains versions of main Maven plugins. This is one of the Spring Boot’s opinionated aspects. I like it, because it provides stable dependencies matrix. You can change the version if you need. When we have artifact in our file system, we can start testing. We need to be able to execute JAR file from command line. I used standard JavaProcessBuilder this way: public class ProcessExecutor { public Process execute(String jarName) throws IOException { Process p = null; ProcessBuilder pb = new ProcessBuilder("java", "-jar", jarName); pb.directory(new File("target/dependency")); File log = new File("log"); pb.redirectErrorStream(true); pb.redirectOutput(Redirect.appendTo(log)); p = pb.start(); return p; } } This class executes given process JAR based on given file name. Location is hard-coded to target/dependency directory, where maven-dependency-plugin located our artifact. Standard and error outputs are redirected to file. Next class needed for testing is DTO (Data transfer object). It is simple POJO that will be used for deserialization from JSON. I use Lombok project to reduce boilerplate code needed for getters, setters, hashCode and equals. @Data @AllArgsConstructor @NoArgsConstructor public class Greeting { private long id; private String content; } Test itself looks like this: public class BlackBoxTest { private static final String RESOURCE_URL = "http://localhost:8080/greeting"; @Test public void contextLoads() throws InterruptedException, IOException { Process process = null; Greeting actualGreeting = null; try { process = new ProcessExecutor().execute("gs-rest-service.jar"); RestTemplate restTemplate = new RestTemplate(); waitForStart(restTemplate); actualGreeting = restTemplate.getForObject(RESOURCE_URL, Greeting.class); } finally { process.destroyForcibly(); } Assert.assertEquals(new Greeting(2L, "Hello, World!"), actualGreeting); } private void waitForStart(RestTemplate restTemplate) { while (true) { try { Thread.sleep(500); restTemplate.getForObject(RESOURCE_URL, String.class); return; } catch (Throwable throwable) { // ignoring errors } } } } It executes Spring Boot microservice process first and wait unit it starts. To verify if microservice is started, it sends HTTP request to URL where it’s expected. The service is ready for testing after first successful response. Microservice should send simple greeting JSON response for HTTP GET request. Deserialization from JSON into our Greeting DTO is verified at the end of the test. Source code is shared on Github.

Scrapy, being based on Twisted, introduces an incredible host of obstacles to easily and efficiently writing self-contained unit tests: 1. You can't call reactor.run() multiple times 2. You can't stop the reactor multiple times, so you can't blindly call "crawler.signals.connect(reactor.stop, signal=signals.spider_closed)" 3. Reactor runs in its own thread, so your failed assertions won't make it to the main unittest thread, so test failures will be thrown as assertion errors but unittest doesn't know about them To get around these hurdles, I created a BaseScrapyTestCase class that uses tl.testing's ThreadAwareTestCase and the following workarounds. class BaseScrapyTestCase(ThreadAwareTestCase): in_suite = False def setUp(self): self.last_crawler = None self.settings = get_project_settings() def run_reactor(self, called_from_suite=False): if not called_from_suite and BaseScrapyTestCase.in_suite: return log.start() self.last_crawler.signals.connect(reactor.stop, signal=signals.spider_closed) reactor.run() def queue_spider(self, spider, callback): crawler = Crawler(self.settings) self.last_crawler = crawler crawler.signals.connect(callback, signal=signals.spider_closed) crawler.configure() crawler.crawl(spider) crawler.start() return crawler def wrap_asserts(self, fn): with ThreadJoiner(1): self.run_in_thread(fn) You'll use it like so: class SimpleScrapyTestCase(BaseScrapyTestCase): def test_suite(self): BaseScrapyTestCase.in_suite = True self.do_test_simple() self.run_reactor(True) def do_test_simple(self): spider = Spider("site.com") def _fn(): def __fn(): self.assertTrue(False) self.wrap_asserts(__fn) self.queue_spider(spider, _fn) self.run_reactor() 1. Call run_reactor() at the end of test method. 2. You have to place your assertions in its own function which gets called in a ThreadJoiner so that unittest knows about assertion failures. 3. If you're testing multiple spiders, just call queue_spider() for each, and run_reactor() at the end. 4. BaseScrapyTestCase keeps track of the crawlers created, and makes sure to only attach a reactor.stop signal to the last one. Let me know if you come up with a better/more elegant way of testing scrapy spiders!

this article represents top 6 popular angularjs topics that has been used most by the angularjs developer community to date. the inference is derived based on number of tagged discussions happening on stackoverflow . clearly, “directive” is the winner and attracts most of them all. the article presents my thoughts on why these topics have been most popular. please feel free to comment/suggest if i missed to mention one or more important points. also, sorry for the typos. following is the list of top 6 popular topics: directives scope object ng-repeat angular ui & bootstrap routing service following plot demonstrates the popularity of different feature/topics in relation with angularjs. angularjs topics popularity inference : some of the following could as well be inferred from the above data/plot. three features which have been most used by the developers and therefore, should be key reasons why you would also want to use angular in next project are following: directives routing ng-repeat one of the pain point (or shortcoming) that have been talked most by the angular developers is the ui widgets related support by angular. this is where most of them have jumped to angular ui and bootstrap. the topic/concept that has intrigued most to several developers is scope object. thoughts on why these topics may be most popular following are top 6 popular topics in angularjs discussed on forums such as stackoverflow: directives : this is, no doubt, the most popular and powerful feature of angularjs directives as also indicated by count of discussion threads posted on stackoverflow as of today. the power of directives lies in the following and this is why it is the most popular topic of angularjs. re-usability : once created a directive as part of a module, all that one need to do to use the directive is include the module as a dependency when defining new module and define the directives wherever required on the page. usability : owing to the fact that one could give intuitive names to directives, directive enhances the readability and understandability of code by a notch. greater adherence to dry principle : the aspect of templating makes directive a very attractive feature. it does reduce the duplication of code as same html template code could be used at several places without the need to write the code in html file. scope object : this is second most popular topic found based on the discussion count. rightfully expected as well! the whole notion of scope object and how it is key to dependency injection makes it one of the most powerful as well as tricky concept of angularjs. also, this is one of the topic which raised the barrier to entry for angularjs and contributed in making steep learning curve for developers. that said, scope is going to r.i.p in angular 2.0 which could be seen as a good sign for those who always struggled with scope object. ng-repeat : the ng-repeat feature brings power to angularjs from the fact that it is one of the feature that removed the need of server-side code required to repeat the html code over multiple iterations. with ng-repeat, one could easily repeat html code multiple times. angular ui & bootstrap : one of the shortcoming of angularjs for good or bad is its inability to be one and all solution to create some great ui along with powerful eventing feature. for creating fancy or great looking ui, one would still have to go to ui frameworks such as bootstrap, kendo-ui etc. this is where people have been looking for angularui and bootstrap. angularui comes with attractive feature set for enhanced routing, grid util, angularjs code editor plugins, bootstrap module etc. routing : routing feature is key to creating single page application. one of key reason why angularjs is very popular is the ease with which one could create single-page application using it. and, routing feature makes it all happen. no doubt, this is why many developers have been looking for it. service : service feature helps one to create reusable components in an angular module. these services could then be injected in another modules using dependency injection feature. the service could be injected in one of the following components: controllers services doing a quick recap, one may recall that for creating a service, one could use factory recipe method and define service that way. you could know details about creating a custom service on our another page dedicated on this.

CompletableFuture from Java 8 is an advanced abstraction over a promise that value of type T will be available in the future. Observable is quite similar, but it promises arbitrary number of items in the future, from 0 to infinity. These two representations of asynchronous results are quite similar to the point where Observable with just one item can be used instead of CompletableFuture and vice-versa. On the other hand CompletableFuture is more specialized and because it's now part of JDK, should become prevalent quite soon. Let's celebrate RxJava 1.0 release with a short article showing how to convert between the two, without loosing asynchronous and event-driven nature of them. From CompletableFuture to Observable CompletableFuture represents one value in the future, so turning it into Observable is rather simple. When Futurecompletes with some value, Observable will emit that value as well immediately and close stream: class FuturesTest extends Specification { public static final String MSG = "Don't panic" def 'should convert completed Future to completed Observable'() { given: CompletableFuture future = CompletableFuture.completedFuture("Abc") when: Observable observable = Futures.toObservable(future) then: observable.toBlocking().toIterable().toList() == ["Abc"] } def 'should convert failed Future into Observable with failure'() { given: CompletableFuture future = failedFuture(new IllegalStateException(MSG)) when: Observable observable = Futures.toObservable(future) then: observable .onErrorReturn({ th -> th.message } as Func1) .toBlocking() .toIterable() .toList() == [MSG] } CompletableFuture failedFuture(Exception error) { CompletableFuture future = new CompletableFuture() future.completeExceptionally(error) return future } } First test of not-yet-implemented Futures.toObservable() converts Future into Observable and makes sure value is propagated correctly. Second test created failed Future, replaces failure with exception's message and makes sure exception was propagated. The implementation is much shorter: public static Observable toObservable(CompletableFuture future) { return Observable.create(subscriber -> future.whenComplete((result, error) -> { if (error != null) { subscriber.onError(error); } else { subscriber.onNext(result); subscriber.onCompleted(); } })); } NB: Observable.fromFuture() exists, however we want to take full advantage of ComplatableFuture's asynchronous operators. From Observable toCompletableFuture> There are actually two ways to convert Observable to Future - creating CompletableFuture> orCompletableFuture (if we assume Observable has just one item). Let's start from the former case, described with the following test cases: def 'should convert Observable with many items to Future of list'() { given: Observable observable = Observable.just(1, 2, 3) when: CompletableFuture> future = Futures.fromObservable(observable) then: future.get() == [1, 2, 3] } def 'should return failed Future when after few items exception was emitted'() { given: Observable observable = Observable.just(1, 2, 3) .concatWith(Observable.error(new IllegalStateException(MSG))) when: Futures.fromObservable(observable) then: def e = thrown(Exception) e.message == MSG } Obviously Future doesn't complete until source Observable signals end of stream. Thus Observable.never() would never complete wrapping Future, rather then completing it with empty list. The implementation is much shorter and sweeter: public static CompletableFuture> fromObservable(Observable observable) { final CompletableFuture> future = new CompletableFuture<>(); observable .doOnError(future::completeExceptionally) .toList() .forEach(future::complete); return future; } The key is Observable.toList() that conveniently converts from Observable and Observable>. The latter emits one item of List type when source Observable finishes. From Observable to CompletableFuture Special case of the previous transformation happens when we know that CompletableFuture will return exactly one item. In that case we can convert it directly to CompletableFuture, rather than CompletableFuture>with one item only. Tests first: def 'should convert Observable with single item to Future'() { given: Observable observable = Observable.just(1) when: CompletableFuture future = Futures.fromSingleObservable(observable) then: future.get() == 1 } def 'should create failed Future when Observable fails'() { given: Observable observable = Observable. error(new IllegalStateException(MSG)) when: Futures.fromSingleObservable(observable) then: def e = thrown(Exception) e.message == MSG } def 'should fail when single Observable produces too many items'() { given: Observable observable = Observable.just(1, 2) when: Futures.fromSingleObservable(observable) then: def e = thrown(Exception) e.message.contains("too many elements") } Again the implementation is quite straightforward and almost identical: public static CompletableFuture> fromObservable(Observable observable) { final CompletableFuture> future = new CompletableFuture<>(); observable .doOnError(future::completeExceptionally) .toList() .forEach(future::complete); return future; } Helpers methods above aren't fully robust yet, but if you ever need to convert between JDK 8 and RxJava style of asynchronous computing, this article should be enough to get you started.

A long while ago a optimisation was added to the JVM so that if the same exception is thrown again and again and again a single instance of the Exception is created without the stack trace filled in in order to increase performance. This is an excellent idea unless you are trying to diagnose a problem and you have missed the original error. If you forgot about this optimisation you send the afternoon looking at the following log output and weeping slightly. (In my defence I have a little one in the house hence the fuzzy brain and lack of blogging action this year) java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException ... java.lang.Exception: Uncaught exceptions during test at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/common-adf/CreateNewCustomAppAndProjectWithName.xml:25) at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/level0/testADFcMATS.xml:94) at oracle.abbot.JDevScriptFixture.runTest(JDevScriptFixture.java:555) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at junit.textui.TestRunner.doRun(TestRunner.java:116) at junit.textui.TestRunner.doRun(TestRunner.java:109) at oracle.abbot.AbbotRunner.run(AbbotRunner.java:614) at oracle.abbot.AbbotAddin$IdeAbbotRunner.run(AbbotAddin.java:634) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException [Crickets] It turns out that you can turn off this optimisation with a simple flag: java -XX:-OmitStackTraceInFastThrow .... In my particular case the actual exception causing this trouble was a NPE from the GlyphView code in JDK8. (One that is being caused by a glitch in hotspot it seems) But that in turn was causing the AIOOBE in some logging code clouding the issue even more. This in particular is a good flag to add by default when running your automated tests, particularly in combination with the stack trace length override I have talked about before.

today i'd like to introduce a new series of posts named untangling concepts . in the it business we are overloaded day by day with more and more information and sometimes it's difficult to handle all this knowledge without making mistakes. the main goal of these posts is to clarify the difference between concepts that sometimes are mixed together. in the first post i'll write about unit testing and acceptance testing. i'd like to emphasize that this post isn't about test driven development . tdd is a subject to other posts. unit tests when we talk about unit tests the first thing that should come to mind is the single responsibility principle . the name unit tests isn't a coincidence, you should test one small piece of code which does just one thing. i guarantee that if you code big classes with big methods which do a lot of stuff, you'll have trouble to write good and useful unit tests. high cohesion matters. so, what is the point in unit tests? imagine the design of a system. if you follow the object oriented software design good practices you will come up to a system where a lot of objects communicate with each other in order to achieve the program's goal. imagine the following situation: you have to implement an e-mail system. the program receives as input a list of e-mail addresses and have to send an e-mail to each one of the recipients. you might have something like this: in this case, someone will call the readrecipientlist method with an inputstream (e.g. a fileinputstream of a txt file with some e-mail addresses). after that, the method sendmails is called, returning the number of e-mails sent. but what should happen if the method sendmails is called before the method readrecipientlist is called? and what if i pass a fileinputstream of an empty file? or even a null inputstream object? how can i trust that the class mailsender is doing it's job right? those are the type of question that unit tests try to answer. when writing unit tests, our concern is to assert that our objects' internal structure is working as expected. it's like a puzzle... if i can assert that each piece is working as expected, i can trust that when the pieces are put together (in the right places... more about that in the next session), the puzzle will be completed correctly. and that's not everything. what if you have to change something inside one piece of code, or if you need to do some refactoring (you should do it a lot)? if you have tests to assert that this piece of code is working as expected you can run these tests after the changes and see if your modifications broke something. you can make changes more confidently. there are a lot to say about unit tests and how they can be beneficial to your code. here i just want that you understand the concept, but i'll put some links in the useful resources section so you can read a lot more about that. acceptance tests now let's talk about acceptance tests. put yourself in the place of a stakeholder. you know what you want the software to do but you know nothing about software development nor programming. how can you believe that the software is doing everything you want, in the way you want? unit tests are good to test small pieces of code and to assert that the code works as expected. but that isn't enough to be sure that the code is satisfying the stakeholders. acceptance tests are test cases written based on scenarios specified by the customer. usually each history will have at least one associated acceptance test. they work as black box tests , meaning that shouldn't be considered implementation details in those tests (one more reason to write them first...). it's like an input-output evaluation: " given some context, when something happen, then i expect the system to answer that (or to be in that state)." we use this given-when-then template to write good acceptance tests for a user history. let's try a little example. remember our e-mail system? which scenario do you think that our customer would like to satisfy? i believe that the most obvious is something like that: given that i am logged in the mail system, when i send an e-mail to [email protected] , then i expect that there is one new mail in someone's inbox. this is just one super basic example. it's not unusual to have acceptance tests with far more given and then clauses. the best way to write acceptance tests is using something that everyone can understand. but that's not easy. we can use, for example, selenium to capture tests based in user's input, but that depends in having at least a prototype interface where the user can navigate. in the last project i have been working on, we have successfully adopted acceptance tests written in gherkin , a language used by cucumber . for example, a more detailed version of our test of sending an e-mail would look like that in gherkin: feature: sending an e-mail to someone scenario: send an e-mail to an existing user given that i am authenticated in the mail system and that exists an account: [email protected] when i send an e-mail to [email protected] then someone's inbox must have one unread mail and my sent itens folder must have the sent mail as you can see, we are writing our tests in plain english. anyone can read them and understand what the system is supposed to do (the bold words have nothing to do with gherkin, it's a bug in the syntax highlight feature). i see two great benefits in writing acceptance tests before the development of a history. the first one is that we can be sure that the customer knows what he wants to be developed and that the developer knows what the customer wants . so we reduce the chance of misunderstandings, increasing customer satisfaction. the second is that we have an easy feedback when any of the user histories are broken during the development cycle. the more we run the acceptance tests, faster we can see if our system is not doing what it is supposed to do. and faster we fix it . conclusion in this article we learned that unit tests are the best tool that developers have to protect themselves from mistakes. the wider our test coverage is, more we can do refactoring with confidence, leading to better code. remember, code rots . acceptance tests is an excelent tool to help to make the gap between the customer desires and the developer understanding smaller. they help in the fast feedback loop, leading to software that makes our customer happier. remember that unit tests and acceptance tests complement each other. our goal is not just to build the right thing but make sure that we build the thing right . useful resources martin fowler's post: unit tests extremeprogramming.org site: page about unit tests extremeprogramming.org site: page about acceptance tests uncle bob's post: the truth about bdd

[This article was written by Barak Merimovich.] We have discussed OpenStack networking extensively in previous posts. In this post, I’d like to dive into a more advanced OpenStack networking scenario. Many cloud images are not configured to automatically bring up all network cards that are available. They will usually only have a single network card configured. To correctly set up a host in the cloud with multiple network cards, log on to the machine and bring up the additional interfaces. echo $'auto eth1\niface eth1 inet dhcp' | sudo tee /etc/network/interfaces.d/eth1.cfg > /dev/null sudo ifup eth1 Networks in the cloud A complex network architecture is a mainstay of modern IaaS clouds. Understanding how to configure your cloud-based networks, and hosts, is critical to getting your application working in the cloud. This is especially true with Cloudify, the open source cloud orchestration platform I work on. The cloud, like the world, used to be flat It was not that long a time ago that most IaaS providers only supported flat networks – all of your hosts were in one large network. Separation between services running in the cloud was enforced in software or with firewalls/security-groups. But technically, all of the hosts were connected to the same network and visible to each other. The flat network model is simple, and therefore easy to reason and understand. It was a good choice for the early days of the IaaS cloud and no doubt helped with getting applications into the cloud in the first place. It was one of the things that made EC2 so easy to use for anyone just starting out with the ‘cloud’. This model is in fact still available on Amazon Web Services under the title ‘EC2-Classic’. And for many applications, a flat network is good enough. But as cloud adoption increases, more complex applications are moving into the clouds, and issues like network separation, security, SLA and broadcast domains make more complex networks models a must. Software Defined Networks (SDN) fill that gap. They are now a staple of most major IaaS clouds. AWS has AWS-VPC, OpenStack has the Neutron project and there are many other implementations. Working with SDN requires knowing a bit more about how information moves around between your cloud resources. In this post I am going to discuss how to set up a host in the cloud so it will play nice with complex networks. I’ll be using OpenStack, but the concepts are similar for other cloud infrastructures. Openstack configuration I am going to start with an empty tenant, only the public network is available. First, lets set up out networks and router: neutron router-create demo-router neutron net-create demo-network-1 neutron net-create demo-network-2 neutron subnet-create --name demo-subnet-1 demo-network-1 10.0.0.0/24 neutron subnet-create --name demo-subnet-2 demo-network-2 10.0.1.0/24 neutron router-interface-add demo-router demo-subnet-1 neutron router-interface-add demo-router demo-subnet-2 neutron router-gateway-set demo-router public Note the network IDs: neutron net-list | id | name | subnets | | 2c33efe2-6204-4125-9716-3bc525630016 | demo-network-1 | 928dafa0-83ef-459c-b20d-71d8ea596fa2 10.0.0.0/24 | | aa30627e-c181-4a4b-89bf-5dd7c26c244e | demo-network-2 | 26d573f7-7953-4a54-825b-ed7bbc0661c7 10.0.1.0/24 | | e502de8d-929a-4ee0-bd18-efa297875cf6 | public | d40dab51-a729-452c-9ee6-b9ad08d10808 | We’ll start with a standard Ubuntu cloud image: glance image-create --name "Ubuntu 12.04 Standard" --location "http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img" --disk-format qcow2 --container-format bare Create the keypair and security group: nova keypair-add demo-keypair > demo-keypair.pem chmod 400 demo-keypair.pem nova secgroup-create demo-security-group "Security group for demo" nova secgroup-add-rule demo-security-group tcp 22 22 0.0.0.0/0 Let’s spin up an instance connected to both our networks: nova boot -flavor m1.small --image "Ubuntu 12.04 Standard" --nic net-id=2c33efe2-6204-4125-9716-3bc525630016 --nic net-id=aa30627e-c181-4a4b-89bf-5dd7c26c244e --security-groups demo-security-group --key-name demo-keypair demo-vm And set up floating IPs for the first network: nova list | ID | Name | Status | Task State | Power State | Networks | 2b17588b-8980-4489-9a04-6539a159dc3c | demo-vm | ACTIVE | None | Running | demo-network-1=10.0.0.2; demo-network-2=10.0.1.2 | neutron floatingip-create public neutron floatingip-list | id | fixed_ip_address | floating_ip_address | port_id | | 49c8b05e-bb8f-4b07-80ed-3155ab6ffc09 | | 192.168.15.42 | | neutron port-list | id | name | mac_address | fixed_ips | | 1ccfd334-7328-4b22-b93e-24a0888276ab | | fa:16:3e:14:39:39 | {"subnet_id": "94598487-c1fc-4f55-ac1f-ef2545d5cfeb", "ip_address": "10.0.1.3"} | | a482c4f6-fa74-476e-b1ce-cd8dd0c70815 | | fa:16:3e:18:92:79 | {"subnet_id": "94598487-c1fc-4f55-ac1f-ef2545d5cfeb", "ip_address": "10.0.1.2"} | | b23d7836-30c5-4bff-b873-15c87ba051f6 | | fa:16:3e:3a:28:40 | {"subnet_id": "dec6ec74-cfa9-4a08-8792-54900631b98e", "ip_address": "10.0.0.3"} | | d421b447-2adf-406f-876b-142238683344 | | fa:16:3e:9d:fc:7f | {"subnet_id": "dec6ec74-cfa9-4a08-8792-54900631b98e", "ip_address": "10.0.0.2"} | | dcf8696b-cc80-4b48-b09c-61c0f8ab02ac | | fa:16:3e:5b:39:fb | {"subnet_id": "94598487-c1fc-4f55-ac1f-ef2545d5cfeb", "ip_address": "10.0.1.1"} | | f6a1666e-495a-4d3f-afa3-754b3cb3cfc0 | | fa:16:3e:8a:1b:fb | {"subnet_id": "dec6ec74-cfa9-4a08-8792-54900631b98e", "ip_address": "10.0.0.1"} | neutron floatingip-associate 49c8b05e-bb8f-4b07-80ed-3155ab6ffc09 d421b447-2adf-406f-876b-142238683344 Note how we matched the VM’s IP to its port, and associated the floating IP to the port. I wish there was an easier way to do this from the CLI… If everything worked correctly, you should have the following setup: Let’s make sure ssh works correctly: ssh -i demo-keypair.pem [email protected] hostname demo-vm Cool, ssh works. Now, we should have two network cards, right? ssh -i demo-keypair.pem [email protected] hostname demo-vm Cool, ssh works. Now, we should have two network cards, right? ssh -i demo-keypair.pem [email protected] ifconfig eth0 Link encap:Ethernet HWaddr fa:16:3e:5f:a2:5f inet addr:10.0.0.4 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe5f:a25f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:230 errors:0 dropped:0 overruns:0 frame:0 TX packets:224 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:46297 (46.2 KB) TX bytes:31130 (31.1 KB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Huh?! The VM only has one working network interface! Where is my second NIC? Was there a configuration problem with the OpenStack network setup? The answer is here: ssh -i demo-keypair.pem [email protected] ifconfig -a eth0 Link encap:Ethernet HWaddr fa:16:3e:5f:a2:5f inet addr:10.0.0.4 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe5f:a25f/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:324 errors:0 dropped:0 overruns:0 frame:0 TX packets:332 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:69973 (69.9 KB) TX bytes:47218 (47.2 KB) eth1 Link encap:Ethernet HWaddr fa:16:3e:29:6d:22 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) The second NIC exists, but is not running. The issue is not with the OpenStack network configuration – it’s with the image. The image itself should be configured to work correctly with multiple NICs. All we have to do is bring up the NIC. So we ssh into the instance: ssh -i demo-keypair.pem [email protected] And run the following commands: echo $'auto eth1\niface eth1 inet dhcp' | sudo tee /etc/network/interfaces.d/eth1.cfg > /dev/null sudo ifup eth1 The second NIC should now be running: ifconfig eth1 eth1 Link encap:Ethernet HWaddr fa:16:3e:18:92:79 inet addr:10.0.1.2 Bcast:10.0.1.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe18:9279/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:81 errors:0 dropped:0 overruns:0 frame:0 TX packets:45 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:15376 (15.3 KB) TX bytes:3960 (3.9 KB) And there you go – your VM can access both networks. This issue can make life complicated when setting up a complex, or even a not very complex, application. When will this issue hurt you? Well, imagine a scenario where you have a web server and a database server. The web server is connected to both Network1 and Network2, and the database server is only connected to Network2. Network1 is connected to the external world over a router, and Network 2 is completely internal, adding another layer of security to the critical database server. So what happens if the web server only has one network card? If only the NIC for Network1 is up, the web server can’t access the database. If only the NIC for Network2 is up, the web server can’t be reached from the external world. Even worse, if this web server is accessed via a floating IP, this IP will also not work, so you won’t be able to access the web server and fix the issue. Tricky. In conclusion The above commands will bring up your additional network card. You will of-course need to repeat this process for each additional network card, and for each VM. You can use a start-up script (a.k.a. user-data script) or system service to run these commands, but there are better ways. I’ll discuss how to automate the network setup in a follow-up post. This was originally posted at Barak's blog Head in the Clouds, find it here.

For a long time JSF for me was just another web framework I didn’t care too much about. This changed. After being forced to use it for a couple of months now, I consider it a major project risk in almost all cases. Here I present the reasons for this verdict. Bad entanglement of UI and processing Logic The official tutorial claims the following about the benefits of JSF: One of the greatest advantages of Java Server Faces technology is that it offers a clean separation between behavior and presentation for web applications. The opposite is the case. Facelets, the preferred presentation technology of JSF looks at first sight like an ordinary templating technology like the good old JSP or Thyme Leaf. But if you look closer the horror becomes obvious. In the same place where you structure your HTML, you also place the logic what parts of the UI should get updated on an action. A clear violation of the separation of concerns principle in my book. Even better is the immediate attribute which changes the server side life cycle! And if this isn’t enough it does it in different waysdepending on what tag you use it on. You can’t make stuff like this up. It tries to abstract what you can not abstract. Except some weird edge cases clients and server of web application are located on rather different computers, separated by some kind of network. From this follows a simple fact: communication between client and server is slow and unreliable. JSF tries to abstract away the separation of client and server. It processes everything on the backend wildly communicating between client and server in a hard to control way. The result are all kind of failure scenarios just popping into existence because you use JSF. For me the most annoying one is this one: If you open a JSF page, let’s say a simple search page, wait an hour, then hit the submit button you will get an exception because the server side state expired. WAT? Why is there server state of any relevance for a trivial search page? (Yes I know you can change that behavior with the latest versions of JSF, but it is still the way JSF is designed to work) I though everybody learned since EJBs: If you want to abstract over the fact, if two parts of an application run on the same machine or not, you have to assume they don’t. Everything else is just hiding problems until they grow so large that they can eat your project for breakfast. Making stuff complex and complicated that was easy to start with.The architecture of the World Wide Web is a simple one. Simple meaning: It consists of a small set of concepts with limited interaction. This is what made it so widely successful. It also makes it not obvious for beginners how to use it to implement certain features. I’m sure most of us remember the first time they tried to implement something like a shopping cart without having session state. But the solutions for almost all these problems are well known and understood by know. And all you need is a little reading and what you gain is a strong conceptual understanding how to solve this kind of issue. And again, the basics are extremely simple: You send a request to an URL, with some headers and content using a HTTP verb. And you reply with some resource containing links and some headers. And you don’t have state in the server session. Making load balancing and fail over rather simple. Making bookmarkable URLs trivial. Making your site searchable for zero costs. Making your site cachable. Allowing the user to use their back buttons, history and tabs as they wish. Making it trivial to have nice URLs Compare that to the live cycle model of JSF: The page from which a user submitted a request will get synchronized with a model on the server side, then submitted values validated, converted, events generated and processed. As mentioned above the order in which things happen, and if they happen at all are controlled by XML Tags hidden away in a document camouflaged as markup. Apart from hardly anybody properly understanding all this (BalusC seems to be the only one available in the interwebs) it has the following effect on your application: The URLs become ugly. You’ll see the URL of the resource you came from instead of the one you are looking at, thus making bookmarking URLs as useful as a doorknob on your knee. Same for caching, fail over, load balancing and so on. Sure you can fix it with some convention here, and an additional library there. Which of course makes perfect sense when you are in the business of breaking stuff so people have to pay you for fixing it. I personally prefer helping to solve real problems. Hindering testability: I can’t speak for most frameworks but I can compare Spring MVC with JSF. Let me tell you this: If anybody is telling you JSF is nicely testable he probably doesn’t know automatic testing. With JSF you can test your backend beans using unit tests. You can test the whole UI, by deploying the application to a server and hitting it with Selenium. That’s basically it. Just in case you are wondering what else one should be able to test: Load a static version of a page in a browser and testing it with selenium, in order test your Client side UI behavior. Test your generated markup without starting a full blown application server. Test the mapping of attributes/parameter to bean methods. Test your generated markup without bootstrapping a complete application. All this is perfectly possible with Spring MVC and probably with many other sane server side frameworks, but not with JSF … Again: I’m aware there are fixes for many issues, but the simplest fix is> Don’t use JSF.

This article explains how errors are handled when using the messaging system with Spring Integration and how to handle route and redirect to specific channel.

This technical tip shows how to read Excel worksheet cells values in multiple threads simultaneously inside android applications. Often you need to read worksheet cells values in multiple threads simultaneously. To do so, set Worksheet.getCells().setMultiThreadReading() to true. If you do not set this property you might get the wrong cell values. Setting it to true, you always get the correct values. To achieve this task first create a workbook and adds a worksheet. Populates the worksheet with some string values then create two threads that simultaneously read values from random cells. If the values read are correct, then nothing happens. But if the values read are incorrect, then a message box shows up in the LogCat window. If you comment this line: testWorkbook.getWorksheets().get(0).getCells().setMultiThreadReading(true); the following message will show up in LogCat window: if (s.equals("R" + row + "C" + col)!=true) { System.out.println("This message box will show up when cells read values are incorrect."); } Otherwise, the program run without showing any message which means all values read from cells are correct. public class ThreadProc implements Runnable { boolean isRunning = true; Workbook testWorkbook; Random r = new Random(); public ThreadProc(Workbook workbook) { this.testWorkbook = workbook; } public int randomNext(int Low, int High) { int R = r.nextInt(High-Low) + Low; return R; } public void kill() { this.isRunning = false; } public void run(){ while(this.isRunning) { int row = randomNext(0, 100); int col = randomNext(0, 10); String s = testWorkbook.getWorksheets().get(0).getCells().get(row, col).getStringValue(); if (s.equals("R" + row + "C" + col)!=true) { System.out.println("This message box will show up when cells read values are incorrect."); } } } } //........MainActivity.java........ //................................. import java.io.File; import java.util.Random; import android.app.Activity; import android.os.Bundle; import android.os.Environment; import android.view.Menu; import com.aspose.cells.CellsHelper; import com.aspose.cells.IWarningCallback; import com.aspose.cells.WarningInfo; import com.aspose.cells.WarningType; import com.aspose.cells.Workbook; public class MainActivity extends Activity { @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); try{ Workbook testWorkbook = new Workbook(); testWorkbook.getWorksheets().clear(); testWorkbook.getWorksheets().add("Sheet1"); for (int row = 0; row < 100; row++) for (int col = 0; col < 10; col++) testWorkbook.getWorksheets().get(0).getCells().get(row, col).setValue("R" + row + "C" + col); //Commenting this line will show a pop-up message testWorkbook.getWorksheets().get(0).getCells().setMultiThreadReading(true); ThreadProc tp = new ThreadProc(testWorkbook); Thread myThread1 = new Thread(tp); myThread1.start(); Thread myThread2 = new Thread(tp); myThread2.start(); Thread.currentThread().sleep(5*1000); tp.kill(); } catch (Exception e) { e.printStackTrace(); } } ......... }

We use the Test task in Gradle to run tests. If we use the System.out.println or System.err.println methods in our test we don't see the output when we execute the tests. We can customize the test task to show any output send to standard out or error in the Gradle output. First we show our test class written with Spock, but it could also be a JUnit or TestNG test: // File: src/test/groovy/com/mrhaki/gradle/SampleSpec.groovy package com.mrhaki.gradle import spock.lang.* class SampleSpec extends Specification { def "check that Gradle is Gr8"() { when: def value = 'Gradle is great!' then: // Include a println statement, so // we have output to show. println "Value = [$value]" value == 'Gradle is great!' } } Now we write a simple Gradle build file which can execute our test: // File: build.gradle apply plugin: 'groovy' // Adds test task repositories.jcenter() dependencies { compile 'org.codehaus.groovy:groovy-all:2.3.7' testCompile 'org.spockframework:spock-core:0.7-groovy-2.0' } Let's run the test task from the command line and look at the output: $ gradle test :compileJava UP-TO-DATE :compileGroovy UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :compileTestJava :compileTestGroovy :processTestResources UP-TO-DATE :testClasses :test BUILD SUCCESSFUL Total time: 7.022 secs $ Well at least our test is successful, but we don't see the output of our println method invocation in the test. We customize the test task and add thetestLogging method with a configuration closure. In the closure we set the property showStandardStreams to the value true. Alternatively we can set the events property or use the events method with the values standard_out and standard_err to achieve the same result. In the next build file we use the showStandardStreams property: view sourceprint? 00.// File: build.gradle 01.apply plugin: 'groovy' // Adds test task 02. 03.repositories.jcenter() 04. 05.dependencies { 06.compile 'org.codehaus.groovy:groovy-all:2.3.7' 07.testCompile 'org.spockframework:spock-core:0.7-groovy-2.0' 08.} 09. 10.test { 11.testLogging { 12.// Make sure output from 13.// standard out or error is shown 14.// in Gradle output. 15.showStandardStreams = true 16. 17.// Or we use events method: 18.// events 'standard_out', 'standard_error' 19. 20.// Or set property events: 21.// events = ['standard_out', 'standard_error'] 22. 23.// Instead of string values we can 24.// use enum values: 25.// events org.gradle.api.tasks.testing.logging.TestLogEvent.STANDARD_OUT, 26.// org.gradle.api.tasks.testing.logging.TestLogEvent.STANDARD_ERROR, 27.} 28.} We re-run the test task from the command line and look at the output to see the result from the println method: $ gradle test :compileJava UP-TO-DATE :compileGroovy UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :compileTestJava :compileTestGroovy :processTestResources UP-TO-DATE :testClasses :test com.mrhaki.gradle.SampleSpec > check that Gradle is Gr8 STANDARD_OUT Value = [Gradle is great!] BUILD SUCCESSFUL Total time: 8.716 secs $ Written with Gradle 2.1.

It is possible to disable HTTP access on S3 bucket, limiting S3 traffic to only HTTPS requests. The documentation is scattered around the Amazon AWS documentation, but the solution is actually straightforward. All you need to do to block HTTP traffic on an S3 bucket is add a Condition in your bucket's policy. AWS supports a global condition for verifying SSL. So you can add a condition like this: "Condition": { "Bool": { "aws:SecureTransport": "true" } } Here's a complete example: { "Version": "2008-10-17", "Id": "some_policy", "Statement": [ { "Sid": "AddPerm", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "s3:GetObject", "Resource": "arn:aws:s3:::my_bucket/*", "Condition": { "Bool": { "aws:SecureTransport": "true" } } } ] } Now accessing the contents of my_bucket over HTTP will produce a 403 error, while using HTTPS will work fine.

This article shows how to mock your JMS infrastructure using MockRunner and test it using Spring.

I use Charles Proxy extensively for debugging all kinds of applications, and lately I've been using it more with mobile devices. One of the killer features of Charles is its ability to intercept SSL traffic. This is hard - and rightly so, it should be difficult to inspect SSL traffic! Charles handles this by using the server's SSL certificate for the connection from Charles to the remote server, and then using Charles' own SSL certificate for the "last mile" back to your browser or device. This means that the connection will be flagged as insecure; Charles' certificates aren't trusted by your browser or device - but we can easily change that. Get Charles' Certificate On Windows, you can grab the certificate from the Help menu in Charles, but for OS X or Linux you'll need to download it from their website. It's linked from this excellent documentation page about SSL:http://www.charlesproxy.com/documentation/using-charles/ssl-certificates/. Install it On Your Android Device Copy the ``.crt`` file into the root of your SD card (I found this didn't work in a subdirectory, although some documentation suggests it will). Under "Security" -> "Settings", choose "install from device storage" and it should pick up the certificate you put on the SD card. ... that's it :) Check you have enabled the destination for SSL proxying under "Proxy Settings" in Charles, and you're all set! Note, I have one device that refused to see the .crt file on the SD card, but when I emailed it to myself and then saved it from there, was added successfully. Thought I'd mention it! Further Reading Using Charles To Debug PHP SOAP Wireshark Capture on Remote Server View Only Headers with Curl

Property based testing is an alternative approach to testing, complementingexample based testing. The latter is what we've been doing all our lives: exercising production code against "examples" - inputs we think are representative. Picking these examples is an art on its own: "ordinary" inputs, edge cases, malformed inputs, etc. But why are we limiting ourselves to just few examples? Why not test hundreds, millions... ALL inputs? There are at least two difficulties with that approach: Scale. A pure function taking just one int input would require 4 billion tests. This means few hundred gigabytes of test source code and several months of execution time. Square it if a function takes two ints. For String it practically goes to infinity. Assume we have these tests, executed on a quantum computer or something. How do you know the expected result for each particular input? You either enter it by hand (good luck) or generate expected output. Bygenerate I mean write a program that produces expected value for every input. But aren't we testing such program already in the first place? Are we suppose to write better, error-free version of code under test just to test it? Also known as ugly mirror antipattern. So you understand testing every single input, although ideal, is just a mental experiment, impossible to implement. That being said property based testing tries to get as close as possible to this testing nirvana. Issue #1 is solved by slamming code under test with hundreds or thousands of random inputs. Not all of them, not even a fraction. But a good, random representation. Issue #2 is surprisingly harder. Property based testing can generate random arguments, but it can't figure out what should be the expected outcome for that random input. Thus we need a different mechanism, giving name to whole philosophy. We have to come up with properties (invariants, behaviours) that code under test exhibits no matter what the input is. This sounds very theoretically, but there are many such properties in various scenarios: Absolute value of any number should never be negative Encoding and decoding any string should yield the same String back for every symmetric encoding Optimized version of some old algorithm should produce the same result as the old one for any input Total money in a bank should remain the same after arbitrary number of intra-bank transactions in any order As you can see there are many properties we can think of that do not mention specific example inputs. This is not exhaustive and strict testing. It's more like sampling and making sure samples are "sane". There are many, many libraries supporting property based testing for virtually every language. In this article we will explore Spock and ScalaCheck later. Spock + custom data generators Spock does not support property based testing out-of-the-box. However with help from data driven testing and 3rd-party data generators we can go quite far. Data tables in Spock can be generalized into so-called data pipes: def 'absolute value of #value should not be negative'() { expect: value.abs() >= 0 where: value << randomInts(100) } private static def List randomInts(int count) { final Random random = new Random() (1..count).collect { random.nextInt() } } Code above will generate 100 random integers and make sure for all of them.abs() is non-negative. You might think this test is quite dumb, but to a great surprise it actually discovers one bug! But first let's kill some boilerplate code. Generating random inputs, especially more complex, is cumbersome and boring. I found two libraries that can help us. spock-genesis: import spock.genesis.Gen def 'absolute value of #value should not be negative'() { expect: value.abs() >= 0 where: value << Gen.int.take(100) } Looks great, but if you want to generate e.g. lists of random integers,net.java.quickcheck has nicer API and is not Groovy-specific: import static net.java.quickcheck.generator.CombinedGeneratorsIterables.someLists import static net.java.quickcheck.generator.PrimitiveGenerators.integers def 'sum of non-negative numbers from #list should not be negative'() { expect: list.findAll{it >= 0}.sum() >= 0 where: list << someLists(integers(), 100) } This test is interesting. It makes sure sum of non-negative numbers is never negative - by generating 100 lists of randoms ints. Sounds reasonable. However multiple tests are failing. First of all due to integer overflow sometimes two positiveints add up to a negative one. Duh! Another type of failure that was discovered is actually frightening. While [1,2,3].sum() is 6, obviously, [].sum() is... null(WAT?) As you can see even silliest and most basic property based tests can be useful in finding unusual corner cases in your data. But wait, I said testing absolute of intdiscovered one bug. Actually it didn't, because of poor (too "random") data generators, not returning known edge values in the first place. We will fix that in the next article.

If you use Java for programming, read on to learn about tools like Eclipse IDE, the Java Development Kit, and other must-know tools.

in “ semihosting with kinetis design studio ” i used printf() to exchange text and data between the target board and the host using the debug connection. kinetis design studio (kds) has that semihosting baked into its libraries. what about if using the gnu arm embedded (launchpad) tools and libraries (see “ switching arm gnu tool chain and libraries in kinetis design studio “)? actually it requires two more steps, but is very easy too. semihosting output there are three things to be in place to use semihosting with the gnu arm embedded (launchpad) libraries: option in the gnu linker settings enabling semihosting in the debugger settings initializing the gnu libraries linker option to enable semihosting for the gnu arm embedded ( launchpad ) libraries, i need to add --specs=rdimon.specs to the linker options: linker option to enable semihosting in case i’m using newlib-nano and want to use printf() and/or scanf() with floating point support, i need to pull in some symbols explicitly with the linker options ‘u': -u _scanf_float -u _printf_float debugger settings in the gnu arm eclipse plugins, i need to enable semihosting. segger j-link for segger j-link, i enable the console in the launch configuration: allocated semihosting console for segger additionally i enable semihosting options in the startup options of the debugger: enabled semihosting in the startup options for segger p&e multilink for p&e the following settings are used: semihosting settings for pne settings for openocd the following settings are used for openocd: openocd semihosting settings initializing the gnu libraries if you would now try to use semihosting with running the debugger, you probably will get error messages like this (e.g. from segger j-link): warning: semihosting command sys_flen failed. handle is 0. warning: semihosting command sys_write failed. handle is 0. warning: semihosting command sys_write failed. handle is 0. warning: semihosting command sys_write failed. handle is 0. the reason is that the semihosting needs to be enabled by the application. i need to call initialise_monitor_handles() before i’m using printf() : 1 2 3 4 5 6 7 8 extern void initialise_monitor_handles( void ); /* prototype */ int main( void ) { initialise_monitor_handles(); /* initialize handles */ for (;;) { printf ( "hello world!\r\n" ); } } with this, i can use printf() and scanf() through a debugger connection. semihosting printf output summary while i don’t like printf() for many reasons, sometimes it is useful to exchange data with the host. using semihosting no physical connection is required, as the communication goes through the debugger. it is somewhat intrusive, and adds code and data overhead, but the gnu arm embedded (launchpad) libraries (both newlib and newlib-nano) have semihosting built-in. it is a matter to enable it in the linker and debugger settings, and to initialize the handles in the application. happy semihosting :-)

Recently I had the opportunity to work on a project were I needed to create a custom SQL Server image for use with Azure VMs. The process was a little more challenging than I initially anticipated. I think this is mostly because I was not familiar with the process of preparing a SQL Server image. Perhaps this isn’t much of a challenge for an experienced SQL Server DBA or IT Pro. For me, it was a great learning experience. Why a Custom SQL Server Image? The Azure VM image gallery already contains a SQL Server image. It’s very easy to create a new SQL Server VM using this image. However, doing so has a few important trade-offs to consider: Unable to fully customize the base install of SQL Server. This is a template/image after all – you get a VM configured the way the image was configured. Unable to use your own SQL Server license. If your company has an Enterprise Agreement (EA) with Microsoft, it’s likely there is already some SQL Server licenses built into that agreement. Depending on the details, it may be significantly cheaper to use the licenses from the EA instead of paying the SQL Server VM image upcharge from Azure. The Basic Steps There are 6 basic steps to creating a custom SQL Server VM image for use in Azure. Provision a new base Windows Server VM Download the SQL Server installation media Run SQL Server setup to prepare an image Configure Windows to complete the installation of SQL Server Capture the image and add it to the Azure VM image gallery Create a new VM instance using the custom SQL Server image The basic idea here is to create a base VM, customize it with a SQL Server image, capture the VM to create an image, and then provision new VMs using that captured VM image. Let’s dive into each of these in a little more detail. Note: the terminology here can be a little confusing. When referring to the VM used to create the template/image, I’ll use the term “base VM”. When referring to the VM created from the base VM, I’ll use the term “VM instance”. 1. Provision a new base Windows Server VM There are multiple ways to create a Windows Server VM in Azure. Creating a VM via the Azure management portal and PowerShell are probably the two most popular options. Be sure to check out this tutorial to learn how to do so via the portal. For the purposes of this post, I’ll do so via PowerShell. $img = Get-AzureVMImage ` | where { ( $_.PublisherName -ilike "Microsoft*" -and $_.ImageFamily -ilike "Windows Server 2012 Datacenter" ) } ` | Sort-Object -Unique -Descending -Property ImageFamily ` | sort -Descending -Property PublishDate ` | select -First(1) $vmConfig = New-AzureVMConfig -Name "sql-1" -InstanceSize Small -ImageName $img.ImageName | Add-AzureProvisioningConfig -Windows -AdminUsername "[admin-username-here]" -Password "[admin-password-here]" New-AzureVM -ServiceName "SQLServerVMTemplate" -VMs $vmConfig -Location "East US" -WaitForBoot 2. Download the SQL Server installation media With the base Windows Server 2012 VM created, we can now get ready to prepare (sysprep) the SQL Server installation. To do that, we need to get the SQL Server installation media onto the machine. The easiest way I found to do this was to leverage Azure blob storage. Upload the SQL Server ISO file to Azure blob storage Remote Desktop (RDP) into the base VM From the VM, download the SQL Server ISO file to the local disk Mount the SQL Server ISO file to the VM Copy the ISO contents (not the ISO file itself) to the VM’s C:\ drive. For example, use C:\sql The SQL Server installation media files need to be copied to the local C: drive so it can be used later to complete the SQL Server installation (when provisioning the actual SQL Server VM instance). 3. Run SQL Server setup to prepare an image In order to prepare the (sysprep’d) SQL Server VM image (which we can use as a template for future VMs), we need to run the SQL Server installation and instruct it topreparean image – not run the full installation. An easy way to do this is with a SQL Server configuration file, an example of which I’ve included below. ConfigurationFile.ini ;SQL Server 2012 Configuration File [OPTIONS] ; Specifies a Setup workflow, like INSTALL, UNINSTALL, or UPGRADE. This is a required parameter. ACTION="PrepareImage" ; Detailed help for command line argument ENU has not been defined yet. ENU="True" ; Parameter that controls the user interface behavior. Valid values are Normal for the full UI, AutoAdvance for a simplified UI, and EnableUIOnServerCore for bypassing Server Core setup GUI block. ;UIMODE="Normal" ; Specifies setup not display any user interface. ;QUIET="False" ; Specifies setup to display progress only, without any user interaction. QUIETSIMPLE="True" ; Specifies whether SQL Server Setup should discover and include product updates. The valid values are True and False or 1 and 0. By default SQL Server Setup will include updates that are found. UpdateEnabled="True" ; Specifies features to install, uninstall, or upgrade. The list of top-level features include SQL, AS, RS, IS, MDS, and Tools. The SQL feature will install the Database Engine, Replication, Full-Text, and Data Quality Services (DQS) server. The Tools feature will install Management Tools, Books online components, SQL Server Data Tools, and other shared components. FEATURES=SQLENGINE ; Specifies the location where SQL Server Setup will obtain product updates. The valid values are "MU" to search Microsoft Update, a valid folder path, a relative path such as .\MyUpdates or a UNC share. By default SQL Server Setup will search Microsoft Update or a Windows Update service through the Window Server Update Services. UpdateSource="MU" ; Displays the command line parameters usage HELP="False" ; Specifies that the detailed Setup log should be piped to the console. INDICATEPROGRESS="False" ; Specifies that Setup should install into WOW64. This command line argument is not supported on an IA64 or a 32-bit system. X86="False" ; Specifies the root installation directory for shared components. This directory remains unchanged after shared components are already installed. INSTALLSHAREDDIR="C:\Program Files\Microsoft SQL Server" ; Specifies the root installation directory for the WOW64 shared components. This directory remains unchanged after WOW64 shared components are already installed. INSTALLSHAREDWOWDIR="C:\Program Files (x86)\Microsoft SQL Server" ; Specifies the Instance ID for the SQL Server features you have specified. SQL Server directory structure, registry structure, and service names will incorporate the instance ID of the SQL Server instance. INSTANCEID="MSSQLSERVER" ; Specifies the installation directory. INSTANCEDIR="C:\Program Files\Microsoft SQL Server" There are two steps in this process: Copy the ConfigurationFile.ini file (from your local PC) to the same location as the SQL Server installation media (i.e.c:\sql) on the base VM. Run SQL Server setup to prepare an image. From a command prompt (on the base VM), navigate to theC:\sqlfolder and then execute the following command: Setup.exe /ConfigurationFile=ConfigurationFile.ini /IAcceptSQLServerLicenseTerms=true 4. Configure Windows to complete the installation of SQL Server At this point the base VM should have an “installation” of SQL Server that is not fully completed. The SQL Server bits are in place, but they’re not configured for a full server install . . . at least not yet. The final configuration of SQL Server will take place when the VM instance (of which this template/image is the base) is provisioned and boots up for the first time. This is accomplished by using a CMD file with the following content: @ECHO OFF && SETLOCAL && SETLOCAL ENABLEDELAYEDEXPANSION && SETLOCAL ENABLEEXTENSIONS REM All commands will be executed during first Virtual Machine boot "C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\SQLServer2012\setup.exe" /QS /ACTION=CompleteImage /INSTANCEID=MSSQLSERVER /INSTANCENAME=MSSQLSERVER /IACCEPTSQLSERVERLICENSETERMS=1 /SQLSYSADMINACCOUNTS=%COMPUTERNAME%\Administrators /BROWSERSVCSTARTUPTYPE=AUTOMATIC /INDICATEPROGRESS /TCPENABLED=1 /PID="[YOUR-SQL-SERVER-PRODUCT-ID-HERE]" On your local PC, save the file as SetupComplete2.cmd RDP / log into the base VM Copy the SetupComplete2.cmd from your local PC file to the c:\Windows\OEM folder on the base VM Change the value for the SQLSYSADMINACCOUNTS value to be that of the administrative account created on the VM (or better yet – the local Administrators group account) If needed, supply the SQL Server product ID (PID) value. When Windows starts on the new VM instance for the first time, the SetupComplete2.cmd file should automatically run. It is invoked by the SetupComplete.cmd file already on the machine. 5. Capture the image and add it to the Azure VM image gallery At this point a base SQL Server VM has been created and the groundwork laid to complete the install. Now it is time to create the VM image from the base VM, and do to that you sysprep and capture the base VM. Please follow the guide on How to Capture a Windows Virtual Machine to Use as a Template. 6. Create a new VM using the custom SQL Server image With a new custom VM image template available in the VM image gallery, you can provision a new VM instance using that custom template. Upon first boot, the newly provisioned VM should complete the full SQL Server installation as laid out in your SetupComplete2.cmd file. Please follow the guide on How to Create a Custom Virtual Machine for more information on creating the VM from the template. Closing Thoughts One of the quirks I noticed when preparing the base SQL Server image is that it was not possible to prepare the image with SQL Server Management Studio (SSMS). I would have to do the install after the newly provisioned VM instance is created. Not hard, but time consuming (an annoying if doing this on multiple VM instances). I later learned that SQL Server 2012 Cumulative Update 1 does allow for preparing a SQL Server image with SSMS installed. I’ve included a link below that describes the process for creating a SQL Server image with CU1. In the end, this process really is not all that hard. Time consuming? Yes! The worst part (at least for me) was really just understanding how the SQL Server installation and sysprep process works. Once I wrapped my head around that, the process was a lot smoother. Helpful Resources While I was learning how to create a custom SQL Server VM image, the following resources were very helpful: How to: Create a Windows Azure Virtual Machine Operating System Image for Microsoft Dynamics NAV. This MSDN article provided the jumping off point on learning how to install SQL Server by using a sysprep image. Install SQL Server 2012 from the Command Prompt Install SQL Server 2012 Using a Configuration File Install SQL Server 2012 Using SysPrep How to create a slipstream SQL Server 2012 and Cumulative Update 1 image –http://sqlperformance.com/2012/12/system-configuration/sql-2012-slipstream I would like to thank Scott Klein for his assistance in verifying these steps. His help was extremely valuable to ensure I was doing this the right way.