This is the fourth in a series of posts on the new JBoss Red Deer automated test framework. This post describes features in Red Deer that make it easier to debug test failures. Writing any automated UI tests can be a challenging task. Your tests have to be able to manipulate the UI, deal with platform-dependent display characteristics, and timing issues. But, what about when things go wrong and you have to debug a failing test? Are you looking at a bug in the program that you’re testing? Or, perish the thought, what if the bug is not in the program you’re testing, but it’s in your test program? The primary goal of Red Deer is to make this task easier by providing a open source solution that is reliable and extensible. Red Deer also makes your life easier when things go wrong by providing your with multiple debugging features. This post describes these features and illustrates how you can enable/disable Red Deer’s debugging features, and how you can use them to debug test failures. Maven and Eclipse Debugging is Still Usable Before discussing Red Deer’s debugging features, it’s important to remember that in no way does Red Deer prevent you from performing the types of debugging that you can use with Maven or Eclipse. You can use the Eclipse debugger to set breakpoints in Red Deer test programs in the same manner as any other programs. Likewise, to debug a Red Deer test program with maven, all you have to do is to run maven with -DdebugPort=, and in Eclipse, select the test class that you want to execute, select "Debug As," and create a new debug configuration. In the debug configuration, specify that you want to run the test class as a Remote Java Application, and the port number. Starting the debugger makes the waiting test execution run and then stop on the first breakpoint: Debugging Features Added by Red Deer The Red Deer debugging features that we’ll discuss in this post are: Red Deer Debug Logging Automatic Screenshots for Failed Tests Pausing Failing Tests Recording Screencasts Let’s start by looking at the simplest of Red Deer’s debugging tools; logging. Red Deer Debug Logging By default, Red Deer creates a DEBUG level log whenever a test is run. The log is written to stdout when you run tests from a shell and to the console view when you run tests in Eclipse. Some example output of a Red Deer log looks like this: INFO [thread][class] Log message-N INFO [thread][class] Log message-N+1 ERROR [thread][class] Hey! Something failed here DEBUG [thread][class] And, here’s some additional debug information INFO [thread][class] Log message-N+2 To disable debug logging, you set the “logDebug” JVM argument to false. For example: -DlogDebug=false In addition, you can also filter the log contents with the “logMessageFilter” JVM argument. The supported filter values are: debug error warn info trace none all You can set multiple filters by specifying filter values in a single string using vertical pipe separators. For example: -DlogMessageFilter=error|warn Note that the filter values are case insensitive. Automatic Screenshots for Failed Tests As is the case with debug logging, screenshot saving for failed tests is enabled by default by Red Deer. What happens is that when a Red Deer test fails, at the point of failure a screenshot is taken and stored for you. The screenshot illustrates the state of the UI when the test failure occured. By default, Red Deer saves the screenshot files in the “target/screenshot” directory. You can select the directory into which Red Deer will save the screenshot files by setting this JVM argument: relativeScreenshotDirectory For example: -DrelativeScreenshotDirectory=/home/jsmith/screenshots Let’s take a look at this in action. The following screenshot was generated when a test failed. The server address in question is valid. The error was due to a temporary network connectivity issue. What happens under the hood here is that when a test fails, the Red Deer watchdog process takes over, and invokes an extension of the org.eclipse.swt.graphics.ImageLoader class to create a screenshot, before it terminates the test. Having a screenshot is helpful, as it shows the state of the UI Red Deer determined that something went very wrong, but it's limited in that after the failure, Red Deer immediately stops the test and exits. In some cases having Red Deer take a screenshot and then exit may be adequate as the source of the failure is obvious. In other cases, however, you might prefer that Red Deer enable you to investigate the cause of the failure before the test is stopped and Red Deer exits. Pausing Failing Tests In some ways, an automated test failure is like an automobile crash. Something unexpected happens, and before you can react, the test goes off the road, rolls over and finishes in a ditch. Wouldn't life be easier if you could press a pause button to avoid bad things like this from happening? Likewise, it would be helpful if Red Deer, instead of terminating the test immediately could enable you to press a "pause" button and freeze the test so that you could examine the state of the accident so that you could understand its causes. Luckily Red Deer is a built-in feature to enable you to pause failing tests. Unlike the screenshot feature, having Red Deer pause when a test fails is not enabled by default. To enable the feature, all you have to do is to set the "pauseFailedTest" JVM argument to “true.” For example: -DpauseFailedTest=true With this argument set to “true,” when a test fails, it connects to the Red Deer watchdog process and test execution is paused. You can cause the test execution to continue by pressing the Enter key. Note that in the current version of Red Deer, pauseFailedTest only works when your test extends the RedDeerTest class. In a future release of Red Deer, it will work on all test types. Recording Screencasts Up to now, the Red Deer debugging features that we’ve discussed have all had one characteristic in common; they all require that you run the test and manually observe the results. This means that for an error that is subtle and difficult to track, you have may have to rerun the test several times in order to be able to see the error. It would be more helpful if you had an easy way to pause the execution of the test at the point where it fails and “rewind” the execution to a point before the failure, without having to take the time, and occupy system resources, to rerun the test from its beginning. Red Deer solves this problem by enabling you to save a screencast of all failed tests. Red Deer performs the screencast captures through an extension of the org.monte.screenrecorder.ScreenRecorder (http://www.randelshofer.ch/monte/) class As is the case with pausing tests, Red Deer’s screencast feature is disabled by default. To enable the the recording of screenshots, you set the recordScreenCast JVM argument to “true.” For example: -DrecordScreenCast=true The screencast files are stored in a subdirectory named, appropriately enough, “screencast.” References https://github.com/jboss-reddeer/reddeer/wiki/Debugging-RedDeer Acknowledgements Many thanks to Jirka Peterka and Vlado Pakan for their input to this post!

1 Start with What You Know Now Traditionally, a product strategy is the result of months of market research and business analysis work. It is intended to be factual, reliable, and ready to be implemented. But in an agile, dynamic environment a product strategy is best created differently: Start with your idea, state the vision behind it, and capture your initial strategy. Then identify the biggest risk or the crucial leap-of-faith assumption, address it, and change and improve your strategy. Repeat this process until you are confident that your product strategy is valid. This iterative approach, piloted by Lean Startup, helps you acquire the new knowledge fast and in a goal-oriented, focused manner addressing the key risks or assumptions. It avoids the danger of carrying out too much and too little research, reduces time-to-market, and increases your chances of creating a successful product. 2 Focus on what Matters Most The term product strategy means different things to different people, and strategies come in different shapes and sizes. While that’s perfectly fine, an initial product strategy that forms the basis for subsequent correction and refinement cycles should focus on what matters most: the market, the value proposition, the product’s unique selling points, and the business goals. This is where my Vision Board comes in. I have designed it as the simplest thing that could possibly work to capture the vision and the product strategy. You can download it from romanpichler.com/tools/vision-board for free. For an introduction to the Vision Board, please see my post “The Product Vision Board”. 3 Create the Product Strategy Collaboratively A great way to create your product strategy is to employ a collaborative workshop. Invite the key people required to develop, market, sell and service your product and the senior management sponsor. Such a workshop generates early buy-in, creates shared ownership, and leverages the collective knowledge and creativity of the group. Selling an existing vision and product strategy can be challenging. Co-creation is often the better option. Your initial Vision Board has to be good enough to create a shared understanding of your vision and initial strategy and to identify the biggest risk so you can start re-working your board. But don’t spend too much time on it and don’t try to make it perfect. Your board will change as you correct, improve and refine it. 4 Let your Vision Guide you The product vision is the very reason for creating your product: It describes your overarching goal. The vision also forms the basis of your product strategy as the path to reach your overall goal. As the vision is so important, you should capture it before you describe your strategy. Here are four tips to help you capture your vision: Make sure that your vision does not restate your product idea but goes beyond it. For instance, the idea for this post is to write about creating an agile product strategy, but my vision is to help you develop awesome and successful products. Choose a broad vision, a vision that engages people and that enables you to pivot – to change the strategy while staying true to your vision. Make your vision statement concise; capture it in one or two sentences; and ensure that it is clear and easy to understand. Try to come up with a motivating and inspiring vision that helps unite everyone working on the product. Choosing an altruistic vision, a vision that focuses on the benefits created for others, can help you with this. 5 Put the Users First Once you have captured your vision, work on your strategy by filling in the lower sections of the Vision Board from left to right. Start with the “Target Group”, the people who should use and buy your product rather than thinking about the cool, amazing product features or the smart business model that will monetise the product. While both aspects are important, capturing the users and customers and their needs forms the basis for making the right product and business model decisions. While it’s tempting to think of all the people who could possibly benefit from your product, it is more helpful to choose a clear-cut and narrow target group instead. Describe the users and customers as clearly as you can and state the relevant demographic characteristics. If there are several segments that your product could serve then choose the most promising one. Working with a focused target group makes it easier to test your assumptions, to select the right test group and test method, and to analyse the resulting feedback and data. If it turns out that you have picked the wrong group or made the segment is too small then simply pivot to a new or bigger one. A large or heterogeneous target group is usually difficult to test. What’s more, it leads to many diverse needs, which make it difficult to determine a clear and convincing value proposition and therefore to market and sell the product. 6 Clearly State the Main Problem or Benefit Once you have captured your target users and customers, describe their needs. Consider why they would purchase and use your product. What problem will your product solve, what pain or discomfort will it remove, what tangible benefit will it create? If you identify several needs, then determine the main problem or the main benefit, for instance, by putting it at the top of the section. This helps you test your ideas and create a convincing value proposition. I find that if I am not able to clearly describe the main problem or benefit, I don’t really understand why people would want to use and to buy a product. 7 Describe the Essence of your Product Once you have captured the needs, use the “Product” section to describe your actual product idea. State the three to five key features of your product, those features that make the product desirable and that set it apart from its competitors. When capturing the features consider not only product functionality but also nonfunctional qualities such as performance and interoperability, and the visual design. Don’t make the mistake of turning this section into a product backlog. The point is not to describe the product comprehensively or in a great amount of detail but to identify those features that really matter to the target group. 8 State your Business Goals and Key Business Model Elements Use the “Value” section to state your business goals such as creating a new revenue stream, entering a new market, meeting a profitability goal, reducing cost, developing the brand, or selling another product. Make explicit why it is worthwhile for your company to invest in the product. Prioritise the business goals and state them in the order of their importance. This will guide your efforts and help you choose the right business model. Once you have captured the business goals, state the key elements of your business model including the main revenue sources and cost factors. This is particularly important when you work with a new or significantly changed business model. 9 Extend your Board The Vision Board’s simplicity is one of its assets, but it can sometimes become restricting: The Product and the Value sections can get crowded as the board does not separately capture the competitors, the partners, the channels, the revenue sources, the cost factors, and other business model elements. Luckily there is a simple solution: Extend your board and add further sections, for instance, “Competitors”, “Channels”, “Revenue Streams”, and “Cost Factors”, or download an extended version from my website. But before using an extended Vision Board make sure that you understand who your customers and users are and why they would buy and use the product. There is no point in worrying about the marketing and the sales channels or the technologies if you are not confident that you have identified a problem that’s worthwhile addressing. Additionally, a more complex board usually contains more risks and assumptions. This makes it harder to identify the biggest risk and leap-of-faith assumption. 10 Put it to the Test Capturing your vision and initial product strategy on the Vision Board is great. But it’s only the beginning of a journey in search of a valid strategy, as your initial board is likely to be wrong. After all, you have based the board on what you know now rather than extensive market research work. You should therefore review your initial Vision Board carefully, identify its critical risks or leap-of-faith assumptions, and select the most crucial risk or assumption. Determine the right test group, for instance, selected target users, and the right test method such as problem interviews. Carry out the test, analyse the feedback or data collected, and change your Vision Board with the newly gained knowledge as the picture below shows. If you find that the key risks and assumptions hard to identify then your board may be too vague. If that’s the case then narrow down the target group, select the main problem or benefit, reduce the key features to no more than five, identify the main business benefit, and remove everything else. Your board may significantly change as you iterate over your strategy, and you may have to pivot, to choose a different strategy to make your vision come true. If your Vision Board does not change at all then you should stop and reflect: Are you addressing the right risks in the right way and are you analysing the feedback and data effectively?

Recently I’ve been butting heads with some people on the subject of Ownership, Responsibility and Accountability. There seems to be a very unhealthy obsession with these things sometimes, and I think this is indicative of a less-than-ideal culture. I don’t want to say that they’re “anti-agile” because that just sounds a bit weak, and because I also think they’re not just bad for agile, they’re bad for pretty much any system. I’m not sure how familiar most people are with the “RACI matrix” concept, but in my eyes it’s downright evil in the wrong hands, and I’ve been hearing “RACI Matrix” a lot recently (it’s now on my Bullshit Bingo card). I’ll start off by clarifying what I mean. I’ve got nothing against people owning actions or being accountable for certain particular (usually small) things, but I do take offence when pretty much everything has to be given an owner, someone accountable and someone to “take responsibility”. It’s divisive and results in lots of finger pointing, in my experience. I much prefer the concept of shared ownership, and collective accountability. As a software delivery team, we should all feel responsible for the quality of the product, as well as the performance and the feature richness. These things shouldn’t be assigned for ownership to individuals, as it’ll create an attitude of “well it’s not my problem” among the other team members. Here’s an example: I’ve worked in a team where one person was made the “owner” of the build system. They busied themselves making sure all the builds passed and that the system was regularly ticking over. Of course, the builds often failed and nobody cared except this one person, who then had to try to get people to fix their broken builds. It almost seemed as if people didn’t care about the fact that their software wasn’t capable of being compiled, or that the tests were failing, and in truth they didn’t. They cared about writing code and checking it in, because they didn’t “own” the build system. One message that I always try to drive home with software delivery teams is that our objective is to make software that works for our users, not just write code. I know how easy it is for developers to just focus on checking in code, or perhaps just make sure it passes the tests in the CI system, but beyond that, their focus drops off. I know because I was once one of those developers :-) These days I try to encourage everyone to care about things such as: How your code builds How the tests execute How good the tests are How good the code is How easy it is to deploy How easy it is to maintain How easy it is to monitor Because it takes all of these things to produce good software that users can enjoy, which means we get paid. Here’s another example of how “ownership” has hurt a product: A large system I once worked on was deployed into production using a complicated system of bash and perl scripts, which were cobbled together by a sysadmin who did the deployments. He became the de facto “owner” of the deployment system. There were untold issues with the running of the application because of permissions, paths etc and so forth. The deployment process was creaky and relatively untested. Since the “ownership” of this system was assigned to the sysadmin, rather than devolved or collectively shared throughout the delivery team, the “deployability” was seen as a second class citizen within the delivery team, because everybody felt like it was “owned” by one person who just happened to be on the periphery of the team at best. So here’s what I think: The ability to monitor, maintain, deploy, test, build and create software should all be treated as first class citizens and should be the collective responsibility of everyone in the team. They should all own it, and they should all be accountable. I would extend this out further, to include supporting systems such as environments, build systems, testing frameworks and so-on. Sure, each team might have an SME or two who focuses more on one of these things than any other, but that doesn’t make that one person accountable, responsible or the owner any more than any particular developer is the “owner” of any particular class, method or function. If I write some code that depends on a method that someone else has written, and that method is failing, I don’t just down tools, shrug my shoulders and say “well I’m not accountable for that”. That would be hugely unhelpful and I’d make no friends either. In the same way, we shouldn’t treat our supporting functions and systems as someone else’s responsibility. If we need it in order to make our software work for the end user, then it’s our collective responsibility, no matter what “it” is.

one of the most dreadful problem of unit testing is slow testing. if your whole suite of tests runs in 10 minutes, it is normal for developers not to run the whole suite at each build. one of the most common question is how can i deal with slow unit tests? here is my actual scenario: in a project i’m working in, we have some multilingual full text search done in elastic search and we have a battery of unit tests that verify that searches work as expected. since each test deletes all documents, insert a bunch of new documents and finally commits lucene index, execution times is high compared to the rest of tests. each test need almost 2 seconds to run on my workstation, where i have really fast ssd and plenty of ram. this kind of tests cannot be run in memory or with some fancy trick to make then run quickly. actually we have about 30 tests that executes in less than one seconds, and another 13 tests that runs in about 23 seconds, this is clearly unacceptable . after few hours of work, we already reached the point where running the whole suite becomes annoying. the solution this is a real common problem and it is quite simple to fix. first of all visual studio test runner actually tells you execution time for each unit test, so you can immediately spot slow tests. when you identify slow tests you can mark them with a specific category, i use slowtest 1 2 3 4 [testfixture] [category("elasticsearch")] [category("slowtest")] public class essearcherfixture : basetestfixturewithhelpers since i know in advance that this test are slow i immediately mark the entire class with the attribute slowtest. if you have no idea what of your tests are slow, i suggest grouping test by duration in visual studio test runner. figure 1: group tests by duration the result is interesting, because visual studio consider every test that needs more than one second to be slow. i tend to agree with this distinction. figure 2: test are now grouped by duration this permits you to immediately spot slow tests, so you can add the category slowtest to them. if you keep your unit tests organized and with a good usage of categories, you can simply ask vs test runner to exclude slow test with filter – traits:”slowtest” figure 3: thanks to filtering i can now execute continuously only test that are not slow. i suggest you to do a periodic check to verify that every developers is using the slowtest category wisely, just group by duration, filters out the slowtest and you should not have no tests that are marked slow. figure 4: removing the slowtest category and grouping by duration should list no slow test. the nice part is that i’m using nunit, because visual studio test runner supports many unit tests frameworks thanks to the concepts of test adapters. if you keep your tests well organized you will gain maximum benefit from them :).

When I am trying to install third party apk using ADB tool, I have faced "Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]" error. To resolve the issue, I have followed few steps. Open command prompt; Go to your debug.keystore location. For eg: You can find the debug.keystore file in the following location C:\Documents and Settings\User\.android 1.Using Zip align copied apk. zipalign -v 4 D:\Test.apk D:\Testc.apk 2.keytool -genkey -v -keystore debug.keystore -alias sampleName -keyalg RSA -keysize 2048 -validity 20000 Now a prompt will ask for Password First and lastname Name of Organization unit Name of Organization City State Country After entering these fields we get our Certificate 3. jarsigner -verbose -keystore debug.keystore D:\Testc.apk sampleName In some cases we need add -sigalg SHA1withRSA -digestalg SHA1 arguments to work out the step 3 jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore debug.keystore D:\Testc.apk sampleNameNow it will ask for the password and then it will replace the apk with the signed one. To check whether it is working or not, you can check using the following command. jarsigner -verify D:\Testc.apk Then I have installed apk using ADB. Adb install D:\Testc.apk

In an earlier blog entry I had touched on a fictitious rube goldberg flow for capitalizing a string through a complicated series of steps, the premise of the article was to introduce Spring Integration Java DSL as an alternative to defining integration flows through xml configuration files. I learned a few new things after writing that blog entry, thanks to Artem Bilan and wanted to document those learnings here: So, first my original sample, here I have the following flow(the one's in bold): Take in a message of this type - "hello from spring integ" Split it up into individual words(hello, from, spring, integ) Send each word to a ActiveMQ queue Pick up the word fragments from the queue and capitalize each word Place the response back into a response queue Pick up the message, re-sequence based on the original sequence of the words Aggregate back into a sentence("HELLO FROM SPRING INTEG") and Return the sentence back to the calling application. EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); jmsOutboundGateway.setReplyChannel(sequenceChannel()); return jmsOutboundGateway; } It turns out, based on Artem Bilan's feedback, that a few things can be optimized here. First notice how I have explicitly defined two direct channels, "requestChannel" for starting the flow that takes in the string message and the "sequenceChannel" to handle the message once it returns back from the jms message queue, these can actually be totally removed and the flow made a little more concise this way: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } @Bean public JmsOutboundGateway jmsOutboundGateway() { JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway(); jmsOutboundGateway.setConnectionFactory(this.connectionFactory); jmsOutboundGateway.setRequestDestinationName("amq.outbound"); return jmsOutboundGateway; } "requestChannel" is now being implicitly created just by declaring a name for it. The sequence channel is more interesting, quoting Artem Bilan - do not specify outputChannel for AbstractReplyProducingMessageHandler and rely on DSL , what it means is that here jmsOutboundGateway is a AbstractReplyProducingMessageHandler and its reply channel is implicitly derived by the DSL. Further, two methods which were earlier handling the flows for sending out the message to the queue and then continuing once the message is back, is collapsed into one. And IMHO it does read a little better because of this change. The second good change and the topic of this article is the introduction of the Jms namespace factories, when I had written the previous blog article, DSL had support for defining the AMQ inbound/outbound adapter/gateway, now there is support for Jms based inbound/adapter adapter/gateways also, this simplifies the flow even further, the flow now looks like this: @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from("requestChannel") .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(Jms.outboundGateway(connectionFactory) .requestDestination("amq.outbound")) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } The inbound Jms part of the flow also simplifies to the following: @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(Jms.inboundGateway(connectionFactory) .destination("amq.outbound")) .transform((String s) -> s.toUpperCase()) .get(); } Thus, to conclude, Spring Integration Java DSL is an exciting new way to concisely configure Spring Integration flows. It is already very impressive in how it simplifies the readability of flows, the introduction of the Jms namespace factories takes it even further for JMS based flows.

Closely approaching the one year mark of when I first joined MongoLab (and the MongoDB community), I had the pleasure of attending the inaugural MongoDB World conference put together by the incredible MongoDB team. Second only to the excitement around major MongoDB feature announcements was the collective disbelief that this was MongoDB’s first multi-day conference ever. A big congratulations to all those that worked hard to put on such a massive (did you see the Intrepid!?) event. All this planning would have been for naught if MongoDB leaders and engineers failed to deliver announcements and features that would meet and exceed expectations. From major public cloud announcements to the reveal of document-level locking in version 2.8, developers and conference goers had plenty to be excited about. There was a lot to digest from the conference… we’ll cover the major highlights in case you missed them. Big announcements in public cloud Our time at the MongoLab booth yielded many high-quality conversations, predominantly those about offloading previously internal processes and workloads to the public cloud. It was remarkable to see and hear so many enterprise teams with the exact same message: the public cloud is the future, and the future is now. It’s no surprise then that MongoDB, Inc. released not one, but two press releases around MongoDB solutions for the public cloud. Fully-managed MongoDB on the Microsoft Azure Store Nearly one year ago, MongoDB, Inc. chose to partner with the MongoLab team to build a production-ready MongoDB solution for developers on Microsoft Azure. On the first day of World, MongoDB, Inc. announced the product of our collaboration – a fully-managed highly available MongoDB-as-a-Service Add-On offering on the Microsoft Azure Store. This new service runs MongoDB Enterprise and offers replication, monitoring and support from MongoDB, Inc. It’s also backed by MongoDB Management Service (MMS), allowing for point-in-time recovery of MongoDB deployments. Now, teams without the expertise or resources to manage their MongoDB deployment(s) can outsource all the database operations (monitoring and alerting, backups, performance tuning, etc.) to both MongoLab and MongoDB’s expert support teams. You can check out the MongoDB add-on in the Azure Store: https://azure.microsoft.com/en-us/gallery/store/mongodb/mongodb-inc/ MongoDB solutions on Google Cloud Platform MongoDB, Inc. also announced the arrival of new resources to help Google Cloud Platform customers deploy MongoDB on Google Compute Engine. These resources include a “Click to Deploy” feature and a MongoDB on Google Compute Engine Solutions paper covering MongoDB best practices. If you are looking for a fully-managed solution, with automated provisioning, backups, integrated monitoring and alerting, along with expert support, MongoLab recently announced the arrival of production-ready replica sets on Google. Product Roadmap – MongoDB version 2.8 On the second day of MongoDB World, Eliot Horowitz, MongoDB, Inc. CTO & Co-founder, took center stage and announced two huge changes to the MongoDB core project: document-level locking and pluggable storage engines. These features not only reflect improvements to the core project, but also signal to the community that the MongoDB team is listening to its users and is capable of delivering the software needed to power the workloads of tomorrow. Document-level locking The slides above from Eliot’s keynote point to a current obstacle (database-level locking) in MongoDB that limits overall scalability. With database-level locking, any write operation to the database holds the write lock and prevents subsequent writes from executing on the database until the original operation holding the write lock completes. Eliot’s announcement of document-level locking moves the write lock contention from the database level to the document (MongoDB equivalent to SQL “records”) level. This change will allow users to achieve much higher write throughput (we saw a 10x performance improvement in the live demo) across their MongoDB deployments, improving write scalability. If you’d like to try out document-level locking, the MongoDB team has already pushed the feature to the master branch on GitHub. This should only be used for experimentation, not to be run in production. Pluggable storage engine As MongoDB matures, feature releases like document level locking will continue to allow developers to build robust systems on top of MongoDB. But as the number of use cases grows, different tooling tailored to specific use cases may prove to be extremely beneficial. For example, if Company X decides that they want to use MongoDB to warehouse some of their data, they would likely want to optimize their database for slow-moving data and storage efficiency (compression). With the introduction of pluggable storage engines, many new possibilities are open to the community. Teams can now write their own storage engine for a particular use case, configure replica set nodes with different storage engines for specific situations, or collaborate with the open-source community to architect innovative solutions. This feature not only allows for more granular control of the database, but also encourages the MongoDB community to work together. Takeaways: A maturing and thriving ecosystem Roughly a year ago, MongoLab CTO Todd Dampier recapped MongoSF 2013 and spoke to the health of the MongoDB ecosystem. How far we’ve come! After attending the inaugural MongoDB World and chatting with MongoDB Masters, interns, hackathon winners, power users and those new to the community, the enthusiasm is still surging and as positive as ever. This enthusiasm is well placed. Developers and hackers use MongoDB because so much rich data on the web is shared as JSON (think Facebook, Twitter, Google, etc.). As a result, MongoDB is the de-facto database for hackathons and bootstrapped projects. Just learn the API for the site you want to mine, throw the JSON in MongoDB and query your data with the rich query language- it’s that easy. The MongoDB ecosystem is maturing as well. Take a look at the Customer Success Stories and you’ll get a feel for the extent in which enterprises leverage the solution and use it in production. To further drive enterprise adoption, MongoDB, Inc.’s public cloud solutions and product roadmap features aim to help teams run MongoDB in production and give teams the confidence that MongoDB will continue to improve scalability and meet their growing project requirements. Congratulations again to the MongoDB team on their big announcements and for creating such a fantastic forum at which to learn and meet fellow MongoDB users. Our team at MongoLab had a great time making new friends and talking shop; we look forward to meeting more MongoDB users soon (at a MongoDB Days near you)! -Chris@MongoLab

NOTE: Always Agile Consulting now offers an Introduction To Trunk Based Development training course! Trunk Based Development supports Optimistic and Pessimistic Release Branching Trunk Based Development is a style of software development in which all developers commit their changes to a single shared trunk in source control, and every commit yields a production-ready build. It is a prerequisite for Continuous Delivery as it ensures that all code is continuously integrated into a single workstream, that developers always work against the latest code, and that merge/integration pain is minimised. It is important to note that Trunk Based Development does not prohibit branching. Trunk Based Development is compatible with a Release Branching strategy of short-lived release branches that are used for post-development defect fixes. That Release Branching strategy might be optimistic and defer branch creation until a defect occurs, or be pessimistic and immediately incur branch creation. For example, consider an application developed using Trunk Based Development. The most recent commits to trunk were source revisions a and b which yielded application versions 610 and 611 respectively, and version 610 is intended to be the next production release. With Optimistic Branching, the release of version 610 is immediate as there is no upfront branching. If a defect is subsequently found then a decision must be made where to commit the fix, as trunk has progressed since 610 from a to b. If the risk of pulling forward from a to b is acceptable then the simple solution is to commit the fix to trunk as c, and consequently release version 612. However, if the risk of pulling forward from a to b is unacceptable then a 610.x release branch is created from a, with the fix committed to the branch as c and released as version 610.1. That fix is then merged back into trunk as d to produce the next release candidate 612, and the 610.x branch is earmarked for termination. With Pessimistic Branching, the release of version 610 is accompanied by the upfront creation of a 610.x release branch in anticipation of defect(s). If a defect is found in version 610 then as with Optimistic Branching a decision must be made as to where the defect fix should be committed. If the risk of pulling forward from a to b is deemed insignificant then trunk can be pulled forward from a to b and the fix committed to trunk as c for release as version 612. The 610.x branch is therefore terminated without ever being used. If on the other hand the risk is deemed significant then the fix is committed to the 610.x branch as c and released as version 610.1. The fix is merged back into trunk as d and version 612, which will also receive its own branch upon release. The choice between Optimistic Branching and Pessimistic Branching for Trunk Based Development is dependent upon product quality and lead times. If product quality is poor and lead times are long, then the upfront cost of Pessimistic Branching may be justifiable. Alternatively, if post-development defects are rare and production releases are frequent then Optimistic Branching may be preferable.

sqs, simple message queue , is a message-queue-as-a-service offering from amazon web services. it supports only a handful of messaging operations, far from the complexity of e.g. amqp , but thanks to the easy to understand interfaces, and the as-a-service nature, it is very useful in a number of situations. but how fast is sqs? how does it scale? is it useful only for low-volume messaging, or can it be used for high-load applications as well? if you know how sqs works, and want to skip the details on the testing methodology, you can jump straight to the test results . sqs semantics sqs exposes an http-based interface. to access it, you need aws credentials to sign the requests. but that’s usually done by a client library (there are libraries for most popular languages; we’ll use the official java sdk ). the basic message-related operations are: send a message, up to 256 kb in size, encoded as a string. messages can be sent in bulks of up to 10 (but the total size is capped at 256 kb). receive a message. up to 10 messages can be received in bulk, if available in the queue long-polling of messages. the request will wait up to 20 seconds for messages, if none are available initially delete a message there are also some other operations, concerning security, delaying message delivery, and changing a messages’ visibility timeout, but we won’t use them in the tests. sqs offers at-least-once delivery guarantee. if a message is received, then it is blocked for a period called “visibility timeout”. unless the message is deleted within that period, it will become available for delivery again. hence if a node processing a message crashes, it will be delivered again. however, we also run into the risk of processing a message twice (if e.g. the network connection when deleting the message dies, or if an sqs server dies), which we have to manage on the application side. sqs is a replicated message queue, so you can be sure that once a message is sent, it is safe and will be delivered; quoting from the website: amazon sqs runs within amazon’s high-availability data centers, so queues will be available whenever applications need them. to prevent messages from being lost or becoming unavailable, all messages are stored redundantly across multiple servers and data centers. testing methodology to test how fast sqs is and how it scales, we will be running various numbers of nodes, each running various number of threads either sending or receiving simple, 100-byte messages. each sending node is parametrised with the number of messages to send, and it tries to do so as fast as possible. messages are sent in bulk, with bulk sizes chosen randomly between 1 and 10. message sends are synchronous, that is we want to be sure that the request completed successfully before sending the next bulk. at the end the node reports the average number of messages per second that were sent. the receiving node receives messages in maximum bulks of 10. the amazonsqsbufferedasyncclient is used, which pre-fetches messages to speed up delivery. after receiving, each message is asynchronously deleted. the node assumes that testing is complete once it didn’t receive any messages within a minute, and reports the average number of messages per second that it received. each test sends from 10 000 to 50 000 messages per thread. so the tests are relatively short, 2-5 minutes. there are also longer tests, which last about 15 minutes. the full (but still short) code is here: sender , receiver , sqsmq . one set of nodes runs the mqsender code, the other runs the mqreceiver code. the sending and receiving nodes are m3.large ec2 servers in the eu-west region, hence with the following parameters: 2 cores intel xeon e5-2670 v2 7.5 gb rams the queue is of course also created in the eu-west region. minimal setup the minimal setup consists of 1 sending node and 1 receiving node, both running a single thread. the results are, in messages/second: average min max sender 429 365 466 receiver 427 363 463 scaling threads how do these results scale when we add more threads (still using one sender and one receiver node)? the tests were run with 1, 5, 25, 50 and 75 threads. the numbers are an average msg/second throughput. number of threads: 1 5 25 50 75 sender per thread 429,33 407,35 354,15 289,88 193,71 sender total 429,33 2 036,76 8 853,75 14 493,83 14 528,25 receiver per thread 427,86 381,55 166,38 83,92 47,46 receiver total 427,86 1 907,76 4 159,50 4 196,17 3 559,50 as you can see, on the sender side, we get near-to-linear scalability as the number of thread increases, peaking at 14k msgs/second sent (on a single node!) with 50 threads. going any further doesn’t seem to make a difference. the receiving side is slower, and that is kind of expected, as receiving a single message is in fact two operations: receive + delete, while sending is a single operation. the scalability is worse, but still we can get as much as 4k msgs/second received. scaling nodes another (more promising) method of scaling is adding nodes, which is quite easy as we are “in the cloud”. the test results when running multiple nodes, each running a single thread are: number of nodes: 1 2 4 8 sender per node 429,33 370,36 350,30 337,84 sender total 429,33 740,71 1 401,19 2 702,75 receiver per node 427,86 360,60 329,54 306,40 receiver total 427,86 721,19 1 318,15 2 451,23 in this case, both on the sending&receiving side, we get near-linear scalability, reaching 2.5k messages sent&received per second with 8 nodes. scaling nodes and threads the natural next step is, of course, to scale up both the nodes, and the threads! here are the results, when using 25 threads on each node: number of nodes: 1 2 4 8 sender per node&thread 354,15 338,52 305,03 317,33 sender total 8 853,75 16 925,83 30 503,33 63 466,00 receiver per node&thread 166,38 159,13 170,09 174,26 receiver total 4 159,50 7 956,33 17 008,67 34 851,33 again, we get great scalability results, with the number of receive operations about half the number of send operations per second. 34k msgs/second processed is a very nice number! to the extreme the highest results i managed to get are: 108k msgs/second sent when using 50 threads and 8 nodes 35k msgs/second received when using 25 threads and 8 nodes i also tried running longer “stress” tests with 200k messages/thread, 8 nodes and 25 threads, and the results were the same as with the shorter tests. running the tests – technically to run the tests, i built docker images containing the sender / receiver binaries, pushed to docker’s hub, and downloaded on the nodes by chef. to provision the servers, i used amazon opsworks. this enabled me to quickly spin up and provision a lot of nodes for testing (up to 16 in the above tests). for details on how this works, see my “cluster-wide java/scala application deployments with docker, chef and amazon opsworks” blog . the sender / receiver daemons monitored (by checking each second the last-modification date) a file on s3. if a modification was detected, the file was downloaded – it contained the test parameters – and the test started. summing up sqs has good performance and really great scalability characteristics. i wasn’t able to reach the peak of its possibilities – which would probably require more than 16 nodes in total. but once your requirements get above 35k messages per second, chances are you need custom solutions anyway; not to mention that while sqs is cheap, it may become expensive with such loads. from the results above, i think it is clear that sqs can be safely used for high-volume messaging applications, and scaled on-demand. together with its reliability guarantees, it is a great fit both for small and large applications, which do any kind of asynchronous processing; especially if your service already resides in the amazon cloud. as benchmarking isn’t easy, any remarks on the testing methodology, ideas how to improve the testing code are welcome!

The Android system software stack is typically divided into the four areas as the following graphic: Terminology Android Software Development Kit (Android SDK) contains the necessary tools to create, compile and package Android applications Android debug bridge (adb), which is a tool that allows you to connect to a virtual or real Android device Google provides two integrated development environments (IDEs) to develop new applications. Android Developer Tools (ADT) are based on the Eclipse IDE Android Studio based on the IntelliJ IDE Android RunTime (ART) uses Ahead Of Time compilation, and optional runtime for Android 4.4 Android Virtual Device (AVD) - The Android SDK contains an Android device emulator. This emulator can be used to run an Android Virtual Device (AVD), which emulates a real Android phone Dalvik Virtual Machine (Dalvik)- The Android system uses a special virtual machine, Dalvik, to run Java-based applications. Dalvik uses a custom bytecode format which is different from Java bytecode. Therefore you cannot run Java class files on Android directly; they need to be converted into the Dalvik bytecode format.

This post exists to help with an MSDN Magazine article that I am authoring It provides some of the low-level details for the article How to install Mono and root certificates on a raspberry pi How to create an Azure mobile service How to create a Custom API inside Azure mobile services that the raspberry pi can call into How to create an Azure storage account MONO - HOW TO INSTALL ON A RASPBERRY PI Why Mono? How to install Mono on a raspberry pi Installing trusted root certificates on to the raspberry pi http://www.mono-project.com/Main_Page An open source, cross-platform, implementation of C# and the CLR that is binary compatible with Microsoft.NET Mono is a free and open source project led by Xamarin (formerly by Novell) that provides a .NET Framework-compatible set of tools including, among others, a C# compiler and a Common Language Runtime WHY MONO? Because it lets us write .net code compiled on Windows We can simply copy the binary files from Windows to Linux and run it as is From a raspberry pi device, it is possible to use a .net application to take a photo and upload it to Windows Azure storage HOW TO INSTALL ON A RASPBERRY PI RUNNING LINUX You will issue the following commands: pi@raspberrypi ~ $ sudo apt-get update pi@raspberrypi ~ $ sudo apt-get install mono-complete The first command makes sure all the local package index are up to date with the changes made in repositories. Second command installs the complete Mono tooling and runtime. MAKING SURE THAT YOUR MONO APPLICATIONS CAN MAKE A HTTPS REST-BASED CALLS This command downloads the trusted root certificates from the Mozilla LXR web site into the Mono certificate store. Once complete, the Raspberry PI will be capable of making web requests using HTTPS requests within Mono. pi@raspberrypi ~ $ mozroots --import --ask-remove --machine CREATING A NEW AZURE MOBILE SERVICES ACCOUNT The mobile services account is needed to host a Node.js application that provides shared access signatures to raspberry pi devices The shared access signature is needed by the raspberry pi, so that it can directly and securely upload photos to Azure storage STEPS TO CREATE AN AZURE MOBILE SERVICE The steps below will create an Azure mobile service The service will be used to host a Node.js application interacting with a raspberry pi devices We will provision a SQL database, although it will not be used initially FOLLOW THESE STEPS TO CREATE THE MOBILE SERVICE Login into the Azure Portal Select MOBILE SERVICES from the left menu pane at the Azure Portal. In the lower left corner select "+NEW" to create a new Azure Mobile Service. Make sure you've selected, "COMPUTE / MOBILE SERVICE / CREATE." You will now enter a url. We will call this service raspberrymobileservice. For the DATABASE, we will choose "Create a new SQL database instance." The REGION we chose is "West US." The BACKEND is "JavaScript." Click the "->" arrow to proceed to the next screen. In this screen you will "Specify database settings." The NAME of your database will based on the URL you entered previously. In this case, the database is called "raspberrymobileservice_db." You will need to choose a SERVER. We will choose "New SQL database server" from the drop-down list. You will need to provide a SERVER LOGIN NAME and a SERVER LOGIN PASSWORD. Take note of the login you provided as it will be needed later CREATING A CUSTOM API Azure mobile services allows you to create a custom API written in JavaScript that can be called from a raspberry pi device using REST This custom API is really just a Node.js application running in the server CREATING THE API TO RESPOND TO THE DEVICE TRYING TO UPLOAD PHOTOS Now that the service is established, we will turn our attention to creating an API that the device can call into to upload a photo. Login into the Azure Portal Your mobile service will take a few minutes to complete, and you should see the "Ready" flag as the "Status" for your service. Once it is ready you can drill into your service to customize its behavior. Just to the right of the service name, click the right arrow key "->" to drill into the service details. The top menu bar will offer many options, but we are interested in the one titled "API." The API allows you to create a series of node.JS API calls that a device can call into using rest-based approaches. Click on "API." from there, select "CREATE A CUSTOM API." You will be asked to provide an API name. Type in "photos" for the API name. Below you will see a series of drop-down combo boxes that relate to permission. We will keep the default value of "Anybody with the application key." This might not be the best option for all scenarios. You can read more about this here. http://msdn.microsoft.com/en-us/library/azure/jj193161.aspx. Click the checkmark to complete the process. The name of the AP you just created, "Photos," should be visible on the portal interface. To drill into the photos API click on the right arrow key "->". The right arrow key will be just to the right of the name of the API "Photos". At this point you should see a basic script that has been provided by default. We will overwrite this default script with our own script as described in the MSDN Magazine article. CREATING A STORAGE ACCOUNT TO STORE THE PHOTOS Navigate to the portal and create a storage account Create a container for the photos Obtain the: Storage Account Name (you will provide a name) Storage Account Access key (generated for you) Container Name (you will create) CREATING A STORAGE ACCOUNT We will need a storage account so that we can upload photos to it. The steps are well documented here: http://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account/ In our case we call the storage account raspberrystorage. This means that the URL that the device will use to upload photos is https://raspberrystorage.blob.core.windows.net/. As you complete these steps make sure that you choose the storage account location to be the same location as was used for your mobile services account. This avoids any unnecessary latency or bandwidth costs between data centers. Once the storage account is created, we will need to create a container within it. Photos or any blob for that matter, are always stored within a container. To create a container drill into your newly created storage account and select CONTAINERS from the top menu. From there, select CREATE A CONTAINER. The new container dialog box will ask for a name for your container. Take note of the name you provide. We are calling our container ?photocontainer.? When the raspberry pi device uploads photos to the storage account, it will target a specific container, such as the one we just created. You will next be asked to indicate ACCESS rights. To keep things simple we will select access rights of Public Blob. ENTERING APP SETTINGS Rather than hard-code storage account information inside your JavaScript/Node.js applications, you should consider using apps settings inside of the Azure mobile services portal This post also discusses it well: http://blogs.msdn.com/b/carlosfigueira/archive/2013/12/09/application-settings-in-azure-mobile-services.aspx ?The idea of application settings is a set of key-value pairs which can be set for the mobile service (either via the portal or via the command-line interface), and those values could be then read in the service runtime.? NAVIGATING TO APP SETTINGS Navigate to the Azure Mobile Services section of the portal. Drill into the specific service by hitting the arrow below Select from the Configure Menu at the top Scroll down to the very bottom to see app settings Note that we need to enter: - We need to get this from Azure Storage - PhotoContainerName - AccountName - AccountKey We get this information from the Azure Storage Section of the Portal. Note that you need to have provisioned a Storage Account to have this information. How to get the AccountKey with Azure Storage Services Now you can get the access keys HOW NODE.JS WILL ACCESS THE APP SETTINGS You will create a Node.js application inside of Azure Mobile Services See previous steps THE NODE.JS APPLICATION READING APP SETTINGS You will starting by going back to Azure Mobile Services and drill down into your newly minted service We called ours raspberrymobileservice Once you click API, you should see: Notice the app settings are being read on lines 12 to 14.

One of the great improvement in Java EE 5 and beyond it is the introduction of CDI (Context and Dependency Injection). CDI is used for injecting dependencies among a lot of other things like events, interceptors, … and can be used in POJOs. In some cases instead of injecting other objects (as a dependency injection), you want to inject a value from a properties file into a class that needs to be configured externally. I have written an example in Tomitribe community zone: https://github.com/tomitribe/community/tree/master/injecting-properties We keep learning, Alex.

A unit test is a set of methods launched frequently to validate your code. It is usually a good idea to write code in this order: Write a class API Write a method to test the API Implement the API Launch the unit tests Why write unit tests? They validate current and future implementations. They measure code quality. They force you to write testable, loosely coupled code. They’re cheaper than manual regression testing. They build confidence in your code. They help teamwork. Why use a checklist? Unit testing can be harder than actual implementation. Unit testing forces you to really think things through. But unit tests should be simple, direct, and easy to read and maintain. You also need to know when to stop writing tests and start writing the implementation. Use this checklist to be sure your tests are really useful and to the point. Remember: the checklist helps you avoid big mistakes, but you need to make sure of the following: □ My test class is testing only one class. o You are testing a class API to be sure the public contract is respected. □ My methods are testing only one method at a time. o Be sure not to test private methods! They are hidden implementation details, not API. □ My variables and method names are explicit. o For example, store an expected value in an expectedFoo variable instead of just foo. If you test many combinations, use composed variable names like inputValue_NotNull, inputValue_ZeroData, inputValue_PastDate, etc. (according to your application’s coding convention). □ My test cases are easy to read by humans. o Future maintainers should be able to read your tests before reading the implementation. This will help them understand a class API before tweaking or debugging it. □ My tests respect the usual clean code standards. o There should be no flow control in a test method (switch, if, etc.). A good test is just a very straightforward sequence of setup/validate instructions. If necessary, use sub-methods to factorize and make your tests easier to read. In case of multiple scenarios, use multiple test methods (one for each case). o For example, a test method should fit on screen without scrolling – 1 to 20 lines long. If the method is longer, consider writing multiple test methods for each case instead of jamming them together. □ My tests are also testing expected exceptions. o In java, use @Test(expected=MyException.class). □ My tests don’t need access to a database. o Or if a test does need database access, then it must be a mocked, “fire and forget” temporary database that you fill with test cases for every new test method (use the Setup/Teardown methods to prepare it). □ My tests don’t need access to network resources. o You can’t rely on third parties like network or device presence to validate a method (use mocks). □ My tests control side effects, limit values (max, min) and null variables (even when they throw exceptions). o You want to make sure these problem cases never occur, even when the test won’t be used during maintenance. □ My tests can be run at any time, at any place without needing configuration or human intervention. □ My tests pass for the current implementation and are easy to evolve. o Tests really exist to support code evolution. If they are too hard to maintain or too light to refine the code, then they are a useless burden. (Many developers avoid unit testing for this reason.) □ My tests are concrete. o In Java: don’t use Date() as input for a method you are testing, but build a concrete date out of Calendar (don’t forget to force the timezone). Other example: use name = “Smith”; instead of name = “name”; or name = “test”; □ My tests use a mock to simulate/stub complex class structure or methods. o Remember to test only one class API at a time. o Never test third-party libraries through your own classes. Libraries should come with their own tests (this is actually a good way to choose a library). □ My tests are never @ignored or commented out. Never. Ever. □ My tests help me validate my architecture. o If you can’t test a method or a class, your design is not agile enough. □ My tests can run on any supported platform, not just the targeted platform. o Don’t expect a particular device or hardware configuration. Otherwise, your tests will make migration tougher and you will be incentivized to disable them. □ My tests are lightning fast! o Slow tests shouldn’t drag you down. Speed encourages you to launch your tests often. It also helps to reduce building time on Continuous Integration systems. o Use a test runner that allows you to launch one test at a time while you are writing it. Use “delay” or “sleep” with caution – i.e., only in some edge cases, like waiting for notifications or clock-based methods.

this post is summary of the “anomaly detection : a survey”. anomaly detection refers to the problem of finding patterns in data that do not conform to expected behavior. these non-conforming patterns are often referred to as anomalies, outliers, discordant observations, exceptions, aberrations, surprises, peculiarities or contaminants in different application domains. anomalies are patterns in data that do not conform to a well defined notion of normal behavior. interesting to analyze unwanted noise in the data also can be found in there. novelty detection which aims at detecting previously unobserved (emergent, novel) patterns in the data challenges for anomaly detection drawing the boundary between normal and anomalous behavior availability of labeled data noisy data type of anomaly anomalies can be classified into following three categories point anomalies - an individual data instance can be considered as anomalous with respect to the rest of data contextual anomalies - a data instance is anomalous in a specific context (but not otherwise), then it is termed as a contextual anomaly (also referred as conditional anomaly). each data instance is defined using following two sets of attributes contextual attributes. the contextual attributes are used to determine the context (or neighborhood) for that instance eg: in time- series data, time is a contextual attribute which determines the position of an instance on the entire sequence behavioral attributes. the behavioral attributes define the non-contextual characteristics of an instance eg: in a spatial data set describing the average rainfall of the entire world, the amount of rainfall at any location is a behavioral attribute to explain this we will look into "exchange rate history for converting united states dollar (usd) to sri lankan rupee (lkr)"[1] contextual anomaly t2 in a exchange rate time series. note that the exchange rate at time t1 is same as that at time t2 but occurs in a different context and hence is not considered as an anomaly 3. collective anomalies - a collection of related data instances is anomalous with respect to the entire data set data labels the labels associated with a data instance denote if that instance is normal or anomalous. depending labels availability, anomaly detection techniques can be operated in one of the following three modes supervised anomaly detection - techniques trained in supervised mode assume the availability of a training data set which has labeled instances for normal as well as anomaly class semi-supervised anomaly detection - techniques that operate in a semi-supervised mode, assume that the training data has labeled instances for only the normal class. since they do not require labels for the anomaly class unsupervised anomaly detection - techniques that operate in unsupervised mode do not require training data, and thus are most widely applicable. the techniques implicit assume that normal instances are far more frequent than anomalies in the test data. if this assumption is not true then such techniques suffer from high false alarm rate output of anomaly detection anomaly detection have two types of output techniques scores. scoring techniques assign an anomaly score to each instance in the test data depending on the degree to which that instance is considered an anomaly labels. techniques in this category assign a label (normal or anomalous) to each test instance applications of anomaly detection intrusion detection intrusion detection refers to detection of malicious activity. the key challenge for anomaly detection in this domain is the huge volume of data. thus, semi-supervised and unsupervised anomaly detection techniques are preferred in this domain.denning[3] classifies intrusion detection systems into host based and net- work based intrusion detection systems. host based intrusion detection systems - this deals with operating system call traces network intrusion detection systems - these systems deal with detecting intrusions in network data. the intrusions typically occur as anomalous patterns (point anomalies) though certain techniques model[4] the data in a sequential fashion and detect anomalous subsequences (collective anomalies). a challenge faced by anomaly detection techniques in this domain is that the nature of anomalies keeps changing over time as the intruders adapt their network attacks to evade the existing intrusion detection solutions. fraud detection fraud detection refers to detection of criminal activities occurring in commercial organizations such as banks, credit card companies, insurance agencies, cell phone companies, stock market, etc. the organizations are interested in immediate detection of such frauds to prevent economic losses. detection techniques used for credit card fraud and network intrusion detection as below. statistical profiling using histograms parametric statisti- cal modeling non-parametric sta- tistical modeling bayesian networks neural networks support vector ma- chines rule-based clustering based nearest neighbor based spectral information theoretic here are some domain in fraud detections credit card fraud detection mobile phone fraud detection insurance claim fraud detection insider trading detection medical and public health anomaly detection anomaly detection in the medical and public health domains typically work with pa- tient records. the data can have anomalies due to several reasons such as abnormal patient condition or instrumentation errors or recording errors. thus the anomaly detection is a very critical problem in this domain and requires high degree of accuracy. industrial damage detection such damages need to be detected early to prevent further escalation and losses. fault detection in mechanical units structural defect detection image processing anomaly detection techniques dealing with images are either interested in any changes in an image over time (motion detection) or in regions which appear ab- normal on the static image. this domain includes satellite imagery. anomaly detection in text data anomaly detection techniques in this domain primarily detect novel topics or events or news stories in a collection of documents or news articles. the anomalies are caused due to a new interesting event or an anomalous topic. sensor networks since the sensor data collected from various wireless sensors has several unique characteristics. references [1] http://themoneyconverter.com/usd/lkr.aspx [2] varun chandola, arindam banerjee, and vipin kumar. 2009. anomaly detection: a survey. acm comput. surv. 41, 3, article 15 (july 2009), 58 pages. doi=10.1145/1541880.1541882 http://doi.acm.org/10.1145/1541880.1541882 [3] denning, d. e. 1987. an intrusion detection model. ieee transactions of software engineer-ing 13, 2, 222–232. [4]gwadera, r., atallah, m. j., and szpankowski, w. 2004. detection of significant sets of episodes in event sequences. in proceedings of the fourth ieee international conference on data mining. ieee computer society, washington, dc, usa, 3–10.

this post will show how to automate the deployment process of a android application using jenkins continuous integration – to build the project, run the unit tests (if any), archive the built artifacts and run the android lint reports. 1. install jenkins as a windows service navigate to jenkins-ci.org website using an internet browser and download the windows native package (the link is underlined for easy identification) as shown from the right side pane of the download jenkins tab. once the download is complete, uncompress the zip file and click on the jenkins-1.xxx.msi file. proceed through the configuration steps to install the jenkins as a windows service. 2. modify default jenkins port by default jenkins runs on the port 8080. in order to avoid conflict with other applications, the default port can be modified by editing the jenkins.xml found under c:\program files (x86)\jenkins location. as shown below, modify the httpport to 8082. jenkins jenkins this service runs jenkins continuous integration system. %base%\jre\bin\java -xrs -xmx256m -dhudson.lifecycle=hudson.lifecycle.windowsservicelifecycle -jar "%base%\jenkins.war" --httpport=8082 rotate once the modification is saved in jenkins.xml file, restart the jenkins service from the windows task manager->services and right clicking on the jenkins service and choose stop service to stop the service as shown below. once the status of the service changes to stopped, restart the service by right clicking on the jenkins service and choose start service to start the service again. navigate to localhost:8082 to verify if the jenkins restart was successful as shown below – jenkins dashboard will be displayed. note that it takes a while before the jenkins service becomes available. 3. install plugins on the jenkins dashboard, navigate to manage jenkins –> manage plugins as shown in the snapshot below. install the following plugins and restart jenkins for the changes to take effect. git plugin (for integrating git with jenkins) gradle plugin (for integrating gradle with jenkins) android lint plugin (for integration lint with jenkins) 4. configure system on the jenkins dashboard, navigate to manage jenkins –> configure system as shown in the snapshot below. navigate to the global properties section and click on add to add an environment variable android_home as shown in the snapshot below. enter the name as android_home and enter the path of the location where the android sdk is stored on windows. navigate to the jdk section and click on “add jdk” to add the jdk installation as shown in the snapshot below. specify a jdk name, choose the jdk version to install and follow the on-screen instructions to save the oracle login credentials. save the changes. next, proceed to the git section and click on “add git” to add the git installation as shown in the snapshot below. specify git name, specify the path to git executable and save the changes. next, proceed to the gradle section and click on “add gradle” to add the gradle installation as shown in the snapshot below. specify gradle name, choose the appropriate version (at the time of writing, i used gradle 1.10) and save the changes. next, proceed to the email notification section and enter the smtp server details as shown below. click on the advanced button to add the further details required and save the changes. click on “test configuration by sending test e-mail”, enter the test e-mail recipient and click on “test configuration” to see if the email is successfully sent. 5. create a new jenkins job from the jenkins dashboard, click on “new job” to create a new job. enter a name for the job and choose “build a free-style software project” as option and click on ok as shown below. from the new job configuration screen, proceed to the source code management section. save the git credentials by clicking on “add” as shown below and entering the details in the following dialog. save the changes by clicking on “add” as shown below. specify the git repository url for the project, choose the saved credentials from the drop-down list as shown in the snapshot below. save the changes. next, from the build triggers section, select the options desired as shown below and save the changes. proceed to the build section, choose “invoke gradle script” from the drop-down list of choices for “add build step”. choose the appropriate gradle version which is configured, enter the tasks to be built and select the options as desired. save the changes. proceed to the post-build actions section, click on “publish android lint results” from the drop-down list of choices for “add post-build action” and specify the location where the lint results should be stored in the jenkins workspace for the job. similarly, click on “archive the artifacts” from the drop-down list of choices for “add post-build action” and the specify the format of apk files to be archived after every build. additionally, options from advanced section such as “discard all but the last successful/stable artifact to save disk space” could be enabled for saving disk space. click on “e-mail notification” from the drop-down list of choices for “add post-build action” and enter the values for the email recipients as shown below. save the changes. 6. build now once the above configuration steps are complete, click on “build now” under the jenkins –> build android application (or the respective job name) to build the project based on the configuration. the console output has the detailed logs of what steps were initiated by the configuration and the outcome of the entire build. clicking on any successful build outcome shows the artifacts that were archived as part of the build, the change that started the build and the lint results as shown below. thus the entire process of building the project an android application project whenever a scm change is triggered or under another condition, running lint reports, archiving the artifacts built, publishing lint reports and triggering emails to the recipients can be automated with a click of a button through jenkins.

This article was originally published in DZone's 2014 Guide to Mobile Development Mobile development has become a ubiquitous part of the software industry, and most developers understand the central dilemma organizations face when building a mobile app: cross-platform development. What options exist for deploying an app to multiple platforms simultaneously? What are the strengths and weaknesses of each platform? The backbone of mobile development is the native application, but there are a growing number of alternatives: web apps provide a browser-based solution, hybrid apps leverage web development skills in a native package, and code translators apply one platform’s native development skillset to the codebase of another. However, the differences can be subtle, and every option carries its own set of drawbacks. NATIVE DEVELOPMENT Native applications are built from the ground up for a specific platform and tailored to fit it. The precise, platform-centered nature of native development means that these apps have no limits in terms of access to APIs and device features, performance optimization, and platform-specific best practices for user interface design. Ideally, every mobile app would be built this way: to suit its exact purpose while utilizing all of the available resources. One of the major benefits of native mobile development is the availability of resources. For example, developers targeting Android have the Android Software Development Kit (SDK) at their disposal, which includes a suite of tools to streamline the development process: the SDK Manager condenses updates and tool installations into a single menu, the AVD Manager provides access to the Android Emulator and other virtual devices, and the Dalvik Debug Monitor Server (DDMS) is a powerful debugging tool, just to name a few. iOS and Windows Phone developers have similar toolsets available in their SDKs, covering everything from the UI and device feature tools of Cocoa Touch in the iOS SDK to the real world testing conditions of the Simulation Dashboard for Windows Phone 8. These toolsets make native SDKs invaluable and thorough resources. Unfortunately, the native SDKs are all robust toolsets that a native developer has to learn for each platform. To develop native apps from scratch (rather than through an intermediate tool), developers must be skilled with the required language, IDE, and development tools for each targeted platform, and if developers with diverse skillsets are not available, additional developers must be hired. This can be a serious problem, given the increasing push to develop on multiple platforms. For example, according to DZone’s 2014 Mobile Developer Survey, 62% of respondents targeted both Android and iOS. The economic constraints of native development are a major factor in the growing popularity of web apps, hybrid apps, code translators, and Mobile Application Development Platforms (MADPs), which allow developers to reach multiple platforms with just one tooling ecosystem. WEB APPS The skillset for building a basic mobile web app is more common than that of native development. Essentially, mobile web apps are just regular websites optimized to look good and function well on mobile devices, and they can provide a quality app-like experience if the developer is very skilled in web technologies. Widely understood front-end web development languages such as HTML, CSS, and JavaScript provide the logic behind a web app, and there are plenty of tools and libraries out there to help web developers direct their skills toward mobile devices. jQuery Mobile and Sencha Touch are two examples of mobile web frameworks that provide UI components and logic for sliders, swipes, and other touch-activated controls that are common to native mobile applications. The community around open source web technologies is another key difference between native and web development. Web technologies like Node.js and AngularJS are some of the most popular projects in the open source community according to GitHub statistics. This suggests that the community support and knowledge base around web technologies is broader than native technologies. In addition to being a more common skill set, mobile web development can also solve a fundamental issue with native application development. Aside from possible browser compatibility issues, web apps present a near-universal cross-platform option. Most APIs and hardware features will not be accessible by web apps, and because they are not discrete applications in the same way that native apps are, web apps cannot be distributed through common means, such as Apple’s App Store and Google’s Android Marketplace. Web apps may be a particularly flexible option, but they lack a presence on fundamental mobile distribution. HYBRID APPS Many of the drawbacks for web apps are alleviated by another cross-platform option built on the same core web development skillset: the hybrid app. Like web apps, hybrid apps require web development skills, but unlike web apps, they include some native features to allow greater flexibility. It gets the name hybrid because it is built with web languages and technologies at its core. With the help of a native packaging tool, it can be deployed just like a native app and access more native device capabilities (device APIs) than a pure web application. A hybrid app is created by first coding the application to run in the device’s native webview, which is basically a stripped-down version of the browser. For iOS this view is called UIWebView, while on Android it’s called WebView. This view can present the HTML and JavaScript files in a full-screen format, and pure web apps can achieve this full-screen view as well. WebKit is the most commonly targeted browser rendering engine because it is used on iOS, Android, and Blackberry. Where a web app really starts to become a hybrid app is when the app is placed inside of a native wrapper, which packages the hybrid app as a discrete application and makes it viable for app store distribution. In addition to the native wrapper, a native bridge allows the app to communicate with device APIs, such as alarm settings, accelerometers, and cameras. The native bridge is an abstraction layer that exposes the device APIs to the hybrid app as a JavaScript API. This is one feature that clearly separates hybrid and pure web apps, because web apps are unable to pass through the security structures between the browser and native device APIs. Access to many of the hardware features on mobile devices makes hybrid apps feel more like native apps than web apps from the user perspective. MADPS AND CODE TRANSLATORS Some tools can go even further in terms of taking a single codebase and deploying it on multiple mobile platforms. MADPs are development tools, sometimes including a mobile middleware server, that build hybrid or native apps for each platform using one codebase. Some MADPs, such as Appcelerator’s Titanium and Trigger.io, can take advantage of native elements where native is necessary or higher performing. UI widgets may be native, for instance, while a more flexible JavaScript API condenses the universal parts of mobile development and maximizes code reuse. As more native elements are introduced, some of the drawbacks of native development reappear, such as the costly need for multiple skillsets. MADPs are most useful in scenarios where an application needs to work with many back-end data sources, many other mobile apps, or many operating systems. (Inspired by Trigger.io) A less comprehensive but more straightforward solution is to use code translators when building native apps for multiple operating systems. These tools take native code and translate it into another platform’s native code, or translate native code into a neutral low-level alternative, such as bytecode. One example is Google’s J2ObjC, which translates Java classes into their Objective-C equivalents, alleviating a lot the initial development of an iOS version of the app. Although it’s much more than a code translator, a product called Xamarin does something similar by allowing developers working with C# and .NET in Visual Studio to produce a native ARM executable. They can then take advantage of ahead-of-time (AOT) or just-in-time (JIT) compilation to run their apps on iOS and Android in addition to Windows Phone. As is the case with hybrid apps, the UI presents a problem. Because UI development cannot be translated between platforms, code translators still require a significant knowledge of the native platform to write the UI. In other words, code translators can provide substantial benefits in terms of cutting down development time, but they’re not necessarily a “write once, run anywhere” solution. NO SILVER BULLETS Between native apps, web apps, hybrid apps, and the growing number of MADPs, there are a lot of options for mobile development. It’s important to note that there is no one solution that does everything. Some sacrifice affordability and accessibility for pure native performance, UI for easy cross-platform deployment, or ease of development for native authenticity. Even the simplest tools come with some degree of a learning curve. If a method with no trade-offs existed, the industry would adopt it en masse, and you would know about it. Because there are trade-offs, developers and decision-makers will have to recognize their needs, and the needs of their users, in order to determine the best way to approach mobile development. Want to read more articles like this? Download the free guide today! 2014 Guide to Mobile Development DZone's 2014 Guide to Mobile Development provides an analysis of the current state of mobile development and important strategies, tools, and insights for accelerating mobile development and includes: In-depth articles written by industry experts Survey results from over 1000 mobile developers Profiles on 39 mobile developement tools and frameworks And much more! DOWNLOAD NOW

Disclaimer: This post is about the Java micro web framework named Spark and not about the data processing engine Apache Spark. In this blog post we will see how Spark can be used to build a simple web service. As mentioned in the disclaimer, Spark is a micro web framework for Java inspired by the Ruby framework Sinatra. Spark aims for simplicity and provides only a minimal set of features. However, it provides everything needed to build a web application in a few lines of Java code. Getting Started Let's assume we have a simple domain class with a few properties and a service that provides some basic CRUDfunctionality: public class User { private String id; private String name; private String email; // getter/setter } public class UserService { // returns a list of all users public List getAllUsers() { .. } // returns a single user by id public User getUser(String id) { .. } // creates a new user public User createUser(String name, String email) { .. } // updates an existing user public User updateUser(String id, String name, String email) { .. } } We now want to expose the functionality of UserService as a RESTful API (For simplicity we will skip the hypermedia part of REST ;-)). For accessing, creating and updating user objects we want to use following URL patterns: GET /users Get a list of all users GET /users/ Get a specific user POST /users Create a new user PUT /users/ Update a user The returned data should be in JSON format. To get started with Spark we need the following Maven dependencies: com.sparkjava spark-core 2.0.0 org.slf4j slf4j-simple 1.7.7 Spark uses SLF4J for logging, so we need to a SLF4J binder to see log and error messages. In this example we use the slf4j-simple dependency for this purpose. However, you can also use Log4j or any other binder you like. Having slf4j-simple in the classpath is enough to see log output in the console. We will also use GSON for generating JSON output and JUnit to write a simple integration tests. You can find these dependencies in the complete pom.xml. Returning All Users Now it is time to create a class that is responsible for handling incoming requests. We start by implementing the GET /users request that should return a list of all users. import static spark.Spark.*; public class UserController { public UserController(final UserService userService) { get("/users", new Route() { @Override public Object handle(Request request, Response response) { // process request return userService.getAllUsers(); } }); // more routes } } Note the static import of spark.Spark.* in the first line. This gives us access to various static methods including get(), post(), put() and more. Within the constructor the get() method is used to register aRoute that listens for GET requests on /users. A Route is responsible for processing requests. Whenever aGET /users request is made, the handle() method will be called. Inside handle() we return an object that should be sent to the client (in this case a list of all users). Spark highly benefits from Java 8 Lambda expressions. Route is a functional interface (it contains only one method), so we can implement it using a Java 8 Lambda expression. Using a Lambda expression the Routedefinition from above looks like this: get("/users", (req, res) -> userService.getAllUsers()); To start the application we have to create a simple main() method. Inside main() we create an instance of our service and pass it to our newly created UserController: public class Main { public static void main(String[] args) { new UserController(new UserService()); } } If we now run main(), Spark will start an embedded Jetty server that listens on Port 4567. We can test our first route by initiating a GET http://localhost:4567/users request. In case the service returns a list with two user objects the response body might look like this: [com.mscharhag.sparkdemo.User@449c23fd, com.mscharhag.sparkdemo.User@437b26fe] Obviously this is not the response we want. Spark uses an interface called ResponseTransformer to convert objects returned by routes to an actual HTTP response. ReponseTransformer looks like this: public interface ResponseTransformer { String render(Object model) throws Exception; } ResponseTransformer has a single method that takes an object and returns a String representation of this object. The default implementation of ResponseTransformer simply calls toString() on the passed object (which creates output like shown above). Since we want to return JSON we have to create a ResponseTransformer that converts the passed objects to JSON. We use a small JsonUtil class with two static methods for this: public class JsonUtil { public static String toJson(Object object) { return new Gson().toJson(object); } public static ResponseTransformer json() { return JsonUtil::toJson; } } toJson() is an universal method that converts an object to JSON using GSON. The second method makes use of Java 8 method references to return a ResponseTransformer instance. ResponseTransformer is again a functional interface, so it can be satisfied by providing an appropriate method implementation (toJson()). So whenever we call json() we get a new ResponseTransformer that makes use of our toJson()method. In our UserController we can pass a ResponseTransformer as a third argument to Spark's get()method: import static com.mscharhag.sparkdemo.JsonUtil.*; public class UserController { public UserController(final UserService userService) { get("/users", (req, res) -> userService.getAllUsers(), json()); ... } } Note again the static import of JsonUtil.* in the first line. This gives us the option to create a newResponseTransformer by simply calling json(). Our response looks now like this: [{ "id": "1866d959-4a52-4409-afc8-4f09896f38b2", "name": "john", "email": "[email protected]" },{ "id": "90d965ad-5bdf-455d-9808-c38b72a5181a", "name": "anna", "email": "[email protected]" }] We still have a small problem. The response is returned with the wrong Content-Type. To fix this, we can register a Filter that sets the JSON Content-Type: after((req, res) -> { res.type("application/json"); }); Filter is again a functional interface and can therefore be implemented by a short Lambda expression. After a request is handled by our Route, the filter changes the Content-Type of every response toapplication/json. We can also use before() instead of after() to register a filter. Then, the Filterwould be called before the request is processed by the Route. The GET /users request should be working now :-) Returning a Specific User To return a specific user we simply create a new route in our UserController: get("/users/:id", (req, res) -> { String id = req.params(":id"); User user = userService.getUser(id); if (user != null) { return user; } res.status(400); return new ResponseError("No user with id '%s' found", id); }, json()); With req.params(":id") we can obtain the :id path parameter from the URL. We pass this parameter to our service to get the corresponding user object. We assume the service returns null if no user with the passed id is found. In this case, we change the HTTP status code to 400 (Bad Request) and return an error object. ResponseError is a small helper class we use to convert error messages and exceptions to JSON. It looks like this: public class ResponseError { private String message; public ResponseError(String message, String... args) { this.message = String.format(message, args); } public ResponseError(Exception e) { this.message = e.getMessage(); } public String getMessage() { return this.message; } } We are now able to query for a single user with a request like this: GET /users/5f45a4ff-35a7-47e8-b731-4339c84962be If an user with this id exists we will get a response that looks somehow like this: { "id": "5f45a4ff-35a7-47e8-b731-4339c84962be", "name": "john", "email": "[email protected]" } If we use an invalid user id, a ResponseError object will be created and converted to JSON. In this case the response looks like this: { "message": "No user with id 'foo' found" } Creating and Updating Users Creating and updating users is again very easy. Like returning the list of all users it is done using a single service call: post("/users", (req, res) -> userService.createUser( req.queryParams("name"), req.queryParams("email") ), json()); put("/users/:id", (req, res) -> userService.updateUser( req.params(":id"), req.queryParams("name"), req.queryParams("email") ), json()); To register a route for HTTP POST or PUT requests we simply use the static post() and put() methods of Spark. Inside a Route we can access HTTP POST parameters using req.queryParams(). For simplicity reasons (and to show another Spark feature) we do not do any validation inside the routes. Instead we assume that the service will throw an IllegalArgumentException if we pass in invalid values. Spark gives us the option to register ExceptionHandlers. An ExceptionHandler will be called if anException is thrown while processing a route. ExceptionHandler is another single method interface we can implement using a Java 8 Lambda expression: exception(IllegalArgumentException.class, (e, req, res) -> { res.status(400); res.body(toJson(new ResponseError(e))); }); Here we create an ExceptionHandler that is called if an IllegalArgumentException is thrown. The caught Exception object is passed as the first parameter. We set the response code to 400 and add an error message to the response body. If the service throws an IllegalArgumentException when the email parameter is empty, we might get a response like this: { "message": "Parameter 'email' cannot be empty" } The complete source the controller can be found here. Testing Because of Spark's simple nature it is very easy to write integration tests for our sample application. Let's start with this basic JUnit test setup: public class UserControllerIntegrationTest { @BeforeClass public static void beforeClass() { Main.main(null); } @AfterClass public static void afterClass() { Spark.stop(); } ... } In beforeClass() we start our application by simply running the main() method. After all tests finished we call Spark.stop(). This stops the embedded server that runs our application. After that we can send HTTP requests within test methods and validate that our application returns the correct response. A simple test that sends a request to create a new user can look like this: @Test public void aNewUserShouldBeCreated() { TestResponse res = request("POST", "/users?name=john&[email protected]"); Map json = res.json(); assertEquals(200, res.status); assertEquals("john", json.get("name")); assertEquals("[email protected]", json.get("email")); assertNotNull(json.get("id")); } request() and TestResponse are two small self made test utilities. request() sends a HTTP request to the passed URL and returns a TestResponse instance. TestResponse is just a small wrapper around some HTTP response data. The source of request() and TestResponse is included in the complete test classfound on GitHub. Conclusion Compared to other web frameworks Spark provides only a small amount of features. However, it is so simple you can build small web applications within a few minutes (even if you have not used Spark before). If you want to look into Spark you should clearly use Java 8, which reduces the amount of code you have to write a lot. You can find the complete source of the sample project on GitHub.

okay, this is the separate blog post that i referred to in software architecture vs code . what exactly do we mean by an "architecturally-evident coding style"? i built a simple content aggregator for the local tech community here in jersey called techtribes.je , which is basically made up of a web server, a couple of databases and a standalone java application that is responsible for actually aggegrating the content displayed on the website. you can read a little more about the software architecture at techtribes.je - containers . the following diagram is a zoom-in of the standalone content updater application, showing how it's been decomposed. this diagram says that the content updater application is made up of a number of core components (which are shown on a separate diagram for brevity) and an additional four components - a scheduled content updater, a twitter connector, a github connector and a news feed connector. this diagram shows a really nice, simple architecture view of how my standalone content updater application has been decomposed into a small number of components. "component" is a hugely overloaded term in the software development industry, but essentially all i'm referring to is a collection of related behaviour sitting behind a nice clean interface. back to the "architecturally-evident coding style" and the basic premise is that the code should reflect the architecture. in other words, if i look at the code, i should be able to clearly identify each of the components that i've shown on the diagram. since the code for techtribes.je is open source and on github, you can go and take a look for yourself as to whether this is the case. and it is ... there's a je.techtribes.component package that contains sub-packages for each of the components shown on the diagram. from a technical perspective, each of these are simply spring beans with a public interface and a package-protected implementation. that's it; the code reflects the architecture as illustrated on the diagram. so what about those core components then? well, here's a diagram showing those. again, this diagram shows a nice simple decomposition of the core of my techtribes.je system into coarse-grained components. and again, browsing the source code will reveal the same one-to-one mapping between boxes on the diagram and packages in the code. this requires conscious effort to do but i like the simple and explicit nature of the relationship between the architecture and the code. when architecture and code don't match the interesting part of this story is that while i'd always viewed my system as a collection of "components", the code didn't actually look like that. to take an example, there's a tweet component on the core components diagram, which basically provides crud access to tweets in a mongodb database. the diagram suggests that it's a single black box component, but my initial implementation was very different. the following diagram illustrates why. my initial implementation of the tweet component looked like the picture on the left - i'd taken a "package by layer" approach and broken my tweet component down into a separate service and data access object. this is your stereotypical layered architecture that many (most?) books and tutorials present as a way to build (e.g.) web applications. it's also pretty much how i've built most software in the past too and i'm sure you've seen the same, especially in systems that use a dependency injection framework where we create a bunch of things in layers and wire them all together. layered architectures have a number of benefits but they aren't a silver bullet . this is a great example of where the code doesn't quite reflect the architecture - the tweet component is a single box on an architecture diagram but implemented as a collection of classes across a layered architecture when you look at the code. imagine having a large, complex codebase where the architecture diagrams tell a different story from the code. the easy way to fix this is to simply redraw the core components diagram to show that it's really a layered architecture made up of services collaborating with data access objects. the result is a much more complex diagram but it also feels like that diagram is starting to show too much detail. the other option is to change the code to match my architectural vision. and that's what i did. i reorganised the code to be packaged by component rather than packaged by layer. in essence, i merged the services and data access objects together into a single package so that i was left with a public interface and a package protected implementation. here's the tweet component on github . but what about... again, there's a clean simple mapping from the diagram into the code and the code cleanly reflects the architecture. it does raise a number of interesting questions though. why aren't you using a layered architecture? where did the tweetdao interface go? how do you mock out your dao implementation to do unit testing? what happens if i want to call the dao directly? what happens if you want to change the way that you store tweets? layers are now an implementation detail this is still a layered architecture, it's just that the layers are now a component implementation detail rather than being first-class architectural building blocks. and that's nice, because i can think about my components as being my architecturally significant structural elements and it's these building blocks that are defined in my dependency injection framework. something i often see in layered architectures is code bypassing a services layer to directly access a dao or repository. these sort of shortcuts are exactly why layered architectures often become corrupted and turn into big balls of mud. in my codebase, if any consumer wants access to tweets, they are forced to use the tweet component in its entirety because the dao is an internal implementation detail. and because i have layers inside my component, i can still switch out my tweet data storage from mongodb to something else. that change is still isolated. component testing vs unit testing ah, unit testing. bundling up my tweet service and dao into a single component makes the resulting tweet component harder to unit test because everything is package protected. sure, it's not impossible to provide a mock implementation of the mongodbtweetdao but i need to jump through some hoops. the other approach is to simply not do unit testing and instead test my tweet component through its public interface. dhh recently published a blog post called test-induced design damage and i agree with the overall message; perhaps we are breaking up our systems unnecessarily just in order to unit test them. there's very little to be gained from unit testing the various sub-parts of my tweet component in isolation, so in this case i've opted to do automated component testing instead where i test the component as a black-box through its component interface. mongodb is lightweight and fast, with the resulting component tests running acceptably quick for me, even on my ageing macbook air. i'm not saying that you should never unit test code in isolation, and indeed there are some situations where component testing isn't feasible. for example, if you're using asynchronous and/or third party services, you probably do want to ability to provide a mock implementation for unit testing. the point is that we shouldn't blindly create designs where everything can be mocked out and unit tested in isolation. food for thought the purpose of this blog post was to provide some more detail around how to ensure that code reflects architecture and to illustrate an approach to do this. i like the structure imposed by forcing my codebase to reflect the architecture. it requires some discipline and thinking about how to neatly carve-up the responsibilities across the codebase, but i think the effort is rewarded. it's also a nice stepping stone towards micro-services. my techtribes.je system is constructed from a number of in-process components that i treat as my architectural building blocks. the thinking behind creating a micro-services architecture is essentially the same, albeit the components (services) are running out-of-process. this isn't a silver bullet by any means, but i hope it's provided some food for thought around designing software and structuring a codebase with an architecturally-evident coding style.

When I am trying to install a third party apk using the ADB tool, I have faced "Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]" error. To resolve the issue, I have followed these few steps. Open command prompt; Go to your debug.keystore location. For eg: You can find the debug.keystore file in the following location C:\Documents and Settings\User\.android 1. Using Zip align copied apk. zipalign -v 4 D:\Test.apk D:\Testc.apk 2. keytool -genkey -v -keystore debug.keystore -alias sampleName -keyalg RSA -keysize 2048 -validity 20000 Now a prompt will ask for Password First and lastname Name of Organization unit Name of Organization City State Country After entering these fields we get our Certificate 3. jarsigner -verbose -keystore debug.keystore D:\Testc.apk sampleName In some cases we need add -sigalg SHA1withRSA -digestalg SHA1 arguments to work out the step 3 jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore debug.keystore D:\Testc.apk sampleName Now it will ask for the password and then it will replace the apk with the signed one. To check whether it is working or not, you can check using the following command. jarsigner -verify D:\Testc.apk Then I have installed apk using ADB. Adb install D:\Testc.apk Thanks for reading :) Origin: Vardhan Blog - "install parse failed no certificates"

Explore different message brokers, and discover how these important web technologies impact a customer's backlog of messages, and cluster/data performance.