I've been working as a Scrum Master and as an Agile Coach for a good few years now, mainly as a contractor. Each time I am interviewed for a new contract I always like to ask if I can meet the teams I’d be working with. You see, time and again it will be a manager who does the interviewing, while the team members themselves are left with little say in whether I should be hired. I think it’s important that they reckon they can get along with me. Of course, it also gives me an opportunity to see them, and to gain a fuller understanding of the situation I’d really be walking into. As we head towards the desks of my prospective team, one of the first things I look for is the board, whether it be a Scrum task board or a Kanban board. Most teams with agile aspirations…or agile pretensions…will have set up a board of some kind. A board is the "grand old dame" of information radiators. No matter how much the details of a sordid past are glossed over, the truth always seems to come out. It's in the nature of a board to tell the truth, since any untruths can be quickly exposed. The story I can piece together from dubious lanes and columns, misplaced or missing tickets, misplaced or missing avatars, and a host of other shibboleths can be far more telling than anything I get to hear from people in an interview situation. Another of the things I look for is a "fast track" lane on a Scrum Team's board. These are very common; you could say it is almost unusual not to see them. From a certain perspective they are good things to have, and they can imply a level of maturity - or at least of pragmatism - on the part of a team. They suggest that the team accepts that not everything can be predicted in Sprint planning. A fast track lane is a nod to the fact that emergencies happen, that support work and unforeseen defect fixes still need to be done, and most importantly, that the team has a way of dealing with all of this. However it also shows that they aren't doing Scrum. There...I've said it. Fast track lanes aren't part of Scrum. It's that simple. I don't mean to say that they are bad practice, or in some sense un-agile. On the contrary, they are part of the Lean Kanban approach to varying the Quality of Service provided to certain backlog items. That's what a fast track lane is...a way of varying the quality of service that a Scrum team gives to certain items. When something hits a fast track lane, a well-trained team will swarm over it and decide who is best qualified to progress the matter. While they do this, their own tasks will be marked as impeded or blocked. Then, the decision made, all others return to their work in progress. So if fast track lanes are a widely understood and practical way of managing operational issues, what is wrong with them, Scrum-wise? The answer is that Scrum - unlike Lean Kanban - doesn't provide for variations in quality of service. Each piece of work is prioritized and negotiated into a Sprint backlog. The team then self-organizes to deliver a corresponding increment of functionality. The team will plan with the Product Owner what it intends to do during a sprint, and the sprint backlog they agree to belongs to them. No-one, not even the CEO of the organization, can override their sprint backlog by introducing work to be "fast tracked". The team wholly owns their sprint backlog. That's Scrum. When I point this out, teams can become crestfallen or even defensive. “What else are we supposed to do”, they say. “We aren’t dedicated 100% to doing project work. We still have support work to do, and serious issues always trump development. We have to fix them and put project work on hold.” My answer to that is that under the circumstances the team is facing, it may indeed be right to vary the quality of service by fast-tracking support work. It just isn’t Scrum, that’s all. It’s a type of "Scrumban", a Scrum variant that includes Kanban characteristics. This is no fault of the team, but it could suggest a problem higher up. Perhaps a dedicated Kanban support team hasn’t been properly resourced and trained so that Scrum development can proceed unimpeded. Perhaps the Product Owner is being undermined by other managers who have separate interests impacting the development. Whatever the situation, it needs to be made transparent and acknowledged by all stakeholders. So, the next step…and the one I’ll often indicate as the interview progresses…is to account for fast track work as impediments against product burndown or velocity. Moreover, these are impediments which are external to the team. It’s essentially a type of waste, or unplanned work, being generated from outside. It needs to be made quite transparent where this waste is coming from and what can be done to mitigate it. What can be done about those other teams, or workflows, or managers, who are undercutting this Scrum team’s ability to plan out their Sprints? Often, the source of these impediments will be the people interviewing me...and that’s when things can start to get really interesting!

Generally, you pick up a subset of components in some integration tests to check if they are glued as expected. To achieve this, they are usually really invoked, but sometimes, it is too expensive to do so. For example, Component A invokes Component B, and Component B has a dependency on an external system which does not have a test server. We really want to verify the configurations, it seems the only way is replacing Component B with test double after wiring Component A and B. Let's start with Strategy A: Manual Injecting @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = "classpath:config.xml") public class SomeAppIntegrationTestsUsingManualReplacing { private Mockery context = new JUnit4Mockery(); （1） private SomeInterface mock = context.mock(SomeInterface.class); （2） @Resource(name = "someApp") private SomeApp someApp; （3） @Before public void replaceDependenceWithMock() { someApp.setDependence(mock); （4） } @DirtiesContext @Test public void returnsHelloWorldIfDependenceIsAvailable() throws Exception { context.checking(new Expectations() { { allowing(mock).isAvailable(); will(returnValue(true)); （5） } }); String actual = someApp.returnHelloWorld(); assertEquals("helloWorld", actual); context.assertIsSatisfied(); （6） } } We get a spring bean someApp(Component A in this case), and it has a denpendence on SomeInterface's(Component B in this case). We inject mock (declare and init at step 4) to someApp, thus the test passes without sending request to the external system. The context.assertIsSatisfied()(at step 6 ) is very important as we use SpringJUnit4ClassRunner as junit runner instead of JMock, so you have to explictly assert that all expectations are satisfied. There are two downsides of the previous strategy: Firstly, if there are more than one mock, you have to inject them one by one, which is very tedious especially when you need to inject mocks into serveral spring bean. Secondly, the wiring is not tested. For example, if I forget to write the integration tests using manual inject strategy is not going to tell. Strategy B: Using predefined BeanPostProcessor Spring provides BeanPostProcessor which is very useful when you want to replace some bean after the wiring is done. According to the reference, application context will auto detect all BeanPostProcessor registered in metadata(usually in xml format). public class PredefinedBeanPostProcessor implements BeanPostProcessor { public Mockery context = new JUnit4Mockery(); （1） public SomeInterface mock = context.mock(SomeInterface.class); （2） @Override public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException { return bean; } @Override public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException { if ("dependence".equals(beanName)) { return mock; } else { return bean; } } } @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(locations = { "classpath:config.xml", "classpath:predefined.xml" }) （1） public class SomeAppIntegrationTestsUsingPredefinedReplacing { @Resource(name = "someApp") private SomeApp someApp; @Resource(name = "predefined") private PredefinedBeanPostProcessor fixture; @Test public void returnsHelloWorldIfDependenceIsAvailable() throws Exception { fixture.context.checking(new Expectations() { { allowing(fixture.mock).isAvailable(); will(returnValue(true)); } }); String actual = someApp.returnHelloWorld(); assertEquals("helloWorld", actual); fixture.context.assertIsSatisfied(); } } Notice there is an extra config xml in which the PredefinedBeanPostProcessor is registered(at step 1). The predefined.xml is placed in src/test/resources/, so it will not be packed into the artifact for production. For each test, using Strategy B requires inputting both a java file and a xml which is quite verbose. Now we have learned the pros and cons of Strategy A and Strategy B. What about a hybrid version -- killing two birds with one stone. Therefore we have the next strategy. Strategy C：Dynamic Injecting public class TestDoubleInjector implements BeanPostProcessor { private static Map MOCKS = new HashMap(); （1） @Override public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException { return bean; } @Override public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException { if (MOCKS.containsKey(beanName)) { return MOCKS.get(beanName); } return bean; } public void addMock(String beanName, Object mock) { MOCKS.put(beanName, mock); } public void clear() { MOCKS.clear(); } } @RunWith(JMock.class) public class SomeAppIntegrationTestsUsingDynamicReplacing { private Mockery context = new JUnit4Mockery(); private SomeInterface mock = context.mock(SomeInterface.class); private SomeApp someApp; private ConfigurableApplicationContext applicationContext; private TestDoubleInjector fixture = new TestDoubleInjector(); （1） @Before public void replaceDependenceWithMock() { fixture.addMock("dependence", mock); （2） applicationContext = new ClassPathXmlApplicationContext(new String[] { "classpath:config.xml", "classpath:dynamic.xml" }); （3） someApp = (SomeApp) applicationContext.getBean("someApp"); } @Test public void returnsHelloWorldIfDependenceIsAvailable() throws Exception { context.checking(new Expectations() { { allowing(mock).isAvailable(); will(returnValue(true)); } }); String actual = someApp.returnHelloWorld(); assertEquals("helloWorld", actual); } @After public void clean() { applicationContext.close(); fixture.clear(); } } The TestDoubleInjector class is an implementation of Monostate pattern. Mocks are added to the static map before the application context being created. When another TestDoubleInjector instance (defined in dynamic.xml) is initiated, it can share the static map for replacement. Just beware to clear the static map after tests. By the way, you could use Stub instead of Mocks with same strategies. Please do not hesitate to contact me if you might have any questions. And I do appreciate it, if you could let me know you have a better idea. Thanks! Resources： http://www.jmock.org http://www.oracle.com/technetwork/articles/entarch/spring-aop-with-ejb5-093994.html(I saw BeanPostProcessor the first time in this post)

Amazon Virtual Private Cloud (VPC) is a great way to setup an isolated portion of AWS and control the network topology. It is a great way to extend your data center and use AWS for burst requirements. With the latest VPC for Everyone announcement, what was earlier "Classic" and "VPC" in AWS will soon be only VPC. That is, every deployment in AWS will be on a VPC even though one might not need all the additional features that VPC provides. One might eventually start looking at utilizing VPC features such as multiple Subnets, Network isolation, Network ACLs, etc.. Those who have already worked with VPC's understand the role of NAT Instance in a VPC. When you create a VPC, you create them with multiple Subnets (Public and Private). Instances launched in the Public Subnet have direct internet connectivity to send and receive internet traffic through the internet gateway of the VPC. Typically, internet facing servers such as web servers are kept in the Public Subnet. A Private Subnet can be used to launch Instances that do not require direct access from the internet. Instances in a Private Subnet can access the Internet without exposing their private IP address by routing their traffic through a Network Address Translation (NAT) instance in the Public Subnet. AWS provides an AMI that can be launched as a NAT Instance. Following diagram is the representation of a standard VPC that gets provisioned through the AWS Management Console wizard. Standard Private and Public Subnets in a VPC The above architecture has A Public Subnet that has direct internet connectivity through the Internet Gateway. Web Instances can be placed within the Public Subnet The custom Route Table associated with Public Subnet will have the necessary routing information to route traffic to the Internet Gateway A NAT Instance is also provisioned in the Public Subnet A Private Subnet that has outbound internet connectivity through the NAT Instance in the Public Subnet The Main Route Table is by default associated with the Private Subnet. This will have necessary routing information to route internet traffic to the NAT Instance Instances in the Private Subnet will use the NAT Instance for outbound internet connectivity. For example, DB backups from standby that needs to be stored in S3. Background programs that make external web services calls Of course, the above architecture has limited High Availability since all the Subnets are created within the same Availability Zone. We can avoid this by creating multiple Subnets in multiple Availability Zones. Public and Private Subnets with multiple Availability Zones Additional Subnets (Public and Private) are created in one another Availability Zone Both Private Subnets are attached to the Main Routing Table Both Public Subnets are attached to the same Custom Routing Table Instances in the Private Subnet still continue to use the NAT Instance for outbound internet connectivity Though we increased the High Availability by utilizing multiple Availability Zones, the NAT Instance is still a Single Point of Failure. NAT Instance is just another EC2 Instance that can become unavailable any time. The updated architecture below uses two NAT Instances to provide failover and High Availability for the NAT Instances NAT Instance High Availability Each Subnet is associated with its own Route Table NAT1 is provisioned in Public Subnet 1 NAT2 is provisioned in Public Subnet 2 Private Subnet 1's Route Table (RT) has routing entry to NAT1 for internet traffic Private Subnet 2's Route Table (RT) has routing entry to NAT2 for internet traffic NAT Instance HA Illustration A script can be installed on both the NAT Instances to monitor each other and swap the routing table association if one of them fails. For example, if NAT1 detects that NAT2 is not responding to its ping requests, it can change the Route Table of Private Subnet 2 to NAT1 for internet traffic. Once NAT2 becomes operational again, a reverse swapping can happen. AWS has a pretty good documentation on this and a sample script for the swapping. Apart from HA, the above architecture also provides better overall throughput, since during normal conditions, both NAT Instances can be used to drive the outbound internet requirements of the VPC. If there are workloads that requires a lot of outbound internet connectivity, having more than one NAT Instance would make sense. Of course, you are still limited with one NAT Instance per Subnet.

If you are using Amazon Web Services(AWS), you are probably aware how to access and use resources like SNS, SQS, S3 using key and secret. With the aws-java-sdk that is straight forward: AmazonSNSClient snsClient = new AmazonSNSClient( new BasicAWSCredentials("your key", "your secret")) One of the difficulties with this approach is storing the key/secret securely especially when there are different set of these for different environments. Using java property files, combined with maven or spring profiles might help a little bit to externalize the key/secret out of your source code, but still doesn't solve the issue of securely accessing these resources. Amazon has another service to help you in this occasion. No, no, this is not one more service to pay for in order to use the previous services. It is a free service, actually it is a feature of the amazon account. AWS Identity and Access Management (IAM) lets you securely control access to AWS services and resources for your users, you can manage users and groups and define permissions for AWS resources. One interesting functionality of IAM is the ability to assign roles to EC2 instances. The idea is you create roles with sets of permissions and you launch an EC2 instance by assigning the role to the instance. And when you deploy an application on that instance, the application doesn't need to have access key and secret in order to access other amazon resource. The application will use the role credentials to sign the requests. This has a number of benefits like a centralized place to control all the instances credentials, reduced risk with auto refreshing credentials and so on. Here is a short video demonstrating how to assign roles to an EC2 instance: Once you have role based security enabled for an instance, to access other resources from that instances you have to create and AwsClient using the chained credential provider: AmazonSNSClient snsClient = new AmazonSNSClient( new DefaultAWSCredentialsProviderChain()) The provider will search your system properties, environment properties and finally call instance metadata API to retrieve the role credentials in chain of responsibility fashion. It will also refresh the credentials in the background periodically depending on its expiration period. And finally, if you want to use role based security from Camel applications running on Amazon, all you have to do is create an instance of the client with configured chained credentials object and don't specify any key or secret: from("direct:start") .to("aws-sns://MyTopic?amazonSNSClient=#snsClient");

What would you expect from someone who is OFBiz and Camel committer? To integrate them for fun? Fine, here it is. In addition to being fun, I believe this integration will be of real benefit for the OFBiz community, because despite the fact of being a complete ERP software, OFBiz lacks the ability to easily integrate with external systems. The goal of this project is instead of reinventing the wheel and trying to integrate OFBiz with each system separately, integrate it with Camel and let Camel do what it does best: connect your application with every possible protocol and system out there. Quick OFBiz introduction The Apache Open For Business Project is an open source, enterprise automation software. It consist mainly from two parts: A full-stack framework for rapid business application development. It has Entity Engine for the data layer (imagine something like iBATIS and Hibernate combined). It is the same entity engine that powers millions of Attlasian Jira instances. But don't get me wrong, it is not meant for usage outside of OFBiz framework, so use it only as OFBiz data layer. Service Engine - this might be hard to grasp for someone only with DDD background, but OFBiz doesn't have any domain objects. Instead for the service layer it uses SOA and has thousands of services that contains the business logic. A service is an atomic bit of isolated business logic, usually reading and updating the database. If you need you can make services triggering each other using ECAs(event-condition-action) which is kind of rule engine that allows define pre/post conditions for triggering other service calls when a service is executed. The service itself can be written written in java, groovy or simple language (an XML DSL for simple database manipulation) and usually requires authentication, authorisation and finally executed in a transaction. UI widgets - an XML DSL which allows you easily create complex pages with tables, forms and trees. And the really great thing about this framework is that 'The whole is greater than the sum of its parts' - all of the above layers works together amazingly: if you have an entity definition (a table) in your data layer, you can use it in your service layer during your service interface definition or its implementation. It takes one line of code(a long one) to create a service which has as input parameters the table columns and return the primary key as result of the service. Then if you are creating a screen with tables or forms, you can base it on your entity definitions or service definitions. It is again only few lines of code to create a form with fields mapping to a service or entity fields. Out of the box business applications. These are vertical applications for managing the full life cycle of a business domain like ordering, accounting, manufacturing and many more in a horizontally integrated manner. So creating an order from order or ecommerce application will interact with facility to check whether a product is available, and after the order is created will create accounting transaction in accounting application. Before the order is shipped from the facility, it will create invoices and once the invoice is paid, it will close the order. You get the idea. Camel in 30 seconds Apache Camel is an integration framework based on known Enterprise Integration Patterns(EIP). Camel can also be presented as consisting of two artifacts: The routing framework which can be defined using java, scala, xml DSL with all the EIPs like Pipe, Filter, Router, Splitter, Aggregator, Throttler, Normalizer and many more. Components and transformers ie all the different connectors to more than 100 different applications and protocols like: AMQP, AWS, web services, REST, MongoDB, Twitter, Websocket, you name it. If you can imagine a tool, that enables you to consume data from one system, then manipulate the data (transform, filter, split, aggregate) and send it to other systems, using a declarative, concise, English-like DSL without any boilerplate code - that's Apache Camel. Let OFBiz talk to all of the systems Camel do The main interaction point with OFBiz are either by using the Entity Engine for direct data manipulation or by calling services through Service Engine. The latter is preferred because it ensures that the user executing the service is authorised to do so, the operation is transactional to ensure data integrity, and also all the business rules are satisfied (there might be other services that have to be executed with ECA rules). So if we can create an OFBiz endpoint in Camel and execute OFBiz services from Camel messages, that would allow OFBiz to receive notifications from Camel endpoints. What about the other way around - making OFBiz notify Camel endpoints? The ideal way would be to have an OFBiz service that sends the IN parameters to Camel endpoints as message body and headers and return the reply message as OFBiz service response. If you are wondering: why is it so great, what is an endpoint, where is the real world, who is gonna win Euro2012... have a look at the complete list of available Camel components, and you will find out the answer. Running Camel in OFBiz container I've started an experimental ofbiz-camel project on github which allows you to do all of the above. It demonstrates how to poll files from a directory using Camel and create notes in OFBiz with the content of the file using createNote service. The project also has an OFBiz service, that enables sending messages from OFBiz to Camel. For example using that service it is possible to send a message to Camel file://data endpoint, and Camel will create a file in the data folder using the service parameters. The integration between OFBiz and Camel is achieved by running Camel in an OFBiz container as part of the OFBiz framework. This makes quite tight integration, but ensures that there will not be any http, rmi or any other overhead in between. It is still WIP and may change totally. Running Camel and OFBiz separately Another approach is KISS: run Camel and OFBiz as they are - separate applications, and let them interact with RMI, WS* or something else. This doesn't require any much coding, but only configuring both systems to talk to each other. I've created a simple demo camel-ofbiz-rmi which demonstrates how to listen for tweets with a specific keyword and store them in OFBiz as notes by calling createNote service using RMI. It uses Camel's twitter and rmi components and requires only configuration. Notice that this example demonstrates only one way interaction: from Camel to OFBiz. In order to invoke a Camel endpoint from OFBiz you can you have to write some RMI, WS* or other code. PS: I'm looking forward to hear your real world integration requirements for OFBiz.

If you are using Visual Studio plugin for Git, but you have also configured Git with MSys git, probably you could be surprised by some Visual Studio behavior.

the most important part of building an integration with an api is actually writing the code that will connect with the web service and invoke its methods. i'll show you why using the façade pattern to decouple calls from your existing code is a good idea and help you identify what kind of problems you might be able to prevent. so, first things first, what is the façade pattern? a façade is an object that provides simple access to complex - or external - functionality. it might be used to group together several methods into a single one, to abstract a very complex method into several simple calls or, more generically, to decouple two pieces of code where there's a strong dependency of one over the other. what happens when you develop api calls inside your code and, suddenly, the api is upgraded and some of its methods or parameters change? you'll have to change your application code to handle those changes. also, by changing your internal application code, you might have to change the way some of your objects behave. it is easy to overlook every instance and can require you to doublecheck multiple lines of code. there's a better way to keep api calls up-to-date. by writing a façade with the single responsibility of interacting with the external web service, you can defend your code from external changes. now, whenever the api changes, all you have to do is update your façade. your internal application code will remain untouched. from a test driven development point-of-view, using a facade offers a big advantage. you're now able to write simple tests against the façade without affecting your internal code test results. by using this strategy, you'll be able to know immediately whenever an api is not working as you expected and make the necessary changes to the façade. this is the approach we follow at cloudwork when building integrations between any third-party apis. api façades act as tight compartments that protect the rest of the application from external changes and simplify the way we interact with different web services. this article is cross-posted at using facades to decouple api integrations .

Introduction The IT buzzword of 2012 is without a doubt Big Data. It’s new and here to stay, and for a good reason. Big data is data that exceeds the processing capacity of conventional database systems. Great examples are CERN with the Large Hadron Collider, whose experiments generate 25 petabytes of data annually, or Walmart, which handles more than one million customer transaction every hour. Problems These vast amounts of data leave us with two problems. Problem 1: To gain value from this data, one must choose an alternative way to process it. The value of big data to an organization falls into two categories: analytical use, and enabling new products. Big data analytics can reveal insights hidden previously by data too costly to process, such as peer influence among customers, revealed by analyzing shoppers’ transactions, social and geographical data. Being able to process every item of data in reasonable time removes the troublesome need for sampling and promotes an investigative approach to data, in contrast to the somewhat static nature of running predetermined reports. Problem 2: The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. Remember the CERN case where the LHC produces over 25 Petabytes of data annually? No “classic” database architecture or setup is capable of holding these amounts of data. Solutions Fortunately, both problems can be solved by implementing the correct infrastructure and rethinking data storage. There are two critical factors in Big Data environments: size and speed. We already discussed the vast amounts of data and desire to be able to access and process the data fast. The latter is the main differentiator from more traditional data warehouses. Just imagine what you can do when you can access all your data real-time. Enter big data. A common Big Data implementation is an in-memory data grid that lives in a distributed cluster, ensuring both speed, by storing data in-memory, and capacity by using scalability features provided by a cluster. As a bonus, availability is ensured by using a distributed cluster. As for the data storage, there are typically two kinds: in-memory databases and in-memory data grids. But first some background. It is not a new attempt to use main memory as a storage area instead of a disk. In our daily lives there are numerous examples of main memory databases (MMDB), as they perform much faster than disk-based databases. An every day example is a mobile phone. When you SMS or call someone most mobile service providers use MMDB to get the information on your contact as soon as possible. The same applies to your phone. When someone calls you, the caller details are looked up in the contacts application, usually providing a name and sometimes a picture. In memory data grids In Memory Data Grid (IMDG) is the same as MMDB in that it stores data in main memory, but it has a totally different architecture. The features of IMDG can be summarized as follows: Data is distributed and stored on multiple servers. Each server operates in the active mode. A data model is usually object-oriented (serialized) and non-relational. According to the necessity, you often need to add or reduce servers. No traditional database features such as tables. In other words, IMDG is designed to store data in main memory, ensure scalability and store an object itself. These days, there are many IMDG products, both commercial and open source. Some of the most commonly used products are: Hazelcast (http://www.hazelcast.com) JBoss Infinispan (http://www.jboss.org/infinispan) GridGain DataGrid (http://www.gridgain.com/features/in-memory-data-grid/) VMware Gemfire (http://www.vmware.com/nl/products/application-platform/vfabric-gemfire/overview.html) Oracle Coherence (http://www.oracle.com/technetwork/middleware/coherence/overview/index.html) Gigaspaces XAP (http://www.gigaspaces.com/datagrid) Terracotta Enterprise Suite (http://terracotta.org/products/enterprise-suite) Why Memory? The main reasons for using main memory for data storage are once again the two main themes of Big Data: speed and capacity. The processing performance of main memory is 800 times faster than an HDD and up to 40 times faster than an. Moreover, the latest x86 server supports main memory of hundreds of GB per server. It is said that the limit of a traditional processing database’s (OLTP) data capacity is approximately 1 TB and that the OLTP processing data capacity would not increment well. If servers using main memory of 1 TB or larger become more commonly used, you will be able to conduct operations with the entire data placed in main memory, at least in the field of OLTP. IMDG Architecture To use main memory as a storage area, two weak points should be overcome: Limited capacity: involves data that exceeds the maximum capacity of the main memory of the server Reliability: involves data loss in case of a (system) failure. IMDG overcomes the limit of capacity by ensuring horizontal scalability using a distributed architecture, and resolves the issue of reliability through a replication system as part of the grid (or a distributed cluster). Now let’s discuss how an IMDG actually works. First of all, it is important to understand that an IMDG is not the same as an in-memory database, also referred to as MMDB (main memory databases). Typical examples of MMDBs are Oracle TimesTen or Sap Hana. MMDBs are full database products that simply reside in memory. As a result of being a full-blown database, they also carry the weight and overhead of database management features. IMDG is different. No tables, indexes, triggers, stored procedures, process managers etc. Just plain storage. The data model used in IMDG is key-value pairs. A key-value pair is a list with only two parts: a key and a value. The key can be used for storing and retrieving the values in the list. A key can be compared to the index or primary key of a table in a database. Note that IMDG are closely tied to development environments such as Java as the key-value pairs are represented by the structures provided by such a programming environment. Most IMDGs are written in Java, and can only be used within other Java applications. Therefore, the values of key-value pairs can be anything supported by Java, ranging from simple data types such as a string or number, to complex objects. This overcomes the two important hurdles: as you can store complex Java objects as value, there’s no need to translate these objects into a relational datamodel (which is the case in more traditional applications using a database for storage). Furthermore, the seeming limitation of being able to store only one value per key, is actually no limitation at all. Large memory sizes Most of the products introduced above use Java as an implementation language. Java reserves and uses a part of the RAM (internal memory) for dynamic memory allocation. This reserved memory space is called the Java heap. All runtime objects created by a Java application are stored in heap. Using large amounts of data causes two problems. Size limitation: By default, the heap size is 128 MB, but for current business applications, this limit is reached easily. Once the heap is “full”, no new objects can be created and the Java application will show some nasty errors. Performance: It is possible to increase the size of the heap, but this introduces some new problems. When a heap reaches a size of more than 4 gigabytes, Java will have serious issues with memory managements, causing your application to slow down or even freeze. Java has a feature called Garbage Collector, which periodically scans the heap and checks each object if it is still valid and being used. If not, the garbage collector removes the object and defragments the newly available space. The problem is, the larger the heap size, the more work to do for the garbage collector, resulting in performance degradation. Imagine a large bank has a Java application that manages customers, accounts and transactions. We have seen that an IMDG allows the application to store and access all data very quickly by caching it in memory, instead of storing the data in relatively slow databases. Let’s assume the combined data has a size of 40 gigabytes. Storing it in heap is simply not possible, considering the performance penalties of Java’s memory management capabilities. The graph below illustrates the garbage collection pause time when placing cached data in heap: Terracotta’s BigMemory product has a method to overcome these limitations. The method is to use an off-heap memory (direct buffer). Data will not be stored in Java’s heap, but directly in the available internal memory (RAM). Since, this is not subject to Java’s garbage collector, there are no performance penalties. The differences on performance are significant, as can be seen in the graph below: Using off-heap storage has some major benefits: You can use all the available memory on your machine, not just the memory that is allocated to the heap (usually less that 512 Mb). This allows you to store more data in a in-memory data grid, greatly speeding up your application. The heap can be relieved by storing data in native memory, speeding up Java applications as less heap space has to be garbage collected. Clustering, fail over and high availability So far, we have seen IMDG features that are applicable to a single server. However, the real power of IMDG lies in it’s networking and clustering capabilities, providing features as data replication, data synchronization between clients, fail over and high availability. To achieve this, a cluster of servers (or server array) acts a backbone of the infrastructure. Applications (that still can have their own IMDG or off-heap cache) that are connected to the cluster can share, replicate and backup their data with either the cluster or other applications. The graph below depicts a typical setup using Terracotta's BigMemory: The caches on the application servers are usually referred to as “level 1” cache, while the data cache on the server array is referred to as “level 2” cache. There are many different scenarios possible for storing, clustering, synchronizing and replicating data. Covering all these topics goes far beyond the scope of this article. For more information, consult the technical documentation of the product of your choice. Conclusion Big Data brings us some new challenges. First of all, storing and accessing vast amounts of data makes us rethink traditional methods and technologies. Next, there’s the question what to do with all the available data. The potential value for marketing, financial and other businesses is huge. In order to facilitate Big Data, in-memory data grids are considered the best option. IMDGs with off-heap storage are even more powerful, allowing data centric enterprise application to overcome certain limits of the Java platform, such as memory and performance constraints. As the amount of data that (large) companies produce and store, grows exponentially, databases will hit a limit. Accessing your data without a performance penalty simply will not be possible. The answer to this is using an IMDG.

Need help dealing with Maven's non-resolvable parent problem? Check out this post to learn how.

It didn’t take long. A few months after we released an open source continuous integration tool (Anthill) in 2001, we were asked, “It’s great that I have the build setup, now how do I deploy to the test lab?” That email started a clear transformation in our thinking. Lesson 1: Builds generally exist to be tested or released to customers.The corollary is: Continuous integration is not about build, it is about quality and checking quality generally requires a deployment or six. In 2005, we updated our AnthillPro 2.5 with a shiny new deployment capability. It worked ok, but with that generation of tool being so build oriented, it was never a clean fit. Lesson 2: Deployments are a serious challenge, and can not be bolted on to a build tool. This lesson has been reinforced over the years as we’ve watched the results of various tools tacking on deployments. In 2006, we released AnthillPro 3. Oh boy, what a change. Now deployments and other stuff you do to builds days and weeks later were their own thing. Demonstrating a stunning lack of marketing savvy, we called them “non-originating workflows” (processes that don’t make builds) and declared a new type of tool, an “Application Lifecycle Automation Server”. Today, you’d call it a Continuous Delivery tool (thank you Jez and Dave for the better name). To my knowledge, AnthillPro was the first tool to really take a build lifecycle or build pipeline seriously and had the ambition to manage from continuous integration build through to production release quickly and with a solid audit trail. That brings us to Lesson 3: When you come from the development side of the house as either an engineer or a vendor, you have a lot to learn about audit and security. Around the same time we also learned a great deal about the Dev / Ops gap. While some of our customers implemented the tool as intended, most of the time it was used either by a dev organization doing continuous integration and deployment to the early test environments or a production support / release management group that only did the “official” builds and focused on the production release. The failure of the developer initiated efforts to get to production provided a key insight. Lesson 4: When it comes to deployment automation, start with production, and work your way to the lower environments, treating them as a simpler case of the hard problem. We also learned about the limitations of a build pipeline approach in complex applications. Lesson 5: Deployments of complex systems often require a coordinated release of many builds. The need to start at production style deployments (and not care as much about the build) and the emphasis on deployment time dependencies focused our efforts on a deployment centric tool, uDeploy to compliment the pipeline approach in AnthillPro. Production and scaled usage come together to make the deployment tool a critical piece of infrastructure. If you lose a datacenter, can you recover the tool and use it to recover other applications into a backup datacenter?Lesson 6: Your release and deployment systems need to be configured with high availability and recoverability in mind. For the better part of the last decade, we’ve been trying to streamline deployments and kill off the release weekend. Many of customers have eliminated or shrunk those big events down to a more manageable size. Big releases though still can require the coordination of dozens of people across various teams. One time events like configuring the deployment tool with the password for a new database in each environment, or routine manual steps like putting eyeballs on the new production system before putting it out to the public still need to be managed. Lesson 7: Deployment is still just one part of release. With that in mind, we’ve added uRelease to the portfolio. Different people in the application delivery value chain need different tools. Developers need continuous integration and great testing. Change management needs great auditing around production deployments. Testers need their environments to be not broken, updated regularly and close enough to production to make their testing worthwhile. The patterns and needs overlap. Everyone needs deployments that work. Everyone benefits from understanding the lifecycle of a build (or other version) as well as what is in a given environment. Everyone benefits when release planning meetings are shorter. Lesson 8: No single tool is enough. An integrated toolchain that exchanges information is required. So there you go. Twelve years of tooling, eight with deployments. Summed up in six over-simplified lessons. I’m sure we’ll be updating this next year. Lesson 1: Builds generally exist to be tested or released to customers. Lesson 2: Deployments are a serious challenge, and can not be bolted on to a build tool. Lesson 3: When you come from the development side of the house as either an engineer or a vendor, you have a lot to learn about audit and security. Lesson 4: When it comes to deployment automation, start with production, and work your way to the lower environments, treating them as a simpler case of the hard problem. Lesson 5: Deployments of complex systems often require a coordinated release of many builds. Lesson 6: Release and deployment systems need to be configured with high availability and recoverability in mind. Lesson 7: Deployment is still just one part of release Lesson 8: No single tool does “DevOps”. An integrated toolchain that exchanges information is required. What are the biggest lessons you’ve learned about deployments in the last few years? Share in the comments below!

talend has several connectors to integrate sap systems. however, this guide is no introduction to talend’s sap components. instead, this guide helps to understand different alternatives to integrate sap systems with talend set up a local sap system configure talend studio for using sap components use talend’s sap wizard run a first talend job which connects to sap all further required information and example use cases for talend’s sap components should be available in the talend component guide at www.help.talend.com . if that’s not the case, please create a jira documentation ticket ( https://jira.talendforge.org/browse/doct )! now let’s take a look at different alternatives for integration of sap systems with talend. alternatives for sap integration three protocols exist for communication between sap and external programs: dynamic information and action gateway (diag): e.g. used by sap gui remote function call (rfc): a function call with input and output parameters (like a java interface) hypertext transfer protocol (http): internet standard the following alternatives are available for integrating sap systems using some of these protocols. file sap supports the direct import of files (call-transaction-program, batch-input, direct input). files have to be in a specific format to be imported. transformation and integration can be realized with talend’s various file components such as tfileinputdelimited. rfc remote function call is the proprietary sap ag interface for communication between a sap system and other sap or third-party compatible system over tcp/ip or cpi-c connections. remote function calls may be associated with sap software and abap programming, and provide a way for an external program (written in languages such as php, asp, java, or c, c++) to use data returned from the server. data transactions are not limited to getting data from the server, but can insert data into server records as well. sap can act as the client or server in an rfc call. a remote function call (rfc) is the call or remote execution of a remote function module in an external system. in the sap system, these functions are provided by the rfc interface system. the rfc interface system enables function calls between two sap systems, or between a sap system and an external system. tsapinput and tsapoutput are talend’s components to use rfcs. business application programming interface (bapi) a bapi is an object-oriented view on most data and transactions of a sap system (called “business objects”). object types of the business objects are stored in the business object repository (bor). bapis are always implemented as rfcs and therefore can be called the same way. additionally, they have the following characteristics (compared to rfcs): stable interface no view layer no exceptions, instead export parameter: “return” most business objects offer the following standard bapis: getlist getdetail change creationfromdata tsapinput and tsapoutput are talend’s components to use bapis. application link enabling (ale) application link enabling (ale) is used for asynchronous messaging between different systems via “intermediate documents (idoc)”. idoc is a sap document format for business transaction data transfers. it is used to realize distributed business processes. idoc is similar to xml in purpose, but differs in syntax. both serve the purpose of data exchange and automation in computer systems, but the idoc technology takes a different approach. while xml allows having some metadata about the document itself, an idoc is obligated to have information at its header like its creator, creation time, etc. while xml has a tag-like tree structure containing data and meta-data, idocs use a table with the data and meta-data. idocs also have a session that explains all the processes which the document passed or will pass, allowing one to debug and trace the status of the document. an idoc consists of control record (it contains the type of idoc, port of the partner, release of sap r/3 which produced the idoc, etc.) data records of different types. the number and type of segments is mostly fixed for each idoc type, but there is some flexibility (for example an sd order can have any number of items). status records containing messages such as 'idoc created', 'the recipient exists', 'idoc was successfully passed to the port', 'could not book the invoice because...' different idoc types are available to handle different types of messages. for example, the idoc format orders01 may be used for both purchase orders and order confirmations. tsapidocinput and tsapidocoutput are talend’s components to use ale / idoc. bapis can also be called asynchronously via ale. all new idocs are even based on bapis. soap web services sap supports soap web services. not just sap as java, but also sap as abap! integration can be realized with talend’s esb / web service components such as tesbrequest, tesbresponse, or tesbconsumer. installation of sap server and client installation can take about 6 to 8 hours, but it is an “all in one installation”, i.e. you can install it overnight. steps for installation: get yourself a windows 7 64 bit laptop or vm with 8+ gb ram and 50+gb free disc space get a sap community account (for free, just register): http://scn.sap.com/welcome download sap netweaver (software downloads --> sap netweaver main releases: http://www.sdn.sap.com/irj/scn/nw-downloads download current version of sap netweaver application server abap 64-bit trial install sap server: follow installation guide – a html website included in the download in root of extracted download folder (start.htm --> there click on “installation” link) install sap gui (rich client frontend): start.htm --> there click on “install sap gui” link and follow instructions download the sap jco for the operating system on which your connector is running. the sap jco is available for download from sap's website at http://service.sap.com/connectors . you must have an sapnet account to access the sap jco (if you do not already have one, contact your local sap basis administrator). usage of sap server hint: you have to use a windows user which has a password (as you need to enter windows credentials when stopping sap). if you have a windows user without a password (for instance if you use windows within a vm on your mac), sap cannot process these credentials (i.e. it cannot process an empty password field) --> change your windows password before starting sap start the management console (windows startmenu --> programs --> sap management console) start and stop the sap server (right click on “nsp” --> start / stop) default user: sap* (sap system super user) password: the one which you entered at installation of sap netweaver, e.g. admin123 usage of sap client a sap client should be used to get information about the sap system (functions, data, etc.) similarly to using e.g. mysql workbench to get information from a mysql database. sap gui (view layer) communicates with sap as abap (business logic layer). the application server communicates with the relational database (db layer). different clients are available for sap: sap gui windows sap gui java web browser external rfc-program for local development demos, sap gui windows is probably the best alternative. start sap gui windows by: clicking shortcut “windows start menu --> sap frontend --> sap logon” entering username and password clicking logon sap transactions in sap, you call sap programs via sap transaction codes. important transactions codes are for example: bapi: bapi explorer, view all sap bapi's se16: data browser, view/add table data se38: program editor here is a list of several other important transaction codes: http://www.sapdev.co.uk/tcodes/tcodes.htm installation of demo data the sap installation includes some demo data. as most people do not want to install “real” sap modules such as sap fi, sap crm or sap bi on their local system, this demo data is perfect for demos using talend’s sap connectors. to install the flight demo on a local sap system, you just have to open the abap editor (transaction: se38) and execute the program sapbc_data_generator. this program generates example data within the flight tables and does some further initializations. here is a good tutorial with more information and how to test the flight application: http://help.sap.com/saphelp_erp60_sp/helpdata/de/db/7c623cf568896be10000000a11405a/content.htm configuration of talend studio to use sap components talend’s sap components are already included in the studio. however, two further steps are required to be able to use them: copy sapjco3.dll to the directory c:/windows/system32 sap java connector jar must be added copy sapjco3.jar to the directory “talend/studio/lib/java” (re-) start talend studio check if sap library is added successfully open view “talend modules” (eclipse --> windows --> show view --> talend --> modules) sort by column “context” look for “tsap*” contexts and check if sapjco3.jar has status “installed” usage of sap components with talend studio this section describes how to use talend’s sap components and the sap wizard in general (using one specific example for calling a bapi). detailed descriptions of all sap components (for using bapis, rfcs, idocs, bw, etc.) are available in the documentation talend_components_rg_x.y.z.pdf at www.help.talend.com . connection to a sap system a connection to a sap system can be done “built-in” or via “metadata --> sap connections” (the latter only in enterprise version). using the latter has several advantages: reuse connection configuration quick check if connection to sap works wizards for retrieving functions from sap (instead of handwriting without wizard) quick test with test parameters if function works before finishing development lifecycle for a sap job development lifecycle for sap job: create connection (if not existing yet) right click on metadata --> sap connection create sap connection follow wizard sap jco version: 3 client: “001” userid: “sap*” password: “admin123” --> as you defined it while installation language: “en” hostname: “localhost” system number: “00” retrieve function (bapi / rfc) right click on created connection click on “retrieve sap function” enter search filter (e.g. bapi_fl*) click on “search” select and double click on your function (e.g bapi_flcust_getlist) you see all input, output and table parameters for this sap function click on “test in” --> here you see parameters in more detail: you now have to define which input and output parameters you want to use --> remove all other by selecting them and clicking “remove” button hint: if you do not remove an input parameter, you usually have to enter a value for it! select the output type - can be a single (single record), a table (list of records), or a structure output hint: difference between table and structure in sap: http://www.sapfans.com/forums/viewtopic.php?f=12&t=119794 if you want to do a quick test: enter values for input parameters (if there are any for your function call), then click “launch” button in this example, there is only an optional input parameter max_rows you should see data in the output fields in this example, you see the record with custname “sap ag” and street “neurottstr. 16” click “finish” button under “metadata --> sap connections --> “your connection” --> sap functions: there you can now see your function (in this example: bapi_flcust_getlist) create sap job drag&drop the created function into a job (without the wizard, you also can enter all data by hand) tsapinput component is proposed automatically. click ok to add it to your job go to “initialize input” and add parameter values in this example, there is just the parameter “max_rows” hint: the parameter value can be changed from a hardcoded value to a variable, of course (just click control space on your keyboard to get access to all available variables via code completion in your studio) go to the tsapinput component and add the desired output mapping (i.e. which values you want to process further with other components scroll to the bottom to “outputs” add the correct table / structure name (in this example: "customer_list") click on mapping (which is empty and has to be filled) click on “mapping”, then click on “…” add the wanted output columns of your sap function add the same names at the column “schema xpathqueries” (do not forget the double quotes here!) click “ok” button connect the tsapinput component to a tlogrowcomponent and synchronize the schema hint: always try out if this works before adding further logic to your job! run and test your job (you will see five rows logged (as you have configured max_rows = 5 that's it. now enjoy talend's sap components :-) best regards, kai wähner (twitter: @kaiwaehner) content from my blog: http://www.kai-waehner.de/blog/2013/03/03/sap-integration-with-talend-components-connectors-bapi-rfc-idoc-bw-soap/

In order to receive JMS messages, Spring provides the concept of message listener containers. These are beans that can be tied to receive messages that arrive at certain destinations. This post will examine the different ways in which containers can be configured. A simple example is below where the DefaultMessageListenerContianer has been configured to watch one queue (the property jms.queue.name) and has a reference to a myMessageListener bean which implements the MessageListener interface (ie onMessage): This is all very well but means that the myMessageListener bean will have to handle the JMS Message object and process accordingly depending upon the type of javax.jms.Message and its payload. For example: if (message instanceof MapMessage) { // cast, get object, do something } An alternative is to use a MessageListenerAdapter. This class abstracts away the above processing and leaves your code to deal with just the message's payload. For example: The delegate is a reference to a myMessageReceiverDelegate bean which has one or more methods called processMessage. It does not need to implement the MessageListener interface. This method can be overload to handle different payload types. Spring behind the scenes will determine which gets called. For example: public void processMessage(final HashMap message) { // do something } public void processMessage(final String message) { // do something } For the given approach though, only one queue can be tied to the container. Another approach is to tie many listeners (therefore many queues) to the one container, The below Spring XML, using the jms namespace, shows how two listeners for different queues can be tied to one container: The myMessageReceiverDelegate bean is treated as an adapter delegate, therefore does not need to implement the MessageListener interface. Each listener can have a different delegate but for the above example, all messages arriving at the two queues are processed by the one receiver bean ie myMessageReceiverDelegate. If there is a need to check the message type and extract the payload, then the listener can use a class which implements the MessageListener interface (eg the myMessageListener bean used in the first example). The onMessage method will then be called when messages arrive at the specified destination:

Testing threads is hard, very hard and this makes writing good integration tests for multithreaded systems under test... hard. This is because in JUnit there's no built in synchronisation between the test code, the object under test and any threads. This means that problems usually arise when you have to write a test for a method that creates and runs a thread. One of the most common scenarios in this domain is in making a call to a method under test, which starts a new thread running before returning. At some point in the future when the thread's job is done you need assert that everything went well. Examples of this scenario could include asynchronously reading data from a socket or carrying out a long and complex set of operations on a database. For example, the ThreadWrapper class below contains a single public method: doWork(). Calling doWork() sets the ball rolling and at some point in the future, at the discretion of the JVM, a thread runs adding data to a database. public class ThreadWrapper { /** * Start the thread running so that it does some work. */ public void doWork() { Thread thread = new Thread() { /** * Run method adding data to a fictitious database */ @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); } private void addDataToDB() { // Dummy Code... try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } }; thread.start(); System.out.println("Off and running..."); } } A straightforward test for this code would be to call the doWork() method and then check the database for the result. The problem is that, owing to the use of a thread, there's no co-ordination between the object under test, the test and the thread. A common way of achieving some co-ordination when writing this kind of test is to put some kind of delay in between the call to the method under test and checking the results in the database as demonstrated below: public class ThreadWrapperTest { @Test public void testDoWork() throws InterruptedException { ThreadWrapper instance = new ThreadWrapper(); instance.doWork(); Thread.sleep(10000); boolean result = getResultFromDatabase(); assertTrue(result); } /** * Dummy database method - just return true */ private boolean getResultFromDatabase() { return true; } } In the code above there is a simple Thread.sleep(10000) between two method calls. This technique has the benefit of being incredabile simple; however it's also very risky. This is because it introduces a race condition between the test and the worker thread as the JVM makes no guarantees about when threads will run. Often it'll work on a developer's machine only to fail consistently on the build machine. Even if it does work on the build machine it atificially lengthens the duration of the test; remember that quick builds are important. The only sure way of getting this right is to synchronise the two different threads and one technique for doing this is to inject a simple CountDownLatch into the instance under test. In the example below I've modified the ThreadWrapper class's doWork() method adding the CountDownLatch as an argument. public class ThreadWrapper { /** * Start the thread running so that it does some work. */ public void doWork(final CountDownLatch latch) { Thread thread = new Thread() { /** * Run method adding data to a fictitious database */ @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); countDown(); } private void addDataToDB() { try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } private void countDown() { if (isNotNull(latch)) { latch.countDown(); } } private boolean isNotNull(Object obj) { return latch != null; } }; thread.start(); System.out.println("Off and running..."); } } he Javadoc API describes a count down latch as: A synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes. A CountDownLatch is initialized with a given count. The await methods block until the current count reaches zero due to invocations of the countDown() method, after which all waiting threads are released and any subsequent invocations of await return immediately. This is a one-shot phenomenon -- the count cannot be reset. If you need a version that resets the count, consider using a CyclicBarrier. A CountDownLatch is a versatile synchronization tool and can be used for a number of purposes. A CountDownLatch initialized with a count of one serves as a simple on/off latch, or gate: all threads invoking await wait at the gate until it is opened by a thread invoking countDown(). A CountDownLatchinitialized to N can be used to make one thread wait until N threads have completed some action, or some action has been completed N times. A useful property of a CountDownLatch is that it doesn't require that threads calling countDown wait for the count to reach zero before proceeding, it simply prevents any thread from proceeding past an await until all threads could pass. The idea here is that the test code will never check the database for the results until the run() method of the worker thread has called latch.countdown(). This is because the test code thread is blocking at the call to latch.await(). latch.countdown() decrements latch's count and once this is zero the blocking call the latch.await() returns and the test code continues executing, safe in the knowledge that any results which should be in the database, are in the database. The test can then retrieve these results and make a valid assertion. Obviously, the code above merely fakes the database connection and operations. The thing is you may not want to, or need to, inject a CountDownLatch directly into your code; after all it's not used in production and it doesn't look particularly clean or elegant. One quick way around this is to simply make the doWork(CountDownLatch latch) method package private and expose it through a public doWork() method. public class ThreadWrapper { /** * Start the thread running so that it does some work. */ public void doWork() { doWork(null); } @VisibleForTesting void doWork(final CountDownLatch latch) { Thread thread = new Thread() { /** * Run method adding data to a fictitious database */ @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); countDown(); } private void addDataToDB() { try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } private void countDown() { if (isNotNull(latch)) { latch.countDown(); } } private boolean isNotNull(Object obj) { return latch != null; } }; thread.start(); System.out.println("Off and running..."); } } The code above uses Google's Guava @VisibleForTesting annotation to tell us that the doWork(CountDownLatch latch) method visibility has been relaxed slightly for testing purposes. Now I realise that making a method call package private for testing purposes in highly controversial; some people hate the idea, whilst others include it everywhere. I could write a whole blog on this subject (and may do one day), but for me it should be used judiciously, when there's no other choice, for example when you're writing characterisation tests for legacy code. If possible it should be avoided, but never ruled out. After all tested code is better than untested code. With this in mind the next iteration of ThreadWrapper designs out the need for a method marked as @VisibleForTesting together with the need to inject a CountDownLatch into your production code. The idea here is to use the Strategy Pattern and separate the Runnable implementation from the Thread. Hence, we have a very simple ThreadWrapper public class ThreadWrapper { /** * Start the thread running so that it does some work. */ public void doWork(Runnable job) { Thread thread = new Thread(job); thread.start(); System.out.println("Off and running..."); } } and a separate job: public class DatabaseJob implements Runnable { /** * Run method adding data to a fictitious database */ @Override public void run() { System.out.println("Start of the thread"); addDataToDB(); System.out.println("End of the thread method"); } private void addDataToDB() { try { Thread.sleep(4000); } catch (InterruptedException e) { e.printStackTrace(); } } } You'll notice that the DatabaseJob class doesn't use a CountDownLatch. How is it synchronised? The answer lies in the test code below... public class ThreadWrapperTest { @Test public void testDoWork() throws InterruptedException { ThreadWrapper instance = new ThreadWrapper(); CountDownLatch latch = new CountDownLatch(1); DatabaseJobTester tester = new DatabaseJobTester(latch); instance.doWork(tester); latch.await(); boolean result = getResultFromDatabase(); assertTrue(result); } /** * Dummy database method - just return true */ private boolean getResultFromDatabase() { return true; } private class DatabaseJobTester extends DatabaseJob { private final CountDownLatch latch; public DatabaseJobTester(CountDownLatch latch) { super(); this.latch = latch; } @Override public void run() { super.run(); latch.countDown(); } } } The test code above contains an inner class DatabaseJobTester, which extends DatabaseJob. In this class the run() method has been overridden to include a call to latch.countDown() after our fake database has been updated via the call to super.run(). This works because the test passes a DatabaseJobTester instance to the doWork(Runnable job) method adding in the required thread testing capability. The idea of sub-classing objects under test is something I've mentioned before in one of my blogs on testing techniques and is a really powerful technique. So, to conclude: Testing threads is hard. Testing anonymous inner classes is almost impossible. Using Thead.sleep(...) is a risky idea and should be avoided. You can refactor out these problems using the Strategy Pattern. Programming is the Art of Making the Right Decision ...and that relaxing a method's visibility for testing may or may not be a good idea, but more on that later... The code above is available on Github in the captain debug repository (git://github.com/roghughe/captaindebug.git) under the unit-testing-threads project.

Curator's Note: This article was co-authored by Andrzej Jarzyna. At 3scale we find Amazon to be a fantastic platform for running APIs due to the complete control you have on the application stack. For people new to AWS the learning curve is quite steep. So we put together our best practices into this short tutorial. Besides Amazon EC2 we will use the Ruby Grape gem to create the API interface and an Nginx proxy to handle access control. Best of all everything in this tutorial is completely FREE! For the purpose of this tutorial you will need a running API based on Ruby and Thin server. If you don’t have one you can simply clone an example repo as described below (in the “Deploying the Application” section). If you are interested in the background of this example (Sentiment API), you can see a couple of previous guides which 3scale has published. Here we use version_1 of the API(‘API up and running in 10 minutes‘) with some extra sentiment analysis functionality (this part is covered in the second tutorial of the Sentiment API tutorial). Now we will start the creation and configuration of the Amazon EC2 instance. If you already have an EC2 instance (micro or not), you can jump to the next step -> Preparing Instance for Deployment. Creating and configuring EC2 Instance Let’s start by signing up for the Amazon Elastic Compute Cloud (Amazon EC2). For our needs the free tier http://aws.amazon.com/free/ is enough, covering all the basic needs. Once the account is created go to the EC2 dashboard under your AWS Management Console and click on the Launch Instance button. That will transfer you to a popup window where you will continue the process: Choose the classic wizard Choose an AMI (Ubuntu Server 12.04.1 LTS 32bit, T1micro instance) leaving all the other settings for Instance Details as default Create a keypair and download it – this will be the key which you will use to make an ssh connection to the server, it’s VERY IMPORTANT! Add inbound rules for the firewall with source always 0.0.0.0/0 (HTTP, HTTPS, ALL ICMP, TCP port 3000 used by the Ruby thin server) Preparing Instance for Deployment Now, as we have the instance created and running, we can directly connect there from our console (Windows users from PuTTY). Right click on your instance, connect and choose Connect with a standalone SSH Client. Follow the steps and change the username to ubuntu (instead of root) in the given example. After executing this step you are connected to your instance. We will have to install new packages. Some of them require root credentials, so you will have to set a new root password: sudo passwd root. Then login as root: su root. Now with root credentials execute: sudo apt-get update and switch back to your normal user with exit command and install all the required packages: install some libraries which will be required by rvm, ruby and git: sudo apt-get install build-essential git zlib1g-dev libssl-dev libreadline-gplv2-dev imagemagick libxml2-dev libxslt1-dev openssl libreadline6 libreadline6-dev zlib1g libyaml-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison libpq-dev libpq5 libeditline-dev install git (on Linux rather than from Source): http://www.git-scm.com/book/en/Getting-Started-Installing-Git install rvm: https://rvm.io/rvm/install/ install ruby rvm install 1.9.3 rvm use 1.9.3 --default Deploying the Application Our sample Sentiment API is located on Github. Try cloning the repository: git clone [email protected]:jerzyn/api-demo.git you can once again review the code and tutorial on creating and deploying this app here: http://www.3scale.net/2012/06/the-10-minute-api-up-running-3scale-grape-heroku-api-10-minutes/ and here http://www.3scale.net/2012/07/how-to-out-of-the-box-api-analytics/ note the changes (we are using only v1, as authentication will go through the proxy). Now you can deploy the app by issuing: bundle install. Now you can start the thin server: thin start. To access the API directly (i.e. without any security or access control) access: your-public-dns:3000/v1/words/awesome.json (you can find your-public-dns in the AWS EC2 Dashboard->Instances in the details window of your instance) For the Nginx integration you will have to create an elastic IP address. Inside the AWS EC2 dashboard create an elastic IP in the same region as your instance and associate that IP to it (you won’t have to pay anything for the elastic IP as long as it is associated with your instance in the same region). OPTIONAL: If you want to assign a custom domain to your amazon instance you will have to do one thing: add an A record to the DNS record of your domain mapping the domain to the elastic IP address you have previously created. Your domain provider should either give you some way to set the A record (the IPv4 address), or it will give you a way to edit the nameservers of your domain. If they do not allow you to set the A record directly, find a DNS management service, register your domain as a zone there and the service will give you the nameservers to enter in the admin panel of your domain provider. You can then add the A record for the domain. Some possible DNS management services include ZoneEdit (basic, free), Amazon route 53, etc. At this point you API is open to the world. This is good and bad – great that you are sharing, but bad in the sense that without rate limits a few apps could kill the resources of your server, and you have no insight into who is using your API and how it is being used. The solution is to add some management for your API… Enabling API Management with 3scale Rather than reinvent the wheel and implement rate limits, access controls and analytics from scratch we will leverage the handy 3scale API Management service. Get your free 3scale account, activate and log-in to the new instance through the provided links. The first time you log-in you can choose the option for some sample data to be created, so you will have some API keys to use later. Next you would probably like to go through the tour to get a glimpse on the system functionality (optional) and then start with the implementation. To get some instant results we will start with the sandbox proxy which can be used while in development. Then we will also configure an Nginx proxy which can scale up for full production deployments. There is some documentation on the configuration of the API proxy at 3scale: https://support.3scale.net/howtos/api-configuration/nginx-proxy and for more advanced configuration options here: https://support.3scale.net/howtos/api-configuration/nginx-proxy-advanced Once you sign into your 3scale account, Launch your API on the main Dashboard screen or Go to API->Select the service (API)->Integration in the sidebar->Proxy Set the address of of your API backend – this has to be the Elastic IP address unless the custom domain has been set, including http protocol and port 3000. Now you can save and turn on the sandbox proxy to test your API by hitting the sandbox endpoint (after creating some app credentials in 3scale): http://sandbox-endpoint/v1/words/awesome.json?app_id=APP_ID&app_key=APP_KEY where, APP_ID and APP_KEY are id and key of one of the sample applications which you created when you first logged into your 3scale account (if you missed that step just create a developer account and an application within that account). Try it without app credentials, next with incorrect credentials, and then once authenticated within and over any rate limits that you have defined. Only once it is working to your satisfaction do you need to download the config files for Nginx. Note: any time you have errors check whether you can access the API directly: your-public-dns:3000/v1/words/awesome.json. If that is not available, then you need to check if the AWS instance is running and if the Thin Server is running on the instance. Implement an Nginx Proxy for Access Control In order to streamline this step we recommend that you install the fantastic OpenResty web application that is basically a bundle of the standard Nginx core with almost all the necessary 3rd party Nginx modules built-in. Install dependencies: sudo apt-get install libreadline-dev libncurses5-dev libpcre3-dev perl Compile and install Nginx: cd ~ sudo wget http://agentzh.org/misc/nginx/ngx_openresty-1.2.3.8.tar.gz sudo tar -zxvf ngx_openresty-1.2.3.8.tar.gz cd ngx_openresty-1.2.3.8/ ./configure --prefix=/opt/openresty --with-luajit --with-http_iconv_module -j2 make sudo make install In the config file make the following changes: edit the .conf file from nginx download in line 28, which is preceded by info to change your server name put the correct domain (of your Elastic IP or custom domain name) in line 78 change the path to the .lua file, downloaded together with the .conf file. We are almost finished! Our last step is to start the NGINX proxy and put some traffic through it. If it is not running yet (remember, that thin server has to be started first), please go to your EC2 instance terminal (the one you were connecting through ssh before) and start it now: sudo /opt/openresty/nginx/sbin/nginx -p /opt/openresty/nginx/ -c /opt/openresty/nginx/conf/YOUR-CONFIG-FILE.conf The last step will be verifying that the traffic goes through with a proper authorization. To do that, access: http://your-public-dns/v1/words/awesome.json?app_id=APP_ID&app_key=APP_KEY where, APP_ID and APP_KEY are key and id of the application you want to access through the API call. Once everything is confirmed as working correctly, you will want to block public access to the API backend on port 3000, which bypasses any access controls. If encounter some problems with the Nginx configuration or need a more detailed guide, I encourage you to check the 3scale guide on configuring Nginx proxy: https://support.3scale.net/howtos/api-configuration/nginx-proxy. You can go completely wild with customization of your API gateway. If you want to dive more into the 3scale system configuration (like usage and monitoring of your API traffic) feel encouraged to browse our Quickstart guides and HowTo’s.

the repository pattern has been discussed a lot lately. especially about it’s usefulness since the introduction of or/m libraries. this post (which is the third in a series about the data layer) aims to explain why it’s still a great choice. let’s start with the definition : a repository mediates between the domain and data mapping layers, acting like an in-memory domain object collection. client objects construct query specifications declaratively and submit them to repository for satisfaction. objects can be added to and removed from the repository, as they can from a simple collection of objects, and the mapping code encapsulated by the repository will carry out the appropriate operations behind the scenes the repository pattern is used to create an abstraction between your domain and data layer. that is, when you use the repository you should not have to have any knowledge about the underlying data source or the data layer (i.e. entity framework, nhibernate or similar). why do we need it? read the abstractions part of my data layer article. it explains the basics to why we should use repositories or similar abstractions. but let’s also examine some simple business logic: var brokentrucks = _session.query().where(x => x.state == 1); foreach (var truck in brokentrucks) { if (truck.calculatereponsetime().totaldays > 30) sendemailtomanager(truck); } what does that give us? broken trucks? well. no. the statement was copied from another place in the code and the developer had forgot to update the query. any unit tests would likely just check that some trucks are returned and that they are emailed to the manager. so we basically have two problems here: a) most developers will likely just check the name of the variable and not on the query. b) any unit tests are against the business logic and not the query. both those problems would have been fixed with repositories. since if we create repositories we also have unit tests which targets the data layer only. implementations here are some different implementations with descriptions. base classes these classes can be reused for all different implementations. unitofwork the unit of work represents a transaction when used in data layers. typically the unit of work will roll back the transaction if savechanges() has not been invoked before being disposed. public interface iunitofwork : idisposable { void savechanges(); } paging we also need to have page results. public class pagedresult { ienumerable _items; int _totalcount; public pagedresult(ienumerable items, int totalcount) { _items = items; _totalcount = totalcount; } public ienumerable items { get { return _items; } } public int totalcount { get { return _totalcount; } } } we can with the help of that create methods like: public class userrepository { public pagedresult find(int pagenumber, int pagesize) { } } sorting finally we prefer to do sorting and page items, right? var constraints = new queryconstraints() .sortby("firstname") .page(1, 20); var page = repository.find("jon", constraints); do note that i used the property name, but i could also have written constraints.sortby(x => x.firstname) . however, that is a bit hard to write in web applications where we get the sort property as a string. the class is a bit big, but you can find it at github . in our repository we can apply the constraints as (if it supports linq): public class userrepository { public pagedresult find(string text, queryconstraints constraints) { var query = _dbcontext.users.where(x => x.firstname.startswith(text) || x.lastname.startswith(text)); var count = query.count(); //easy var items = constraints.applyto(query).tolist(); return new pagedresult(items, count); } } the extension methods are also available at github . basic contract i usually start use a small definition for the repository, since it makes my other contracts less verbose. do note that some of my repository contracts do not implement this interface (for instance if any of the methods do not apply). public interface irepository where tentity : class { tentity getbyid(tkey id); void create(tentity entity); void update(tentity entity); void delete(tentity entity); } i then specialize it per domain model: public interface itruckrepository : irepository { ienumerable findbrokentrucks(); ienumerable find(string text); } that specialization is important. it keeps the contract simple. only create methods that you know that you need. entity framework do note that the repository pattern is only useful if you have pocos which are mapped using code first. otherwise you’ll just break the abstraction using the entities. the repository pattern isn’t very useful then. what i mean is that if you use the model designer you’ll always get a perfect representation of the database (but as classes). the problem is that those classes might not be a perfect representation of your domain model. hence you got to cut corners in the domain model to be able to use your generated db classes. if you on the other hand uses code first you can modify the models to be a perfect representation of your domain model (if the db is reasonable similar to it). you don’t have to worry about your changes being overwritten as they would have been by the model designer. you can follow this article if you want to get a foundation generated for you. base class public class entityframeworkrepository where tentity : class { private readonly dbcontext _dbcontext; public entityframeworkrepository(dbcontext dbcontext) { if (dbcontext == null) throw new argumentnullexception("dbcontext"); _dbcontext = dbcontext; } protected dbcontext dbcontext { get { return _dbcontext; } } public void create(tentity entity) { if (entity == null) throw new argumentnullexception("entity"); dbcontext.set().add(entity); } public tentity getbyid(tkey id) { return _dbcontext.set().find(id); } public void delete(tentity entity) { if (entity == null) throw new argumentnullexception("entity"); dbcontext.set().attach(entity); dbcontext.set().remove(entity); } public void update(tentity entity) { if (entity == null) throw new argumentnullexception("entity"); dbcontext.set().attach(entity); dbcontext.entry(entity).state = entitystate.modified; } } then i go about and do the implementation: public class truckrepository : entityframeworkrepository, itruckrepository { private readonly truckerdbcontext _dbcontext; public truckrepository(truckerdbcontext dbcontext) { _dbcontext = dbcontext; } public ienumerable findbrokentrucks() { //compare having this statement in a business class compared //to invoking the repository methods. which says more? return _dbcontext.trucks.where(x => x.state == 3).tolist(); } public ienumerable find(string text) { return _dbcontext.trucks.where(x => x.modelname.startswith(text)).tolist(); } } unit of work the unit of work implementation is simple for entity framework: public class entityframeworkunitofwork : iunitofwork { private readonly dbcontext _context; public entityframeworkunitofwork(dbcontext context) { _context = context; } public void dispose() { } public void savechanges() { _context.savechanges(); } } nhibernate i usually use fluent nhibernate to map my entities. imho it got a much nicer syntax than the built in code mappings. you can use nhibernate mapping generator to get a foundation created for you. but you do most often have to clean up the generated files a bit. base class public class nhibernaterepository where tentity : class { isession _session; public nhibernaterepository(isession session) { _session = session; } protected isession session { get { return _session; } } public tentity getbyid(string id) { return _session.get(id); } public void create(tentity entity) { _session.saveorupdate(entity); } public void update(tentity entity) { _session.saveorupdate(entity); } public void delete(tentity entity) { _session.delete(entity); } } implementation public class truckrepository : nhibernaterepository, itruckrepository { public truckrepository(isession session) : base(session) { } public ienumerable findbrokentrucks() { return _session.query().where(x => x.state == 3).tolist(); } public ienumerable find(string text) { return _session.query().where(x => x.modelname.startswith(text)).tolist(); } } unit of work public class nhibernateunitofwork : iunitofwork { private readonly isession _session; private itransaction _transaction; public nhibernateunitofwork(isession session) { _session = session; _transaction = _session.begintransaction(); } public void dispose() { if (_transaction != null) _transaction.rollback(); } public void savechanges() { if (_transaction == null) throw new invalidoperationexception("unitofwork have already been saved."); _transaction.commit(); _transaction = null; } } typical mistakes here are some mistakes which can be stumbled upon when using or/ms. do not expose linq methods let’s get it straight. there are no complete linq to sql implementations. they all are either missing features or implement things like eager/lazy loading in their own way. that means that they all are leaky abstractions. so if you expose linq outside your repository you get a leaky abstraction. you could really stop using the repository pattern then and use the or/m directly. public interface irepository { iqueryable query(); // [...] } those repositories really do not serve any purpose. they are just lipstick on a pig (yay, my favorite) those who use them probably don’t want to face the truth: or are just not reading very good: learn about lazy loading lazy loading can be great. but it’s a curse for all which are not aware of it. if you don’t know what it is, google . if you are not careful you could get 101 executed queries instead of 1 if you traverse a list of 100 items. invoke tolist() before returning the query is not executed in the database until you invoke tolist() , firstordefault() etc. so if you want to be able to keep all data related exceptions in the repositories you have to invoke those methods. get is not the same as search there are to types of reads which are made in the database. the first one is to search after items. i.e. the user want to identify the items that he/she like to work with. the second one is when the user has identified the item and want to work with it. those queries are different. in the first one, the user only want’s to get the most relevant information. in the second one, the user likely want’s to get all information. hence in the former one you should probably return userlistitem or similar while the other case returns user . that also helps you to avoid the lazy loading problems. i usually let search methods start with findxxxx() while those getting the entire item starts with getxxxx() . also don’t be afraid of creating specialized pocos for the searches. two searches doesn’t necessarily have to return the same kind of entity information. summary don’t be lazy and try to make too generic repositories. it gives you no upsides compared to using the or/m directly. if you want to use the repository pattern, make sure that you do it properly.

In this post we will Utilize GitHub and/or BitBucket's static web page hosting capabilities to publish our project's Maven 3 Site Documentation. Each of the two SCM providers offer a slightly different solution to host static pages. The approach spelled out in this post would also be a viable solution to "backup" your site documentation in a supported SCM like Git or SVN. This solution does not directly cover site documentation deployment covered by the maven-site-plugin and the Wagon library (scp, WebDAV or FTP). There is one main project hosted on GitHub that I have posted with the full solution. The project URL is https://github.com/mike-ensor/clickconcepts-master-pom/. The POM has been pushed to Maven Central and will continue to be updated and maintained. com.clickconcepts.project master-site-pom 0.16 GitHub Pages GitHub hosts static pages by using a special branch "gh-pages" available to each GitHub project. This special branch can host any HTML and local resources like JavaScript, images and CSS. There is no server side development. To navigate to your static pages, the URL structure is as follows: http://.github.com/ An example of the project I am using in this blog post: http://mike-ensor.github.com/clickconcepts-master-pom/ where the first bold URL segment is a username and the second bold URL segment is the project. GitHub does allow you to create a base static hosted static site for your username by creating a repository with your username.github.com. The contents would be all of your HTML and associated static resources. This is not required to post documentation for your project, unlike the BitBucket solution. There is a GitHub Site plugin that publishes site documentation via GitHub's object API but this is outside the scope of this blog post because it does not provide a single solution for GitHub and BitBucket projects using Maven 3. BitBucket BitBucket provides a similar service to GitHub in that it hosts static HTML pages and their associated static resources. However, there is one large difference in how those pages are stored. Unlike GitHub, BitBucket requires you to create a new repository with a name fitting the convention. The files will be located on the master branch and each project will need to be a directory off of the root. mikeensor.bitbucket.org/ /some-project +index.html +... /css /img /some-other-project +index.html +... /css /img index.html .git .gitignore The naming convention is as follows: .bitbucket.org An example of a BitBucket static pages repository for me would be: http://mikeensor.bitbucket.org/. The structure does not require that you create an index.html page at the root of the project, but it would be advisable to avoid 404s. Generating Site Documentation Maven provides the ability to post documentation for your project by using the maven-site-plugin. This plugin is difficult to use due to the many configuration options that oftentimes are not well documented. There are many blog posts that can help you write your documentation including my post on maven site documentation. I did not mention how to use "xdoc", "apt" or other templating technologies to create documentation pages, but not to fear, I have provided this in my GitHub project. Putting it all Together The Maven SCM Publish plugin (http://maven.apache.org/plugins/maven-scm-publish-plugin/ publishes site documentation to a supported SCM. In our case, we are going to use Git through BitBucket or GitHub. Maven SCM Plugin does allow you to publish multi-module site documentation through the various properties, but the scope of this blog post is to cover single/mono module projects and the process is a bit painful. Take a moment to look at the POM file located in the clickconcepts-master-pom project. This master POM is rather comprehensive and the site documentation is only one portion of the project, but we will focus on the site documentation. There are a few things to point out here, first, the scm-publish plugin and the idiosyncronies when implementing the plugin. In order to create the site documentation, the "site" plugin must first be run. This is accomplished by running site:site. The plugin will generate the documentation into the "target/site" folder by default. The SCM Publish Plugin, by default, looks for the site documents to be in "target/staging" and is controlled by the content parameter. As you can see, there is a mismatch between folders. NOTE: My first approach was to run the site:stage command which is supposed to put the site documents into the "target/staging" folder. This is not entirely correct, the site plugin combines with the distributionManagement.site.url property to stage the documents, but there is very strange behavior and it is not documented well. In order to get the site plugin's site documents and the SCM Publish's location to match up, use the content property and set that to the location of the Site Plugin output (). If you are using GitHub, there is no modification to the siteOutputDirectory needed, however, if you are using BitBucket, you will need to modify the property to add in a directory layer into the site documentation generation (see above for differences between GitHub and BitBucket pages). The second property will tell the SCM Publish Plugin to look at the root "site" folder so that when the files are copied into the repository, the project folder will be the containing folder. The property will look like: ${project.build.directory}/site/ ${project.artifactId} ${project.build.directory} /site Next we will take a look at the custom properties defined in the master POM and used by the SCM Publish Plugin above. Each project will need to define several properties to use the Master POM that are used within the plugins during the site publishing. Fill in the variables with your own settings. BitBucket ... ... master scm:git:[email protected]:mikeensor/mikeensor.bitbucket.org.git ${project.build.directory}/site/${project.artifactId} ${project.build.directory}/site ${changelog.bitbucket.fileUri} ${changelog.revision.bitbucket.fileUri} ... ... GitHub ... ... gh-pages scm:git:[email protected]:mikeensor/clickconcepts-master-pom.git ${changelog.github.fileUri} ${changelog.revision.github.fileUri} ... ... NOTE: changelog parameters are required to use the Master POM and are not directly related to publishing site docs to GitHub or BitBucket How to Generate If you are using the Master POM (or have abstracted out the Site Plugin and the SCM Plugin) then to generate and publish the documentation is simple. mvn clean site:site scm-publish:publish-scm mvn clean site:site scm-publish:publish-scm -Dscmpublish.dryRun=true Gotchas In the SCM Publish Plugin documentation's "tips" they recommend creating a location to place the repository so that the repo is not cloned each time. There is a risk here in that if there is a git repository already in the folder, the plugin will overwrite the repository with the new site documentation. This was discovered by publishing two different projects and having my root repository wiped out by documentation from the second project. There are ways to mitigate this by adding in another folder layer, but make sure you test often! Another gotcha is to use the -Dscmpublish.dryRun=true to test out the site documentation process without making the SCM commit and push Project and Documentation URLs Here is a list of the fully working projects used to create this blog post: Master POM with Site and SCM Publish plugins &ndash https://github.com/mike-ensor/clickconcepts-master-pom. Documentation URL: http://mike-ensor.github.com/clickconcepts-master-pom/ Child Project using Master Pom &ndash http://mikeensor.bitbucket.org/fest-expected-exception. Documentation URL: http://mikeensor.bitbucket.org/fest-expected-exception/

as described in my previous post the ip (and dns) of your running ec2 ami will change after a reboot of that instance. of course this makes it very hard to make your applications on that machine available for the outside world, like in this case our wordpress blog. that is where elastic ip comes to the rescue. with this feature you can assign a static ip to your instance. assign one to your application as follows: click on the elastic ips link in the aws console allocate a new address associate the address with a running instance right click to associate the ip with an instance: pick the instance to assign this ip to: note the ip being assigned to your instance if you go to the ip address you were assigned then you see the home page of your server: and the nicest thing is that if you stop and start your instance you will receive a new public dns but your instance is still assigned to the elastic ip address: one important note: as long as an elastic ip address is associated with a running instance, there is no charge for it. however an address that is not associated with a running instance costs $0.01/hour. this prevents users from ‘reserving’ addresses while they are not being used.

In this example I will show how to test Spring Integration flow using Mock SftpServer.

for people in hurry, get the latest code and the steps in github . to run the junit test, run “mvn test” and understand the test flow. introduction: fakeftpserver in this spring integration fakeftpserver example, i will demonstrate using spring fakeftpserver to junit test a spring integration flow. this is an interesting topic, and there are few articles on unit testing file transfers , which gives some insight on this topic. in this blog, we will test a spring integration flow which checks for a list of files, apply a splitter to separate each file and start downloading them into a local location. once the download is complete, it will delete the files on the ftp server. in my next blog, i will show how to do junit testing of spring integration flow with sftp server. spring integration flow spring integration fakeftpserver example in order to use fakeftpserver we need to have maven dependency as below, org.mockftpserver mockftpserver 2.3 test the first step to this is to create a fakeftpserver before every test runs as below, @before public void setup() throws exception { fakeftpserver = new fakeftpserver(); fakeftpserver.setservercontrolport(9999); // use any free port filesystem filesystem = new unixfakefilesystem(); filesystem.add(new fileentry(file, contents)); fakeftpserver.setfilesystem(filesystem); useraccount useraccount = new useraccount("user", "password", home_dir); fakeftpserver.adduseraccount(useraccount); fakeftpserver.start(); } @after public void teardown() throws exception { fakeftpserver.stop(); } finally run the junit test case as seen below, @autowired private filedownloadutil downloadutil; @test public void testftpdownload() throws exception { file file = new file("src/test/resources/output"); delete(file); ftpclient client = new ftpclient(); client.connect("localhost", 9999); client.login("user", "password"); string files[] = client.listnames("/dir"); client.help(); logger.debug("before delete" + files[0]); assertequals(1, files.length); downloadutil.downloadfilesfromremotedirectory(); logger.debug("after delete"); files = client.listnames("/dir"); client.help(); assertequals(0, files.length); assertequals(1, file.list().length); } i hope this blog helped.

That’s a pretty long title, I agree. When working on my implementation of RFC2324, also known as the HyperText Coffee Pot Control Protocol, I’ve been struggling with something that you will struggle with as well in your ASP.NET Web API’s: supporting additional HTTP methods like HEAD, PATCH or PROPFIND. ASP.NET Web API has no issue with those, but when hosting them on IIS you’ll find yourself in Yellow-screen-of-death heaven. The reason why IIS blocks these methods (or fails to route them to ASP.NET) is because it may happen that your IIS installation has some configuration leftovers from another API: WebDAV. WebDAV allows you to work with a virtual filesystem (and others) using a HTTP API. IIS of course supports this (because flagship product “SharePoint” uses it, probably) and gets in the way of your API. Bottom line of the story: if you need those methods or want to provide your own HTTP methods, here’s the bit of configuration to add to your Web.config file: Here’s what each part does: Under modules, the WebDAVModule is being removed. Just to make sure that it’s not going to get in our way ever again. The security/requestFiltering element I’ve added only applies if you want to define your own HTTP methods. So unless you need the XYZ method I’ve defined here, don’t add it to your config. Under handlers, I’m removing the default handlers that route into ASP.NET. Then, I’m adding them again. The important part? The "verb attribute. You can provide a list of comma-separated methods that you want to route into ASP.NET. Again, I’ve added my XYZ methodbut you probably don’t need it. This will work on any IIS server as well as on Windows Azure Websites. It will make your API… happy.