Converters are an essential building block in XAML interfaces with one simple task: converting values of one type to another. Since they have a input, usually a view model property, and an output, it would be wonderful if we could somehow chain them to create a new converter that processes all internal converters. Luckily, this is quite simple to do, but we do need to create a new converter which will hold other converters and whose implementation will iterate over nested converters. Full code can be found over at Github repository here, only interesting parts will be highlighted in this blog post. Our combining converter class is also a converter itself, but it can contain other converters inside it: [ContentProperty("Converters")] public class ChainingConverter : IValueConverter { public Collection Converters { get; set; } } Converter functions are trivially implemented and iteratively go through the converters list and apply the converter on the previous value. public object Convert(object value, Type targetType, object parameter, CultureInfo culture) { foreach (var converter in Converters) { value = converter.Convert(value, targetType, parameter, culture); } return value; } ConvertBack is implemented in the same fashion. This allows us to create new converters in XAML with the following syntax: But what if we need to send parameters to some of the converters, how can we do that when the same parameter is used throughout the ChainingConverter implementation? To provide custom parameter for individual converters, we can create a wrapper converter around existing converter and specify parameter on that wrapper. Here is a skeleton for such wrapper converter, notice that the wrapper is also a converter: [ContentProperty("Converter")] public class ParameterizedConverterWrapper : DependencyObject, IValueConverter { // IValueConverter Converter dependency property // object Parameter dependency property // object DefaultReturnValue dependency property public object Convert(object value, Type targetType, object parameter, CultureInfo culture) { if (Converter != null) return Converter.Convert(value, targetType, Parameter ?? parameter, culture); return DefaultReturnValue; } } Converter wrappers allow us to create complex converters such as this one: The final converter should be self explanatory even though you probably haven’t seen these converters before. You can see that unlike other converters, the wrapper is a dependency object which allows us to use bindings on the Parameter property since it is in fact a dependency property. More complex converters should be created from ordinary converters whenever possible, especially when working with primitive types such as bool, string, enums and null values. What’s next? The last example looked like a small DSL embedded in XAML. We could create converters that simulate flow control or conditionals. We could even create converters that switch depending on the property before it, essentially coding logic inside such converters. Whether that is desirable is debatable, but it can be done. The full code with sample application can be found at the following Github repository: MassivePixel/wp-common.

In my previous articles I compared 4 frameworks commonly used in communicating with MongoDB from the JVM and found out that in that use-case, Spring Data for MongoDB was the easiest solution. However I did make the remark that it doesn’t use the GeoJSON format to store geolocation coordinates and geometries. I tried to add GeoJSON support before, but couldn’t get the conversion to work propertly. But after some extensive searching I found out that the reason for it not working was my use of Spring Boot: its autoconfiguration for MongoDB does not support custom conversion out of the box. Luckily, the solution was simple: provide an extra configuration that extends from AbstractMongoConfiguration and import that in the Boot application. In that configuration you can override the customConversions() and add your converters. When you compare the geo classes in Spring Data and GeoJSON, I noticed that only a subset of GeoJSON geometries can be mapped on Spring Data geo classes: Point and Polygon. Spring Boot does not support LineString, MultiLineString, MultiPolygon or MultiPoint. However, in your mapped domain classes, you won’t use these normally. Creating a converter that adheres to the GeoJSON format is quite straightforward. import com.mongodb.BasicDBObject import com.mongodb.DBObject import org.springframework.core.convert.converter.Converter import org.springframework.data.convert.ReadingConverter import org.springframework.data.convert.WritingConverter import org.springframework.data.geo.Point import org.springframework.data.geo.Polygon final class GeoJsonConverters { static List> getConvertersToRegister() { return [ GeoJsonDBObjectToPointConverter.INSTANCE, GeoJsonDBObjectToPolygonConverter.INSTANCE, GeoJsonPointToDBObjectConverter.INSTANCE, GeoJsonPolygonToDBObjectConverter.INSTANCE ] } @WritingConverter static enum GeoJsonPointToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Point source) { return new BasicDBObject([type: 'Point', coordinates: [source.x, source.y]]) } } @ReadingConverter static enum GeoJsonDBObjectToPointConverter implements Converter { INSTANCE; @Override Point convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } @WritingConverter static enum GeoJsonPolygonToDBObjectConverter implements Converter { INSTANCE; @Override DBObject convert(Polygon source) { def coordinates = source.points.collect { [it.x, it.y] } return new BasicDBObject([type: 'Polygon', coordinates: coordinates]) } } @ReadingConverter static enum GeoJsonDBObjectToPolygonConverter implements Converter { INSTANCE; @Override Polygon convert(DBObject source) { def coordinates = source.coordinates as double[] return new Point(coordinates[0], coordinates[1]) } } } To add those converters to the Spring context, you’ll have to override some methods in your MongoDB spring configuration class. import com.mongodb.Mongo import org.springframework.beans.factory.annotation.* import org.springframework.boot.SpringApplication import org.springframework.boot.autoconfigure.EnableAutoConfiguration import org.springframework.context.annotation.* import org.springframework.data.mongodb.config.AbstractMongoConfiguration import org.springframework.data.mongodb.core.convert.* @EnableAutoConfiguration @ComponentScan @Configuration @Import([MongoComparisonMongoConfiguration]) class MongoComparison { static void main(String[] args) { SpringApplication.run(MongoComparison, args); } } @Configuration class MongoComparisonMongoConfiguration extends AbstractMongoConfiguration { @Autowired Mongo mongo; @Value("\${spring.data.mongodb.database}") String databaseName; @Override protected String getDatabaseName() { return databaseName } @Override Mongo mongo() throws Exception { return mongo } @Override CustomConversions customConversions() { def customConverters = [] customConverters << GeoJsonConverters.convertersToRegister return new CustomConversions(customConverters.flatten()) } } As Spring Boot already provides the configuration of the Mongo instance and the name of the database, we can reuse these in the MongoDB configuration class. The custom conversions take preference over the existing ones for Point and Polygon. I’ll be writing a library this weekend to add support for all GeoJSON geometries in Spring Data for MongoDB. However, I already noticed it’ll be very hard to provide support for those in generated query methods in repositories, but with annotated queries being possible, I don’t think this will be a big issue but we’ll see.

Actually, its nothing to do with REST for PUT and POST. In general how HTTP PUT works and how POST work, is what I want to demonstrate through code. Why REST is considered, usually we get confused while developing REST API, that when to use PUT and when to use POST for an update and insert resource. Let's start with the actual definition of these methods (copied formhttp://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html) POST The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line. The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database. The action performed by the POST method might not result in a resource that can be identified by a URI. In this case, either 200 (OK) or 204 (No Content) is the appropriate response status, depending on whether or not the response includes an entity that describes the result. If a resource has been created on the origin server, the response SHOULD be 201 (Created) and contain an entity which describes the status of the request and refers to the new resource, and a Location header (see section 14.30). Responses to this method are not cacheable unless the response includes appropriate Cache-Control or Expires header fields. However, the 303 (See Other) response can be used to direct the user agent to retrieve a cacheable resource. PUT The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI. If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource is modified, either the 200 (OK) or 204 (No Content) response code SHOULD be sent to indicate successful completion of the request. If the resource could not be created or modified with the Request-URI, an appropriate error response SHOULD be given that reflects the nature of the problem. The recipient of the entity MUST NOT ignore any Content-* (e.g. Content-Range) headers that it does not understand or implement and MUST return a 501 (Not Implemented) response in such cases. If the request passes through a cache and the Request-URI identifies one or more currently cached entities, those entries SHOULD be treated as stale. Responses to this method are not cacheable. The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request — the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource. If the server desires that the request is applied to a different URI. Let's Go back to our REST example Ok, now to make it more clear in REST terms, let's consider an example of Customer and Order scenario, so we have API to create/modify/get a customer but for order, we do have to create order for customer and when we call GET /CustomerOrders API will get the customer orders. APIs we have GET /Customer/{custID} PUT /Customer/{custID} POST /Customer/{custID} (to demonstrate difference between POST and PUT, otherwise for the UC we are considering, it won't be required) POST /Order/{custID} GET /CustomerOrders/{custID} I have enabled browser cache by adding header “Cache-Control”. so lets first see the flow of PUT and GET for customer Initial load, I called PUT /Customer/1 which placed new resource on the server and then called GET /Customer/1 which returned me the customer I placed. now when I again call the GET /Customer/1 I will get the browser “Cached” instance of a customer. Now you call PUT /Customer/1 with updated values of a customer and then call GET /Customer/1, you will observe that browser makes calls to the server to get new changed values. and if you add debug point or increase the wait time you PUT, and make a parallel request for GET (Ajax), then GET request will be pending till PUT is served, so browser makes a cached instance of a resource to stale. In the case of POST, the new resource will be posted to the server, but if POST request is not served, and you request for the same resource using GET, the cached instance will be returned. Once the post is successful and you make GET call to the resource, the browser will hit the server to get a new resource. I added delay of 100 milliseconds in both PUT and POST and made request as 1) Called GET /Customer/1 multiple times to check if I am getting the cached resource. Then I called PUT, and immediately called GET, and GET was pending till PUT is served. below if the screen shot which explains it. 2) Called GET /Customer/1 multiple times to check if I am getting the cached resource. Then I called POST, and immediately called GET, and GET was served from cache. below is the screen shot which explains it. In our customer and order case, the customer should be PUT for a new customer and for updating customer as we are retrieving the customer using same resource URI but for Order, we used POST as we don’t have same URI for GET orders.

This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications. Authenticating iOS app users with Azure Active Directory How to Best handle AAD access tokens in native mobile apps Using Azure SSO tokens for Multiple AAD Resources From Native Mobile Apps(this post) Sharing Azure SSO access tokens across multiple native mobile apps. Brief Start In an enterprise context, it is highly likely that you would have multiple web services that your native mobile app needs to consume. I had exactly this scenario, where one of my clients had asked if they could maintain the same token in the background in the mobile app to use it for accessing multiple web services. I spent some time digging through the documentation and conducting some experiments to confirm some points. Therefore, this post is to share my findings on accessing multiple Azure AD resources from native mobile apps using ADAL. In the previous two posts, we looked at implementing Azure AD SSO login on native mobile apps, then we looked at how to best maintain these access tokens. This post discusses how to use Azure AD SSO tokens to manage access to multiple AAD resources. Let’s assume that we have 2 web services sitting in Azure (ie WebApi1, and WebApi2), both of which are set to use Azure AD authentication. Then, we have the native mobile app, which needs access to both web services (WebApi1, and WebApi2). Let’s look at what we can and cannot do. Cannot Use the Same Azure AD Access-Token for Multiple Resources The first thing that comes to mind is to use the same access token for multiple Azure AD resources, and that is what the client asked about. However, this is not allowed. Azure AD issues a token for certain resource (which is mapped to an Azure AD app). When we call AcquireToken(), we need to provide a resourceID, only ONE resourceID. The result would have a token that can only be used for the supplied resource (id). There are ways where you could use the same token (as we will see later in this post), but it is not recommended as it complicates operations logging, authentication process tracing, etc. Therefore it is better to look at the other options provided by Azure and the ADAL library. Use the Refresh-Token to Acquire Tokens for Multiple Resources The ADAL library supports acquiring multiple access-Tokens for multiple resources using a refresh token. This means once a user is authenticated, the ADAL’s authentication context, would be able to generate an access-token to multiple resources without authenticating the user again. This was mentioned briefly by the MSDN documentation here. The refresh token issued by Azure AD can be used to access multiple resources. For example, if you have a client application that has permission to call two web APIs, the refresh token can be used to get an access token to the other web API as well. (MSDN documentation) public async Task RefreshTokens() { var tokenEntry = await tokensRepository.GetTokens(); var authorizationParameters = new AuthorizationParameters (_controller); var result = "Refreshed an existing Token"; bool hasARefreshToken = true; if (tokenEntry == null) { var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId1, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); tokenEntry = new Tokens { WebApi1AccessToken = localAuthResult.AccessToken, RefreshToken = localAuthResult.RefreshToken, Email = localAuthResult.UserInfo.DisplayableId, ExpiresOn = localAuthResult.ExpiresOn }; hasARefreshToken = false; result = "Acquired a new Token"; } var refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync(tokenEntry.RefreshToken, clientId, resourceId2); tokenEntry.WebApi2AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; if (hasARefreshToken) { // this will only be called when we try refreshing the tokens (not when we are acquiring new tokens. refreshAuthResult = await _authContext.AcquireTokenByRefreshTokenAsync (refreshAuthResult.RefreshToken, clientId, resourceId1); tokenEntry.WebApi1AccessToken = refreshAuthResult.AccessToken; tokenEntry.RefreshToken = refreshAuthResult.RefreshToken; tokenEntry.ExpiresOn = refreshAuthResult.ExpiresOn; } await tokensRepository.InsertOrUpdateAsync (tokenEntry); return result; } As you can see from above, we check if we have an access-token from previous runs, and if we do, we refresh the access-tokens for both web services. Notice how the _authContext.AcquireTokenByRefreshTokenAsync() provides an overloading parameter that takes a resourceId. This enables us to get multiple access tokens for multiple resources without having to re-authenticate the user. The rest of the code is similar to what we have seen in the previous two posts. ADAL Library Can Produce New Tokens For Other Resources In the previous two posts, we looked at ADAL library and how it uses TokenCache. Although ADAL does not support persistent caching of tokens yet on mobile apps, it still uses the TokenCache for in-memory caching. This enables ADAL library to generate new access-tokens if the context (AuthenticationContext) still exists from previous authentications. Remember in the previous post we said it is recommended to keep a reference to the authentication-context? Here it comes in handy, as it enables us to generate new access-tokens for accessing multiple Azure AD resources. var localAuthResult = await _authContext.AcquireTokenAsync ( resourceId2, clientId, new Uri (redirectUrl), authorizationParameters, UserIdentifier.AnyUser, null); Calling AcquireToken() (even with no refresh-token) would give us a new access-token to webApi2. This is due to ADAL great goodness where it checks if we have a refresh-token in-memory (managed by ADAL), then it uses that to generate a new access-token for webApi2. An alternative The third alternative option is the simplest, but not necessarily the best. In this option, we could use the same access token to consume multiple Azure AD resources. To do this, we need to use the same Azure AD app ID when setting the web application’s authentication. This requires some understanding of how the Azure AD authentication happens on our web apps. If you refer to Taiseer Joudeh’s tutorial, which we mentioned before, you will see that in our web app, we need to tell the authentication framework what’s our Authority and the Audience (Azure AD app Id). If we set up both of our web apps, to use the same Audience (Azure AD app Id), meaning that we link them both into the same Azure AD application, then we could use the same access-token to use both web services. // linking our web app authentication to an Azure AD application private void ConfigureAuth(IAppBuilder app) { app.UseWindowsAzureActiveDirectoryBearerAuthentication( new WindowsAzureActiveDirectoryBearerAuthenticationOptions { Audience = ConfigurationManager.AppSettings["Audience"], Tenant = ConfigurationManager.AppSettings["Tenant"] }); } As we said before, this is very simple and requires less code, but could cause complications in terms of security logging and maintenance. At the end of the day, it depends on your context and what you are trying to achieve. Therefore, I thought it would be worth mentioning and I will leave the judgement for you on which option you choose. Conclusions We looked at how we could use Azure AD SSO with ADAL to access multiple resources from native mobile apps. As we saw, there are three main options, and the choice could be made based on the context of your app. I hope you find this useful and if you have any questions or you need help with some development that you are doing, then just get in touch. This blog post is the third in a series that cover Azure Active Directory Single Sign-On (SSO) authentication in native mobile applications.

recently some pwc tech supremos wrote an article: agile coding in enterprise it: code small and local . subsections: moving away from the monolith why microservices? msa: a think-small approach for rapid development thinking the msa way: minimalism is a must where msa makes sense in msa, integration is the problem, not the solution conclusion msa is short for microservices architecture(s), in the above article. the article posits that microservices is the antidote to monoliths. it doesn’t mention cookie cutter scaling at all, which is another antidote to monoliths, with the right build infrastructure and devops. here’s a view of hypothetical architecture a company could deploy if they were doing microservices: w is web server. p and q don’t stand for anything in particular. here’s the same solution as cookie-cutter scaling, and the alternate (historical) choice of monolith to the right of it: the cookie cutter approach will often leverage components that are dependency injected into each other, and though monoliths might be the same today, pre 2004 they were probably hairballs of singletons (the design patten, not the springframework idiom). continuous delivery, agile? here’s one excerpt that confuses me: " … makes no sense to design and develop software over an 18-month process to accommodate all possible use cases when those use cases can change unexpectedly and the life span of code modules might be less than 18 months…. as i recall, the 18 month-delay problem was solved previously. agile methodologies principally, and continuous delivery/deployment in more recent times. it does not matter whether you’re compiling a monolith, a cookie-cutter solution, old soa services, or microservices, the 18-month fear isn’t real if you’re doing agile and/or cd. agile and cd were increasing the release cadence, and allowing the organization to pivot faster before microservices. it doesn’t matter whether you’ve got a monolith, something cookie-cutter scaled, or soa (micro or not), you’re going to be able to benefit from agile practices and devops setup that facilitates cd. in something like 30 thoughtworks client engagements since 2002, i have not seen the 18-month process at all. in fact i last encountered it in 1997 on an as/400 project, which was the last time i saw a waterfall process being championed. build(s) and trunk elsewhere there is a suggestion: “each microservice [has] its own build, to avoid trunk conflict”. that isn’t unique to microservices, of course. component based systems today also have a multiple build file (module) structure in a source tree. hopefully “trunk” mentioned is alluding to trunk based development, as i would recommend. build technologies this is a expansion on the above, and you can skip this paragraph if you want. hierarchical build systems like maven has allow you to have one build file per module (whether that’s a service or a simple jar destined for the classpath of a bigger thing). buck has a build grammar that allows for a build to grow/shrink/change based on what is being built (from implicitly shared source). maven is for the java ecosystem, while buck promises to be multi-language. both are doing multi-module builds for the sake of a composed or servicified deployment. both maven and buck are presently competing to draw the most reduced set of compile/test/deploy operations for the changes since last build for a hierarchy of modules. anyway, what is it we are striving for? what we want is to develop cheaply, and to deploy smoothly and often, without defect. we want the ability to deploy without large permanent or temporary headcount overseeing or participating in deployment. aside from development costs, and support/operation, deployment costs are a potentially big factor in total cost of ownership. what i like about cookie-cutter is the uniformity of the deployable things. the team size for deployment of such a thing doesn’t grow with the numbers of nodes that binary is being deployed to. at least, if you’re able to automate the deployment to those nodes, and have a strategy for handling the users connected to the stack at redeployment time somehow (sessions or stateless). the uniformity of the deployment is a cheapener, i think. when you have a number of dissimilar services, you might be able to minimize release personnel if you’re only doing one service. if more than one service is being updated in a particular deployment, you’re going to have to concentrate to make sure you don’t experience a multiplier effect for the participants. it is possible of course, to keep the headcount small, but the practice needed beforehand is bigger, which in turn allows for some calmness around the actual deployment. if we’ve stepped away from the project management office thinking that suggests three buggy releases a year (which is more usual than 18 month schedules of old), then we can employ continuous deployment to further eliminate personnel costs around going live. this is something that microservices does well at, but because the most adept proponents design forwards & backwards compatibility into the permutations most likely to co-exist in production. it is at least much quicker to redeploy and bounce one small service, n times than the the cookie-cutter uniform deployment.

both high availability (ha) and disaster recovery (dr) have been essential it topics. fundamentally ha is about fault tolerance relevant to the availability of an examined subject like application, database, vms, etc. while dr roots on the ability to resume operations in the aftermath of a catastrophic event. a fundamental difference of these two is that ha expects no down time and no data loss, while dr does. they are different issues and should be addressed separately. background for many it shops, either ha or dr has been a high risk and high cost item. both are essential to business continuity, while traditionally tough technical problems to solve with very significant and long-term commitments on resources. not only they are technically challenging, but a continual cost-cutting which has become an it standard practice in the past two decades makes purchasing hardware/software and constructing either ha or dr solution on premises further distant from it’s financial and technical realties. sense of urgency too often, the technical challenges and resource commitments overwhelm it and turn ha and dr into academic discussions, or symbolic items on a project checklist. at the same time, information is rapidly exploding as internet, mobility and social-network are becoming integral in our daily lives and businesses. there are progressively more data to process and store. for many businesses, the needs for ha and dr is urgent for better managing risks. and continual availability and on-demand recoverability of it are becoming increasingly critical. this is the reality, now the good news is that the recent introduction of cloud computing has fundamentally changed how an ha or dr solution can be implemented. microsoft azure is a vivid example of ha and dr solutions with significantly reduced the required financial commitment and involved technical complexities. the traditional approach by establishing redundancy and acquiring a physical dr site with long-term resources and financial commitments is now largely replaced with consumable services which can be configured in minutes by mouse-clicking and with a manageable cost structure based on usage. ha and dr have become it solutions which are financially realistic and technically feasible for businesses in all sizes. ha, redundancy, and microsoft azure lrs ha is to eliminate a single point of failure of an examined component, an application for example. it denotes a strategy to employ redundancy such that a target application can and will continue being available without downtime while experiencing a failure of hosting hardware or software. there are various and well-developed ha solutions like a hyper-v host cluster using redundant hardware to eliminate a single point of failure of hosting os or hardware, and an application cluster for eliminating a single point of failure by running the application in multiple vm instances with a synchronous state. although ha implementations may vary, the fundamental principle nevertheless remains the same. ha expects neither downtime nor data loss while experiencing an outage of a target hardware or software. ha has become dramatically simple in microsoft azure. basically, all data written to disk in microsoft azure are kept at least in the so-called lrs, locally redundant storage. lrs replicates a transaction synchronously to three different storage nodes across fault domains and upgrade domains within the same region for durability. in layman’s terms, microsoft azure by default maintains at least three copies of user data to achieve ha. dr, replication, and microsoft azure grs dr is about having a plan and backups in place to resume operations in the aftermath of a catastrophic event. unplanned outage is assumed in a dr scenario, therefore some data loss is also expected. notice that ha and dr are different business problems and addressed differently. while both ha and dr are based on applying redundancy, i.e. a source and replicas, or multiple identical nodes of an examines component like application instance, databases, or vms, there are however differences between the two. a dr solution generally employs replicas or backups, are implemented with asynchronous processes, and expects an outage of a source and with some data loss in transit while the outage occurs. while ha requires a logical representation with a real-time integrity using synchronous processes across all participating nodes, expects neither downtime nor data loss while experiencing an outage of a participating node. for a critical workload, one approach of dr is to establish geo-replication to address an outage of an entire geographic area caused by a natural disaster, for example. the concern is that a catastrophic event may impact an entire geographic area causing a datacenter where a mission critical application is being hosted becomes unavailable for an extended period of time. in microsoft azure, geo redundant storage or grs is the default and an optional setting, as shown above, while configuring a storage account. grs will queue a transaction committed to lrs as an asynchronous replication to a secondary region, a few hundreds miles away from the primary region where a storage account is originated. at the secondary region, data is also stored in lrs, i.e. made durable by replicating it to three storage nodes. specifically, a microsoft azure storage account configured with grs essentially maintains three replicas locally for high availability, and replicates the content and maintains three replicas at a secondary datacenter a few hundreds miles away for dr. so all are six copies, three locally and three remotely. all these are configured by one, yes one mouse click from a dropdown list while creating a storage account. the above is a conceptual model illustrated a data flow of grs. grs replication has little performance impact on an application since application data are committed to lrs in real-time while replication to grs is queued, i.e. asynchronously. a write to lrs is synchronous and in real-time, once committed, the changes are expected within 15 minutes to be asynchronously replicated to the secondary site. for a ra-grs storage account, in addition to one primary endpoint for read/write operations as it is in a grs, there is also one secondary endpoint as read only becomes available as shown below. the cost implications of grs or ra-grs include the additional storage and the transmission costs for egress traffic, as applicable, of the secondary datacenter. ingress traffic is free . and microsoft azure storage sla offers 99.9% availability and a cost calculator is also available. microsoft azure recovery services so far, much is about backing up or replicating data. to successfully restore, a dr plan must be put in place and ensure its availability upon a dr scenario in progress. either placing a dr plan at a primary site where the source is or a secondary site where a replica stays has some issues and concerns. keeping a dr plan at the source site where all the resources are in place and on-the-job trainings seems logical. or does it? dr is assuming a catastrophic event over an extended geographic areas where the source site is experiencing an outage. in such case, keeping a dr plan in the source site defeats the purpose. maintaining a dr plan at the secondary site is the choice then. in a dr scenario, a recovery site is to be brought on line within a expected period of time according to a dr plan, and having the dr plan right there and then at a recovery site makes all the sense. or does it? this decision introduces a number of requirements including the physical readiness, the timeliness, and the financial implications on securing and maintaining a dr plan at a remote physical facility. for a vmm server running on system center 2012 sp1 or later, an idea, reliable and straightforward way is to use azure recovery services to maintain a dr plan as shown below. and for any backup needs, using cloud as a backup site makes backing up and restoring data an anytime anywhere operation. azure site recovery vault this service essentially acts as the director of a dr process. it orchestrates and manages the protection and failover of vms in clouds managed by virtual machine manager 2012 sp1 or later. a noticeable advantage is the ability to test a recovery configuration, exercise a proactive failover and recovery, and automate recovery in the event of a site outage. the sla of site recovery services is 99.9% availability to ensure a configured dr plan is always in place with expected updates. this is a dr solution that it can implement, simulate, verify, bring online and be absolutely confident with the readiness. azure backup vault this is a reliable, scalable and inexpensive data protection solution with zero capital investment and extremely low operational expense. like other secure communication with microsoft azure, you will first upload a public certificate to microsoft azure. then download the backup agent to register a target server with the backup vault. then select what to be backed up. both microsoft azure backup sla (99.9% availability) and cost calculator are available for better assessing the solution. closing thoughts form an application’s view, ha is an on-going event while dr is an anticipation. ha and dr are different business problems and should be addressed differently. nevertheless, microsoft azure provides a single platform to gracefully address ha with lrs, dr with grs, and dr orchestration with recovery services, and all with published sla s and a predictable cost structure . going forward, it pros can now include ha and dr as a reliable, scalable and relatively inexpensive proposition by employing microsoft azure as a solution platform. call to action register at microsoft virtual academy, http://aka.ms/mva1 , and train yourself on microsoft azure by taking the track of courses. go to http://aka.ms/azure200 and acquire a free trial subscription and assess microsoft azure for ha and dr solutions. review my recommended content at http://aka.ms/recommended .

This article represents command set in R programming language, which could be used to extract rows and columns from a given data frame.

When I needed to do prototyping, proof of concept or play with some new technology in free time, starting new project was always a little annoying barrier with Maven. Have to say that setting up Maven project is not hard and you can use Maven Archetypes. But Archetypes are often out of date. Who wants to play with old technologies? So I always end up wiring in dependencies I wanted to play with. Not very productive spent time. But than Spring Boot came to my way. I fell in love. In last few months I created at least 50 small playground projects, prototypes with Spring Boot. Also incorporated it at work. It’s just perfect for prototyping, learning, microservices, web, batch, enterprise, message flow or command line applications. You have to be dinosaur or be blind not to evaluate Spring Boot for your next Spring project. And when you finish evaluate it, you will go for it. I promise. I feel a need to highlight how easy is Black Box Testing of Spring Boot microservice. Black Box Testing refers to testing without any poking with application artifact. Such testing can be called also integration testing. You can also perform performance or stress testing way I am going to demonstrate. Spring Boot Microservice is usually web application with embedded Tomcat. So it is executed as JAR from command line. There is possibility to convert Spring Boot project into WAR artifact, that can be hosted on shared Servlet container. But we don’t want that now. It’s better when microservice has its own little embedded container. I used existing Spring’s REST service guide as testing target. Focus is mostly on testing project, so it is handy to use this “Hello World” REST application as example. I expect these two common tools are set up and installed on your machine: Maven 3 Git So we’ll need to download source code and install JAR artifact into our local repository. I am going to use command line to download and install the microservice. Let’s go to some directory where we download source code. Use these commands: git clone [email protected]:spring-guides/gs-rest-service.git cd gs-rest-service/complete mvn clean install If everything went OK, Spring Boot microservice JAR artifact is now installed in our local Maven repository. In serious Java development, it would be rather installed into shared repository (e.g. Artifactory, Nexus,… ). When our microservice is installed, we can focus on testing project. It is also Maven and Spring Boot based. Black box testing will be achieved by downloading the artifact from Maven repository (doesn’t matter if it is local or remote). Maven-dependency-plugin can help us this way: org.apache.maven.plugins maven-dependency-plugin copy-dependencies compile copy-dependencies gs-rest-service true It downloads microservice artifact into target/dependency directory by default. As you can see, it’s hooked to compile phase of Maven lifecycle, so that downloaded artifact is available during test phase. Artifact version is stripped from version information. We use latest version. It makes usage of JAR artifact easier during testing. Readers skilled with Maven may notice missing plugin version. Spring Boot driven project is inherited from parent Maven project called spring-boot-starter-parent. It contains versions of main Maven plugins. This is one of the Spring Boot’s opinionated aspects. I like it, because it provides stable dependencies matrix. You can change the version if you need. When we have artifact in our file system, we can start testing. We need to be able to execute JAR file from command line. I used standard JavaProcessBuilder this way: public class ProcessExecutor { public Process execute(String jarName) throws IOException { Process p = null; ProcessBuilder pb = new ProcessBuilder("java", "-jar", jarName); pb.directory(new File("target/dependency")); File log = new File("log"); pb.redirectErrorStream(true); pb.redirectOutput(Redirect.appendTo(log)); p = pb.start(); return p; } } This class executes given process JAR based on given file name. Location is hard-coded to target/dependency directory, where maven-dependency-plugin located our artifact. Standard and error outputs are redirected to file. Next class needed for testing is DTO (Data transfer object). It is simple POJO that will be used for deserialization from JSON. I use Lombok project to reduce boilerplate code needed for getters, setters, hashCode and equals. @Data @AllArgsConstructor @NoArgsConstructor public class Greeting { private long id; private String content; } Test itself looks like this: public class BlackBoxTest { private static final String RESOURCE_URL = "http://localhost:8080/greeting"; @Test public void contextLoads() throws InterruptedException, IOException { Process process = null; Greeting actualGreeting = null; try { process = new ProcessExecutor().execute("gs-rest-service.jar"); RestTemplate restTemplate = new RestTemplate(); waitForStart(restTemplate); actualGreeting = restTemplate.getForObject(RESOURCE_URL, Greeting.class); } finally { process.destroyForcibly(); } Assert.assertEquals(new Greeting(2L, "Hello, World!"), actualGreeting); } private void waitForStart(RestTemplate restTemplate) { while (true) { try { Thread.sleep(500); restTemplate.getForObject(RESOURCE_URL, String.class); return; } catch (Throwable throwable) { // ignoring errors } } } } It executes Spring Boot microservice process first and wait unit it starts. To verify if microservice is started, it sends HTTP request to URL where it’s expected. The service is ready for testing after first successful response. Microservice should send simple greeting JSON response for HTTP GET request. Deserialization from JSON into our Greeting DTO is verified at the end of the test. Source code is shared on Github.

This article represents the first Proof of Concept from series described in the previous article 4 Hands-On Approaches to Improve Your Data Access Layer Implementation and it presents how to implement Ehcache over MyBatis, how to achieve an optim configuration for it and personal opinions of the author about the chosen approach for the Data Access Layer. Throughout my research on caching over MyBatis I have discovered that Ehcache is the first option among developers when they need to implement a cache mechanism over MyBatis, using a 3rd party library. Ehcache is probably so popular because it represents an open source, java-based cache, available under an Apache 2 license. Also, it scales from in-process with one or more nodes through to a mixed in-process/out-of-process configuration with terabyte-sized caches. In addition, for those applications needing a coherent distributed cache, Ehcache uses the open source Terracotta Server Array. Last but not least, among its adopters is the Wikimedia Foundation that uses Ehcache to improve the performance of its wiki projects. Within this article, the following aspects will be addressed: 1. How will an application benefit from caching using Ehcache? Ehcache's features will be detailed in this section. 2. Hands-on implementation of the EhCachePOC project - in this section the key concepts of EhCache will be explored through a hands on implementation. 3. Summary - How has the application performance been improved after this implementation? Code of all the projects that will be implemented can be found at https://github.com/ammbra/CacherPoc or if you are interested only in the current implementation, you can access it here: https://github.com/ammbra/CacherPoc/tree/master/EhCachePoc How will an application benefit from caching using Ehcache? The time taken for an application to process a request principally depends on the speed of the CPU and main memory. In order to "speed up" your application you can perform one or more of the following: improve the algorithm performance achieve parallelisation of the computations across multiple CPUs or multiple machines upgrade the CPU speed As explained in the previous article, high availability applications should perform a small amount of actions with the database. Since the time taken to complete a computation depends principally on the rate at which data can be obtained, then the application should be able to temporarily store computations that may be reused again. Caching may be able to reduce the workload required, this means a caching mechanism should be created! Ehcache is described as : Fast and Light Weight , having a simple API and requiring only a dependency on SLF4J. Scalable to hundreds of nodes with the Terracotta Server Array, but also because provides Memory and Disk store for scalability into gigabytes Flexible because supports Object or Serializable caching; also provides LRU, LFU and FIFO cache eviction policies Standards Based having a full implementation of JSR107 JCACHE API Application Persistence Provider because it offers persistent disk store which stores data between VM restarts JMX Enabled Distributed Caching Enabler because it offers clustered caching via Terracotta and replicated caching via RMI, JGroups, or JMS Cache Server (RESTful, SOAP cache Server) Search Compatible, having a standalone and distributed search using a fluent query language Hands-on implementation of the EhCachePOC project The implementation of EhCachePoc will look as described in the diagram below: In order to test Ehcache performance through a POC(proof of concept) project the following project setup is performed: 1. Create a new Maven EJB Project from your IDE (this kind of project is platform provided by NetBeans but for those that use eclipse, here is an usefull tutorial) . In the article this project is named EhCachePOC. 2. Edit the project's pom by adding required jars : org.mybatis mybatis 3.2.6 org.mybatis.caches mybatis-ehcache 1.0.2 log4j log4j 1.2.17 net.sf.ehcache ehcache 2.7.0 org.slf4j slf4j-log4j12 1.7.5 3.Add your database connection driver, in this case apache derby: org.apache.derby derbyclient 10.11.1.1 4. Run mvn clean and mvn install commands on your project. Now the project setup is in place, let's go ahead with MyBatis implementation : 1. Configure under resources/com/tutorial/ehcachepoc/xml folder the Configuration.xml file with : 2. Create in java your own SQLSessionFactory implementation. For example, create something similar to com.tutorial.ehcachepoc.config. SQLSessionFactory : public class SQLSessionFactory { private static final SqlSessionFactory FACTORY; static { try { Reader reader = Resources.getResourceAsReader("com/tutorial/ehcachepoc/xml/Configuration.xml"); FACTORY = new SqlSessionFactoryBuilder().build(reader); } catch (Exception e){ throw new RuntimeException("Fatal Error. Cause: " + e, e); } } public static SqlSessionFactory getSqlSessionFactory() { return FACTORY; } } 3. Create the necessary bean classes, those that will map to your sql results, like Employee: public class Employee implements Serializable { private static final long serialVersionUID = 1L; private Integer id; private String firstName; private String lastName; private String adress; private Date hiringDate; private String sex; private String phone; private int positionId; private int deptId; public Employee() { } public Employee(Integer id) { this.id = id; } @Override public String toString() { return "com.tutorial.ehcachepoc.bean.Employee[ id=" + id + " ]"; } } 4. Create the IEmployeeDAO interface that will expose the ejb implementation when injected: public interface IEmployeeDAO { public List getEmployees(); } 5. Implement the above inteface and expose the implementation as a Stateless EJB (this kind of EJB preserves only its state, but there is no need to preserve its associated client state): @Stateless(name = "ehcacheDAO") @TransactionManagement(TransactionManagementType.CONTAINER) public class EmployeeDAO implements IEmployeeDAO { private static Logger logger = Logger.getLogger(EmployeeDAO.class); private SqlSessionFactory sqlSessionFactory; @PostConstruct public void init() { sqlSessionFactory = SQLSessionFactory.getSqlSessionFactory(); } @Override public List getEmployees() { logger.info("Getting employees....."); SqlSession sqlSession = sqlSessionFactory.openSession(); List results = sqlSession.selectList("retrieveEmployees"); sqlSession.close(); return results; } } 5. Create the EmployeeMapper.xml that contains the query named "retrieveEmployees" select id, first_name, last_name, hiring_date, sex, dept_id from employee If you remember the CacherPOC setup from the previously article, then you can test your implementation if you add EhCachePOC project as dependency and inject the IEmployeeDAO inside the EhCacheServlet. Your CacherPOC pom.xml file should contain : ${project.groupId} EhCachePoc ${project.version} and your servlet should look like: @WebServlet("/EhCacheServlet") public class EhCacheServlet extends HttpServlet { private static Logger logger = Logger.getLogger(EhCacheServlet.class); @EJB(beanName ="ehcacheDAO") IEmployeeDAO employeeDAO; private static final String LIST_USER = "/listEmployee.jsp"; @Override protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { String forward= LIST_USER; List results = new ArrayList(); for (int i = 0; i < 10; i++) { for (Employee emp : employeeDAO.getEmployees()) { logger.debug(emp); results.add(emp); } try { Thread.sleep(3000); } catch (Exception e) { logger.error(e, e); } } req.setAttribute("employees", results); RequestDispatcher view = req.getRequestDispatcher(forward); view.forward(req, resp); } } Run your CacherPoc implementation to check if your Data Access Layer with MyBatis is working or download the code provided at https://github.com/ammbra/CacherPoc But if a great amount of employees is stored in database, or perhaps the retrieval of a number of 10xemployeesNo represents a lot of workload for the database. Also, can be noticed that the query from the EmployeeMapper.xml retrieves data that almost never changes (id, first_name, last_name, hiring_date, sex cannot change; the only value that might change in time is dept_id); so a caching mechanism can be used. Below is described how this can be achieved using EhCache: 1. Configure directly under the resources folder the ehcache.xml file with: This xml explains that the Memory Store is used for an LRU (Last Recently Used) caching strategy, sets the limits for the number of elements allowed for storage, their time to be idle and their time to live. The Memory Store strategy is often chosen because is fast and thread safe for use by multiple concurrent threads, being backed by LinkedHashMap. Also, all elements involved in the caching process are suitable for placement in the Memory Store. Another approach can be tried: storing cache on disk. This can be done by replacing the ehcache tag content with: diskStore path="F:\\cache" /> Unlike the memory store strategy, the disk store implementation is suitable only for elements which are serializable can be placed in the off-heap; if any non serializable elements are encountered, those will be removed and WARNING level log message emitted. The eviction is made using the LFU algorithm and it is not configurable or changeable. From persistency point of view, this method of caching allows control of the cache by the disk persistent configuration; if false or omitted, disk store will not persist between CacheManager restarts. 2. Update EmployeeMapper.xml to use the previous implemented caching strategy: select id, first_name, last_name, hiring_date, sex, dept_id from employee By adding the line and specifying on the query useCache="true" you are binding the ehcache.xml configuration to your DataAccessLayer implementation. Clean, build and redeploy both EhCachePOC and CacherPoc projects; now retrieve your employees for two times in order to allow the in-memory cache to store your values. When you run your query for the first time, your application will execute the query on the database and retrieve the results. Second time you access the employee list, your application will access the in-memory storage. Summary - How has the application performance been improved after this implementation? An application's performances depend on a multitude of factors how many times a cached piece of data can and is reduced by the application the proportion of the response time that is alleviated by caching Amdhal's law can be used to estimate the system's speed up : where P is proportion speed up and S is speed up. Let's take the application from this article as example and calculate the speed up. When the application ran the query without caching,a JDBC transaction is performed and in your log will be something similar to : INFO: 2014-11-27 18:01:30,020 [EmployeeDAO] INFO com.tutorial.hazelcastpoc.dao.EmployeeDAO:38 - Getting employees..... INFO: 2014-11-27 18:01:39,148 [JdbcTransaction] DEBUG org.apache.ibatis.transaction.jdbc.JdbcTransaction:98 - Setting autocommit to false on JDBC Connection [org.apache.derby.client.net.NetConnection40@1c374fd] INFO: 2014-11-27 18:01:39,159 [retrieveEmployees] DEBUG com.tutorial.hazelcastpoc.mapper.EmployeeMapper.retrieveEmployees:139 - ==> Preparing: select id, first_name, last_name, hiring_date, sex, dept_id from employee INFO: 2014-11-27 18:01:39,220 [retrieveEmployees] DEBUG com.tutorial.hazelcastpoc.mapper.EmployeeMapper.retrieveEmployees:139 - ==> Parameters: INFO: 2014-11-27 18:01:39,316 [retrieveEmployees] DEBUG com.tutorial.hazelcastpoc.mapper.EmployeeMapper.retrieveEmployees:139 - <== Total: 13 while running the queries with Ehcache caching the JDBC transaction is performed only once (to initialize the cache) and after that the log will look like : INFO: 2014-11-28 18:04:50,020 [EmployeeDAO] INFO com.tutorial.ehcachepoc.dao.EmployeeDAO:38 - Getting employees..... INFO: 2014-11-28 18:04:50,020 [EhCacheServlet] DEBUG com.tutorial.cacherpoc.EhCacheServlet:41 - com.tutorial.crudwithjsp.model.Employee[ id=1 ] Let's look at the time that each of our 10 times requests has scored: the first not cached version of 10 times requests took about 57 seconds and 51 milliseconds, while the cached requests scored a time of 27seconds and 86 miliseconds. In order to apply Amdhal's law for the system the following input is needed: Un-cached page time: 60 seconds Database time : 58 seconds Cache retrieval time: 28seconds Proportion: 96.6% (58/60) (P) The expected system speedup is thus: 1 / (( 1 – 0.966) + 0.966 / (58/28)) = 1 / (0.034 + 0. 966/2.07) = 2 times system speedup This result can be improved of course, but the purpose of this article was to prove that caching using Ehcache over MyBatis offers a significant improvement to what used to be available before its implementation. Learn more from: MyBatis Documentation MyBatis Ehcache Adapter EhCache website

Recently I’ve worked on a part of project where are a lot of entities. As in many other projects with the same feature there was implemented “soft delete” approach. That’s mean that when someone deletes any entity it remains in a database but a special field (e.g. ‘isDeleted’) changes its value to true. As you’ve already guessed in every SELECT operation for this kind of entities we need to apply condition: WHERE isDeleted = false It’s a little bit redundant and boring to append each time this condition to a SQL query. So I started look at solutions which could give me some elegant solution of the problem. Fortunately a colleague of mine have given me a hint how to deal with such cases. The answer is covered behind the Hibernate‘s annotation @Where. Let’s consider how we can decorate an entity with the @Where annotation to avoid extra condition in regular SQL queries: import org.hibernate.annotations.Where; import javax.persistence.*; @Entity @Table @Where(clause = "isDeleted='false'") public class Customer { @Id @GeneratedValue @Column private Integer id; @Column private String name; @Column private Boolean isDeleted; //Getters and setters } Now when you want to select Customer on JPA level you will always get only isDeleted=false records. It’s very convenient when you are working with “soft delete” or any other situation which requires permanent application of some condition. I hope it will be useful for your projects.

Here’s something that stumped me for a while today. I’ve got the following Linq query in my repository (this is using the ORM from DevExpress, XPO, but the basic idea is the same) internal virtual IOrderedQueryable GetMyData(string keyVal) { return (from MyEntity ent in new XPQuery(Context) where ent.Key == keyVal orderby ent.SortCol select end); } The problem I was having was in mocking the return value from this method. One cannot create an interface so I could not create a list of items to return from the mocked method. I finally hit on this magic combination of linq queries that lets me return a set built by hand for the mock. var emptyLst = new List(); var lst = (from d in emptyLst select d).AsQueryable().OrderBy(x => x.Key ); _mockRepo.Setup(r => r.MyMockedEvent).Returns(lst); This seems to work like a charm

I’m a huge Vaadin fan and I’ve created a Github workshop I can demo at conferences. A common issue with such kind of workshops is that attendees have to prepare their workstations in advance… and there’s always a significant part of them that comes with not everything ready. At this point, two options are available to the speaker: either wait for each of the attendee to finish the preparation – too bad for the people who took the time at home to do that, or start anyway – and lose the not-ready part. Given the current buzz around Docker, I thought that could be a very good way to make the workshop preparation quicker – only one step, and hasslefree – no problem regarding the quirks of your operation system. The required steps I ask the attendees are the following: Install Git Install Java, Maven and Tomcat Clone the git repo Build the project (to prepare the Maven repository) Deploy the built webapp Start Tomcat These should directly be automated into Docker. As I wasted much time getting this to work, here’s the tale of my journey in achieving this (be warned, it’s quite long). If you’ve got similar use-cases, I hope it will be useful in you getting things done faster. Starting with Docker The first step was to get to know the basics about Docker. Fortunately, I had the chance to attend a Docker workshop by David Gageot at Duchess Swiss. This included both Docker installation and basics of Dockerfile. I assume readers have likewise a basic understanding of Docker. For those who don’t, I guess browsing the Docker’s official documentation is a nice idea: Installation Dockerfile reference Building my first Dockerfile The Docker image can be built with the following command ran into the directory of the Dockerfile: $ docker build -t vaadinworkshop . The first issues one can encounter when playing with Docker the first time, is to get the following error message: Get http:///var/run/docker.sock/v1.14/containers/json: dial unix /var/run/docker.sock: no such file or directory The reason is because one didn’t export the required environment variables displayed by the boot2docker information message. If you lost the exact data, no worry, just use the shellinit boot2docker parameter: $ boot2docker shellinit Writing /Users/i303869/.docker/boot2docker-vm/ca.pem: Writing /Users/i303869/.docker/boot2docker-vm/cert.pem: Writing /Users/i303869/.docker/boot2docker-vm/key.pem: export DOCKER_HOST=tcp://192.168.59.103:2376 export DOCKER_CERT_PATH=/Users/i303869/.docker/boot2docker-vm Copy-paste the export lines above will solve the issue. These can also be set in one’s .bashrc script as it seems these values seldom change. Next in line is the following error: Get http://192.168.59.103:2376/v1.14/containers/json: malformed HTTP response "x15x03x01x00x02x02" This error message seems to be because of a mismatch between versions of the client and the server. It seems it is because of a bug on Mac OSX when upgrading. For a long term solution, reinstall Docker from scratch; for a quick fix, use the --tls flag with the docker command. As it is quite cumbersome to type it everything, one can alias it: $ alias docker="docker --tls" My last mistake when building the image comes from building the Dockerfile from a not empty directory. Docker sends every file it finds in the directory of the Dockerfile to the Docker container for build: $ docker --tls build -t vaadinworkshop . Sending build context to Docker daemon Too many kB Fix: do not try this at home and start from a directory container the Dockerfile only. Starting from scratch Dockerfiles describe images – images are built as a layered list of instructions. Docker images are designed around single inheritance: one image has to be set a single parent. An image requiring no parent starts from scratch, but Docker provides 4 base official distributions: busybox, debian, ubuntu and centos (operating systems are generally a good start). Whatever you want to achieve, it is necessary to choose the right parent. Given the requirements I set for myself (Java, Maven, Tomcat and Git), I tried to find the right starting image. Many Dockerfiles are already available online on the Docker hub. The browsing app is quite good, but to be really honest, the search can really be improved. My intention was to use the image that matched the most of my requirements, then fill the gap. I could find no image providing Git, but I thought the dgageot/maven Dockerfile would be a nice starting point. The problem is that the base image is a busybox and provides no installer out-of-the-box (apt-get, yum, whatever). For this reason, David uses a lot of curl to get Java 8 and Maven in his Dockerfiles. I foolishly thought I could use a different flavor of busybox that provides the opkg installer. After a while, I accumulated many problems, resolving one heading to another. In the end, I finally decided to use the OS I was most comfortable with and to install everything myself: FROM ubuntu:utopic Scripting Java installation Installing git, maven and tomcat packages is very straightforward (if you don’t forget to use the non-interactive options) with RUN and apt-get: RUN apt-get update && \ apt-get install -y --force-yes git maven tomcat8 Java doesn’t fall into this nice pattern, as Oracle wants you to accept the license. Nice people did however publish it to a third-party repo. Steps are the following: Add the needed package repository Configure the system to automatically accept the license Configure the system to add un-certified packages Update the list of repositories At last, install the package Also add a package for Java 8 system configuration. RUN echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list && \ echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \ apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886 RUN apt-get update && \ apt-get install -y --force-yes oracle-java8-installer oracle-java8-set-default Building the sources Getting the workshop’s sources and building them is quite straightforward with the following instructions: RUN git clone https://github.com/nfrankel/vaadin7-workshop.git WORKDIR /vaadin7-workshop RUN mvn package The drawback of this approach is that Maven will start from a fresh repository, and thus download the Internet the first time it is launched. At first, I wanted to mount a volume from the host to the container to share the ~/.m2/repository folder to avoid this, but I noticed this could only be done at runtime through the -v option as the VOLUME instruction cannot point to a host directory. Starting the image The simplest command to start the created Docker image is the following: $ docker run -p 8080:8080 Do not forget the port forwarding from the container to the host, 8080 for the standard HTTP port. Also, note that it’s not necessary to run the container as a daemon (with the -d option). The added value of that is that the standard output of the CMD (see below) will be redirected to the host. When running as a daemon and wanting to check the logs, one has to execute bash in the container, which requires a sequence of cumbersome manipulations. Configuring and launching Tomcat Tomcat can be launched when starting the container by just adding the following instruction to the Dockerfile: CMD ["catalina.sh", "run"] However, trying to start the container at this point will result in the following error: Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/common/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/common], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/server/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/server], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/shared/classes], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.ClassLoaderFactory validateFile WARNING: Problem with directory [/usr/share/tomcat8/shared], exists: [false], isDirectory: [false], canRead: [false] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina initDirs SEVERE: Cannot find specified temporary folder at /usr/share/tomcat8/temp Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina load WARNING: Unable to load server configuration from [/usr/share/tomcat8/conf/server.xml] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina initDirs SEVERE: Cannot find specified temporary folder at /usr/share/tomcat8/temp Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina load WARNING: Unable to load server configuration from [/usr/share/tomcat8/conf/server.xml] Nov 15, 2014 9:24:18 PM org.apache.catalina.startup.Catalina start SEVERE: Cannot start server. Server instance is not configured. I have no idea why, but it seems Tomcat 8 on Ubuntu is not configured in any meaningful way. Everything is available but we need some symbolic links here and there as well as creating the temp directory. This translates into the following instruction in the Dockerfile: RUN ln -s /var/lib/tomcat8/common $CATALINA_HOME/common && \ ln -s /var/lib/tomcat8/server $CATALINA_HOME/server && \ ln -s /var/lib/tomcat8/shared $CATALINA_HOME/shared && \ ln -s /etc/tomcat8 $CATALINA_HOME/conf && \ mkdir $CATALINA_HOME/temp The final trick is to connect the exploded webapp folder created by Maven to Tomcat’s webapps folder, which it looks for deployments: RUN mkdir $CATALINA_HOME/webapps && \ ln -s /vaadin7-workshop/target/workshop-7.2-1.0-SNAPSHOT/ $CATALINA_HOME/webapps/vaadinworkshop At this point, the Holy Grail is not far away, you just have to browse the URL… if only we knew what the IP was. Since running on Mac, there’s an additional VM beside the host and the container that’s involved. To get this IP, type: $ boot2docker ip The VM's Host only interface IP address is: 192.168.59.103 Now, browsing http://192.168.59.103:8080/vaadinworkshop/ will bring us to the familiar workshop screen: Developing from there Everything works fine but didn’t we just forget about one important thing, like how workshop attendees are supposed to work on the sources? Easy enough, just mount the volume when starting the container: docker run -v /Users//vaadin7-workshop:/vaadin7-workshop -p 8080:8080 vaadinworkshop Note that the host volume must be part of /Users and if on OSX, it must use boot2docker v. 1.3+. Unfortunately, it seems now is the showstopper, as mounting an empty directory from the host to the container will not make the container’s directory available from the host. On the contrary, it will empty the container’s directory given that the host’s directory doesn’t exist… It seems there’s an issue in Docker on Mac. The installation of JHipster runs into the same problem, and proposes to use the Samba Docker folder sharing project. I’m afraid I was too lazy to go further at this point. However, this taught me much about Docker, its usages and use-cases (as well as OSX integration limitations). For those who are interested, you’ll find below the Docker file. Happy Docker! FROM ubuntu:utopic MAINTAINER Nicolas Frankel # Config to get to install Java 8 w/o interaction RUN echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list && echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EEA14886 RUN apt-get update && apt-get install -y --force-yes git oracle-java8-installer oracle-java8-set-default maven tomcat8 RUN git clone https://github.com/nfrankel/vaadin7-workshop.git WORKDIR /vaadin7-workshop RUN git checkout v7.2-1 RUN mvn package ENV JAVA_HOME /usr/lib/jvm/java-8-oracle ENV CATALINA_HOME /usr/share/tomcat8 ENV PATH $PATH:$CATALINA_HOME/bin # Configure Tomcat 8 directories RUN ln -s /var/lib/tomcat8/common $CATALINA_HOME/common && ln -s /var/lib/tomcat8/server $CATALINA_HOME/server && ln -s /var/lib/tomcat8/shared $CATALINA_HOME/shared && ln -s /etc/tomcat8 $CATALINA_HOME/conf && mkdir $CATALINA_HOME/temp && mkdir $CATALINA_HOME/webapps && ln -s /vaadin7-workshop/target/workshop-7.2-1.0-SNAPSHOT/ $CATALINA_HOME/webapps/vaadinworkshop VOLUME ["/vaadin7-workshop"] CMD ["catalina.sh", "run"] # docker build -t vaadinworkshop . # docker run -v ~/vaadin7-workshop training/webapp -p 8080:8080 vaadinworkshop

MongoDB provides a lot of query selectors for filtering documents from a collection. Writing complex queries for MongoDB in Java can be tricky sometimes. Consider below data present in student_marks collection {"sid" : 1,"fname" : "Tom","lname" : "Ford","marks" : [ {"english" : 48}, {"maths" : 49}, {"science" : 50}]} {"sid" : 2,"fname" : "Tim","lname" : "Walker","marks" : [ {"english" : 35}, {"maths" : 42}, {"science" : 37}]} {"sid" : 3,"fname" : "John","lname" : "Ward","marks" : [ {"english" : 45}, {"maths" : 41}, {"science" : 37}]} If we want to get students whose last name is Ford and have obtained more than 35 marks in english then the MongoDB shell command for this will be - db.student_marks.find({$and:[{"lname":"Ford"},{"marks.english": {$gt:35}]}) The same query written in Java will look something like this - DBObject query = new BasicDBObject(); List andQuery = new ArrayList(); andQuery.add(new BasicDBObject("lname", "Ford")); andQuery.add(new BasicDBObject("marks.english", new BasicDBObject("$gt", 35))); query.put("$and", andQuery); Using MongoDB QueryBuilder we can rewrite above query as - DBObject query = new QueryBuilder() .start() .and(new QueryBuilder().start().put("lname").is("Ford").get(), new QueryBuilder().start().put("marks.english") .greaterThan(35).get()).get(); You can see that by using QueryBuilder we can write complex queries with ease. QueryBuilder class provides many methods like and, not, greaterThan, exists, etc. which helps in writing MongoDB queries more efficiently and less prone to error/mistakes. If you enjoyed this article and want to learn more about MongoDB, check out this collection of tutorials and articles on all things MongoDB.

there is currently a strong trend for microservice based architectures and frequent discussions comparing them to monoliths. there is much advice about breaking-up monoliths into microservices and also some amusing fights between proponents of the two paradigms - see the great microservices vs monolithic melee . the term 'monolith' is increasingly being used as a generic insult in the same way that 'legacy' is! however, i believe that there is a great deal of misunderstanding about exactly what a 'monolith' is and those discussing it are often talking about completely different things. a monolith can be considered an architectural style or a software development pattern (or anti-pattern if you view it negatively). styles and patterns usually fit into different viewtypes (a viewtype is a set, or category, of views that can be easily reconciled with each other [clements et al., 2010]) and some basic viewtypes we can discuss are: module - the code units and their relation to each other at compile time. allocation - the mapping of the software onto its environment. runtime - the static structure of the software elements and how they interact at runtime. a monolith could refer to any of the basic viewtypes above. module monolith if you have a module monolith then all of the code for a system is in a single codebase that is compiled together and produces a single artifact. the code may still be well structured (classes and packages that are coherent and decoupled at a source level rather than a big-ball-of-mud) but it is not split into separate modules for compilation. conversely a non-monolithic module design may have code split into multiple modules or libraries that can be compiled separately, stored in repositories and referenced when required. there are advantages and disadvantages to both but this tells you very little about how the code is used - it is primarily done for development management. allocation monolith for an allocation monolith, all of the code is shipped/deployed at the same time. in other words once the compiled code is 'ready for release' then a single version is shipped to all nodes. all running components have the same version of the software running at any point in time. this is independent of whether the module structure is a monolith. you may have compiled the entire codebase at once before deployment or you may have created a set of deployment artifacts from multiple sources and versions. either way this version for the system is deployed everywhere at once (often by stopping the entire system, rolling out the software and then restarting). a non-monolithic allocation would involve deploying different versions to individual nodes at different times. this is again independent of the module structure as different versions of a module monolith could be deployed individually. runtime monolith a runtime monolith will have a single application or process performing the work for the system (although the system may have multiple, external dependencies). many systems have traditionally been written like this (especially line-of-business systems such as payroll, accounts payable, cms etc). whether the runtime is a monolith is independent of whether the system code is a module monolith or not. a runtime monolith often implies an allocation monolith if there is only one main node/component to be deployed (although this is not the case if a new version of software is rolled out across regions, with separate users, over a period of time). note that my examples above are slightly forced for the viewtypes and it won't be as hard-and-fast in the real world. conclusion be very carefully when arguing about 'microservices vs monoliths'. a direct comparison is only possible when discussing the runtime viewtype and properties. you should also not assume that moving away from a module or allocation monolith will magically enable a microservice architecture (although it will probably help). if you are moving to a microservice architecture then i'd advise you to consider all these viewtypes and align your boundaries across them i.e. don't just code, build and distribute a monolith that exposes subsets of itself on different nodes.

There may be cases when your REST api provides responses that are very long, and we all know how important transfer speed and bandwidth still are on mobile devices/networks. I think this is the first performance optimization point one needs to address, when developing REST apis that support mobile apps. Guess what? Because responses are text, we can compress them. And with today’s power of smartphones and tablets uncompressing them on the client side should not be a big deal… So in this post I will present how you can SELECTIVELY compress your REST API responses, if you’ve built it in Java with Jersey, which is the JAX-RS Reference Implementation (and more)… 1. Jersey filters and interceptors Well, thanks to Jersey’s powerful Filters and Interceptors features, the implementation is fairly easy. Whereas filters are primarily intended to manipulate request and response parameters like HTTP headers, URIs and/or HTTP methods, interceptors are intended to manipulate entities, via manipulating entity input/output streams. You’ve seen the power of filters in my posts How to add CORS support on the server side in Java with Jersey, where I’ve shown how to CORS-enable a REST API and How to log in Spring with SLF4J and Logback, where I’ve shown how to log requests and responses from the REST API , but for compressing will be using a GZip WriterInterceptor. A writer interceptor is used for cases where entity is written to the “wire”, which on the server side as in this case, means when writing out a response entity. 1.1. GZip Writer Interceptor So let’s have a look at our GZip Writer Interceptor: package org.codingpedia.demo.rest.interceptors; import java.io.IOException; import java.io.OutputStream; import java.util.zip.GZIPOutputStream; import javax.ws.rs.WebApplicationException; import javax.ws.rs.core.MultivaluedMap; import javax.ws.rs.ext.WriterInterceptor; import javax.ws.rs.ext.WriterInterceptorContext; @Provider @Compress public class GZIPWriterInterceptor implements WriterInterceptor { @Override public void aroundWriteTo(WriterInterceptorContext context) throws IOException, WebApplicationException { MultivaluedMap headers = context.getHeaders(); headers.add("Content-Encoding", "gzip"); final OutputStream outputStream = context.getOutputStream(); context.setOutputStream(new GZIPOutputStream(outputStream)); context.proceed(); } } Note: it implements the WriterInterceptor, which is an interface for message body writer interceptors that wrap around calls to javax.ws.rs.ext.MessageBodyWriter.writeTo providers implementing WriterInterceptor contract must be either programmatically registered in a JAX-RS runtime or must be annotated with @Provider annotation to be automatically discovered by the JAX-RS runtime during a provider scanning phase. @Compress is the name binding annotation, which we will discuss more detailed in the coming paragraph “The interceptor gets a output stream from the WriterInterceptorContext and sets a new one which is a GZIP wrapper of the original output stream. After all interceptors are executed the output stream lastly set to the WriterInterceptorContext will be used for serialization of the entity. In the example above the entity bytes will be written to the GZIPOutputStream which will compress the stream data and write them to the original output stream. The original stream is always the stream which writes the data to the “wire”. When the interceptor is used on the server, the original output stream is the stream into which writes data to the underlying server container stream that sends the response to the client.” [2] “The overridden method aroundWriteTo() gets WriterInterceptorContext as a parameter. This context contains getters and setters for header parameters, request properties, entity, entity stream and other properties.” [2]; when you compress your response you should set the “Content-Encoding” header to “gzip” 1.2. Compress annotation Filters and interceptors can be name-bound. Name binding is a concept that allows to say to a JAX-RS runtime that a specific filter or interceptor will be executed only for a specific resource method. When a filter or an interceptor is limited only to a specific resource method we say that it is name-bound. Filters and interceptors that do not have such a limitation are called global. In our case we’ve built the @Compress annotation: package org.codingpedia.demo.rest.interceptors; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import javax.ws.rs.NameBinding; //@Compress annotation is the name binding annotation @NameBinding @Retention(RetentionPolicy.RUNTIME) public @interface Compress {} and used it to mark methods on resources which should be gzipped (e.g. when GET-ing all the podcasts with the PodcastsResource): @Component @Path("/podcasts") public class PodcastsResource { @Autowired private PodcastService podcastService; ........................... /* * *********************************** READ *********************************** */ /** * Returns all resources (podcasts) from the database * * @return * @throws IOException * @throws JsonMappingException * @throws JsonGenerationException * @throws AppException */ @GET @Compress @Produces({ MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML }) public List getPodcasts( @QueryParam("orderByInsertionDate") String orderByInsertionDate, @QueryParam("numberDaysToLookBack") Integer numberDaysToLookBack) throws IOException, AppException { List podcasts = podcastService.getPodcasts( orderByInsertionDate, numberDaysToLookBack); return podcasts; } ........................... } 2. Testing 2.1. SOAPui Well, if you are testing with SOAPui, you can issue the following request against the PodcastsResource Request: GET http://localhost:8888/demo-rest-jersey-spring/podcasts/?orderByInsertionDate=DESC HTTP/1.1 Accept-Encoding: gzip,deflate Accept: application/json, application/xml Host: localhost:8888 Connection: Keep-Alive User-Agent: Apache-HttpClient/4.1.1 (java 1.5) Response: HTTP/1.1 200 OK Content-Type: application/json Content-Encoding: gzip Content-Length: 409 Server: Jetty(9.0.7.v20131107) [ { "id": 2, "title": "Quarks & Co - zum Mitnehmen", "linkOnPodcastpedia": "http://www.podcastpedia.org/quarks", "feed": "http://podcast.wdr.de/quarks.xml", "description": "Quarks & Co: Das Wissenschaftsmagazin", "insertionDate": "2014-10-29T10:46:13.00+0100" }, { "id": 1, "title": "- The Naked Scientists Podcast - Stripping Down Science", "linkOnPodcastpedia": "http://www.podcastpedia.org/podcasts/792/-The-Naked-Scientists-Podcast-Stripping-Down-Science", "feed": "feed_placeholder", "description": "The Naked Scientists flagship science show brings you a lighthearted look at the latest scientific breakthroughs, interviews with the world top scientists, answers to your science questions and science experiments to try at home.", "insertionDate": "2014-10-29T10:46:02.00+0100" } ] SOAPui recognizes the Content-Type: gzip header, we’ve added in the GZIPWriterInterceptor and automatically uncompresses the response and displays it readable to the human eye. Well, that’s it. You’ve learned how Jersey makes it straightforward to compress the REST api responses. Tip: If you want really learn how to design and implement REST API in Java read the following Tutorial – REST API design and implementation in Java with Jersey and Spring

A reader pinged me yesterday with a simple problem that I thought would be good to share on the blog. He had a query of events that he wanted to use with jQuery UI's Accordion control. The Accordion control simply takes content and splits into various "panes" with one visible at a time. For his data, he wanted to split his content into panes designated by a unique month and year. Here is a quick demo of that in action. I began by creating a query to store my data. I created a query with a date and title property and then random chose to add 0 to 3 "events" over the next twelve months. I specifically wanted to support 0 to ensure my demo handled noticing months without any data. 01. 04. 05.q = queryNew("date,title"); 06.for(i=1; i<12; i++) { 07. //for each month, we add 0-3 events (some months may not have data) 08. toAdd = randRange(0, 3); 09. 10. for(k=0; k To handle creating the accordion, I had to follow the rules jQuery UI set up for the control. Basically - wrap the entire set of data in a div, and separate each "pane" with an h3 and inner div. To handle this, I have to know when a new unique month/year "block" starts. I store this in a variable, lastDateStr, and just check it in every iteration over the query. I also need to ensure that on the last row I close the div. 01. 02. 03. 04. 05. 06. 07. 08. 09. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. #thisDateStr# 30. 31. 32. 33. 34. 35. 36. #title# 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. And the end result: So, not rocket science, but hopefully helpful to someone. Here is the entire template if you want to try it yourself. 01. 04. 05.q = queryNew("date,title"); 06.for(i=1; i<12; i++) { 07. //for each month, we add 0-3 events (some months may not have data) 08. toAdd = randRange(0, 3); 09. 10. for(k=0; k 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. #thisDateStr# 46. 47. 48. 49. 50. 51. 52. #title# 53. 54. 55. 56. 57. 58. 59. 60. 61. 62. 63.

Originally Written by Peiran Song A deadlock in MySQL happens when two or more transactions mutually hold and request for locks, creating a cycle of dependencies. In a transaction system, deadlocks are a fact of life and not completely avoidable. InnoDB automatically detects transaction deadlocks, rollbacks a transaction immediately and returns an error. It uses a metric to pick the easiest transaction to rollback. Though an occasional deadlock is not something to worry about, frequent occurrences call for attention. Before MySQL 5.6, only the latest deadlock can be reviewed using SHOW ENGINE INNODB STATUS command. But with Percona Toolkit’s pt-deadlock-logger you can have deadlock information retrieved from SHOW ENGINE INNODB STATUS at a given interval and saved to a file or table for late diagnosis. For more information on using pt-deadlock-logger, see this post. With MySQL 5.6, you can enable a new variable innodb_print_all_deadlocks to have all deadlocks in InnoDB recorded in mysqld error log. Before and above all diagnosis, it is always an important practice to have the applications catch deadlock error (MySQL error no. 1213) and handle it by retrying the transaction. How to diagnose a MySQL deadlock A MySQL deadlock could involve more than two transactions, but the LATEST DETECTED DEADLOCK section only shows the last two transactions. Also it only shows the last statement executed in the two transactions, and locks from the two transactions that created the cycle. What are missed are the earlier statements that might have really acquired the locks. I will show some tips on how to collect the missed statements. Let’s look at two examples to see what information is given. Example 1: 1 141013 6:06:22 2 *** (1) TRANSACTION: 3 TRANSACTION 876726B90, ACTIVE 7 sec setting auto-inc lock 4 mysql tables in use 1, locked 1 5 LOCK WAIT 9 lock struct(s), heap size 1248, 4 row lock(s), undo log entries 4 6 MySQL thread id 155118366, OS thread handle 0x7f59e638a700, query id 87987781416 localhost msandbox update 7 INSERT INTO t1 (col1, col2, col3, col4) values (10, 20, 30, 'hello') 8 *** (1) WAITING FOR THIS LOCK TO BE GRANTED: 9 TABLE LOCK table `mydb`.`t1` trx id 876726B90 lock mode AUTO-INC waiting 10 *** (2) TRANSACTION: 11 TRANSACTION 876725B2D, ACTIVE 9 sec inserting 12 mysql tables in use 1, locked 1 13 876 lock struct(s), heap size 80312, 1022 row lock(s), undo log entries 1002 14 MySQL thread id 155097580, OS thread handle 0x7f585be79700, query id 87987761732 localhost msandbox update 15 INSERT INTO t1 (col1, col2, col3, col4) values (7, 86, 62, "a lot of things"), (7, 76, 62, "many more") 16 *** (2) HOLDS THE LOCK(S): 17 TABLE LOCK table `mydb`.`t1` trx id 876725B2D lock mode AUTO-INC 18 *** (2) WAITING FOR THIS LOCK TO BE GRANTED: 19 RECORD LOCKS space id 44917 page no 529635 n bits 112 index `PRIMARY` of table `mydb`.`t2` trx id 876725B2D lock mode S locks rec but not gap waiting 20 *** WE ROLL BACK TRANSACTION (1) Line 1 gives the time when the deadlock happened. If your application code catches and logs deadlock errors,which it should, then you can match this timestamp with the timestamps of deadlock errors in application log. You would have the transaction that got rolled back. From there, retrieve all statements from that transaction. Line 3 & 11, take note of Transaction number and ACTIVE time. If you log SHOW ENGINE INNODB STATUS output periodically(which is a good practice), then you can search previous outputs with Transaction number to hopefully see more statements from the same transaction. The ACTIVE sec gives a hint on whether the transaction is a single statement or multi-statement one. Line 4 & 12, the tables in use and locked are only with respect to the current statement. So having 1 table in use does not necessarily mean that the transaction involves 1 table only. Line 5 & 13, this is worth of attention as it tells how many changes the transaction had made, which is the “undo log entries” and how many row locks it held which is “row lock(s)”. These info hints the complexity of the transaction. Line 6 & 14, take note of thread id, connecting host and connecting user. If you use different MySQL users for different application functions which is another good practice, then you can tell which application area the transaction comes from based on the connecting host and user. Line 9, for the first transaction, it only shows the lock it was waiting for, in this case the AUTO-INC lock on table t1. Other possible values are S for shared lock and X for exclusive with or without gap locks. Line 16 & 17, for the second transaction, it shows the lock(s) it held, in this case the AUTO-INC lock which was what TRANSACTION (1) was waiting for. Line 18 & 19 shows which lock TRANSACTION (2) was waiting for. In this case, it was a shared not gap record lock on another table’s primary key. There are only a few sources for a shared record lock in InnoDB: 1) use of SELECT … LOCK IN SHARE MODE 2) on foreign key referenced record(s) 3) with INSERT INTO… SELECT, shared locks on source table The current statement of trx(2) is a simple insert to table t1, so 1 and 3 are eliminated. By checking SHOW CREATE TABLE t1, you could confirm that the S lock was due to a foreign key constraint to the parent table t2. Example 2: With MySQL community version, each record lock has the record content printed: 1 2014-10-11 10:41:12 7f6f912d7700 2 *** (1) TRANSACTION: 3 TRANSACTION 2164000, ACTIVE 27 sec starting index read 4 mysql tables in use 1, locked 1 5 LOCK WAIT 3 lock struct(s), heap size 360, 2 row lock(s), undo log entries 1 6 MySQL thread id 9, OS thread handle 0x7f6f91296700, query id 87 localhost ro ot updating 7 update t1 set name = 'b' where id = 3 8 *** (1) WAITING FOR THIS LOCK TO BE GRANTED: 9 RECORD LOCKS space id 1704 page no 3 n bits 72 index `PRIMARY` of table `tes t`.`t1` trx id 2164000 lock_mode X locks rec but not gap waiting 10 Record lock, heap no 4 PHYSICAL RECORD: n_fields 5; compact format; info bit s 0 11 0: len 4; hex 80000003; asc ;; 12 1: len 6; hex 000000210521; asc ! !;; 13 2: len 7; hex 180000122117cb; asc ! ;; 14 3: len 4; hex 80000008; asc ;; 15 4: len 1; hex 63; asc c;; 16 17 *** (2) TRANSACTION: 18 TRANSACTION 2164001, ACTIVE 18 sec starting index read 19 mysql tables in use 1, locked 1 20 3 lock struct(s), heap size 360, 2 row lock(s), undo log entries 1 21 MySQL thread id 10, OS thread handle 0x7f6f912d7700, query id 88 localhost r oot updating 22 update t1 set name = 'c' where id = 2 23 *** (2) HOLDS THE LOCK(S): 24 RECORD LOCKS space id 1704 page no 3 n bits 72 index `PRIMARY` of table `tes t`.`t1` trx id 2164001 lock_mode X locks rec but not gap 25 Record lock, heap no 4 PHYSICAL RECORD: n_fields 5; compact format; info bit s 0 26 0: len 4; hex 80000003; asc ;; 27 1: len 6; hex 000000210521; asc ! !;; 28 2: len 7; hex 180000122117cb; asc ! ;; 29 3: len 4; hex 80000008; asc ;; 30 4: len 1; hex 63; asc c;; 31 32 *** (2) WAITING FOR THIS LOCK TO BE GRANTED: 33 RECORD LOCKS space id 1704 page no 3 n bits 72 index `PRIMARY` of table `tes t`.`t1` trx id 2164001 lock_mode X locks rec but not gap waiting 34 Record lock, heap no 3 PHYSICAL RECORD: n_fields 5; compact format; info bit s 0 35 0: len 4; hex 80000002; asc ;; 36 1: len 6; hex 000000210520; asc ! ;; 37 2: len 7; hex 17000001c510f5; asc ;; 38 3: len 4; hex 80000009; asc ;; 39 4: len 1; hex 62; asc b;; Line 9 & 10: The ‘space id’ is tablespace id, ‘page no’ gives which page the record lock is on inside the tablespace. The ‘n bits’ is not the page offset, instead the number of bits in the lock bitmap. The page offset is the ‘heap no’ on line 10, Line 11~15: It shows the record data in hex numbers. Field 0 is the cluster index(primary key). Ignore the highest bit, the value is 3. Field 1 is the transaction id of the transaction which last modified this record, decimal value is 2164001 which is TRANSACTION (2). Field 2 is the rollback pointer. Starting from field 3 is the rest of the row data. Field 3 is integer column, value 8. Field 4 is string column with character ‘c’. By reading the data, we know exactly which row is locked and what is the current value. What else can we learn from analysis? Since most MySQL deadlocks happen between two transactions, we could start the analysis based on that assumption. In Example 1, trx (2) was waiting on a shared lock, so trx (1) either held a shared or exclusive lock on that primary key record of table t2. Let’s say col2 is the foreign key column, by checking the current statement of trx(1), we know it did not require the same record lock, so it must be some previous statement in trx(1) that required S or X lock(s) on t2’s PK record(s). Trx (1) only made 4 row changes in 7 seconds. Then you learned a few characteristics of trx(1): it does a lot of processing but a few changes; changes involve table t1 and t2, a single record insertion to t2. These information combined with other data could help developers to locate the transaction. Where else can we find previous statements of the transactions? Besides application log and previous SHOW ENGINE INNODB STATUS output, you may also leverage binlog, slow log and/or general query log. With binlog, if binlog_format=statement, each binlog event would have the thread_id. Only committed transactions are logged into binlog, so we could only look for Trx(2) in binlog. In the case of Example 1, we know when the deadlock happened, and we know Trx(2) started 9 seconds ago. We can run mysqlbinlog on the right binlog file and look for statements with thread_id = 155097580. It is always good to then cross refer the statements with the application code to confirm. $ mysqlbinlog -vvv --start-datetime=“2014-10-13 6:06:12” --stop-datatime=“2014-10-13 6:06:22” mysql-bin.000010 > binlog_1013_0606.out With Percona Server 5.5 and above, you can set log_slow_verbosity to include InnoDB transaction id in slow log. Then if you have long_query_time = 0, you would be able to catch all statements including those rolled back into slow log file. With general query log, the thread id is included and could be used to look for related statements. How to avoid a MySQL deadlock There are things we could do to eliminate a deadlock after we understand it. – Make changes to the application. In some cases, you could greatly reduce the frequency of deadlocks by splitting a long transaction into smaller ones, so locks are released sooner. In other cases, the deadlock rises because two transactions touch the same sets of data, either in one or more tables, with different orders. Then change them to access data in the same order, in another word, serialize the access. That way you would have lock wait instead of deadlock when the transactions happen concurrently. – Make changes to the table schema, such as removing foreign key constraint to detach two tables, or adding indexes to minimize the rows scanned and locked. – In case of gap locking, you may change transaction isolation level to read committed for the session or transaction to avoid it. But then the binlog format for the session or transaction would have to be ROW or MIXED.

In the modern world of microservices it's important to provide strict and polyglot clients for your service. It's better if your API is self-documented. One of the best tools for it is Apache Thrift. I want to explain how to use it with my favorite platform for microservices - Spring Boot. All project source code is available on GitHub: https://github.com/bsideup/spring-boot-thrift Project skeleton I will use Gradle to build our application. First, we need our main build.gradle file: buildscript { repositories { jcenter() } dependencies { classpath("org.springframework.boot:spring-boot-gradle-plugin:1.1.8.RELEASE") } } allprojects { repositories { jcenter() } apply plugin:'base' apply plugin: 'idea' } subprojects { apply plugin: 'java' } Nothing special for a Spring Boot project. Then we need a gradle file for thrift protocol modules (we will reuse it in next part): import org.gradle.internal.os.OperatingSystem repositories { ivy { artifactPattern "http://dl.bintray.com/bsideup/thirdparty/[artifact]-[revision](-[classifier]).[ext]" } } buildscript { repositories { jcenter() } dependencies { classpath "ru.trylogic.gradle.plugins:gradle-thrift-plugin:0.1.1" } } apply plugin: ru.trylogic.gradle.thrift.plugins.ThriftPlugin task generateThrift(type : ru.trylogic.gradle.thrift.tasks.ThriftCompileTask) { generator = 'java:beans,hashcode' destinationDir = file("generated-src/main/java") } sourceSets { main { java { srcDir generateThrift.destinationDir } } } clean { delete generateThrift.destinationDir } idea { module { sourceDirs += [file('src/main/thrift'), generateThrift.destinationDir] } } compileJava.dependsOn generateThrift dependencies { def thriftVersion = '0.9.1'; Map platformMapping = [ (OperatingSystem.WINDOWS) : 'win', (OperatingSystem.MAC_OS) : 'osx' ].withDefault { 'nix' } thrift "org.apache.thrift:thrift:$thriftVersion:${platformMapping.get(OperatingSystem.current())}@bin" compile "org.apache.thrift:libthrift:$thriftVersion" compile 'org.slf4j:slf4j-api:1.7.7' } We're using my Thrift plugin for Gradle. Thrift will generate source to the "generated-src/main/java" directory. By default, Thrift uses slf4j v1.5.8, while Spring Boot uses v1.7.7. It will cause an error in runtime when you will run your application, that's why we have to force a slf4j api dependency. Calculator service Let's start with a simple calculator service. It will have 2 modules: protocol and app.We will start with protocol. Your project should look as follows: calculator/ protocol/ src/ main/ thrift/ calculator.thrift build.gradle build.gradle settings.gradle thrift.gradle Where calculator/protocol/build.gradle contains only one line: apply from: rootProject.file('thrift.gradle') Don't forget to put these lines to settings.gradle, otherwise your modules will not be visible to Gradle: include 'calculator:protocol' include 'calculator:app' Calculator protocol Even if you're not familiar with Thrift, its protocol description file (calculator/protocol/src/main/thrift/calculator.thrift) should be very clear to you: namespace cpp com.example.calculator namespace d com.example.calculator namespace java com.example.calculator namespace php com.example.calculator namespace perl com.example.calculator namespace as3 com.example.calculator enum TOperation { ADD = 1, SUBTRACT = 2, MULTIPLY = 3, DIVIDE = 4 } exception TDivisionByZeroException { } service TCalculatorService { i32 calculate(1:i32 num1, 2:i32 num2, 3:TOperation op) throws (1:TDivisionByZeroException divisionByZero); } Here we define TCalculatorService with only one method - calculate. It can throw an exception of type TDivisionByZeroException. Note how many languages we're supporting out of the box (in this example we will use only Java as a target, though) Now run ./gradlew generateThrift, you will get generated Java protocol source in the calculator/protocol/generated-src/main/java/ folder. Calculator application Next, we need to create the service application itself. Just create calculator/app/ folder with the following structure: src/ main/ java/ com/ example/ calculator/ handler/ CalculatorServiceHandler.java service/ CalculatorService.java CalculatorApplication.java build.gradle Our build.gradle file for app module should look like this: apply plugin: 'spring-boot' dependencies { compile project(':calculator:protocol') compile 'org.springframework.boot:spring-boot-starter-web' testCompile 'org.springframework.boot:spring-boot-starter-test' } Here we have a dependency on protocol and typical starters for Spring Boot web app. CalculatorApplication is our main class. In this example I will configure Spring in the same file, but in your apps you should use another config class instead. package com.example.calculator; import com.example.calculator.handler.CalculatorServiceHandler; import org.apache.thrift.protocol.*; import org.apache.thrift.server.TServlet; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.EnableAutoConfiguration; import org.springframework.context.annotation.*; import javax.servlet.Servlet; @Configuration @EnableAutoConfiguration @ComponentScan public class CalculatorApplication { public static void main(String[] args) { SpringApplication.run(CalculatorApplication.class, args); } @Bean public TProtocolFactory tProtocolFactory() { //We will use binary protocol, but it's possible to use JSON and few others as well return new TBinaryProtocol.Factory(); } @Bean public Servlet calculator(TProtocolFactory protocolFactory, CalculatorServiceHandler handler) { return new TServlet(new TCalculatorService.Processor(handler), protocolFactory); } } You may ask why Thrift servlet bean is called "calculator". In Spring Boot, it will register your servlet bean in context of the bean name and our servlet will be available at /calculator/. After that we need a Thrift handler class: package com.example.calculator.handler; import com.example.calculator.*; import com.example.calculator.service.CalculatorService; import org.apache.thrift.TException; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; @Component public class CalculatorServiceHandler implements TCalculatorService.Iface { @Autowired CalculatorService calculatorService; @Override public int calculate(int num1, int num2, TOperation op) throws TException { switch(op) { case ADD: return calculatorService.add(num1, num2); case SUBTRACT: return calculatorService.subtract(num1, num2); case MULTIPLY: return calculatorService.multiply(num1, num2); case DIVIDE: try { return calculatorService.divide(num1, num2); } catch(IllegalArgumentException e) { throw new TDivisionByZeroException(); } default: throw new TException("Unknown operation " + op); } } } In this example I want to show you that Thrift handler can be a normal Spring bean and you can inject dependencies in it. Now we need to implement CalculatorService itself: package com.example.calculator.service; import org.springframework.stereotype.Component; @Component public class CalculatorService { public int add(int num1, int num2) { return num1 + num2; } public int subtract(int num1, int num2) { return num1 - num2; } public int multiply(int num1, int num2) { return num1 * num2; } public int divide(int num1, int num2) { if(num2 == 0) { throw new IllegalArgumentException("num2 must not be zero"); } return num1 / num2; } } That's it. Well... almost. We still need to test our service somehow. And it should be an integration test. Usually, even if your application is providing JSON REST API, you still have to implement a client for it. Thrift will do it for you. We don't have to care about it. Also, it will support different protocols. Let's use a generated client in our test: package com.example.calculator; import org.apache.thrift.protocol.*; import org.apache.thrift.transport.THttpClient; import org.apache.thrift.transport.TTransport; import org.junit.*; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.*; import org.springframework.boot.test.IntegrationTest; import org.springframework.boot.test.SpringApplicationConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.context.web.WebAppConfiguration; import static org.junit.Assert.*; @RunWith(SpringJUnit4ClassRunner.class) @SpringApplicationConfiguration(classes = CalculatorApplication.class) @WebAppConfiguration @IntegrationTest("server.port:0") public class CalculatorApplicationTest { @Autowired protected TProtocolFactory protocolFactory; @Value("${local.server.port}") protected int port; protected TCalculatorService.Client client; @Before public void setUp() throws Exception { TTransport transport = new THttpClient("http://localhost:" + port + "/calculator/"); TProtocol protocol = protocolFactory.getProtocol(transport); client = new TCalculatorService.Client(protocol); } @Test public void testAdd() throws Exception { assertEquals(5, client.calculate(2, 3, TOperation.ADD)); } @Test public void testSubtract() throws Exception { assertEquals(3, client.calculate(5, 2, TOperation.SUBTRACT)); } @Test public void testMultiply() throws Exception { assertEquals(10, client.calculate(5, 2, TOperation.MULTIPLY)); } @Test public void testDivide() throws Exception { assertEquals(2, client.calculate(10, 5, TOperation.DIVIDE)); } @Test(expected = TDivisionByZeroException.class) public void testDivisionByZero() throws Exception { client.calculate(10, 0, TOperation.DIVIDE); } } This test will run your Spring Boot application, bind it to a random port and test it. All client-server communications will be performed in the same way real world clients are. Note how easy to use our service is from the client side. We're just calling methods and catching exceptions.

daniel bryant , simon and i recently had a discussion about how to represent system communication with external apis. the requirement for integration with external apis is now extremely common but it's not immediately obvious how to clearly show them in architectural diagrams. how to represent an external system? the first thing we discussed was what symbol to use for a system supplying an api. traditionally, uml has used the actor (stick man) symbol to represent a "user or any other system that interacts with the subject" (uml superstructure specification, v2.1.2). therefore a system providing an api may look like this: i've found that this symbol tends to confuse those who aren't well versed in uml as most people assume that the actor symbol always represents a *person* rather than a system. sometimes this is stereotyped to make it more obvious e.g. however the symbol is very powerful and tends to overpower the stereotype. therefore i prefer to use a stereotyped box for an external system supplying an api. let's compare two context diagrams using boxes vs stick actors. in which diagram is it more obvious what are systems or people? note that archimate has a specific symbol for application service that can be used to represent an api: (application service notation from the open group's archimate 2.1 specification) an api or the system that supplies it? whatever symbol we choose, what we've done is to show the *system* rather than the actual api. the api is a definition of a service provided by the system in question. how should we provide more details about the api? there are a number of ways we could do this but my preference is to give details of the api on the connector (line connecting two elements/boxes). in c4 the guidelines for a container diagram includes listing protocol information on the connector and an api can be viewed as the layer above the protocol. for example: multiple apis per external system many api providers supply multiple services/apis (i'm not referring to different operations within an api but multiple sets of operations in different apis, which may even use different underlying protocols.) for example a financial marketplace may have apis that do the following: allow a bulk, batch download of static data (such as details of companies listed on a stock market) via xml over http. supply real time, low latency updates of market prices via bespoke messages over udp. allow entry of trades via industry standard fpml over a queuing system. supply a bulk, batch download of trades for end-of-day reconciliation via fpml over http. two of the services use the same protocol (xml over http) but have very different content and use. one of the apis is used to constantly supply information after user subscription (market data) and the last service involves the user supplying all the information with no acknowledgment (although it should reconcile at eod). there are multiple ways of showing this. we could: have a single service element, list the apis on it and have all components linking to it. show each service/api as a separate box and connect the components that use the individual service to the relevant box. show a single service element with multiple connections. each connection is labeled and represents an api. use a port and connector style notation to represent each api from the service provider. provide a key for the ports. use a uml style 'cup and ball' notation to define interfaces and their usage. some examples are below: a single service element and simple description in the above diagram the containers are stating what they are using but contain no information about how to use the apis. we don't know if it is a single api (with different operations) or anything about the mechanisms used to transport the data. this isn't very useful for anyone implementing a solution or resolving operational issues. single, service box with descriptive connectors in this diagram there is a single, service box with descriptive connectors. the above diagram shows all the information so is much more useful as a diagnostic or implementation tool. however it does look quite crowded. services/apis shown as separate boxes here the external system has its services/apis shown as separate boxes. this contains all the information but might be mistaken as defining the internal structure of the external system. we want to show the services it provides but we know nothing about the internal structure. using ports to represent apis in the above diagram the services/apis are shown as 'ports' on the external system and the details have been moved into a separate key/table. this is less likely to be mistaken as showing any internal structure of the external service. (note that i could have also shown outgoing rports from the brokerage system.) uml interfaces this final diagram is using a uml style interface provider and requirer. this is a clean diagram but requires the user to be aware of what the cup and ball means (although i could have explained this in the key). conclusion any of these solutions could be appropriate depending on the complexity of the api set you are trying to represent. i'd suggest starting with a simple representation (i.e. fully labeled connections) and moving to a more complex one if needed but remember to use a key to explain any elements you use!

Introduction Hibernate provides an optimistic locking mechanism to prevent lost updates even for long-conversations. In conjunction with an entity storage, spanning over multiple user requests (extended persistence context or detached entities) Hibernate can guarantee application-level repeatable-reads. The dirty checking mechanism detects entity state changes and increments the entity version. While basic property changes are always taken into consideration, Hibernate collections are more subtle in this regard. Owned vs. Inverse Collections In relational databases, two records are associated through a foreign key reference. In this relationship, the referenced record is the parent while the referencing row (the foreign key side) is the child. A non-null foreign key may only reference an existing parent record. In the Object-oriented space this association can be represented in both directions. We can have a many-to-one reference from a child to parent and the parent can also have a one-to-many children collection. Because both sides could potentially control the database foreign key state, we must ensure that only one side is the owner of this association. Only the owningside state changes are propagated to the database. The non-owning side has been traditionally referred as the inverse side. Next I’ll describe the most common ways of modelling this association. The Unidirectional Parent-Owning-Side-Child Association Mapping Only the parent side has a @OneToMany non-inverse children collection. The child entity doesn’t reference the parent entity at all. @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true) private List comments = new ArrayList (); ... } The Unidirectional Parent-Owning-Side-Child Component Association Mapping Mapping The child side doesn’t always have to be an entity and we might model it as acomponent type instead. An Embeddable object (component type) may contain both basic types and association mappings but it can never contain an @Id. The Embeddable object is persisted/removed along with its owning entity. The parent has an @ElementCollection children association. The child entity may only reference the parent through the non-queryable Hibernate specific @Parentannotation. @Entity(name = "post") public class Post { ... @ElementCollection @JoinTable(name = "post_comments", joinColumns = @JoinColumn(name = "post_id")) @OrderColumn(name = "comment_index") private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Embeddable public class Comment { ... @Parent private Post post; ... } The Bidirectional Parent-Owning-Side-Child Association Mapping The parent is the owning side so it has a @OneToMany non-inverse (without a mappedBy directive) children collection. The child entity references the parent entity through a @ManyToOne association that’s neither insertable nor updatable: @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true) private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Entity(name = "comment") public class Comment ... @ManyToOne @JoinColumn(name = "post_id", insertable = false, updatable = false) private Post post; ... } The Bidirectional Parent-Owning-Side-Child Association Mapping The child entity references the parent entity through a @ManyToOne association, and the parent has a mappedBy @OneToMany children collection. The parent side is the inverse side so only the @ManyToOne state changes are propagated to the database. Even if there’s only one owning side, it’s always a good practice to keep both sides in sync by using the add/removeChild() methods. @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true, mappedBy = "post") private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Entity(name = "comment") public class Comment { ... @ManyToOne private Post post; ... } The Unidirectional Parent-Owning-Side-Child Association Mapping The child entity references the parent through a @ManyToOne association. The parent doesn’t have a @OneToMany children collection so the child entity becomes the owning side. This association mapping resembles the relational data foreign key linkage. @Entity(name = "comment") public class Comment { ... @ManyToOne private Post post; ... } Collection Versioning The 3.4.2 section of the JPA 2.1 specification defines optimistic locking as: The version attribute is updated by the persistence provider runtime when the object is written to the database. All non-relationship fields and proper ties and all relationships owned by the entity are included in version checks[35]. [35] This includes owned relationships maintained in join tables N.B. Only owning-side children collection can update the parent version. Testing Time Let’s test how the parent-child association type affects the parent versioning. Because we are interested in the children collection dirty checking, theunidirectional child-owning-side-parent association is going to be skipped, as in that case the parent doesn’t contain a children collection. Test Case The following test case is going to be used for all collection type use cases: protected void simulateConcurrentTransactions(final boolean shouldIncrementParentVersion) { final ExecutorService executorService = Executors.newSingleThreadExecutor(); doInTransaction(new TransactionCallable () { @Override public Void execute(Session session) { try { P post = postClass.newInstance(); post.setId(1L); post.setName("Hibernate training"); session.persist(post); return null; } catch (Exception e) { throw new IllegalArgumentException(e); } } }); doInTransaction(new TransactionCallable () { @Override public Void execute(final Session session) { final P post = (P) session.get(postClass, 1L); try { executorService.submit(new Callable () { @Override public Void call() throws Exception { return doInTransaction(new TransactionCallable () { @Override public Void execute(Session _session) { try { P otherThreadPost = (P) _session.get(postClass, 1L); int loadTimeVersion = otherThreadPost.getVersion(); assertNotSame(post, otherThreadPost); assertEquals(0L, otherThreadPost.getVersion()); C comment = commentClass.newInstance(); comment.setReview("Good post!"); otherThreadPost.addComment(comment); _session.flush(); if (shouldIncrementParentVersion) { assertEquals(otherThreadPost.getVersion(), loadTimeVersion + 1); } else { assertEquals(otherThreadPost.getVersion(), loadTimeVersion); } return null; } catch (Exception e) { throw new IllegalArgumentException(e); } } }); } }).get(); } catch (Exception e) { throw new IllegalArgumentException(e); } post.setName("Hibernate Master Class"); session.flush(); return null; } }); } The Unidirectional Parent-Owning-Side-Child Association Testing #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comment (post_id bigint not null, comments_id bigint not null, comment_index integer not null, primary key (post_id, comment_index))][]} Query:{[alter table post_comment add constraint FK_se9l149iyyao6va95afioxsrl foreign key (comments_id) references comment][]} Query:{[alter table post_comment add constraint FK_6o1igdm04v78cwqre59or1yj1 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} #insert comment in secondary transaction #optimistic locking post version update in secondary transaction Query:{[insert into comment (id, review) values (default, ?)][Good post!]} Query:{[update post setname=?, version=? where id=? and version=?][Hibernate training,1,1,0]} Query:{[insert into post_comment (post_id, comment_index, comments_id) values (?, ?, ?)][1,0,1]} #optimistic locking exception in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.vladmihalcea.hibernate.masterclass.laboratory.concurrency.EntityOptimisticLockingOnUnidirectionalCollectionTest$Post#1] The Unidirectional Parent-Owning-Side-Child Component Association Testing #create tables Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comments (post_id bigint not null, review varchar(255), comment_index integer not null, primary key (post_id, comment_index))][]} Query:{[alter table post_comments add constraint FK_gh9apqeduab8cs0ohcq1dgukp foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_0_0_, entityopti0_.name as name2_0_0_, entityopti0_.version as version3_0_0_ from post entityopti0_ where entityopti0_.id=?][1]} Query:{[selectcomments0_.post_id as post_id1_0_0_, comments0_.review as review2_1_0_, comments0_.comment_index as comment_3_0_ from post_comments comments0_ where comments0_.post_id=?][1]} #insert comment in secondary transaction #optimistic locking post version update in secondary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate training,1,1,0]} Query:{[insert into post_comments (post_id, comment_index, review) values (?, ?, ?)][1,0,Good post!]} #optimistic locking exception in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.vladmihalcea.hibernate.masterclass.laboratory.concurrency.EntityOptimisticLockingOnComponentCollectionTest$Post#1] The Bidirectional Parent-Owning-Side-Child Association Testing #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), post_id bigint, primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comment (post_id bigint not null, comments_id bigint not null)][]} Query:{[alter table post_comment add constraint UK_se9l149iyyao6va95afioxsrl unique (comments_id)][]} Query:{[alter table comment add constraint FK_f1sl0xkd2lucs7bve3ktt3tu5 foreign key (post_id) references post][]} Query:{[alter table post_comment add constraint FK_se9l149iyyao6va95afioxsrl foreign key (comments_id) references comment][]} Query:{[alter table post_comment add constraint FK_6o1igdm04v78cwqre59or1yj1 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} Query:{[selectcomments0_.post_id as post_id1_1_0_, comments0_.comments_id as comments2_2_0_, entityopti1_.idas id1_0_1_, entityopti1_.post_id as post_id3_0_1_, entityopti1_.review as review2_0_1_, entityopti2_.idas id1_1_2_, entityopti2_.name as name2_1_2_, entityopti2_.version as version3_1_2_ from post_comment comments0_ inner joincomment entityopti1_ on comments0_.comments_id=entityopti1_.idleft outer joinpost entityopti2_ on entityopti1_.post_id=entityopti2_.idwhere comments0_.post_id=?][1]} #insert comment in secondary transaction #optimistic locking post version update in secondary transaction Query:{[insert into comment (id, review) values (default, ?)][Good post!]} Query:{[update post setname=?, version=? where id=? and version=?][Hibernate training,1,1,0]} Query:{[insert into post_comment (post_id, comments_id) values (?, ?)][1,1]} #optimistic locking exception in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.vladmihalcea.hibernate.masterclass.laboratory.concurrency.EntityOptimisticLockingOnBidirectionalParentOwningCollectionTest$Post#1] The Bidirectional Parent-Owning-Side-Child Association Testing #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), post_id bigint, primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[alter table comment add constraint FK_f1sl0xkd2lucs7bve3ktt3tu5 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} #insert comment in secondary transaction #post version is not incremented in secondary transaction Query:{[insert into comment (id, post_id, review) values (default, ?, ?)][1,Good post!]} Query:{[selectcount(id) from comment where post_id =?][1]} #update works in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} If you enjoy reading this article, you might want to subscribe to my newsletter and get a discount for my book as well. Overruling Default Collection Versioning If the default owning-side collection versioning is not suitable for your use case, you can always overrule it with Hibernate [a href="http://docs.jboss.org/hibernate/annotations/3.5/reference/en/html_single/#d0e2903" style="font-family: inherit; font-size: 14px; font-style: inherit; font-weight: inherit; text-decoration: none; color: rgb(1, 160, 219); -webkit-tap-highlight-color: rgb(240, 29, 79); background: transparent;"]@OptimisticLock annotation. Let’s overrule the default parent version update mechanism for bidirectional parent-owning-side-child association: @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true) @OptimisticLock(excluded = true) private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Entity(name = "comment") public class Comment { ... @ManyToOne @JoinColumn(name = "post_id", insertable = false, updatable = false) private Post post; ... } This time, the children collection changes won’t trigger a parent version update: #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), post_id bigint, primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comment (post_id bigint not null, comments_id bigint not null)][]} Query:{[]} Query:{[alter table comment add constraint FK_f1sl0xkd2lucs7bve3ktt3tu5 foreign key (post_id) references post][]} Query:{[alter table post_comment add constraint FK_se9l149iyyao6va95afioxsrl foreign key (comments_id) references comment][]} Query:{[alter table post_comment add constraint FK_6o1igdm04v78cwqre59or1yj1 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} Query:{[selectcomments0_.post_id as post_id1_1_0_, comments0_.comments_id as comments2_2_0_, entityopti1_.idas id1_0_1_, entityopti1_.post_id as post_id3_0_1_, entityopti1_.review as review2_0_1_, entityopti2_.idas id1_1_2_, entityopti2_.name as name2_1_2_, entityopti2_.version as version3_1_2_ from post_comment comments0_ inner joincomment entityopti1_ on comments0_.comments_id=entityopti1_.idleft outer joinpost entityopti2_ on entityopti1_.post_id=entityopti2_.idwhere comments0_.post_id=?][1]} #insert comment in secondary transaction Query:{[insert into comment (id, review) values (default, ?)][Good post!]} Query:{[insert into post_comment (post_id, comments_id) values (?, ?)][1,1]} #update works in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} If you enjoyed this article, I bet you are going to love my book as well. Conclusion It’s very important to understand how various modeling structures impact concurrency patterns. The owning side collections changes are taken into consideration when incrementing the parent version number, and you can always bypass it using the @OptimisticLock annotation. Code available on GitHub. If you have enjoyed reading my article and you’re looking forward to getting instant email notifications of my latest posts, you just need to follow my blog.