[This article was written by Artem Bilan.] Dear Spring community, As we promised in the Release Candidate blog post, we are pleased to announce that the Spring Integration Java DSL 1.0 GA is now available. As usual, use the Release Repository with Maven or Gradle, or download a distribution archive, to give it a spin. See the project home page for more information. First of all, we are glad to share with you that on Nov 12, 2014, DZone research recognized Spring Integration as the leader in the ESB / Integration framework space, leading with 42% marketshare, in a publication of their recent survey results. And the report is the most popular DZone Guide in November, with more than 12 000 downloads already! Don't miss it: very exciting. We hope the release of the Spring Integration Java DSL adds more excitement!. Many thanks to all contributors, including several who are new to the community. The release includes just a few bug fixes, since the release candidate, and a lot of JavaDocs! Not specifically related to the the release, I want to present here some resources on the matter. We are observing many valuable DSL questions on Stack Overflow. Josh Long's tech tip showing how we can use together Spring Boot, REST, Spring Integration 4.1 WebSocket support and Spring Integration Java DSL plus Java 8 features. The Jdbc Splitter implementation in the project tests. My gist to demonstrate how we can use Reactor Streams together with the Spring Integration Java DSL. Dave Syer has started to use Spring Integration Java DSL in the Spring Cloud Bus project. Don't miss the si4demo to see the evolution of Spring Integration including the Java DSL, as shown at the 2014 SpringOne/2GX Conference. (Video should be available soon). Especial thanks to Biju Kunjummen who has done some nice articles on DZone to introduce Spring Integration Java DSL: https://dzone.com/articles/spring-integration-java-dsl, https://dzone.com/articles/spring-integration-java-dsl-0. And of course, with the latest Spring XD, we can build Modules based on @Configuration including Spring Integration Java DSL IntegrationFlow definitions. Just after this announcement I'm going to publish a DSL Tutorial to explain concepts and features using the Java DSL version of the Cafe Demo sample as material. As always, we look forward to your comments and feedback (StackOverflow (spring-integration tag), Spring JIRA, GitHub) and we very much welcome contributions!

Cross Site Scripting, or XSS, is a fairly common vector used to attack web sites. It involves user generated code being redisplayed by a website with all the privileges and security rights that a browser assigns to code originating from the current host. If the user code is something like , then you have a problem. OWASP is an organisation that provides guidance on web security, and they have a page that provides a suggested method for avoiding XSS in JavaEE web app. You can read this document at https://www.owasp.org/index.php/How_to_add_validation_logic_to_HttpServletRequest. The library being demonstrated here is based off the ideas presented in that article, but fleshed out to be more flexible and easy to deploy. We call this library the (unimaginatively named) Parameter Validation Filter, or PVF. PVF is implemented as a Servlet filter that intercepts requests to web pages, runs submitted parameters through a configurable sequence of validation rules, and either sanitises the parameters before they are sent through to the web application, or returns a HTTP error code if validation errors were detected. We have made the following assumptions when developing this library: Client side validation will prevent legitimate users from submitting invalid data. The PVF library should prevent further processing if invalid data is submitted in the majority of cases. Occasionally it might be appropriate to sanitise submitted data, but any sanitisation should be trivial (like the removal of whitespace). To make use of the PVF library, you’ll need to add it to your project. This artifact is currently in the Sonatype staging repo, so you'll need to add that repo to your Maven config. See http://stackoverflow.com/questions/13945757/how-do-you-import-a-maven-dependency-from-sonatype-org for details. com.matthewcasperson parameter_validation_filter LATEST The filter then needs to be added to the web.xml file with the following settings. You may want to configure the url-pattern to match the pages that you actually want to protect. ParameterValidationFilter com.matthewcasperson.validation.filter.ParameterValidationFilter configFile /WEB-INF/xml/pvf.xml ParameterValidationFilter *.jsp Finally you need to create a file called WEB-INF/xml/pvf.xml. This file defines the custom validation rules applied to the parameters being sent to your web applications. true com.matthewcasperson.validation.ruleimpl.TrimTextValidationRule com.matthewcasperson.validation.ruleimpl.FailIfNotCanonicalizedValidationRule com.matthewcasperson.validation.ruleimpl.FailIfContainsHTMLValidationRule .* .* false false The XML has been commented to make it easier to understand, but there are a few interesting elements: paramNamePatternString, which has been configured to enable the validation chain to match all parameters requestURIPatternString, which has been configured to enable the chain to match all URIs The three elements called validationRuleName, which reference the full class name of the validation rules that will be applied to each parameter passed into our web application Although this is a simple example, the three validation rules that have been implemented (TrimTextValidationRule, FailIfNotCanonicalizedValidationRule and FailIfContainsHTMLValidationRule) are quite effective at preventing a malicious user from submitting parameters that contain XSS code. The first rule, TrimTextValidationRule, simply strips away any whitespace on either side of the parameter. This uses the trim() function any developer should be familiar with. The second rule, FailIfNotCanonicalizedValidationRule, will prevent further processing if the supplied parameter has already been encoded. No legitimate user will have a need to supply text like %3Cscript%3EdoEvil()%3B%3C%2Fscript%3E, so any time encoded text is found we simply return with a HTTP 400 error code. This rule makes use of the ESAPI library supplied by OWASP. Like the second rule, the third rule will prevent further processing if the supplied parameter has any special HTML characters. If you would like your customers to be able to pass through characters like &, this rule is too broad. However, it is almost always valid to block special HTML characters. If you want to see how effective this simple validation chain is, check out the live demo at http://pvftest-matthewcasperson.rhcloud.com/. You may want to take a look at https://www.owasp.org/index.php/XSS_Filter_Evasion_Cheat_Sheet to find some XSS patterns that are often used to bypass XSS filters. Moving forward we will be looking to implement more targeted validation rules, especially those that can’t be easily implemented as regex matches (like making sure a date if after today, or that a number is between two values etc). If you have any suggestions, or find any bugs, feel free to fork the code from our GitHub repo . We do hope to get some public feedback in order to make this library as robust as it can be.

This article represents code samples along with related concepts for two different ways in which an Angular app can be defined.

My last post talked about what’s wrong with type classes (in general, but also specifically in Haskell). This post generated some great feedback on Reddit, including some valid criticism that I didn’t explain why I hated on newtypes so much. I took some of that feedback and incorporated it into a revised version of the post, but I have even more to say about “newtypes," so I decided to write another blog post. What’s in a Newtype Newtypes are a feature of Haskell that let you define a new type in terms of an existing type. In the following example, I create a newtype for Email, which “holds” a String. newtype Email = Email String They are similar to type synonyms (type Email = String), except that type synonyms don’t create new types, they just allow you to refer to existing types by other names. Every newtype can be easily translated into a data declaration. In fact, only the keyword changes: data Email = Email String There’s a slight semantic difference between the two, but for purposes of this blog post, any criticism I have against newtypes apply equally to similar constructs modeled with type or data. The Promise of Newtypes Newtypes are used to provide and select between alternate implementations of type classes for some base types. I think that’s a hack (albeit a necessary one), but I’ve already talked about this so I won’t belabor it here. The other promise of newtypes is that we can use them to make our code more type safe. Instead of passing around String as an email, for example, we can create a super lightweight “wrapper” around String called Email, and make it an error to use a String wherever an Email is expected. This practice isn’t restricted to Haskell. Even in Java, it’s considered good coding practice to wrap primitives with classes whose names denote the meaning of the wrapper (Email, SSN, Address, etc.). There’s a part of this promise that’s certainly true. If I have to define a function accepting four parameters, and three of them are strings, but one of those strings denotes an email, then I have two choices: Model the email parameter with a String. In this case, I may accidentally use the email where I intended to use the other two string parameters, or I may use one of the other two string parameters where I intended to use the email. Considering just these choices, there are five ways my program may go wrong if I use the wrong name in the wrong position. Model the email parameter with a newtype. In this case, I cannot use the email where I intended to use the other two string parameters, because the compiler may stop me. Similarly, I cannot use the other two string parameters where I intended to use the email, for the same reason. Looking at just these choices, there are 0 ways my program may go wrong. Thus, newtypes, like all good FP practices, reduce the number of ways my program can go wrong. Unfortunately, in my opinion, they don’t go nearly far enough. False Security For most intent and purposes, newtypes are isomorphic to the single value they hold. In my preceding example, given a String, I can get an email (Email "foo"). Given an Email, I can also get a String, e.g. by pattern matching on the Email constructor. Stated differently, and also approximately because I’m ignoring bottom: the String and Email types are isomorphic; they contain the same inhabitants, for any useful definition of “same”. The only substantive difference between the preceding String and Email is the name of the data constructor (call Email an AbergrackleFoozyWatzit, and what has changed?). Hence, my previous criticism of newtypes as “programming by name”. By themselves, newtypes don’t really reduce the number of ways my program can go wrong. They just make it a bit harder to go wrong. But any newtype is isomorphic to the value it holds, and it’s trivial to convert between the two. In fact, if my code doesn’t need to convert between the two (either directly or indirectly), then it’s better off generic. That is, if I never need to convert an Email to a String, or a String to an Email, then I should really write the code generically to work with any value (even if that means making data structures or functions more polymorphic). Parametricity provides a massive reduction in the number of ways my program can go wrong. Newtypes, on the other hand, just make it a bit harder to go wrong, by adding one layer of indirection. In this example, as with many newtypes, I’ve created a bad isomorphism. The domain model of an email is not isomorphic to the data model of a string. But by using a newtype, I have implicitly declared that they are isomorphic. Calling a string an email may make me feel better, because of the different name, but fundamentally, with a newtype, it’s still a string, and I’m only ever one more step away from going wrong. In my experience, too many newtypes create an isomorphism between things that, properly modeled, are not isomorphic. Fortunately, there’s a well-worn workaround that lets us get more mileage out of newtypes. Smart Constructors If I define Email in a module, I can make its data constructor private, and export a helper function to construct an Email. Such helper functions are called smart constructors. They can be used to break the natural isomorphisms created by newtyping. An example is shown below: newtype Email = MkEmail String mkEmail :: String -> Maybe Email mkEmail s = ... In this example, I create a smart constructor which does not promise that it can turn every string into an email. It promises only that it might be able to turn a string into an email, by returning a Maybe Email. With the smart constructor approach, I’ve modeled the fact that while every email has a string representation, not every string has an email representation. Going back to my earlier example of passing same-typed parameters to a function, if I use a smart constructor, then while I can still use an email anywhere a string is expected (by converting), I can’t use a string anywhere an email is expected. (Well, ignoring the fromJust abomination!) Smart Constructors, Dumb Data Smart constructors take us one step closer toward modeling data in a type safe fashion. Unfortunately, I still don’t think it’s far enough. With smart constructors, our data model is fundamentally underconstrained, so we patch that up by restricting who can create the data. That’s putting a band-aid on the real problem. Why not just solve the root issue — viz., that our data model is underconstrained? Dumb Constructors, Smart Data The best solution to a great many newtype problems, I believe, is creating a data model where there is a true isomorphism between the entity modeled by our data and the values passed to the data constructor. That is, creating a data model such that there exists no regions in our data’s state space which correspond to invalid states. Email is a simple example, because there are well-defined models for what constitutes a data model, which can be translated into data declarations in straightforward, if tedious fashion. (To some extent, it’s a failure of most languages I know that such specifications cannot be easily translated into code without tedious boilerplate!) When our data declaration precisely fits our data model specification, there’s no need for smart constructors, and no need for newtypes. There’s far fewer ways that code can go wrong, and because our domain model is captured precisely by our data model, we can transform that data model in ways that make semantic sense (e.g. transforming just the name part of an email, since we’re now in the realm of structured data). Summary As I’ve explained in this blog post, I don’t really hate newtypes. I think they’re very useful, and I do use them, because they make it more difficult for my programs to go wrong. Ultimately, however, I think a lot of problems solved with newtypes (modeling coordinates, positions, emails, etc.) are better solved by more precise data modeling. That is, by making our programs stop lying about isomorphisms. Precision may be tedious due to limitations of the languages we work in, but honestly, what’s more tedious than debugging broken code?

in this brief tutorial you will learn how to develop a camera viewer application in c# that allows you to display the image of your usb webcam and to stream the camera image to remote pcs and smartphones. instead of presenting a long article, i would rather show how to implement such application with a few lines of c# code by using the prewritten components of a c# camera library. prerequisites a visual c# wpf application created in visual studio the voipsdk.dll added to the references. (it can be found on the official website of this c# camera library .) a media player supporting rtsp streaming (e.g. vlc) installed on a remote pc first of all let’s build the gui. if you follow the content of the mainwindow.xaml file line-by-line, you will see how to create user all the necessary gui elements that allows the user to be able to connect to a usb camera and display its image, and to set the listen address (including 2 textboxes for the ip address and the port number) that makes rtsp streaming possible. (the following figure illustrates the gui that can be created by using this code snippet.) in the mainwindow.xaml.cs file you will see how to implement the camera viewer functionality and how to turn your application as a video server. to test your application run the program, click the connect button, then when the camera image is displayed, enter the ipv4 address of your pc as listening address, and specify ’554’ as a port number. thereafter open the vlc media player on an other pc or smartphone, and open the network media stream by entering the following network url: rtsp://192.168.115.1:554 (that is: rtsp://youripv4address/portnumber). the result can be seen below: i hope my code snippet was useful! happy programming! // mainwindow.xaml // mainwindow.xaml.cs using system; using system.collections.generic; using system.linq; using system.runtime.interopservices; using system.text; using system.threading.tasks; using system.windows; using system.windows.controls; using system.windows.data; using system.windows.documents; using system.windows.input; using system.windows.media; using system.windows.media.imaging; using system.windows.navigation; using system.windows.shapes; using ozeki.media.ipcamera; using ozeki.media.mediahandlers; using ozeki.media.mediahandlers.video; using ozeki.media.mjpegstreaming; using ozeki.media.video.controls; namespace basic_cameraviewer { /// /// interaction logic for mainwindow.xaml /// public partial class mainwindow : window { private videoviewerwpf _videoviewerwpf; private bitmapsourceprovider _provider; private iipcamera _ipcamera; private webcamera _webcamera; private mediaconnector _connector; private myserver _server; private ivideosender _videosender; public mainwindow() { initializecomponent(); _connector = new mediaconnector(); _provider = new bitmapsourceprovider(); _server = new myserver(); setvideoviewer(); } private void setvideoviewer() { _videoviewerwpf = new videoviewerwpf { horizontalalignment = horizontalalignment.stretch, verticalalignment = verticalalignment.stretch, background = brushes.black }; camerabox.children.add(_videoviewerwpf); _videoviewerwpf.setimageprovider(_provider); } #region usb camera connect/disconnect private void connectusbcamera_click(object sender, routedeventargs e) { _webcamera = webcamera.getdefaultdevice(); if (_webcamera == null) return; _connector.connect(_webcamera, _provider); _videosender = _webcamera; _webcamera.start(); _videoviewerwpf.start(); } private void disconnectusbcamera_click(object sender, routedeventargs e) { if (_webcamera == null) return; _videoviewerwpf.stop(); _webcamera.stop(); _webcamera.dispose(); _connector.disconnect(_webcamera, _provider); } #endregion private void guithread(action action) { dispatcher.begininvoke(action); } private void startserver_click(object sender, routedeventargs e) { var ipadress = ipaddresstext.text; var port = int.parse(porttext.text); _server.videosender = _videosender; _server.onclientcountchanged += server_onclientcountchanged; _server.start(); _server.setlistenaddress(ipadress, port); } void server_onclientcountchanged(object sender, eventargs e) { guithread(() => { connectedclientlist.items.clear(); foreach (var client in _server.connectedclients) connectedclientlist.items.add("end point: " + client.transportinfo.remoteendpoint); }); } private void stopserver_click(object sender, routedeventargs e) { _server.onclientcountchanged -= server_onclientcountchanged; _server.stop(); } } }

There may be cases when your REST api provides responses that are very long, and we all know how important transfer speed and bandwidth still are on mobile devices/networks. I think this is the first performance optimization point one needs to address, when developing REST apis that support mobile apps. Guess what? Because responses are text, we can compress them. And with today’s power of smartphones and tablets uncompressing them on the client side should not be a big deal… So in this post I will present how you can SELECTIVELY compress your REST API responses, if you’ve built it in Java with Jersey, which is the JAX-RS Reference Implementation (and more)… 1. Jersey filters and interceptors Well, thanks to Jersey’s powerful Filters and Interceptors features, the implementation is fairly easy. Whereas filters are primarily intended to manipulate request and response parameters like HTTP headers, URIs and/or HTTP methods, interceptors are intended to manipulate entities, via manipulating entity input/output streams. You’ve seen the power of filters in my posts How to add CORS support on the server side in Java with Jersey, where I’ve shown how to CORS-enable a REST API and How to log in Spring with SLF4J and Logback, where I’ve shown how to log requests and responses from the REST API , but for compressing will be using a GZip WriterInterceptor. A writer interceptor is used for cases where entity is written to the “wire”, which on the server side as in this case, means when writing out a response entity. 1.1. GZip Writer Interceptor So let’s have a look at our GZip Writer Interceptor: package org.codingpedia.demo.rest.interceptors; import java.io.IOException; import java.io.OutputStream; import java.util.zip.GZIPOutputStream; import javax.ws.rs.WebApplicationException; import javax.ws.rs.core.MultivaluedMap; import javax.ws.rs.ext.WriterInterceptor; import javax.ws.rs.ext.WriterInterceptorContext; @Provider @Compress public class GZIPWriterInterceptor implements WriterInterceptor { @Override public void aroundWriteTo(WriterInterceptorContext context) throws IOException, WebApplicationException { MultivaluedMap headers = context.getHeaders(); headers.add("Content-Encoding", "gzip"); final OutputStream outputStream = context.getOutputStream(); context.setOutputStream(new GZIPOutputStream(outputStream)); context.proceed(); } } Note: it implements the WriterInterceptor, which is an interface for message body writer interceptors that wrap around calls to javax.ws.rs.ext.MessageBodyWriter.writeTo providers implementing WriterInterceptor contract must be either programmatically registered in a JAX-RS runtime or must be annotated with @Provider annotation to be automatically discovered by the JAX-RS runtime during a provider scanning phase. @Compress is the name binding annotation, which we will discuss more detailed in the coming paragraph “The interceptor gets a output stream from the WriterInterceptorContext and sets a new one which is a GZIP wrapper of the original output stream. After all interceptors are executed the output stream lastly set to the WriterInterceptorContext will be used for serialization of the entity. In the example above the entity bytes will be written to the GZIPOutputStream which will compress the stream data and write them to the original output stream. The original stream is always the stream which writes the data to the “wire”. When the interceptor is used on the server, the original output stream is the stream into which writes data to the underlying server container stream that sends the response to the client.” [2] “The overridden method aroundWriteTo() gets WriterInterceptorContext as a parameter. This context contains getters and setters for header parameters, request properties, entity, entity stream and other properties.” [2]; when you compress your response you should set the “Content-Encoding” header to “gzip” 1.2. Compress annotation Filters and interceptors can be name-bound. Name binding is a concept that allows to say to a JAX-RS runtime that a specific filter or interceptor will be executed only for a specific resource method. When a filter or an interceptor is limited only to a specific resource method we say that it is name-bound. Filters and interceptors that do not have such a limitation are called global. In our case we’ve built the @Compress annotation: package org.codingpedia.demo.rest.interceptors; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import javax.ws.rs.NameBinding; //@Compress annotation is the name binding annotation @NameBinding @Retention(RetentionPolicy.RUNTIME) public @interface Compress {} and used it to mark methods on resources which should be gzipped (e.g. when GET-ing all the podcasts with the PodcastsResource): @Component @Path("/podcasts") public class PodcastsResource { @Autowired private PodcastService podcastService; ........................... /* * *********************************** READ *********************************** */ /** * Returns all resources (podcasts) from the database * * @return * @throws IOException * @throws JsonMappingException * @throws JsonGenerationException * @throws AppException */ @GET @Compress @Produces({ MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML }) public List getPodcasts( @QueryParam("orderByInsertionDate") String orderByInsertionDate, @QueryParam("numberDaysToLookBack") Integer numberDaysToLookBack) throws IOException, AppException { List podcasts = podcastService.getPodcasts( orderByInsertionDate, numberDaysToLookBack); return podcasts; } ........................... } 2. Testing 2.1. SOAPui Well, if you are testing with SOAPui, you can issue the following request against the PodcastsResource Request: GET http://localhost:8888/demo-rest-jersey-spring/podcasts/?orderByInsertionDate=DESC HTTP/1.1 Accept-Encoding: gzip,deflate Accept: application/json, application/xml Host: localhost:8888 Connection: Keep-Alive User-Agent: Apache-HttpClient/4.1.1 (java 1.5) Response: HTTP/1.1 200 OK Content-Type: application/json Content-Encoding: gzip Content-Length: 409 Server: Jetty(9.0.7.v20131107) [ { "id": 2, "title": "Quarks & Co - zum Mitnehmen", "linkOnPodcastpedia": "http://www.podcastpedia.org/quarks", "feed": "http://podcast.wdr.de/quarks.xml", "description": "Quarks & Co: Das Wissenschaftsmagazin", "insertionDate": "2014-10-29T10:46:13.00+0100" }, { "id": 1, "title": "- The Naked Scientists Podcast - Stripping Down Science", "linkOnPodcastpedia": "http://www.podcastpedia.org/podcasts/792/-The-Naked-Scientists-Podcast-Stripping-Down-Science", "feed": "feed_placeholder", "description": "The Naked Scientists flagship science show brings you a lighthearted look at the latest scientific breakthroughs, interviews with the world top scientists, answers to your science questions and science experiments to try at home.", "insertionDate": "2014-10-29T10:46:02.00+0100" } ] SOAPui recognizes the Content-Type: gzip header, we’ve added in the GZIPWriterInterceptor and automatically uncompresses the response and displays it readable to the human eye. Well, that’s it. You’ve learned how Jersey makes it straightforward to compress the REST api responses. Tip: If you want really learn how to design and implement REST API in Java read the following Tutorial – REST API design and implementation in Java with Jersey and Spring

NOTE: Apparently a bundle with a linked source will not be exported or built in an update site built. also Tycho will complain that it can't find linked sources, which severely limits the possibilities. A workaround is to export the bundles as plain old jars (this works fine for some reason), but the problem is far from ideal. See bug reports: https://bugs.eclipse.org/bugs/show_bug.cgi?id=457192 https://bugs.eclipse.org/bugs/show_bug.cgi?id=66177 Introduction I've been using Eclipse for more than ten years now, and so I like to think I know my way around it's offerings, but every now and then I get pleasantly surprised by discovering a feature – which usually had been there all along- but for which I finally have made the time to investigate. In this case, I am talking about the 'Link Source' feature in the Project Properties tab. Most experienced Eclipse users will at some point wander through the project properties, for instance when certain libraries are not found by the compiler, or when a plugin project starts to behave unexpectedly. The Project Properties tab comes into play when the Manifest.MF file no longer provides the answers for certain problems you face, and you need to delve deeper into the classpath and project settings. It also becomes topical when you need to make a custom project. At Project Chaupal we are currently maintaining and updating the code from Project JXTA. JXTA has been around for quite a bit in the open source community, and the development has had its ups and downs, so the code could do with a makeover here and there. I've been involved with keeping the code available in OSGI since 2006 or so – also with its ups and downs- and one of my ideals would be to automatically generate the OSGI bundles straight from the JXTA sources, without any handwork. The JXTA jar ships with a large number of third party libraries (e.g. Jetty and Log4j), some of which are available as OSGI bundles, so I don't want to include them in the JXTA OSGI bundle I make. A list of dependencies in the Manifest should be enough! Some of the third party libraries also aim to provide the same functionality (e.g. the database functionality provided by Derby and H2), so I would prefer to divide this over two bundles, and then just select the bundle that is needed. Ever since JXTA 2.6, the code has been mavenised, and so the code now conforms to the typical structure that Maven requires, with a specific location for JAVA code (src/main/java), resources (src/main/resources) and tests (src/main/test). I prefer to use Tycho for my OSGI bundles, so the regular Eclipse tooling is leading. As a result my goal is to: Create separate bundles for the core and the test source code Add the required resources, such as .properties files and the likes Split the core source code over different bundles, so that every bundle depends on one third party library at the most. It took me a day or two to get everything the way I wanted it, but in the end it was surprisingly easy, so I thought I'd share the experience.This tutorial assumes that you are well-versed in Eclipse and OSGI development. If not, a good tutorial on the subject can be found here. Preparation As was described earlier, the plan is to use a Maven project (available on GitHub) as a source for a number of OSGI bundles. For starters, we need to do the following; Prepare an Eclipse IDE with Egit and, optionally, with support for GitHub. As always, Lars Vogel's tutorial provides an excellent guide to achieve this. With the GitHub support you can actually search for the required repository, and clone it in your workspace within minutes. Add Maven Support for Eclipse. The tutorial can be found here. We now have an Eclipse IDE with one project loaded in the workspace, which conforms to the Maven structure. Now we can start to do the magic! Extracting Source Files in an OSGI Project First create a new plugin project, using the wizard (File → new → Plugin Project). Fill in the required details as requested (target platform→OSGI framework) and press 'finish'. We now have a standard textbook OSGI bundle project. For the sake of argument, let's call this bundle org.mybundle.host. Now we are going to add java source files and resources from the Maven project: open the project properties tab (right mouse click → properties) select the 'Java build path' option and choose the 'source' tab press 'Link Source' and browse to the 'main/java' subfolder of the Maven project. Close the project properties Include the source folder in the build.properties file and clean the workspace Update the Manifest to include the required dependencies, and export the packages as needed As you can see, the java files have been included in the bundle project, and will compile in a normal fashion. TIP: Currently the source file will by default have the same name as the folder. You can change this in the 'link source' wizard. For instance, you can delete the 'src' folder that is created by default, and replace it with the linked source if you want. This should only be considered if you are not going to make specific java files in the bundle. Next we are going to include the resources, such as .properties files that are included in the Maven project. As an exercise, we will exclude all html files that may be included. Open the project properties and follow the steps described previously, but now select the main/resources folder. Then press the 'next' tab, instead of closing You can now select which files to include or exclude in your bundle. Select the 'exclude' tab, and enter the following pattern: **/*.htm*. Close the project properties, update the build path and clean the project We have now included the desired resources, and in principle the bundle should now work as desired. With the include and exclude tabs, you can determine which files and folders you want to add to your bundle. The inclusion and exclusion patterns follow the conventions used by Apache ANT. TIP: You can check if the bundle has the correct source and resource files by opening a file explorer (in Windows) and browsing to the 'bin' folder of your bundle project. If you first refresh your bundle project (F5 in the Eclipse IDE), the correct class and resource files should be present there. NOTE: Although I would not recommend it, it is possible to link the sources of multiple non-OSGI code sources this way. Even though the folders need different names, they will be built as if they are one source folder. Now we create a second plugin project for the test files. We will call this bundle org.mybundle.test Open the project properties and follow the steps described previously, but now select the main/test folder. If required, you can exclude certain tests in the 'exclude' tab Close the project properties, update the build path and clean the project In the manifest editor, include the dependencies to org.mybundle.host, and for instance JUnit and JMock. When there are no more compiler errors, your two bundles should behave as regular OSGI bundles, with the only difference that the sources are extracted from the Maven project. TIP: It is also possible to make fragment bundles this way, and you can include library resources in your bundle (such as third party jars). This way you can restructure a non-OSGI project at will Using Variables When you store your projects in the cloud, such as on gitHub, you may often find multiple versions of your workspace scattered over different computers, and your repositories stored on different drives. This means that linking your sources with absolute paths, as we have done previously, is not a very versatile approach. Especially with linking this may become problematic, as the linked source project (e.g. the maven project) can be stored on a different location than the project that uses the source files. Luckily eclipse allows you to define variables in your project, which can help either to standardise the relative locations or, if this is not possible, to easily modify the links. In order to achive this, follow these steps: Select the project properties, and select the 'Java Build Path' option. Add or Edit a link source, and select the 'variables' button. Add one or more new variables by pressing the 'New' button and entering the locations. Then press 'OK' As an example, the following three variables point to two GitHub projects, one which holds the Maven project, while the bundle project is located in the same subfolder: - GITHUB_LOC: C:/Users/MyName/MyGithubLocation - MYPROJECT_LOC: ${GITHUB_LOC}/MyProject - MYSOURCEPROJECT_LOC: ${MYPROJECT_LOC}/MySource Now all you have to do is change the 'Linked folder location' to: MYSOURCEPROJECT_LOC/src/main/java in order to include the Maven project in your bundle.You can then also add a resource folder: MYSOURCEPROJECT_LOC/src/main/resources TIP: In the above example with a Maven project, the linked folder name will default to 'java' (and the resources to 'resources'). It is recommended to leave it that way, because you can then use the 'src' folder for bundle specific code that yuo may want to add, like a decalarative service. Also remember to update the build path to include the folders your project needs. Conclusion The 'link source' option provides a powerful way to make non-OSGI code accessible as OSGI bundles. The inclusion and exclusion patterns allow you to customise the bundles to your needs.

A long while ago a optimisation was added to the JVM so that if the same exception is thrown again and again and again a single instance of the Exception is created without the stack trace filled in in order to increase performance. This is an excellent idea unless you are trying to diagnose a problem and you have missed the original error. If you forgot about this optimisation you send the afternoon looking at the following log output and weeping slightly. (In my defence I have a little one in the house hence the fuzzy brain and lack of blogging action this year) java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException java.lang.ArrayIndexOutOfBoundsException ... java.lang.Exception: Uncaught exceptions during test at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/common-adf/CreateNewCustomAppAndProjectWithName.xml:25) at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/level0/testADFcMATS.xml:94) at oracle.abbot.JDevScriptFixture.runTest(JDevScriptFixture.java:555) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at junit.textui.TestRunner.doRun(TestRunner.java:116) at junit.textui.TestRunner.doRun(TestRunner.java:109) at oracle.abbot.AbbotRunner.run(AbbotRunner.java:614) at oracle.abbot.AbbotAddin$IdeAbbotRunner.run(AbbotAddin.java:634) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException [Crickets] It turns out that you can turn off this optimisation with a simple flag: java -XX:-OmitStackTraceInFastThrow .... In my particular case the actual exception causing this trouble was a NPE from the GlyphView code in JDK8. (One that is being caused by a glitch in hotspot it seems) But that in turn was causing the AIOOBE in some logging code clouding the issue even more. This in particular is a good flag to add by default when running your automated tests, particularly in combination with the stack trace length override I have talked about before.

The article represents the 3rd set of 10 interview questions. The following are previous two sets that have been published earlier on our website. Following are other sets that we recommend you to go through. Interview questions Set 1 Interview questions Set 2 Q1: Directives can be applied to which all element type? Ans: Following represents the element type and directive declaration style: `E` – Element name: `` `A` – Attribute (default): `` `C` – Class: `` `M` – Comment: `` Q2. What is notion of “isolate” scope object when creating a custom directive? How is it different from the normal scope object? Ans: When creating a custom directive, there is a property called as “scope” which can be assigned different values such as true/false or {}. When assigned with the value “{}”, then a new “isolate” scope is created. The ‘isolate’ scope differs from normal scope in that it does not prototypically inherit from the parent scope. This is useful when creating reusable components, which should not accidentally read or modify data in the parent scope. Q3. What are different return types from compile function? Ans: A compile function can have a return value which can be either a function or an object. A (post-link) function: It is equivalent to registering the linking function via the `link` property of the config object when the compile function is empty. An object with function(s) registered via `pre` and `post` properties. It allows you to control when a linking function should be called during the linking phase. Q4. WHich API need to be invoked on the rootScope service to get the child scopes? Ans: $new Q5. Explain the relationship between scope.$apply & scope.$digest? Ans: As an event such as text change in a textfield happens, the event is caught with an eventhandler which then invokes $apply method on the scope object. The $apply method in turn evaluates the expression and finally invokes $digest method on the scope object. Following code does it all: $apply: function(expr) { try { beginPhase('$apply'); return this.$eval(expr); } catch (e) { $exceptionHandler(e); } finally { clearPhase(); try { $rootScope.$digest(); } catch (e) { $exceptionHandler(e); throw e; } } } Q6. Which angular module is loaded by default? Ans: ng Q7. What angular function is used to manually start an application? Ans: angular.bootstrap Q8. Name some of the methods that could be called on a module instance? For example, say, you instantiated a module such as ‘var helloApp = angular.module( “helloApp”, [] );’. What are different methods that could be called on helloApp instance? Ans: Following are some of the methods: controller factory directive filter constant service provider config Q9. Which angular function is used to wrap a raw DOM element or HTML string as a jQuery element? Ans: angular.element; If jQuery is available, `angular.element` is an alias for the jQuery function. If jQuery is not available, `angular.element` delegates to Angular’s built-in subset of jQuery, called “jQuery lite” or “jqLite.” Q10. Write sample code representing an injector that could be used to kick off your application? var $injector = angular.injector(['ng', 'appName']); $injector.invoke(function($rootScope, $compile, $document){ $compile($document)($rootScope); $rootScope.$digest(); }); Feel free to suggest any changes in above answers if you feel so.

A hashmap which maintains keys as regular expressions. Any pattern matching the expression will be able to retrieve the same value. Internally it maintains two maps, one containing the regex to value, and another containing matched pattern to regex. Whenever there is a new pattern to 'get', there will be a O(n) search through the compiled regex(s) (which have been 'put' as keys) to find a match. Existing patterns will have constant time lookup through two maps. import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.Set; import java.util.WeakHashMap; import java.util.regex.Pattern; public class RegexHashMap implements Map { private class PatternMatcher { private final String regex; private final Pattern compiled; PatternMatcher(String name) { regex = name; compiled = Pattern.compile(regex); } boolean matched(String string) { if(compiled.matcher(string).matches()) { ref.put(string, regex); return true; } return false; } } /** * Map of input to pattern */ private final Map ref; /** * Map of pattern to value */ private final Map map; /** * Compiled patterns */ private final List matchers; @Override public String toString() { return "RegexHashMap [ref=" + ref + ", map=" + map + "]"; } /** * */ public RegexHashMap() { ref = new WeakHashMap(); map = new HashMap(); matchers = new ArrayList(); } /** * Returns the value to which the specified key pattern is mapped, or null if this map contains no mapping for the key pattern */ @Override public V get(Object weakKey) { if(!ref.containsKey(weakKey)) { for(PatternMatcher matcher : matchers) { if(matcher.matched((String) weakKey)) { break; } } } if(ref.containsKey(weakKey)) { return map.get(ref.get(weakKey)); } return null; } /** * Associates a specified regular expression to a particular value */ @Override public V put(String key, V value) { V v = map.put(key, value); if (v == null) { matchers.add(new PatternMatcher(key)); } return v; } /** * Removes the regular expression key */ @Override public V remove(Object key) { V v = map.remove(key); if(v != null) { for(Iterator iter = matchers.iterator(); iter.hasNext();) { PatternMatcher matcher = iter.next(); if(matcher.regex.equals(key)) { iter.remove(); break; } } for(Iterator> iter = ref.entrySet().iterator(); iter.hasNext();) { Entry entry = iter.next(); if(entry.getValue().equals(key)) { iter.remove(); } } } return v; } /** * Set of view on the regular expression keys */ @Override public Set> entrySet() { return map.entrySet(); } @Override public void putAll(Map m) { for(Entry entry : m.entrySet()) { put(entry.getKey(), entry.getValue()); } } @Override public int size() { return map.size(); } @Override public boolean isEmpty() { return map.isEmpty(); } /** * Returns true if this map contains a mapping for the specified regular expression key. */ @Override public boolean containsKey(Object key) { return map.containsKey(key); } /** * Returns true if this map contains a mapping for the specified regular expression matched pattern. * @param key * @return */ public boolean containsKeyPattern(Object key) { return ref.containsKey(key); } @Override public boolean containsValue(Object value) { return map.containsValue(value); } @Override public void clear() { map.clear(); matchers.clear(); ref.clear(); } /** * Returns a Set view of the regular expression keys contained in this map. */ @Override public Set keySet() { return map.keySet(); } /** * Returns a Set view of the regex matched patterns contained in this map. The set is backed by the map, so changes to the map are reflected in the set, and vice-versa. * @return */ public Set keySetPattern() { return ref.keySet(); } @Override public Collection values() { return map.values(); } /** * Produces a map of patterns to values, based on the regex put in this map * @param patterns * @return */ public Map transform(List patterns) { for(String pattern : patterns) { get(pattern); } Map transformed = new HashMap(); for(Entry entry : ref.entrySet()) { transformed.put(entry.getKey(), map.get(entry.getValue())); } return transformed; } public static void main(String...strings) { RegexHashMap rh = new RegexHashMap(); rh.put("[o|O][s|S][e|E].?[1|2]", "This is a regex match"); rh.put("account", "This is a direct match"); System.out.println(rh); System.out.println("get:ose-1 -> "+rh.get("ose-1")); System.out.println("get:OSE2 -> "+rh.get("OSE2")); System.out.println("get:OSE112 -> "+rh.get("OSE112")); System.out.println("get:ose-2 -> "+rh.get("ose-2")); System.out.println("get:account -> "+rh.get("account")); System.out.println(rh); } }

Introduction Hibernate provides an optimistic locking mechanism to prevent lost updates even for long-conversations. In conjunction with an entity storage, spanning over multiple user requests (extended persistence context or detached entities) Hibernate can guarantee application-level repeatable-reads. The dirty checking mechanism detects entity state changes and increments the entity version. While basic property changes are always taken into consideration, Hibernate collections are more subtle in this regard. Owned vs. Inverse Collections In relational databases, two records are associated through a foreign key reference. In this relationship, the referenced record is the parent while the referencing row (the foreign key side) is the child. A non-null foreign key may only reference an existing parent record. In the Object-oriented space this association can be represented in both directions. We can have a many-to-one reference from a child to parent and the parent can also have a one-to-many children collection. Because both sides could potentially control the database foreign key state, we must ensure that only one side is the owner of this association. Only the owningside state changes are propagated to the database. The non-owning side has been traditionally referred as the inverse side. Next I’ll describe the most common ways of modelling this association. The Unidirectional Parent-Owning-Side-Child Association Mapping Only the parent side has a @OneToMany non-inverse children collection. The child entity doesn’t reference the parent entity at all. @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true) private List comments = new ArrayList (); ... } The Unidirectional Parent-Owning-Side-Child Component Association Mapping Mapping The child side doesn’t always have to be an entity and we might model it as acomponent type instead. An Embeddable object (component type) may contain both basic types and association mappings but it can never contain an @Id. The Embeddable object is persisted/removed along with its owning entity. The parent has an @ElementCollection children association. The child entity may only reference the parent through the non-queryable Hibernate specific @Parentannotation. @Entity(name = "post") public class Post { ... @ElementCollection @JoinTable(name = "post_comments", joinColumns = @JoinColumn(name = "post_id")) @OrderColumn(name = "comment_index") private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Embeddable public class Comment { ... @Parent private Post post; ... } The Bidirectional Parent-Owning-Side-Child Association Mapping The parent is the owning side so it has a @OneToMany non-inverse (without a mappedBy directive) children collection. The child entity references the parent entity through a @ManyToOne association that’s neither insertable nor updatable: @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true) private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Entity(name = "comment") public class Comment ... @ManyToOne @JoinColumn(name = "post_id", insertable = false, updatable = false) private Post post; ... } The Bidirectional Parent-Owning-Side-Child Association Mapping The child entity references the parent entity through a @ManyToOne association, and the parent has a mappedBy @OneToMany children collection. The parent side is the inverse side so only the @ManyToOne state changes are propagated to the database. Even if there’s only one owning side, it’s always a good practice to keep both sides in sync by using the add/removeChild() methods. @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true, mappedBy = "post") private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Entity(name = "comment") public class Comment { ... @ManyToOne private Post post; ... } The Unidirectional Parent-Owning-Side-Child Association Mapping The child entity references the parent through a @ManyToOne association. The parent doesn’t have a @OneToMany children collection so the child entity becomes the owning side. This association mapping resembles the relational data foreign key linkage. @Entity(name = "comment") public class Comment { ... @ManyToOne private Post post; ... } Collection Versioning The 3.4.2 section of the JPA 2.1 specification defines optimistic locking as: The version attribute is updated by the persistence provider runtime when the object is written to the database. All non-relationship fields and proper ties and all relationships owned by the entity are included in version checks[35]. [35] This includes owned relationships maintained in join tables N.B. Only owning-side children collection can update the parent version. Testing Time Let’s test how the parent-child association type affects the parent versioning. Because we are interested in the children collection dirty checking, theunidirectional child-owning-side-parent association is going to be skipped, as in that case the parent doesn’t contain a children collection. Test Case The following test case is going to be used for all collection type use cases: protected void simulateConcurrentTransactions(final boolean shouldIncrementParentVersion) { final ExecutorService executorService = Executors.newSingleThreadExecutor(); doInTransaction(new TransactionCallable () { @Override public Void execute(Session session) { try { P post = postClass.newInstance(); post.setId(1L); post.setName("Hibernate training"); session.persist(post); return null; } catch (Exception e) { throw new IllegalArgumentException(e); } } }); doInTransaction(new TransactionCallable () { @Override public Void execute(final Session session) { final P post = (P) session.get(postClass, 1L); try { executorService.submit(new Callable () { @Override public Void call() throws Exception { return doInTransaction(new TransactionCallable () { @Override public Void execute(Session _session) { try { P otherThreadPost = (P) _session.get(postClass, 1L); int loadTimeVersion = otherThreadPost.getVersion(); assertNotSame(post, otherThreadPost); assertEquals(0L, otherThreadPost.getVersion()); C comment = commentClass.newInstance(); comment.setReview("Good post!"); otherThreadPost.addComment(comment); _session.flush(); if (shouldIncrementParentVersion) { assertEquals(otherThreadPost.getVersion(), loadTimeVersion + 1); } else { assertEquals(otherThreadPost.getVersion(), loadTimeVersion); } return null; } catch (Exception e) { throw new IllegalArgumentException(e); } } }); } }).get(); } catch (Exception e) { throw new IllegalArgumentException(e); } post.setName("Hibernate Master Class"); session.flush(); return null; } }); } The Unidirectional Parent-Owning-Side-Child Association Testing #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comment (post_id bigint not null, comments_id bigint not null, comment_index integer not null, primary key (post_id, comment_index))][]} Query:{[alter table post_comment add constraint FK_se9l149iyyao6va95afioxsrl foreign key (comments_id) references comment][]} Query:{[alter table post_comment add constraint FK_6o1igdm04v78cwqre59or1yj1 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} #insert comment in secondary transaction #optimistic locking post version update in secondary transaction Query:{[insert into comment (id, review) values (default, ?)][Good post!]} Query:{[update post setname=?, version=? where id=? and version=?][Hibernate training,1,1,0]} Query:{[insert into post_comment (post_id, comment_index, comments_id) values (?, ?, ?)][1,0,1]} #optimistic locking exception in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.vladmihalcea.hibernate.masterclass.laboratory.concurrency.EntityOptimisticLockingOnUnidirectionalCollectionTest$Post#1] The Unidirectional Parent-Owning-Side-Child Component Association Testing #create tables Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comments (post_id bigint not null, review varchar(255), comment_index integer not null, primary key (post_id, comment_index))][]} Query:{[alter table post_comments add constraint FK_gh9apqeduab8cs0ohcq1dgukp foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_0_0_, entityopti0_.name as name2_0_0_, entityopti0_.version as version3_0_0_ from post entityopti0_ where entityopti0_.id=?][1]} Query:{[selectcomments0_.post_id as post_id1_0_0_, comments0_.review as review2_1_0_, comments0_.comment_index as comment_3_0_ from post_comments comments0_ where comments0_.post_id=?][1]} #insert comment in secondary transaction #optimistic locking post version update in secondary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate training,1,1,0]} Query:{[insert into post_comments (post_id, comment_index, review) values (?, ?, ?)][1,0,Good post!]} #optimistic locking exception in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.vladmihalcea.hibernate.masterclass.laboratory.concurrency.EntityOptimisticLockingOnComponentCollectionTest$Post#1] The Bidirectional Parent-Owning-Side-Child Association Testing #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), post_id bigint, primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comment (post_id bigint not null, comments_id bigint not null)][]} Query:{[alter table post_comment add constraint UK_se9l149iyyao6va95afioxsrl unique (comments_id)][]} Query:{[alter table comment add constraint FK_f1sl0xkd2lucs7bve3ktt3tu5 foreign key (post_id) references post][]} Query:{[alter table post_comment add constraint FK_se9l149iyyao6va95afioxsrl foreign key (comments_id) references comment][]} Query:{[alter table post_comment add constraint FK_6o1igdm04v78cwqre59or1yj1 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} Query:{[selectcomments0_.post_id as post_id1_1_0_, comments0_.comments_id as comments2_2_0_, entityopti1_.idas id1_0_1_, entityopti1_.post_id as post_id3_0_1_, entityopti1_.review as review2_0_1_, entityopti2_.idas id1_1_2_, entityopti2_.name as name2_1_2_, entityopti2_.version as version3_1_2_ from post_comment comments0_ inner joincomment entityopti1_ on comments0_.comments_id=entityopti1_.idleft outer joinpost entityopti2_ on entityopti1_.post_id=entityopti2_.idwhere comments0_.post_id=?][1]} #insert comment in secondary transaction #optimistic locking post version update in secondary transaction Query:{[insert into comment (id, review) values (default, ?)][Good post!]} Query:{[update post setname=?, version=? where id=? and version=?][Hibernate training,1,1,0]} Query:{[insert into post_comment (post_id, comments_id) values (?, ?)][1,1]} #optimistic locking exception in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.vladmihalcea.hibernate.masterclass.laboratory.concurrency.EntityOptimisticLockingOnBidirectionalParentOwningCollectionTest$Post#1] The Bidirectional Parent-Owning-Side-Child Association Testing #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), post_id bigint, primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[alter table comment add constraint FK_f1sl0xkd2lucs7bve3ktt3tu5 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} #insert comment in secondary transaction #post version is not incremented in secondary transaction Query:{[insert into comment (id, post_id, review) values (default, ?, ?)][1,Good post!]} Query:{[selectcount(id) from comment where post_id =?][1]} #update works in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} If you enjoy reading this article, you might want to subscribe to my newsletter and get a discount for my book as well. Overruling Default Collection Versioning If the default owning-side collection versioning is not suitable for your use case, you can always overrule it with Hibernate [a href="http://docs.jboss.org/hibernate/annotations/3.5/reference/en/html_single/#d0e2903" style="font-family: inherit; font-size: 14px; font-style: inherit; font-weight: inherit; text-decoration: none; color: rgb(1, 160, 219); -webkit-tap-highlight-color: rgb(240, 29, 79); background: transparent;"]@OptimisticLock annotation. Let’s overrule the default parent version update mechanism for bidirectional parent-owning-side-child association: @Entity(name = "post") public class Post { ... @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true) @OptimisticLock(excluded = true) private List comments = new ArrayList (); ... public void addComment(Comment comment) { comment.setPost(this); comments.add(comment); } } @Entity(name = "comment") public class Comment { ... @ManyToOne @JoinColumn(name = "post_id", insertable = false, updatable = false) private Post post; ... } This time, the children collection changes won’t trigger a parent version update: #create tables Query:{[create table comment (idbigint generated by default as identity (start with 1), review varchar(255), post_id bigint, primary key (id))][]} Query:{[create table post (idbigint not null, name varchar(255), version integer not null, primary key (id))][]} Query:{[create table post_comment (post_id bigint not null, comments_id bigint not null)][]} Query:{[]} Query:{[alter table comment add constraint FK_f1sl0xkd2lucs7bve3ktt3tu5 foreign key (post_id) references post][]} Query:{[alter table post_comment add constraint FK_se9l149iyyao6va95afioxsrl foreign key (comments_id) references comment][]} Query:{[alter table post_comment add constraint FK_6o1igdm04v78cwqre59or1yj1 foreign key (post_id) references post][]} #insert post in primary transaction Query:{[insert into post (name, version, id) values (?, ?, ?)][Hibernate training,0,1]} #select post in secondary transaction Query:{[selectentityopti0_.idas id1_1_0_, entityopti0_.name as name2_1_0_, entityopti0_.version as version3_1_0_ from post entityopti0_ where entityopti0_.id=?][1]} Query:{[selectcomments0_.post_id as post_id1_1_0_, comments0_.comments_id as comments2_2_0_, entityopti1_.idas id1_0_1_, entityopti1_.post_id as post_id3_0_1_, entityopti1_.review as review2_0_1_, entityopti2_.idas id1_1_2_, entityopti2_.name as name2_1_2_, entityopti2_.version as version3_1_2_ from post_comment comments0_ inner joincomment entityopti1_ on comments0_.comments_id=entityopti1_.idleft outer joinpost entityopti2_ on entityopti1_.post_id=entityopti2_.idwhere comments0_.post_id=?][1]} #insert comment in secondary transaction Query:{[insert into comment (id, review) values (default, ?)][Good post!]} Query:{[insert into post_comment (post_id, comments_id) values (?, ?)][1,1]} #update works in primary transaction Query:{[update post setname=?, version=? where id=? and version=?][Hibernate Master Class,1,1,0]} If you enjoyed this article, I bet you are going to love my book as well. Conclusion It’s very important to understand how various modeling structures impact concurrency patterns. The owning side collections changes are taken into consideration when incrementing the parent version number, and you can always bypass it using the @OptimisticLock annotation. Code available on GitHub. If you have enjoyed reading my article and you’re looking forward to getting instant email notifications of my latest posts, you just need to follow my blog.

Spring provides a great out of the box support for caching expensive method calls. The caching abstraction is covered in a great detail here. My objective here is to cover one of the newer cache implementations that Spring now provides with 4.0+ version of the framework - using Google Guava Cache In brief, consider a service which has a few slow methods: public class DummyBookService implements BookService { @Override public Book loadBook(String isbn) { // Slow method 1. } @Override public List loadBookByAuthor(String author) { // Slow method 2 } } With Spring Caching abstraction, repeated calls with the same parameter can be sped up by an annotation on the method along these lines - here the result of loadBook is being cached in to a "book" cache and listing of books cached into another "books" cache: public class DummyBookService implements BookService { @Override @Cacheable("book") public Book loadBook(String isbn) { // slow response time.. } @Override @Cacheable("books") public List loadBookByAuthor(String author) { // Slow listing } } Now, Caching abstraction support requires a CacheManager to be available which is responsible for managing the underlying caches to store the cached results, with the new Guava Cache support the CacheManager is along these lines: @Bean public CacheManager cacheManager() { return new GuavaCacheManager("books", "book"); } Google Guava Cache provides a rich API to be able to pre-load the cache, set eviction duration based on last access or created time, set the size of the cache etc, if the cache is to be customized then a guava CacheBuilder can be passed to the CacheManager for this customization: @Bean public CacheManager cacheManager() { GuavaCacheManager guavaCacheManager = new GuavaCacheManager(); guavaCacheManager.setCacheBuilder(CacheBuilder.newBuilder().expireAfterAccess(30, TimeUnit.MINUTES)); return guavaCacheManager; } This works well if all the caches have a similar configuration, what if the caches need to be configured differently - for eg. in the sample above, I may want the "book" cache to never expire but the "books" cache to have an expiration of 30 mins, then the GuavaCacheManager abstraction does not work well, instead a better solution is actually to use a SimpleCacheManager which provides a more direct way to get to the cache and can be configured this way: @Bean public CacheManager cacheManager() { SimpleCacheManager simpleCacheManager = new SimpleCacheManager(); GuavaCache cache1 = new GuavaCache("book", CacheBuilder.newBuilder().build()); GuavaCache cache2 = new GuavaCache("books", CacheBuilder.newBuilder() .expireAfterAccess(30, TimeUnit.MINUTES) .build()); simpleCacheManager.setCaches(Arrays.asList(cache1, cache2)); return simpleCacheManager; } This approach works very nicely, if required certain caches can be configured to be backed by a different caching engines itself, say a simple hashmap, some by Guava or EhCache some by distributed caches like Gemfire.

As memory gets cheaper and cheaper, our applications can keep more data readily available in main memory, or even all as in case of in-memory databases. To make real use of the growing heap memory, appropriate data structures must be used. Interesting enough, there seem to be no specialized implementations for lists - by far the most used collection. This article introduces BigList, a list designed for handling large collections where large means that all data still fit completely in the heap memory. The article will show the special requirements for handling large collections, how BigList is implemented and how it compares to other list implementations. 1. Requirements What are the special requirements we need to handle large collections efficiently? Memory: Sparing use of memory: The list should need little memory for its own implementation so memory can be used for storing application data. Specialized versions for primitives: It must be possible to store common primitives like ints in a memory saving way. Avoid copying large data blocks: If the list grows or shrinks, only a small part of the data must be copied around, as this operation becomes expensive and needs the same amount of memory again. Data sharing: copying collections is a frequent operation which should be efficiently possible even if the collection is large. An efficient implementation requires some sort of data sharing as copying all elements is per se a costly operation. Performance: Good performance for normal operations like reading, storing, adding or removing single elements. Great performance for bulk operations like adding or removing multiple elements. Predictable overhead of operations, so similar operations should need a similar amount of time without excessive worst case scenarios. If an implementation does not offer these features, some operations will not only be slow for really large collections, but will becomse just not feasible because memory or CPU usage will be too exhaustive. Introduction to BigList BigList is a member of the Brownies Collections library which also includes GapList, the fastest list implementation known. GapList is a drop-in replacement for ArrayList, LinkedList, or ArrayDequeue and offers fast access by index and fast insertion/removal at the beginning and at the end at the same time. GapList however has not been designed to cope with large collections, so adding or removing elements can make it necessary to copy a lot of elements around which will lead to performance problems. Also copying a large collection becomes an expensive operation, both in term of time and memory consumption. It will simply not be possible to make a copy of a large collections if not the same amount of memory is available a second time. And this is a common operation as you often want to return a copy of an internal list through your API which has no reference the original list. BigList addresses both problems. The first problem is solved by storing the collection elements in fixed size blocks. Add or remove operations are then implemented to use only data from one block. The copying problem is solved by maintaining a reference count on the fixed size blocks which allows to implement a copy-on-write approach. For efficient access to the fixed size blocks, they are maintained in a specialized tree structure. 2. BigList Details Each BigList instance stores the following information: Elements are stored in in fixed-size blocks A single block is implemented as GapList with a reference count for sharing All blocks are maintained in a tree for fast access Access information for the current block is cached for better performance The following illustration shows these details for two instances of BigList which share one block. 2.1 Use of Blocks Elements are stored in in fixed-size blocks with a default block size of 1000. Where this default may look pretty small, it is most of the time a good choice because it guarantees that write operation only need to move few elements. Read operations will profit from locality of reference by using the currently cached block to be fast. It is however possible to specify the block size for each created BigList instance. All blocks except the first are allocated with this fixed size and will not grow or shrink. The first block will grow to the specified block size to save memory for small lists. If a block has reached its maximum size and more data must be stored within, the block needs to be split up in two blocks before more elements can be stored. If elements are added to the head or tail of the list, the block will only be filled up to a threshold of 95%. This allows inserts into the block without the immediate need for split operations. To save memory, blocks are also merged. This happens automatically if two adjacent blocks are both filled less than 35% after a remove operation. 2.2 Locality of Reference For each operation on BigList, the affected block must be determined first. As predicted by locality of reference, most of the time the affected block will be the same as for the last operation. The implementation of BigList has therefore been designed to profit from locality of reference which makes common operations like iterating over a list very efficient. Instead of always traversing the block tree to determine the block needed for an operation, lower and upper index of the last used block are cached. So if the next operation happens near to the previous one, the same block can be used again without need to traverse the tree. 2.3 Reference Counting To support a copy-on-write approach, BigList stores a reference count for each fixed size blocks indicating whether this block is private or shared. Initially all lists are private having a reference count of 0, so that modification are allowed. If a list is copied, the reference count is incremented which prohibits further modifications. Before a modification then can be made, the block must be copied decrementing the block's reference count and setting the reference count of the copy to 0. The reference count of a block is then decremented by the finalizer of BigList. 3. Benchmarks To prove the excellence of BigList in time and memory consumption, we compare it with some other List implementations. And here are the nominees: Type Library Description BigList brownie-collections List optimized for storing large number of elements. Elements stored in fixed size blocks which are maintained in a tree. GapList brownie-collections Fastest list implementation known. Fast access by index and fast insertion/removal at end and at beginning. ArrayList JDK Maintains elements in a single array. Fast access by index, fast insertion/removal at end, but slow at beginning. LinkedList JDK Elements stored using a linked list. Slow access by index. Memory overhead for each element stored. TreeList commons-collections Elements stored in a tree. All operations are not really fast, but there are no very slow operations. Memory overhead for each element stored. FastTable javolution Elements stored in a "fractal"-like data structure. Good performance and use of memory. However no bulk operations and collection does not shrink. 3.1 Handling Objects In the first part of the benchmark, we compare memory consumption and performance of the different list implementations. Let's first have a look at the memory consumption. The following table shows the bytes used to hold a list with 1'000'000 null elements: BigList GapList ArrayList LinkedList TreeList FastTable 32 bit 4'298'466 4'021'296 4'861'992 8'000'028 18'000'028 4'142'892 64 bit 8'544'254 8'042'552 9'723'964 16'000'044 26'000'044 8'222'988 We can see that BigList, GapList, ArrayList, and FastTable only add small overhead to the stored elements, where as Linkedlist needs twice the memory and TreeList even more. Now to the performance. Here are the results of 9 benchmarks which have been run for each of the 6 candidates with JDK 8 in a 32 bit Windows environment and a list of 1'000'000 elements: The result table can be read as follows: the fastest candidate for each test has a relative performance indicator of 1 the value for the other candidates indicate how many times they have been slower, so a factor of 3 means that this implementation was 3 times slower than the best one The different factor are colored like this: 
factor 1: green (best)
 factor <5: blue (good) 
factor <25: yellow (moderate) 
factor >25: red (poor) If we look the benchmark result, we can see that the performance of BigList is best for all expect two benchmarks. The only moderate result is produces in getting elements in a totally random order. This could be expected as there is no locality of reference which can be exploited, so for each access, the block tree must be traversed to find the correct block. Luckily this is a rare use case in real applications. And the benchmark "Get local" shows that performance is back to good as soon as elements next to each other must be retrieved - as it is the case if we iterate over a range. 3.2 Handling Primitives In the second part of the benchmark, we want see how big the savings are if we use a data structure specialized for storing primitives compared to strong wrapped objects. For this reason, we compare IntBigList and BigList. The following table shows memory needed to store 1'000'000 integer values: BigList IntBigList 32 bit 16'298'454 4'534'840 64 bit 28'544'234 4'570'432 Obviously it is easy to save a lot of memory. In a 32 bit environment, IntBigList just needs 25% percent of memory, in a 64 bit environment only 14%! These figures become plausible if you recall that a simple object needs 8 bytes in a 32 bit, but already 16 bytes in a 64 bit environment, where as a primitive integer value always only needs 4 bytes. The measurable performance gain is not so impressive, it is something below 10% for simple get operations and something above 10% for add and remove operations. These numbers show that the JVM is impressively fast in creating wrapper objects and boxing and unboxing primitive values. We must however also consider that each created object will need to be garbage collected once and therefore adds to the total load of the JVM. 4. Summary BigList is a scalable high-performance list for storing large collections. Its design guarantees that all operations will be predictable and efficient both in term of performance and memory consumption, even copying large collections is tremendous fast. Benchmarks haven proven this and shown that BigList outperform other known list implementations. The library also offers specialized implementations for primitive types like IntBigList which save much memory and provide superior performance. BigList for handling objects and the specializations for handling primitives are part of the Brownies Collections library and can be downloaded from http://www.magicwerk.org/collections.

We can see from javadoc that javax.net.ssl.HttpsURLConnection provided a static method to override withsetDefaultSSLSocketFory() method. This allow you to supply a custom javax.net.ssl.TrustManagerthat may verify your own CA certs handshake and validation etc. But this will override the default for all "https" URLs per your JVM! So how can we override just a single https URL? Looking at javax.net.ssl.HttpsURLConnection again we see instance method for setSSLSocketFactory(), but we can't instantiate HttpsURLConnection object directly! It took me some digging to realized that the java.net.URL is actually an factory class for its implementation! One can get an instance like this using new URL("https://localhost").openConnection() To complete this article, I will provide a simple working example that demonstrate this. package zemian; import java.io.BufferedReader; import java.io.InputStream; import java.io.InputStreamReader; import java.net.URL; import java.net.URLConnection; import java.security.SecureRandom; import java.security.cert.X509Certificate; import javax.net.ssl.HttpsURLConnection; import javax.net.ssl.SSLContext; import javax.net.ssl.SSLSocketFactory; import javax.net.ssl.TrustManager; import javax.net.ssl.X509TrustManager; public class WGetText { public static void main(String[] args) throws Exception { String urlString = System.getProperty("url", "https://google.com"); URL url = new URL(urlString); URLConnection urlConnection = url.openConnection(); HttpsURLConnection httpsUrlConnection = (HttpsURLConnection) urlConnection; SSLSocketFactory sslSocketFactory = createSslSocketFactory(); httpsUrlConnection.setSSLSocketFactory(sslSocketFactory); try (InputStream inputStream = httpsUrlConnection.getInputStream()) { BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream)); String line = null; while ((line = reader.readLine()) != null) { System.out.println(line); } } } private static SSLSocketFactory createSslSocketFactory() throws Exception { TrustManager[] byPassTrustManagers = new TrustManager[] { new X509TrustManager() { public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; } public void checkClientTrusted(X509Certificate[] chain, String authType) { } public void checkServerTrusted(X509Certificate[] chain, String authType) { } } }; SSLContext sslContext = SSLContext.getInstance("TLS"); sslContext.init(null, byPassTrustManagers, new SecureRandom()); return sslContext.getSocketFactory(); } }

This article explains how errors are handled when using the messaging system with Spring Integration and how to handle route and redirect to specific channel.

if someone asked you to develop a rest api on the jvm, which frameworks would you use? i was recently tasked with such a project. my client asked me to implement a rest api to ingest requests from a 3rd party. the project entailed consuming xml requests, storing the data in a database, then exposing the data to internal application with a json endpoint. finally, it would allow taking in a json request and turning it into an xml request back to the 3rd party. with the recent release of apache camel 2.14 and my success using it , i started by copying my apache camel / cxf / spring boot project and trimming it down to the bare essentials. i whipped together a simple hello world service using camel and spring mvc. i also integrated swagger into both. both implementations were pretty easy to create ( sample code ), but i decided to use spring mvc. my reasons were simple: its rest support was more mature, i knew it well, and spring mvc test makes it easy to test apis. camel's swagger support without web.xml as part of the aforementioned spike, i learned out how to configure camel's rest and swagger support using spring's javaconfig and no web.xml. i made this into a sample project and put it on github as camel-rest-swagger . this article shows how i built a rest api with java 8, spring boot/mvc, jaxb and spring data (jpa and rest components). i stumbled a few times while developing this project, but figured out how to get over all the hurdles. i hope this helps the team that's now maintaining this project (my last day was friday) and those that are trying to do something similar. xml to java with jaxb the data we needed to ingest from a 3rd party was based on the ncpdp standards. as a member, we were able to download a number of xsd files, put them in our project and generate java classes to handle the incoming/outgoing requests. i used the maven-jaxb2-plugin to generate the java classes. org.jvnet.jaxb2.maven2 maven-jaxb2-plugin 0.8.3 generate -xtostring -xequals -xhashcode -xcopyable org.jvnet.jaxb2_commons jaxb2-basics 0.6.4 src/main/resources/schemas/ncpdp the first error i ran into was about a property already being defined. [info] --- maven-jaxb2-plugin:0.8.3:generate (default) @ spring-app --- [error] error while parsing schema(s).location [ file:/users/mraible/dev/spring-app/src/main/resources/schemas/ncpdp/structures.xsd{1811,48}]. com.sun.istack.saxparseexception2; systemid: file:/users/mraible/dev/spring-app/src/main/resources/schemas/ncpdp/structures.xsd; linenumber: 1811; columnnumber: 48; property "multipletimingmodifierandtimingandduration" is already defined. use to resolve this conflict. at com.sun.tools.xjc.errorreceiver.error(errorreceiver.java:86) i was able to workaround this by upgrading to maven-jaxb2-plugin version 0.9.1. i created a controller and stubbed out a response with hard-coded data. i confirmed the incoming xml-to-java marshalling worked by testing with a sample request provided by our 3rd party customer. i started with a curl command, because it was easy to use and could be run by anyone with the file and curl installed. curl -x post -h 'accept: application/xml' -h 'content-type: application/xml' \ --data-binary @sample-request.xml http://localhost:8080/api/message -v this is when i ran into another stumbling block: the response wasn't getting marshalled back to xml correctly. after some research, i found out this was caused by the lack of @xmlrootelement annotations on my generated classes. i posted a question to stack overflow titled returning jaxb-generated elements from spring boot controller . after banging my head against the wall for a couple days, i figured out the solution . i created a bindings.xjb file in the same directory as my schemas. this causes jaxb to generate @xmlrootelement on classes. to add namespaces prefixes to the returned xml, i had to modify the maven-jaxb2-plugin to add a couple arguments. -extension -xnamespace-prefix and add a dependency: org.jvnet.jaxb2_commons jaxb2-namespace-prefix 1.1 then i modified bindings.xjb to include the package and prefix settings. i also moved into a global setting. i eventually had to add prefixes for all schemas and their packages. i learned how to add prefixes from the namespace-prefix plugins page . finally, i customized the code-generation process to generate joda-time's datetime instead of the default xmlgregoriancalendar . this involved a couple custom xmladapters and a couple additional lines in bindings.xjb . you can see the adapters and bindings.xjb with all necessary prefixes in this gist . nicolas fränkel's customize your jaxb bindings was a great resource for making all this work. i wrote a test to prove that the ingest api worked as desired. @runwith(springjunit4classrunner.class) @springapplicationconfiguration(classes = application.class) @webappconfiguration @dirtiescontext(classmode = dirtiescontext.classmode.after_class) public class initiaterequestcontrollertest { @inject private initiaterequestcontroller controller; private mockmvc mockmvc; @before public void setup() { mockitoannotations.initmocks(this); this.mockmvc = mockmvcbuilders.standalonesetup(controller).build(); } @test public void testgetnotallowedonmessagesapi() throws exception { mockmvc.perform(get("/api/initiate") .accept(mediatype.application_xml)) .andexpect(status().ismethodnotallowed()); } @test public void testpostpainitiationrequest() throws exception { string request = new scanner(new classpathresource("sample-request.xml").getfile()).usedelimiter("\\z").next(); mockmvc.perform(post("/api/initiate") .accept(mediatype.application_xml) .contenttype(mediatype.application_xml) .content(request)) .andexpect(status().isok()) .andexpect(content().contenttype(mediatype.application_xml)) .andexpect(xpath("/message/header/to").string("3rdparty")) .andexpect(xpath("/message/header/sendersoftware/sendersoftwaredeveloper").string("hid")) .andexpect(xpath("/message/body/status/code").string("010")); } } spring data for jpa and rest with jaxb out of the way, i turned to creating an internal api that could be used by another application. spring data was fresh in my mind after reading about it last summer. i created classes for entities i wanted to persist, using lombok's @data to reduce boilerplate. i read the accessing data with jpa guide, created a couple repositories and wrote some tests to prove they worked. i ran into an issue trying to persist joda's datetime and found jadira provided a solution. i added its usertype.core as a dependency to my pom.xml: org.jadira.usertype usertype.core 3.2.0.ga ... and annotated datetime variables accordingly. @column(name = "last_modified", nullable = false) @type(type="org.jadira.usertype.dateandtime.joda.persistentdatetime") private datetime lastmodified; with jpa working, i turned to exposing rest endpoints. i used accessing jpa data with rest as a guide and was looking at json in my browser in a matter of minutes. i was surprised to see a "profile" service listed next to mine, and posted a question to the spring boot team. oliver gierke provided an excellent answer . swagger spring mvc's integration for swagger has greatly improved since i last wrote about it . now you can enable it with a @enableswagger annotation. below is the swaggerconfig class i used to configure swagger and read properties from application.yml . @configuration @enableswagger public class swaggerconfig implements environmentaware { public static final string default_include_pattern = "/api/.*"; private relaxedpropertyresolver propertyresolver; @override public void setenvironment(environment environment) { this.propertyresolver = new relaxedpropertyresolver(environment, "swagger."); } /** * swagger spring mvc configuration */ @bean public swaggerspringmvcplugin swaggerspringmvcplugin(springswaggerconfig springswaggerconfig) { return new swaggerspringmvcplugin(springswaggerconfig) .apiinfo(apiinfo()) .genericmodelsubstitutes(responseentity.class) .includepatterns(default_include_pattern); } /** * api info as it appears on the swagger-ui page */ private apiinfo apiinfo() { return new apiinfo( propertyresolver.getproperty("title"), propertyresolver.getproperty("description"), propertyresolver.getproperty("termsofserviceurl"), propertyresolver.getproperty("contact"), propertyresolver.getproperty("license"), propertyresolver.getproperty("licenseurl")); } } after getting swagger to work, i discovered that endpoints published with @repositoryrestresource aren't picked up by swagger. there is an open issue for spring data support in the swagger-springmvc project. liquibase integration i configured this project to use h2 in development and postgresql in production. i used spring profiles to do this and copied xml/yaml (for maven and application*.yml files) from a previously created jhipster project. next, i needed to create a database. i decided to use liquibase to create tables, rather than hibernate's schema-export. i chose liquibase over flyway based of discussions in the jhipster project . to use liquibase with spring boot is dead simple: add the following dependency to pom.xml, then place changelog files in src/main/resources/db/changelog . org.liquibase liquibase-core i started by using hibernate's schema-export and changing hibernate.ddl-auto to "create-drop" in application-dev.yml . i also commented out the liquibase-core dependency. then i setup a postgresql database and started the app with "mvn spring-boot:run -pprod". i generated the liquibase changelog from an existing schema using the following command (after downloading and installing liquibase). liquibase --driver=org.postgresql.driver --classpath="/users/mraible/.m2/repository/org/postgresql/postgresql/9.3-1102-jdbc41/postgresql-9.3-1102-jdbc41.jar:/users/mraible/snakeyaml-1.11.jar" --changelogfile=/users/mraible/dev/spring-app/src/main/resources/db/changelog/db.changelog-02.yaml --url="jdbc:postgresql://localhost:5432/mydb" --username=user --password=pass generatechangelog i did find one bug - the generatechangelog command generates too many constraints in version 3.2.2 . i was able to fix this by manually editing the generated yaml file. tip: if you want to drop all tables in your database to verify liquibase creation is working in postgesql, run the following commands: psql -d mydb drop schema public cascade; create schema public; after writing minimal code for spring data and configuring liquibase to create tables/relationships, i relaxed a bit, documented how everything worked and added a loggingfilter . the loggingfilter was handy for viewing api requests and responses. @bean public filterregistrationbean loggingfilter() { loggingfilter filter = new loggingfilter(); filterregistrationbean registrationbean = new filterregistrationbean(); registrationbean.setfilter(filter); registrationbean.seturlpatterns(arrays.aslist("/api/*")); return registrationbean; } accessing api with resttemplate the final step i needed to do was figure out how to access my new and fancy api with resttemplate . at first, i thought it would be easy. then i realized that spring data produces a hal -compliant api, so its content is embedded inside an "_embedded" json key. after much trial and error, i discovered i needed to create a resttemplate with hal and joda-time awareness. @bean public resttemplate resttemplate() { objectmapper mapper = new objectmapper(); mapper.configure(deserializationfeature.fail_on_unknown_properties, false); mapper.registermodule(new jackson2halmodule()); mapper.registermodule(new jodamodule()); mappingjackson2httpmessageconverter converter = new mappingjackson2httpmessageconverter(); converter.setsupportedmediatypes(mediatype.parsemediatypes("application/hal+json")); converter.setobjectmapper(mapper); stringhttpmessageconverter stringconverter = new stringhttpmessageconverter(); stringconverter.setsupportedmediatypes(mediatype.parsemediatypes("application/xml")); list> converters = new arraylist<>(); converters.add(converter); converters.add(stringconverter); return new resttemplate(converters); } the jodamodule was provided by the following dependency: com.fasterxml.jackson.datatype jackson-datatype-joda with the configuration complete, i was able to write a messagesapiitest integration test that posts a request and retrieves it using the api. the api was secured using basic authentication, so it took me a bit to figure out how to make that work with resttemplate. willie wheeler's basic authentication with spring resttemplate was a big help. @runwith(springjunit4classrunner.class) @contextconfiguration(classes = integrationtestconfig.class) public class messagesapiitest { private final static log log = logfactory.getlog(messagesapiitest.class); @value("http://${app.host}/api/initiate") private string initiateapi; @value("http://${app.host}/api/messages") private string messagesapi; @value("${app.host}") private string host; @inject private resttemplate resttemplate; @before public void setup() throws exception { string request = new scanner(new classpathresource("sample-request.xml").getfile()).usedelimiter("\\z").next(); responseentity response = resttemplate.exchange(gettesturl(initiateapi), httpmethod.post, getbasicauthheaders(request), org.ncpdp.schema.transport.message.class, collections.emptymap()); assertequals(httpstatus.ok, response.getstatuscode()); } @test public void testgetmessages() { httpentity request = getbasicauthheaders(null); responseentity> result = resttemplate.exchange(gettesturl(messagesapi), httpmethod.get, request, new parameterizedtypereference>() {}); httpstatus status = result.getstatuscode(); collection messages = result.getbody().getcontent(); log.debug("messages found: " + messages.size()); assertequals(httpstatus.ok, status); for (message message : messages) { log.debug("message.id: " + message.getid()); log.debug("message.datecreated: " + message.getdatecreated()); } } private httpentity getbasicauthheaders(string body) { string plaincreds = "user:pass"; byte[] plaincredsbytes = plaincreds.getbytes(); byte[] base64credsbytes = base64.encodebase64(plaincredsbytes); string base64creds = new string(base64credsbytes); httpheaders headers = new httpheaders(); headers.add("authorization", "basic " + base64creds); headers.add("content-type", "application/xml"); if (body == null) { return new httpentity<>(headers); } else { return new httpentity<>(body, headers); } } } to get spring data to populate the message id, i created a custom restconfig class to expose it. i learned how to do this from tommy ziegler . /** * used to expose ids for resources. */ @configuration public class restconfig extends repositoryrestmvcconfiguration { @override protected void configurerepositoryrestconfiguration(repositoryrestconfiguration config) { config.exposeidsfor(message.class); config.setbaseuri("/api"); } } summary this article explains how i built a rest api using jaxb, spring boot, spring data and liquibase. it was relatively easy to build, but required some tricks to access it with spring's resttemplate. figuring out how to customize jaxb's code generation was also essential to make things work. i started developing the project with spring boot 1.1.7, but upgraded to 1.2.0.m2 after i found it supported log4j2 and configuring spring data rest's base uri in application.yml. when i handed the project off to my client last week, it was using 1.2.0.build-snapshot because of a bug when running in tomcat . this was an enjoyable project to work on. i especially liked how easy spring data makes it to expose jpa entities in an api. spring boot made things easy to configure once again and liquibase seems like a nice tool for database migrations. if someone asked me to develop a rest api on the jvm, which frameworks would i use? spring boot, spring data, jackson, joda-time, lombok and liquibase. these frameworks worked really well for me on this particular project.

I’ve been playing around with Python over the last few days while cleaning up a data set and one thing I wanted to do was translate date strings into a timestamp. I started with a date in this format: date_text = "13SEP2014" So the first step is to translate that into a Python date – the strftime section of the documentation is useful for figuring out which format code is needed: import datetime date_text = "13SEP2014" date = datetime.datetime.strptime(date_text, "%d%b%Y") print(date) $ python dates.py 2014-09-13 00:00:00 The next step was to translate that to a UNIX timestamp. I thought there might be a method or property on the Date object that I could access but I couldn’t find one and so ended up using calendar to do the transformation: import datetime import calendar date_text = "13SEP2014" date = datetime.datetime.strptime(date_text, "%d%b%Y") print(date) print(calendar.timegm(date.utctimetuple())) $ python dates.py 2014-09-13 00:00:00 1410566400 It’s not too tricky so hopefully I shall remember next time.

In this article, we learn how to utilize AngularJS's array filter features in a variety of use cases. Read on to get started.

I prefer OpenCSV for CSV parsing in Java. That library also supports parsing of tab delimited files, here's how: Just a quick gist: import au.com.bytecode.opencsv.CSVReader; public class LoadTest { @Test public void testLoad(String row) throws IOException, JobNotFoundException, InterruptedException { CSVReader reader = new CSVReader(new FileReader("/Users/bone/Desktop/foo_data.tab"), '\t'); String[] record; while ((record = reader.readNext()) != null) { for (String value : record) { System.out.println(value); // Debug only } } } }

introduction in this tutorial, the apache lucene and apache tika frameworks will be explained through their core concepts (e.g. parsing, mime detection, content analysis, indexing, scoring, boosting) via illustrative examples that should be applicable to not only seasoned software developers but to beginners to content analysis and programming as well. we assume you have a working knowledge of the java™ programming language and plenty of content to analyze. throughout this tutorial, you will learn: how to use apache tika's api and its most relevant functions how to develop code with apache lucene api and its most important modules how to integrate apache lucene and apache tika in order to build your own piece of software that stores and retrieves information efficiently. (project code is available for download) what are lucene and tika? according to apache lucene's site, apache lucene represents an open source java library for indexing and searching from within large collections of documents. the index size represents roughly 20-30% the size of text indexed and the search algorithms provide features like: ranked searching - best results returned first many powerful query types: phrase queries, wildcard queries, proximity queries, range queries and more. in this tutorial we will demonstrate only phrase queries. fielded search (e.g. title, author, contents) sorting by any field flexible faceting, highlighting, joins and result grouping pluggable ranking models, including the vector space model and okapi bm25 but lucene's main purpose is to deal directly with text and we want to manipulate documents, who have various formats and encoding. for parsing document content and their properties the apache tika library it is necessary. apache tika is a library that provides a flexible and robust set of interfaces that can be used in any context where metadata analyzis and structured text extraction is needed. the key component of apache tika is the parser (org.apache.tika.parser.parser ) interface because it hides the complexity of different file formats while providing a simple and powerful mechanism to extract structured text content and metadata from all sorts of documents. criterias for tika parsing design streamed parsing the interface should require neither the client application nor the parser implementation to keep the full document content in memory or spooled to disk. this allows even huge documents to be parsed without excessive resource requirements. structured content a parser implementation should be able to include structural information (headings, links, etc.) in the extracted content. a client application can use this information for example to better judge the relevance of different parts of the parsed document. input metadata a client application should be able to include metadata like the file name or declared content type with the document to be parsed. the parser implementation can use this information to better guide the parsing process. output metadata a parser implementation should be able to return document metadata in addition to document content. many document formats contain metadata like the name of the author that may be useful to client applications. context sensitivity while the default settings and behaviour of tika parsers should work well for most use cases, there are still situations where more fine-grained control over the parsing process is desirable. it should be easy to inject such context-specific information to the parsing process without breaking the layers of abstraction. requirements maven 2.0 or higher java 1.6 se or higher lesson 1: automate metadata extraction from any file type our premisses are the following: we have a collection of documents stored on disk/database and we would like to index them; these documents can be word documents, pdfs, htmls, plain text files etc. as we are developers, we would like to write reusable code that extracts file properties regarding format (metadata) and file content. apache tika has a mimetype repository and a set of schemes (any combination of mime magic, url patterns, xml root characters, or file extensions) to determine if a particular file, url, or piece of content matches one of its known types. if the content does match, tika has detected its mimetype and can proceed to select the appropriate parser. in the sample code, the file type detection and its parsing is being covered inside the class com.retriever.lucene.index.indexcreator , method indexfile. listing 1.1 analyzing a file with tika public static documentwithabstract indexfile(analyzer analyzer, file file) throws ioexception { metadata metadata = new metadata(); contenthandler handler = new bodycontenthandler(10 * 1024 * 1024); parsecontext context = new parsecontext(); parser parser = new autodetectparser(); inputstream stream = new fileinputstream(file); //open stream try { parser.parse(stream, handler, metadata, context); //parse the stream } catch (tikaexception e) { e.printstacktrace(); } catch (saxexception e) { e.printstacktrace(); } finally { stream.close(); //close the stream } //more code here } the above code displays how a file it is being parsed using org.apache.tika.parser.autodetectparser; this kind of implementation was chosen because we would like to achieve parsing documents disregarding their format. also, for handling the content the org.apache.tika.sax.bodycontenthandler wasconstructed with a writelimit given as parameter ( 10*1024*1024); this type of constructor creates a content handler that writes xhtml body character events to an internal string buffer and in case of documents with large content is less likely to throw a saxexception (thrown when the default write limit is reached). as a result of our parsing we have obtained a metadata object that we can now use to detect file properties (title or any other header specific to a document format). metadata processing can be done as described below ( com.retriever.lucene.index.indexcreator , method indexfiledescriptors) : listing 1.2 processing metadata private static document indexfiledescriptors(string filename, metadata metadata) { document doc = new document(); //store file name in a separate textfield doc.add(new textfield(isearchconstants.field_file, filename, store.yes)); for (string key : metadata.names()) { string name = key.tolowercase(); string value = metadata.get(key); if (stringutils.isblank(value)) { continue; } if ("keywords".equalsignorecase(key)) { for (string keyword : value.split(",?(\\s+)")) { doc.add(new textfield(name, keyword, store.yes)); } } else if (isearchconstants.field_title.equalsignorecase(key)) { doc.add(new textfield(name, value, store.yes)); } else { doc.add(new textfield(name, filename, store.no)); } } in the method presented above we store the file name in a separate field and also the document's title ( a document can have a title different from its file name); we are not interested in storing other informations.