DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Newsletter
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Java Topics

article thumbnail
Making Operations on Volatile Fields Atomic
The expected behaviour for volatile fields is that they should behave in a multi-threaded application the same as they do in a single threaded application. They are not forbidden to behave the same way, but they are not guaranteed to behave the same way. The solution in Java 5.0+ is to use AtomicXxxx classes however these are relatively inefficient in terms of memory (they add a header and padding), performance (they add a references and little control over their relative positions), and syntactically they are not as clear to use. IMHO A simple solution if for volatile fields to act as they might be expected to do, the way JVM must support in AtomicFields which is not forbidden in the current JMM (Java- Memory Model) but not guaranteed. Why make fields volatile? The benefit of volatile fields is that they are visible across threads and some optimisations which avoid re-reading them are disabled so you always check again the current value even if you didn't change them. e.g. without volatile Thread 2: int a = 5; Thread 1: a = 6; (later) Thread 2: System.out.println(a); // prints 5 With volatile Thread 2: volatile int a = 5; Thread 1: a = 6; (later) Thread 2: System.out.println(a); // prints 6 Why not use volatile all the time? Volatile read and write access is substantially slower. When you write to a volatile field it stalls the entire CPU pipeline to ensure the data has been written to cache. Without this, there is a risk the next read of the value sees an old value, even in the same thread (See AtomicLong.lazySet() which avoids stalling the pipeline) The penalty can be in the order of 10x slower which you don't want to be doing on every access. What are the limitations of volatile? A significant limitation is that operations on the field is not atomic, even when you might think it is. Even worse than that is that usually, there is no difference. I.e. it can appear to work for a long time even years and suddenly/randomly break due to an incidental change such as the version of Java used, or even where the object is loaded into memory. e.g. which programs you loaded before running the program. e.g. updating a value Thread 2: volatile int a = 5; Thread 1: a += 1; Thread 2: a += 2; (later) Thread 2: System.out.println(a); // prints 6, 7 or 8 This is an issue because the read of a and the write of a are done separately and you can get a race condition. 99%+ of the time it will behave as expect, but sometimes it won't/ What can you do about it? You need to use AtomicXxxx classes. These wrap volatile fields with operations which behave as expected. Thread 2: AtomicInteger a = new AtomicInteger(5); Thread 1: a.incrementAndGet(); Thread 2: a.addAndGet(2); (later) Thread 2: System.out.println(a); // prints 8 What do I propose? The JVM has a means to behave as expected, the only surprising thing is you need to use a special class to do what the JMM won't guarantee for you. What I propose is that the JMM be changed to support the behaviour currently provided by the concurrency AtomicClasses. In each case the single threaded behaviour is unchanged. A multi-threaded program which does not see a race condition will behave the same. The difference is that a multi-threaded program does not have to see a race condition but changing the underlying behaviour current method suggested syntax notes x.getAndIncrement() x++ or x += 1 x.incrementAndGet() ++x x.getAndDecrment() x-- or x -= 1 x.decrementAndGet() --x x.addAndGet(y) (x += y) x.getAndAdd(y) ((x += y)-y) x.compareAndSet(e, y) (x == e ? x = y, true : false) Need to add the comma syntax used in other languages. These operations could be supported for all the primitive types such as boolean, byte, short, int, long, float and double. Additional assignment operators could be supported such as current method suggested syntax notes Atomic multiplication x *= 2; Atomic subtraction x -= y; Atomic division x /= y; Atomic modulus x %= y; Atomic shift x <<= y; Atomic shift x >>= z; Atomic shift x >>>= w; Atomic and x &= ~y; clears bits Atomic or x |= z; sets bits Atomic xor x ^= w; flips bits What is the risk? This could break code which relies on these operations occasionally failing due to race conditions. It might not be possible to support more complex expressions in a thread safe manner. This could lead to surprising bugs as the code can look like the works, but it doesn't. Never the less it will be no worse than the current state. JEP 193 - Enhanced Volatiles There is a JEP 193 to add this functionality to Java. An example is; class Usage { volatile int count; int incrementCount() { return count.volatile.incrementAndGet(); } } IMHO there is a few limitations in this approach. The syntax is fairly significant change. Changing the JMM might not require many changes the the Java syntax and possibly no changes to the compiler. It is a less general solution. It can be useful to support operations like volume += quantity; where these are double types. It places more burden on the developer to understand why he/she should use this instead of x++; Conclusion Much of the syntactic and performance overhead of using AtomicInteger and AtomicLong could be removed if the JMM guaranteed the equivalent single threaded operations behaved as expected for multi-threaded code. This feature could be added to earlier versions of Java by using byte code instrumentation.
June 27, 2014
by Peter Lawrey
· 12,095 Views
article thumbnail
Eclipse Community Survey 2014 Results
We have published the results of the Eclipse Community Survey 2014. Thank you to everyone who participated in the survey this year. The complete results and data are available for anyone to download [xls] [ods]. As in other years, I think the results provide an interesting perspective on what tools software developers are using and the type of applications they are building. Here are some key highlights from the results this year: 1) Git #1 Code Management Tool. Git has finally surpassed Subversion to be the top code management tool used by software developers. A third of developers (33.3%) report they use Git as their primary code management tool compared to 30.7% using Subversion. Subversion continues to show a downward trend from previous years when it was used by more than half the developers. Of note, 9.6% claim GitHub is their primary code management tool so the prevalence of overall Git usage is becoming dominate. 2) Maven and Jenkins Key Tools. For Build and Release tools, Maven and Jenkins continue to be key tools used by developers. Of interest is the growth of Gradle from 2013 (4.5%) to 2014 (11%). 3) Top 3 Application Servers. Tomcat (32.6%), JBoss (11.8%) and Jetty (7.2%) continue to be the top 3 application servers. 4) Java 8 Adoption. Java 8 was released in March 2014 and already 9.2% of Java developers have migrated to Java 8 as their primary version of Java. 59.2% are using Java 7 but close to a quarter are using Java 6 or before. 5) Majority of Developers Use JavaScript. More and more software developers use multiple languages to develop software. Due to the Eclipse biased of the survey, Java is not surprisingly the top primary language. However, when asked what other languages developers might use, JavaScript stands out to be a popular language with a the majority of developers (56.2%) claiming it as a secondary language. 6) Developers Experimenting With Open Hardware. The Internet of Things (IoT) and Open Hardware have become important industry trends in the last couple of years. Over a third (35.7%) of software developers are spending their own personal time learning about devices like the BeagleBone, Arduino and Raspberry Pi. Thanks again to everyone who participated in the survey. I hope everyone finds the results of interest.
June 25, 2014
by Ian Skerrett
· 14,469 Views
article thumbnail
Using Markdown Syntax in Javadoc
In this post we will see how we can write Javadoc comments using Markdown instead of the typical Javadoc syntax. So what is Markdown? Markdown is a plain text formatting syntax designed so that it optionally can be converted to HTML using a tool by the same name. Markdown is popularly used to format readme files, for writing messages in online discussion forums or in text editors for the quick creation of rich text documents. (Wikipedia: Markdown) Markdown is a very easy to read formatting syntax. Different variations of Markdown can be used on Stack Overflow or GitHub to format user generated content. Setup By default the Javadoc tool uses Javadoc comments to generate API documentation in HTML form. This process can be customized used Doclets. Doclets are Java programs that specify the content and format of the output of the Javadoc tool. The markdown-doclet is a replacement for the standard Java Doclet which gives developers the option to use Markdown syntax in their Javadoc comments. We can set up this doclet in Maven using the maven-javadoc-plugin. maven-javadoc-plugin 2.9 ch.raffael.doclets.pegdown.PegdownDoclet ch.raffael.pegdown-doclet pegdown-doclet 1.1 true Writing comments in Markdown Now we can use Markdown syntax in Javadoc comments: /** * ## Large headline * ### Smaller headline * * This is a comment that contains `code` parts. * * Code blocks: * * ```java * int foo = 42; * System.out.println(foo); * ``` * * Quote blocks: * * > This is a block quote * * lists: * * - first item * - second item * - third item * * This is a text that contains an [external link][link]. * * [link]: http://external-link.com/ * * @param id the user id * @return the user object with the passed `id` or `null` if no user with this `id` is found */ public User findUser(long id) { ... } After running mvn javadoc:javadoc we can find the generated HTML API documentation in target/site/apidocs. The generated documentation for the method shown above looks like this: As we can see the Javadoc comments get nicely converted to HTML. Conclusion Markdown has the clear advantage over standard Javadoc syntax that the source it is far easier to read. Just have a look at some of the method comments of java.util.Map. Many Javadoc comments are full with formatting tags and are barely readable without any tool. But be aware that Markdown can cause problems with tools and IDEs that expect standard Javadoc syntax.
June 23, 2014
by Michael Scharhag
· 15,905 Views · 1 Like
article thumbnail
Java Code Review Checklist
Utilize this checklist to review the quality of your Java code, including security, performance, and static code analysis.
June 20, 2014
by Mahesh Chopker
· 212,088 Views · 22 Likes
article thumbnail
Spring Security Misconfiguration
I recently saw Mike Wienser’s SpringOne2GX talk about Application Security Pitfalls. It is very informative and worth watching if you are using Spring’s stack on servlet container. It reminded me one serious Spring Security Misconfiguration I was facing once. Going to explain it on Spring’s Guide Project called Securing a Web Application. This project uses Spring Boot, Spring Integration and Spring MVC. Project uses these views: @Configuration public class MvcConfig extends WebMvcConfigurerAdapter { @Override public void addViewControllers(ViewControllerRegistry registry) { registry.addViewController("/home").setViewName("home"); registry.addViewController("/").setViewName("home"); registry.addViewController("/hello").setViewName("hello"); registry.addViewController("/login").setViewName("login"); } } Where “/home”, “/” and “/login” URLs should be publicly accessible and “/hello” should be accessible only to authenticated user. Here is original Spring Security configuration from Guide: @Configuration @EnableWebMvcSecurity public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Override protected void configure(HttpSecurity http) throws Exception { http .authorizeRequests() .antMatchers("/", "/home").permitAll() .anyRequest().authenticated(); http .formLogin() .loginPage("/login") .permitAll() .and() .logout() .permitAll(); } @Override protected void configure(AuthenticationManagerBuilder auth) throws Exception { auth .inMemoryAuthentication() .withUser("user").password("password").roles("USER"); } } Nice and explanatory as all Spring’s Guides are. First “configure” method registers “/” and “home” as public and specifies that everything else should be authenticated. It also registers login URL. Second “configure” method specifies authentication method for role “USER”. Of course you don’t want to use it like this in production :). Now I am going to slightly amend this code. @Override protected void configure(HttpSecurity http) throws Exception { //!!! Don't use this example !!! http .authorizeRequests() .antMatchers("/hello").hasRole("USER"); //... same as above ... } Everything is public and private endpoints have to be listed. You can see that my amended code have the same behavior as original. In fact it saved one line of code. But there is serious problem with this. What if my I need to introduce new private endpoint? Let’s say I am not aware of the fact that it needs to be registered in Spring Security configuration. My new endpoint would be public. Such misconfiguration is really hard to catch and can lead to unwanted exposure of URLs. So conclusion is: Always authenticate all endpoints by default.
June 17, 2014
by Lubos Krnac
· 12,222 Views · 1 Like
article thumbnail
Higher-order Functions, Functions Composition, and Currying in Java 8
The main concept behind functional programming is that data and behaviors can be treated and manipulated uniformly and in the same way. In practical terms this means that it is possible to pass to a method both values and functions and in the same way the method itself can return either a value or a function. With this regard a function accepting one or more functions as argument and/or returning another function as result is called an higher-order function. The purpose of this article is explaining what functions composition and currying are demonstrating why they can be both seen as special cases of higher-order functions and showing how to use them with a practical example in Java 8. Let's define a Converter as an object with a single method that takes as input a conversion rate and the value to be converted and returns the conversion's result that is obtained by simply multiplying these 2 arguments. Given this description the Converter can be seen as a function of 2 parameters and then defined as a particular implementation of the BiFunction interface taking 2 Doubles, the conversion rate and the value to be converted, as arguments and returning a third Double that is the converted value. public class Converter implements BiFunction { @Override public Double apply(Double conversionRate, Double value) { return conversionRate * value; } } In this way we could for instance convert 10, 20 or 50 miles in kilometers by instancing a new Converter and passing to it the conversion rate between miles and kilometers together with the number of miles to be converted. Converter converter = new Converter(); double tenMilesInKm = converter.apply(1.609, 10.0); double twentyMilesInKm = converter.apply(1.609, 20.0); double fiftyMilesInKm = converter.apply(1.609, 50.0); This works, but there is something not very practical with this approach: we are obliged to repeat the same conversion rate for all the invocation of our converter. It would be better if our converter could offer an easy way to obtain a specialized version of it just converting miles to kilometers. In other words we would like to create a Function out of the original BiFunction having set one of the 2 arguments of the BiFunction to a fixed value, that in our case is the miles/km conversion rate. In functional programming this particular operation is called currying. Currying Unfortunately the native Java 8 BiFunction interface doesn't provide currying out of the box. Nevertheless it is very easy to develop a custom extension of the original BiFunction interface having 2 additional default methods that allow to set one of the 2 arguments of the BiFunction to a fixed value. @FunctionalInterface public interface ExtendedBiFunction extends BiFunction { default Function curry1(T t) { return u -> apply(t, u); } default Function curry2(U u) { return t -> apply(t, u); } } The methods curry1 and curry2 fix respectively the first and the second argument of the BiFunction and return a Function having as single argument the one of the two that is still remained unfixed. If we now change our Converter just making it to implement this ExtendedBiFunction interface public class Converter implements ExtendedBiFunction it's possible to obtain a Function converting miles to kilometers fixing the first argument of the converter to the appropriate conversion rate. Function mi2kmConverter = converter.curry1(1.609); double tenMilesInKm = mi2kmConverter.apply(10.0); double twentyMilesInKm = mi2kmConverter.apply(20.0); double fiftyMilesInKm = mi2kmConverter.apply(50.0); Of course we can repeat this specialization process how many times we want for example to obtain a converter from ounces to grams. Function ou2grConverter = converter.curry1(28.345); double tenOuncesInGr = ou2grConverter.apply(10.0); double twentyOuncesInGr = ou2grConverter.apply(20.0); double fiftyOuncesInGr = ou2grConverter.apply(50.0); More formally currying is a technique where a function f of two arguments (x and y say) is seen instead as a function g of one argument which returns a function also of one argument. The value returned by the latter function is the same as the value of the original function. f(x,y) = (g(x))(y) Of course the same principle can be generalized to transform a function of n arguments in one of n-1 arguments having fixed the remaining one. It's also easy to see that the curry1 and curry2 methods are examples of higher-order function since they returns a Function as result. Function composition In the biggest part of cases a multiplication, as the one performed by our Converter class, is all you need to do to convert a unit measure into another. However there are some situations for which this couldn't be enough. For instance the formula to convert a Celsius temperature in a Farenheit one is: F = C * 9/5 + 32 After having multiplied the Celsius temperature by the 9/5 factor, you still have to add 32 in order to obtain the corresponding Farenheit value. Is there a way to obtain a Function converting Celsius to Farenheit degrees from our generic Converter? This is a case where functions composition can come to the rescue. In fact the native Java 8 Function interface already provides a default method named andThen accepting another Function as argument and returning a third Function as result (another higher-order function) that is the composition of the first 2. The resulting composed function first applies the original function to its input, and then applies the function passed as argument to the result. In this way the Celsius to Farenheit conversion function can be obtained currying the generic converter with the 9/5 factor and then applying a further function that increments the result of the first one of 32. Function celsius2farenheitConverter = converter.curry1(9.0/5).andThen(n -> n + 32); double tenCInF = celsius2farenheitConverter.apply(10.0); double twentyCInF = celsius2farenheitConverter.apply(20.0); double fiftyCInF = celsius2farenheitConverter.apply(50.0); What about the opposite conversion? We have to use the formula: C = (F - 32) * 5/9 meaning that this time the subtraction has to be performed before the multiplication by the conversion rate. In other words we need to compose the original conversion function with another function (the subtraction) that has to be applied before it and not after as we did above. The native Java 8 Function interface already provides a compose method allowing to achieve this, but for some reason the same method is not available also on the BiFunction interface. This is not too bad since we can add it (actually we need 2 of them, one for each argument) to our ExtendedBiFunction interface. @FunctionalInterface public interface ExtendedBiFunction extends BiFunction { default Function curry1(T t) { return u -> apply(t, u); } default Function curry2(U u) { return t -> apply(t, u); } default ExtendedBiFunction compose1(Function before) { return (v, u) -> apply(before.apply(v), u); } default ExtendedBiFunction compose2(Function before) { return (t, v) -> apply(t, before.apply(v)); } } Here the compose1 higher-order function composes the BiFuction with the before Function and returns another BiFunction that, when applied, first transform its first argument with the before Function and then applies the original BiFunction with the transformed first argument and the unchanged second one. The compose2 method implements exactly the same principle to the second argument of the BiFunction leaving unchanged the first one. We can now obtain the Farenheit to Celsius converter composing the generic Converter with a Function that subtract 32 from the value to be converted before before to currying the first argument fixing it to 5/9. Function farenheit2celsiusConverter = converter.compose2((Double n) -> n - 32).curry1(5.0/9); double tenFInC = farenheit2celsiusConverter.apply(10.0); double twentyFInC = farenheit2celsiusConverter.apply(20.0); double fiftyFInC = farenheit2celsiusConverter.apply(50.0); If you want you can both deepen your knowledge of the new API introduced into Java in its 8th major release and learn how to leverage the new functional features of the language reading the Java 8 in Action book that I just finished to write together with Raoul-Gabriel Urma and Alan Mycroft.
June 17, 2014
by Mario Fusco
· 55,135 Views · 11 Likes
article thumbnail
Java 8 Friday: 10 Subtle Mistakes When Using the Streams API
at data geekery , we love java. and as we’re really into jooq’s fluent api and query dsl , we’re absolutely thrilled about what java 8 will bring to our ecosystem. java 8 friday every friday, we’re showing you a couple of nice new tutorial-style java 8 features, which take advantage of lambda expressions, extension methods, and other great stuff. you’ll find the source code on github . 10 subtle mistakes when using the streams api we’ve done all the sql mistakes lists: 10 common mistakes java developers make when writing sql 10 more common mistakes java developers make when writing sql yet another 10 common mistakes java developers make when writing sql (you won’t believe the last one) but we haven’t done a top 10 mistakes list with java 8 yet! for today’s occasion ( it’s friday the 13th ), we’ll catch up with what will go wrong in your application when you’re working with java 8. (it won’t happen to us, as we’re stuck with java 6 for another while) 1. accidentally reusing streams wanna bet, this will happen to everyone at least once. like the existing “streams” (e.g. inputstream ), you can consume streams only once. the following code won’t work: intstream stream = intstream.of(1, 2); stream.foreach(system.out::println); // that was fun! let's do it again! stream.foreach(system.out::println); you’ll get a java.lang.illegalstateexception: stream has already been operated upon or closed so be careful when consuming your stream. it can be done only once 2. accidentally creating “infinite” streams you can create infinite streams quite easily without noticing. take the following example: // will run indefinitely intstream.iterate(0, i -> i + 1) .foreach(system.out::println); the whole point of streams is the fact that they can be infinite, if you design them to be. the only problem is, that you might not have wanted that. so, be sure to always put proper limits: // that's better intstream.iterate(0, i -> i + 1) .limit(10) .foreach(system.out::println); 3. accidentally creating “subtle” infinite streams we can’t say this enough. you will eventually create an infinite stream, accidentally. take the following stream, for instance: intstream.iterate(0, i -> ( i + 1) % 2) .distinct() .limit(10) .foreach(system.out::println); so… we generate alternating 0′s and 1′s then we keep only distinct values, i.e. a single 0 and a single 1 then we limit the stream to a size of 10 then we consume it well… the distinct() operation doesn’t know that the function supplied to the iterate() method will produce only two distinct values. it might expect more than that. so it’ll forever consume new values from the stream, and the limit(10) will never be reached. tough luck, your application stalls. 4. accidentally creating “subtle” parallel infinite streams we really need to insist that you might accidentally try to consume an infinite stream. let’s assume you believe that the distinct() operation should be performed in parallel. you might be writing this: intstream.iterate(0, i -> ( i + 1) % 2) .parallel() .distinct() .limit(10) .foreach(system.out::println); now, we’ve already seen that this will turn forever. but previously, at least, you only consumed one cpu on your machine. now, you’ll probably consume four of them, potentially occupying pretty much all of your system with an accidental infinite stream consumption. that’s pretty bad. you can probably hard-reboot your server / development machine after that. have a last look at what my laptop looked like prior to exploding: if i were a laptop, this is how i’d like to go. 5. mixing up the order of operations so, why did we insist on your definitely accidentally creating infinite streams? it’s simple. because you may just accidentally do it. the above stream can be perfectly consumed if you switch the order of limit() and distinct() : intstream.iterate(0, i -> ( i + 1) % 2) .limit(10) .distinct() .foreach(system.out::println); this now yields: 0 1 why? because we first limit the infinite stream to 10 values (0 1 0 1 0 1 0 1 0 1), before we reduce the limited stream to the distinct values contained in it (0 1). of course, this may no longer be semantically correct, because you really wanted the first 10 distinct values from a set of data (you just happened to have “forgotten” that the data is infinite). no one really wants 10 random values, and only then reduce them to be distinct. if you’re coming from a sql background, you might not expect such differences. take sql server 2012, for instance. the following two sql statements are the same: -- using top selectdistincttop10 * fromi orderby.. -- using fetch select* fromi orderby.. offset 0 rows fetchnext10 rowsonly so, as a sql person, you might not be as aware of the importance of the order of streams operations. 6. mixing up the order of operations (again) speaking of sql, if you’re a mysql or postgresql person, you might be used to the limit .. offset clause. sql is full of subtle quirks, and this is one of them. the offset clause is applied first , as suggested in sql server 2012′s (i.e. the sql:2008 standard’s) syntax. if you translate mysql / postgresql’s dialect directly to streams, you’ll probably get it wrong: intstream.iterate(0, i -> i + 1) .limit(10) // limit .skip(5) // offset .foreach(system.out::println); the above yields 5 6 7 8 9 yes. it doesn’t continue after 9 , because the limit() is now applied first , producing (0 1 2 3 4 5 6 7 8 9). skip() is applied after, reducing the stream to (5 6 7 8 9). not what you may have intended. beware of the limit .. offset vs. "offset .. limit" trap! 7. walking the file system with filters we’ve blogged about this before . what appears to be a good idea is to walk the file system using filters: files.walk(paths.get(".")) .filter(p -> !p.tofile().getname().startswith(".")) .foreach(system.out::println); the above stream appears to be walking only through non-hidden directories, i.e. directories that do not start with a dot. unfortunately, you’ve again made mistake #5 and #6. walk() has already produced the whole stream of subdirectories of the current directory. lazily, though, but logically containing all sub-paths. now, the filter will correctly filter out paths whose names start with a dot “.”. e.g. .git or .idea will not be part of the resulting stream. but these paths will be: .\.git\refs , or .\.idea\libraries . not what you intended. now, don’t fix this by writing the following: files.walk(paths.get(".")) .filter(p -> !p.tostring().contains(file.separator + ".")) .foreach(system.out::println); while that will produce the correct output, it will still do so by traversing the complete directory subtree, recursing into all subdirectories of “hidden” directories. i guess you’ll have to resort to good old jdk 1.0 file.list() again. the good news is, filenamefilter and filefilter are both functional interfaces. 8. modifying the backing collection of a stream while you’re iterating a list , you must not modify that same list in the iteration body. that was true before java 8, but it might become more tricky with java 8 streams. consider the following list from 0..9: // of course, we create this list using streams: list list = intstream.range(0, 10) .boxed() .collect(tocollection(arraylist::new)); now, let’s assume that we want to remove each element while consuming it: list.stream() // remove(object), not remove(int)! .peek(list::remove) .foreach(system.out::println); interestingly enough, this will work for some of the elements! the output you might get is this one: 0 2 4 6 8 null null null null null java.util.concurrentmodificationexception if we introspect the list after catching that exception, there’s a funny finding. we’ll get: [1, 3, 5, 7, 9] heh, it “worked” for all the odd numbers. is this a bug? no, it looks like a feature. if you’re delving into the jdk code, you’ll find this comment in arraylist.arralistspliterator : /* * if arraylists were immutable, or structurally immutable (no * adds, removes, etc), we could implement their spliterators * with arrays.spliterator. instead we detect as much * interference during traversal as practical without * sacrificing much performance. we rely primarily on * modcounts. these are not guaranteed to detect concurrency * violations, and are sometimes overly conservative about * within-thread interference, but detect enough problems to * be worthwhile in practice. to carry this out, we (1) lazily * initialize fence and expectedmodcount until the latest * point that we need to commit to the state we are checking * against; thus improving precision. (this doesn't apply to * sublists, that create spliterators with current non-lazy * values). (2) we perform only a single * concurrentmodificationexception check at the end of foreach * (the most performance-sensitive method). when using foreach * (as opposed to iterators), we can normally only detect * interference after actions, not before. further * cme-triggering checks apply to all other possible * violations of assumptions for example null or too-small * elementdata array given its size(), that could only have * occurred due to interference. this allows the inner loop * of foreach to run without any further checks, and * simplifies lambda-resolution. while this does entail a * number of checks, note that in the common case of * list.stream().foreach(a), no checks or other computation * occur anywhere other than inside foreach itself. the other * less-often-used methods cannot take advantage of most of * these streamlinings. */ now, check out what happens when we tell the stream to produce sorted() results: list.stream() .sorted() .peek(list::remove) .foreach(system.out::println); this will now produce the following, “expected” output 0 1 2 3 4 5 6 7 8 9 and the list after stream consumption? it is empty: [] so, all elements are consumed, and removed correctly. the sorted() operation is a “stateful intermediate operation” , which means that subsequent operations no longer operate on the backing collection, but on an internal state. it is now “safe” to remove elements from the list! well… can we really? let’s proceed with parallel() , sorted() removal: list.stream() .sorted() .parallel() .peek(list::remove) .foreach(system.out::println); this now yields: 7 6 2 5 8 4 1 0 9 3 and the list contains [8] eek. we didn’t remove all elements!? free beers ( and jooq stickers ) go to anyone who solves this streams puzzler! this all appears quite random and subtle, we can only suggest that you never actually do modify a backing collection while consuming a stream. it just doesn’t work. 9. forgetting to actually consume the stream what do you think the following stream does? intstream.range(1, 5) .peek(system.out::println) .peek(i -> { if(i == 5) thrownewruntimeexception("bang"); }); when you read this, you might think that it will print (1 2 3 4 5) and then throw an exception. but that’s not correct. it won’t do anything. the stream just sits there, never having been consumed. as with any fluent api or dsl, you might actually forget to call the “terminal” operation. this might be particularly true when you use peek() , as peek() is an aweful lot similar to foreach() . this can happen with jooq just the same, when you forget to call execute() or fetch() : dsl.using(configuration) .update(table) .set(table.col1, 1) .set(table.col2, "abc") .where(table.id.eq(3)); oops. no execute() yes, the “best” way – with 1-2 caveats ;-) 10. parallel stream deadlock this is now a real goodie for the end! all concurrent systems can run into deadlocks, if you don’t properly synchronise things. while finding a real-world example isn’t obvious, finding a forced example is. the following parallel() stream is guaranteed to run into a deadlock: object[] locks = { newobject(), newobject() }; intstream .range(1, 5) .parallel() .peek(unchecked.intconsumer(i -> { synchronized(locks[i % locks.length]) { thread.sleep(100); synchronized(locks[(i + 1) % locks.length]) { thread.sleep(50); } } })) .foreach(system.out::println); note the use of unchecked.intconsumer() , which transforms the functional intconsumer interface into a org.jooq.lambda.fi.util.function.checkedintconsumer , which is allowed to throw checked exceptions. well. tough luck for your machine. those threads will be blocked forever :-) the good news is, it has never been easier to produce a schoolbook example of a deadlock in java! for more details, see also brian goetz’s answer to this question on stack overflow . conclusion with streams and functional thinking, we’ll run into a massive amount of new, subtle bugs. few of these bugs can be prevented, except through practice and staying focused. you have to think about how to order your operations. you have to think about whether your streams may be infinite. streams (and lambdas) are a very powerful tool. but a tool which we need to get a hang of, first.
June 16, 2014
by Lukas Eder
· 10,347 Views · 2 Likes
article thumbnail
Converting ListenableFutures to CompletableFutures and back
Java 8 introduced CompletableFutures. They build on standard Futures and add completion callbacks, chaining and other useful stuff. But the world did not wait for Java 8 and lot of libraries added different variants of ListenableFutures which serve the same purpose. Some library authors are reluctant to add support for CompletableFutures even today. It makes sense, Java 8 is quite new and it's not easy to add support for CompletableFutures and be compatible with Java 7 at the same time. Luckily it's easy to convert to CompletableFutures and back. Let's take Spring 4 ListenableFutures as an example. How to convert it to CompletableFuture? static CompletableFuture buildCompletableFuture( final ListenableFuture listenableFuture ) { //create an instance of CompletableFuture CompletableFuture completable = new CompletableFuture() { @Override public boolean cancel(boolean mayInterruptIfRunning) { // propagate cancel to the listenable future boolean result = listenableFuture.cancel(mayInterruptIfRunning); super.cancel(mayInterruptIfRunning); return result; } }; // add callback listenableFuture.addCallback(new ListenableFutureCallback() { @Override public void onSuccess(T result) { completable.complete(result); } @Override public void onFailure(Throwable t) { completable.completeExceptionally(t); } }); return completable; } We just create a CompletableFuture instance and add a callback to the ListenableFuture. In the callback method we just notify the CompletableFuture that the underlying task has finished. We can even propagate call to cancel method if we want to. That's all you need to convert to CompletableFuture. What about the opposite direction? The approach is a bit different, but it's more or less straightforward as well class ListenableCompletableFutureWrapper implements ListenableFuture { private final ListenableFutureCallbackRegistry callbackRegistry = new ListenableFutureCallbackRegistry<>(); private final Future wrappedFuture; ListenableCompletableFutureWrapper( CompletableFuture wrappedFuture ) { this.wrappedFuture = wrappedFuture; wrappedFuture.whenComplete((result, ex) -> { if (ex != null) { if (ex instanceof CompletionException && ex.getCause() != null ) { callbackRegistry.failure(ex.getCause()); } else { callbackRegistry.failure(ex); } } else { callbackRegistry.success(result); } }); } @Override public void addCallback( ListenableFutureCallback callback ) { callbackRegistry.addCallback(callback); } @Override public boolean cancel(boolean mayInterruptIfRunning) { return wrappedFuture.cancel(mayInterruptIfRunning); } @Override public boolean isCancelled() { return wrappedFuture.isCancelled(); } @Override public boolean isDone() { return wrappedFuture.isDone(); } @Override public T get() throws InterruptedException, ExecutionException { return wrappedFuture.get(); } @Override public T get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, TimeoutException { return wrappedFuture.get(timeout, unit); } } We just wrap the CompletableFuture and again register a callback. The only non-obvious part is the use of ListenableFutureCallbackRegistry which keeps track of registered ListenableFutureCallbacks. We also have to do some exception processing, but that's all. If you need to do something like this, I have good news. I have wrapped the code to a reusable library, so you do not have to copy and paste the code, you can just use it as described in the library documentation.
June 16, 2014
by Lukas Krecan
· 30,006 Views · 23 Likes
article thumbnail
What's Wrong in Java 8, Part V: Tuples
In part 5 of our "What's Wrong in Java 8" series, we turn to Tuples.
June 10, 2014
by Pierre-Yves Saumont
· 132,964 Views · 4 Likes
article thumbnail
Building a Simple RESTful API with Java Spark
Disclaimer: This post is about the Java micro web framework named Spark and not about the data processing engine Apache Spark. In this blog post we will see how Spark can be used to build a simple web service. As mentioned in the disclaimer, Spark is a micro web framework for Java inspired by the Ruby framework Sinatra. Spark aims for simplicity and provides only a minimal set of features. However, it provides everything needed to build a web application in a few lines of Java code. Getting Started Let's assume we have a simple domain class with a few properties and a service that provides some basic CRUDfunctionality: public class User { private String id; private String name; private String email; // getter/setter } public class UserService { // returns a list of all users public List getAllUsers() { .. } // returns a single user by id public User getUser(String id) { .. } // creates a new user public User createUser(String name, String email) { .. } // updates an existing user public User updateUser(String id, String name, String email) { .. } } We now want to expose the functionality of UserService as a RESTful API (For simplicity we will skip the hypermedia part of REST ;-)). For accessing, creating and updating user objects we want to use following URL patterns: GET /users Get a list of all users GET /users/ Get a specific user POST /users Create a new user PUT /users/ Update a user The returned data should be in JSON format. To get started with Spark we need the following Maven dependencies: com.sparkjava spark-core 2.0.0 org.slf4j slf4j-simple 1.7.7 Spark uses SLF4J for logging, so we need to a SLF4J binder to see log and error messages. In this example we use the slf4j-simple dependency for this purpose. However, you can also use Log4j or any other binder you like. Having slf4j-simple in the classpath is enough to see log output in the console. We will also use GSON for generating JSON output and JUnit to write a simple integration tests. You can find these dependencies in the complete pom.xml. Returning All Users Now it is time to create a class that is responsible for handling incoming requests. We start by implementing the GET /users request that should return a list of all users. import static spark.Spark.*; public class UserController { public UserController(final UserService userService) { get("/users", new Route() { @Override public Object handle(Request request, Response response) { // process request return userService.getAllUsers(); } }); // more routes } } Note the static import of spark.Spark.* in the first line. This gives us access to various static methods including get(), post(), put() and more. Within the constructor the get() method is used to register aRoute that listens for GET requests on /users. A Route is responsible for processing requests. Whenever aGET /users request is made, the handle() method will be called. Inside handle() we return an object that should be sent to the client (in this case a list of all users). Spark highly benefits from Java 8 Lambda expressions. Route is a functional interface (it contains only one method), so we can implement it using a Java 8 Lambda expression. Using a Lambda expression the Routedefinition from above looks like this: get("/users", (req, res) -> userService.getAllUsers()); To start the application we have to create a simple main() method. Inside main() we create an instance of our service and pass it to our newly created UserController: public class Main { public static void main(String[] args) { new UserController(new UserService()); } } If we now run main(), Spark will start an embedded Jetty server that listens on Port 4567. We can test our first route by initiating a GET http://localhost:4567/users request. In case the service returns a list with two user objects the response body might look like this: [com.mscharhag.sparkdemo.User@449c23fd, com.mscharhag.sparkdemo.User@437b26fe] Obviously this is not the response we want. Spark uses an interface called ResponseTransformer to convert objects returned by routes to an actual HTTP response. ReponseTransformer looks like this: public interface ResponseTransformer { String render(Object model) throws Exception; } ResponseTransformer has a single method that takes an object and returns a String representation of this object. The default implementation of ResponseTransformer simply calls toString() on the passed object (which creates output like shown above). Since we want to return JSON we have to create a ResponseTransformer that converts the passed objects to JSON. We use a small JsonUtil class with two static methods for this: public class JsonUtil { public static String toJson(Object object) { return new Gson().toJson(object); } public static ResponseTransformer json() { return JsonUtil::toJson; } } toJson() is an universal method that converts an object to JSON using GSON. The second method makes use of Java 8 method references to return a ResponseTransformer instance. ResponseTransformer is again a functional interface, so it can be satisfied by providing an appropriate method implementation (toJson()). So whenever we call json() we get a new ResponseTransformer that makes use of our toJson()method. In our UserController we can pass a ResponseTransformer as a third argument to Spark's get()method: import static com.mscharhag.sparkdemo.JsonUtil.*; public class UserController { public UserController(final UserService userService) { get("/users", (req, res) -> userService.getAllUsers(), json()); ... } } Note again the static import of JsonUtil.* in the first line. This gives us the option to create a newResponseTransformer by simply calling json(). Our response looks now like this: [{ "id": "1866d959-4a52-4409-afc8-4f09896f38b2", "name": "john", "email": "[email protected]" },{ "id": "90d965ad-5bdf-455d-9808-c38b72a5181a", "name": "anna", "email": "[email protected]" }] We still have a small problem. The response is returned with the wrong Content-Type. To fix this, we can register a Filter that sets the JSON Content-Type: after((req, res) -> { res.type("application/json"); }); Filter is again a functional interface and can therefore be implemented by a short Lambda expression. After a request is handled by our Route, the filter changes the Content-Type of every response toapplication/json. We can also use before() instead of after() to register a filter. Then, the Filterwould be called before the request is processed by the Route. The GET /users request should be working now :-) Returning a Specific User To return a specific user we simply create a new route in our UserController: get("/users/:id", (req, res) -> { String id = req.params(":id"); User user = userService.getUser(id); if (user != null) { return user; } res.status(400); return new ResponseError("No user with id '%s' found", id); }, json()); With req.params(":id") we can obtain the :id path parameter from the URL. We pass this parameter to our service to get the corresponding user object. We assume the service returns null if no user with the passed id is found. In this case, we change the HTTP status code to 400 (Bad Request) and return an error object. ResponseError is a small helper class we use to convert error messages and exceptions to JSON. It looks like this: public class ResponseError { private String message; public ResponseError(String message, String... args) { this.message = String.format(message, args); } public ResponseError(Exception e) { this.message = e.getMessage(); } public String getMessage() { return this.message; } } We are now able to query for a single user with a request like this: GET /users/5f45a4ff-35a7-47e8-b731-4339c84962be If an user with this id exists we will get a response that looks somehow like this: { "id": "5f45a4ff-35a7-47e8-b731-4339c84962be", "name": "john", "email": "[email protected]" } If we use an invalid user id, a ResponseError object will be created and converted to JSON. In this case the response looks like this: { "message": "No user with id 'foo' found" } Creating and Updating Users Creating and updating users is again very easy. Like returning the list of all users it is done using a single service call: post("/users", (req, res) -> userService.createUser( req.queryParams("name"), req.queryParams("email") ), json()); put("/users/:id", (req, res) -> userService.updateUser( req.params(":id"), req.queryParams("name"), req.queryParams("email") ), json()); To register a route for HTTP POST or PUT requests we simply use the static post() and put() methods of Spark. Inside a Route we can access HTTP POST parameters using req.queryParams(). For simplicity reasons (and to show another Spark feature) we do not do any validation inside the routes. Instead we assume that the service will throw an IllegalArgumentException if we pass in invalid values. Spark gives us the option to register ExceptionHandlers. An ExceptionHandler will be called if anException is thrown while processing a route. ExceptionHandler is another single method interface we can implement using a Java 8 Lambda expression: exception(IllegalArgumentException.class, (e, req, res) -> { res.status(400); res.body(toJson(new ResponseError(e))); }); Here we create an ExceptionHandler that is called if an IllegalArgumentException is thrown. The caught Exception object is passed as the first parameter. We set the response code to 400 and add an error message to the response body. If the service throws an IllegalArgumentException when the email parameter is empty, we might get a response like this: { "message": "Parameter 'email' cannot be empty" } The complete source the controller can be found here. Testing Because of Spark's simple nature it is very easy to write integration tests for our sample application. Let's start with this basic JUnit test setup: public class UserControllerIntegrationTest { @BeforeClass public static void beforeClass() { Main.main(null); } @AfterClass public static void afterClass() { Spark.stop(); } ... } In beforeClass() we start our application by simply running the main() method. After all tests finished we call Spark.stop(). This stops the embedded server that runs our application. After that we can send HTTP requests within test methods and validate that our application returns the correct response. A simple test that sends a request to create a new user can look like this: @Test public void aNewUserShouldBeCreated() { TestResponse res = request("POST", "/users?name=john&[email protected]"); Map json = res.json(); assertEquals(200, res.status); assertEquals("john", json.get("name")); assertEquals("[email protected]", json.get("email")); assertNotNull(json.get("id")); } request() and TestResponse are two small self made test utilities. request() sends a HTTP request to the passed URL and returns a TestResponse instance. TestResponse is just a small wrapper around some HTTP response data. The source of request() and TestResponse is included in the complete test classfound on GitHub. Conclusion Compared to other web frameworks Spark provides only a small amount of features. However, it is so simple you can build small web applications within a few minutes (even if you have not used Spark before). If you want to look into Spark you should clearly use Java 8, which reduces the amount of code you have to write a lot. You can find the complete source of the sample project on GitHub.
June 9, 2014
by Michael Scharhag
· 111,821 Views · 3 Likes
article thumbnail
Working with ZeroMQ, Java, and JZMQ on a CentOS Platform
Recently I decided to port some of my development using ZeroMQ onto my CentOS development machine and I ran into some challenges. I’m documenting those challenges so that if someone else runs into the same pitfalls I did, they can avoid it. In this example today, we will work with the first “HelloWorld” examples in the ZeroMQ guide found here. I added a few modifications to the sample such as a package name and a try-catch around the Thread and an exception.tostring() to display any stack-trace. Source code for src/zmq/hwserver.java package zmq; import java.io.PrintWriter; import java.io.StringWriter; import org.zeromq.ZMQ; // // Hello World server in Java // Binds REP socket to tcp://*:5555 // Expects "Hello" from client, replies with "World" // public class hwserver { /** * @param args */ public static void main(String[] args) { ZMQ.Context context = ZMQ.context(1); // Socket to talk to clients ZMQ.Socket socket = context.socket(ZMQ.REP); socket.bind ("tcp://*:5555"); try { while (!Thread.currentThread ().isInterrupted ()) { byte[] reply = socket.recv(0); System.out.println("Received Hello"); String request = "World" ; socket.send(request.getBytes (), 0); Thread.sleep(1000); // Do some 'work' } } catch(Exception e) { StringWriter sw = new StringWriter(); PrintWriter pw = new PrintWriter(sw); e.printStackTrace(pw); System.out.println(sw.toString()); } socket.close(); context.term(); } } Similarly, source code for the client, src/zmq/hwclient.java package zmq; import org.zeromq.ZMQ; public class hwclient { /** * @param args */ public static void main(String[] args) { ZMQ.Context context = ZMQ.context(1); // Socket to talk to server System.out.println("Connecting to hello world server"); ZMQ.Socket socket = context.socket(ZMQ.REQ); socket.connect ("tcp://localhost:5555"); for(int requestNbr = 0; requestNbr != 10; requestNbr++) { String request = "Hello" ; System.out.println("Sending Hello " + requestNbr ); socket.send(request.getBytes (), 0); byte[] reply = socket.recv(0); System.out.println("Received " + new String (reply) + " " + requestNbr); } socket.close(); context.term(); } } Now that you have the sample code, how do you compile using the ZeroMQ? Assumption: You have installed Java (1.7 or above) Step-1: Installing ZeroMQ onto CentOS [Following steps are performed under root account] Install “Development Tools” if it’s not already installed on your CentOS as root: yum groupinstall “Development Tools” Download the “POSIX tarball” ZeroMQ source code onto your CentOS development machine from here. At the time of writing this article, ZeroMQ version 3.2.3 was the stable release. You might want to download the latest stable release. Unpack the .tar.gz source archive. Run ./configure, followed by “make” then “make install“. Run ldconfig after installation. Step-2: Installing a Language Binding for Java. In this case, we will use JZMQ from https://github.com/zeromq/jzmq Download the latest stable release from GITHub link above. (git clone git://github.com/zeromq/jzmq.git) Change directory, cd jzmq Compile and Install: 1 2 3 4 ./autogen.sh ./configure make make install Where did it install? 1 2 # JAR is located here: /usr/local/share/java/zmq.jar # .so link files are located here: /usr/local/lib Important Step: Add /usr/local/lib to a line in /etc/ld.so.conf (here is my copy after editing) 1 2 include ld.so.conf.d/*.conf /usr/local/lib Reload “ldconfig“. This clears the cache. Step-3: Compile and run the Java examples above. cd ~/dev/zeromq/example/ # Compile hwserver.java javac -classpath /usr/local/share/java/zmq.jar ./zmq/hwserver.java # Compile hwclient.java javac -classpath /usr/local/share/java/zmq.jar ./zmq/hwclient.java # Run hwserver in a separate prompt java -classpath .: /usr/local/share/java/zmq.jar -Djava.library.path=/usr/local/lib zmq.hwserver # Run hwclient in a seperate prompt java -classpath .:/usr/local/share/java/zmq.jar -Djava.library.path=/usr/local/lib zmq.hwclient Output on the hwserver console: Received Hello Received Hello Received Hello Received Hello Received Hello Received Hello Received Hello Received Hello Received Hello Received Hello output on the hwclient console: Connecting to hello world server Sending Hello 0 Received World 0 Sending Hello 1 Received World 1 Sending Hello 2 Received World 2 Sending Hello 3 Received World 3 Sending Hello 4 Received World 4 Sending Hello 5 Received World 5 Sending Hello 6 Received World 6 Sending Hello 7 Received World 7 Sending Hello 8 Received World 8 Sending Hello 9 Received World 9 Few interesting points to note are as follows: What happens if you started the client first and then the server? Well, the client waits until the server becomes available (or in other words, until some process connects to socket port 5555) and then sends the message. When you say socket.send(…), ZeroMQ actually enqueues a message to be sent later by a dedicated communication thread and this thread waits until a bind on port 5555 happens by “server”. Also observe that the “server” is doing the connecting, and the “client” is doing the binding. What is ZeroMQ (ØMQ)? (Excerpt from the ZeroMQ website!) ØMQ (also seen as ZeroMQ, 0MQ, zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fanout, pub-sub, task distribution, and request-reply. It’s fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multicore applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. ØMQ is from iMatix and is LGPLv3 open source.
June 6, 2014
by Venkatt Guhesan
· 25,233 Views
article thumbnail
MapDB: The Agile Java Data Engine
MapDB is a pure Java database, specifically designed for the Java developer. The fundamental concept for MapDB is very clever yet natural to use: provide a reliable, full-featured and “tune-able” database engine using the Java Collections API. MapDB 1.0 has just been released, this is the culmination of years of research and development to get the project to this point. Jan Kotek, the primary developer for MapDB, also worked on predecessor projects (JDBM), starting MapDB as an entire from-scratch rewrite. Jan’s expertise and dedication to low-level debugging has yielded excellent results, producting an easy-to-use database for Java with comparable performance to many C-based engines. What sets MapDB apart is the “map” concept. The idea is to leverage the totally natural Java Collections API – so familiar to Java developers that most of them literally use it daily in their work. For most database interactions with a Java application, some sort of translator is required. There are many Object-Relational Mapping (ORM) tools to name just one category of such components. The goal has always been in the direction of making it natural to code in objects in the Java language, and translate them to a specific database syntax (such as SQL). However, such efforts have always come up short, adding complexity for both the application developer and the data architect. When using MapDB there is no object “translation layer” – developers just access data in familiar structures like Maps, Sets, Queues, etc. There is no change in syntax from typical Java coding, other than a brief initialization syntax and transaction management. A developer can literally transform memory-limited maps into a high-speed persistent store in seconds (typically changing just one line of code). A MapDB Example Here is a simple MapDB example, showing how easy and intuitive it is to use in a Java application: // Initialize a MapDB database DB db = DBMaker.newFileDB(new File("testdb")) .closeOnJvmShutdown() .make(); // Create a Map: Map myMap = db.getTreeMap(“testmap”); // Work with the Map using the normal Map API. myMap.put(“key1”, “value1”); myMap.put(“key2”, “value2”); String value = myMap.get(“key1”); ... That’s all you need to do, now you have a file-backed Map of virtually any size. Note the “builder-style” initialization syntax, enabling MapDB as the agile database choice for Java. There are many builder options that let you tune your database for the specific requirements at hand. Just a small subset of options include: In-memory implementation Enable transactions Configurable caching This means that you can configure your database just for what you need, effectively making MapDB serve the job of many other databases. MapDB comes with a set of powerful configuration options, and you can even extend the product to make your own data implementations if necessary. Another very powerful feature is that MapDB utilizes some of the advanced Java Collections variants, such as ConcurrentNavigableMap. With this type of Map you can go beyond simple key-value semantics, as it is also a sorted Map allowing you to access data in order, and find values near a key. Not many people are aware of this extension to the Collections API, but it is extremely powerful and allows you to do a lot with your MapDB database (I will cover more of these capabilities in a future article). The Agile Aspect of MapDB When I first met Jan and started talking with him about MapDB he said something that made a very important impression: If you know what data structure you want, MapDB allows you to tailor the structure and database characteristics to your exact application needs. In other words, the schema and ways you can structure your data is very flexible. The configuration of the physical data store is just as flexible, making a perfect combination for meeting almost any database need. They key to this capability is inherent in MapDB’s architecture, and how it translates to the MapDB API itself. Here is a simple diagram of the MapDB architecture: As you can see from the diagram, there are 3 tiers in MapDB: Collections API: This is the familiar Java Collections API that every Java developer uses for maintaining application state. It has a simple builder-style extension to allow you to control the exact characteristics of a given database (including its internal format or record structure). Engine: The Engine is the real key to MapDB, this is where the records for a database – including their internal structure, concurrency control, transactional semantics – are controlled. MapDB ships with several engines already, and it is straightforward to add your own Engine if needed for specialized data handling. Volume: This is the physical storage layer (e.g., on-disk or in-memory). MapDB has a few standard Volume implementations, and they should suffice for most projects. The main point is that the development API is completely distinct from the Engine implementation (the heart of MapDB), and both are separate from the actual physical storage layer. This offers a very agile approach, allowing developers to exactly control what type of internal structure is needed for a given database, and what the actual data structure looks like from the top-level Collections API. To make things even more extensible and agile, MapDB uses a concept of Engine Wrappers. An Engine Wrapper allows adding additional features and options on top of a specific engine layer. For example, if the standard Map engine is utilized for creating a B-Tree backed Map, it is feasible to enable (or disable) caching support. This caching feature is done through an Engine Wrapper, and that is what shows up in the builder-style API used to configure a given database. While a whole article could be written just about this, the point here is that this adds to MapDB’s inherent agile nature. By way of example, here is how you configure a pure in-memory database, without transactional capabilities: // Initialize an in-memory MapDB database // without transactions DB db = DBMaker.newMemoryDB() .transactionDisable() .closeOnJvmShutdown() .make(); // Create a Map: Map myMap = db.getTreeMap(“testmap”); // Work with the Map using the normal Map API. myMap.put(“key1”, “value1”); myMap.put(“key2”, “value2”); String value = myMap.get(“key1”); ... That’s it! All that was needed was to change the DBMaker call to add the new options, everything else works exactly the same as in the example shown earlier. Agile Data Model In addition to customizing the features and performance characteristics of a given database instance, MapDB allows you to create an agile data model, with a schema exactly matching your application requirements. This is probably similar to how you write your code when creating standard Java in-memory structures. For example, let’s say you need to lookup a Person object by username, or by personID. Simply create a Person object and two Maps to meet your needs: public class Person { private Integer personID; private String username; ... // Setters and getters go here ... } // Create a Map of Person by username. Map personByUsernameMap = ... // Create a Map of Person by personID. Map personByPersonIDMap = ... This is a very trivial example, but now you can easily write to both maps for each new Person instance, and subsequently retrieve a Person by either key. Another interesting concept with MapDB data structures are some key extensions to the normal Java Collections API. A common requirement in applications is to have a Map with a key/value, and in addition to finding the value for a key to be able to perform the inverse: lookup the key for a given value. We can easily do this using the MapDB extension for bi-directional maps: // Create a primary map HTreeMap map = DBMaker.newTempHashMap(); // Create the inverse mapping for primary map NavigableSet> inverseMapping = new TreeSet>(); // Bind the inverse mapping to primary map, so it is auto-updated each time the primary map gets a new key/value Bind.mapInverse(map, inverseMapping); map.put(10L,"value2"); map.put(1111L,"value"); map.put(1112L,"value"); map.put(11L,"val"); // Now find a key by a given value. Long keyValue = Fun.filter(inverseMapping.get(“value2”); MapDB supports many constructs for the interaction of Maps or other collections, allowing you to create a schema of related structures that can automatically be kept in sync. This avoids a lot of scanning of structures, makes coding fast and convenient, and can keep things very fast. Wrapping it up I have shown a very brief introduction on MapDB and how the product works. As you can see its strengths are its use of the natural Java Collections API, the agile nature of the engine itself, and the support for virtually any type of data model or schema that your application needs. MapDB is freely available for any use under the Apache 2.0 license. To learn more, check out: www.mapdb.org.
June 5, 2014
by Cory Isaacson
· 29,003 Views · 3 Likes
article thumbnail
Spring Integration Java DSL sample
A new Java based DSL has now been introduced for Spring Integration which makes it possible to define the Spring Integration message flows using pure java based configuration instead of using the Spring XML based configuration. I tried the DSL for a sample Integration flow that I have - I call it the Rube Goldberg flow, for it follows a convoluted path in trying to capitalize a string passed in as input. The flow looks like this and does some crazy things to perform a simple task: It takes in a message of this type - "hello from spring integ" splits it up into individual words(hello, from, spring, integ) sends each word to a ActiveMQ queue from the queue the word fragments are picked up by a enricher to capitalize each word placing the response back into a response queue It is picked up, resequenced based on the original sequence of the words aggregated back into a sentence("HELLO FROM SPRING INTEG") and returned back to the application. To start with Spring Integration Java DSL, a simple Xml based configuration to capitalize a String would look like this: There is nothing much going on here, a messaging gateway takes in the message passed in from the application, capitalizes it in a transformer and this is returned back to the application. Expressing this in Spring Integration Java DSL: @Configuration @EnableIntegration @IntegrationComponentScan @ComponentScan public class EchoFlow { @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow simpleEchoFlow() { return IntegrationFlows.from(requestChannel()) .transform((String s) -> s.toUpperCase()) .get(); } } @MessagingGateway public interface EchoGateway { @Gateway(requestChannel = "requestChannel") String echo(String message); } Do note that @MessagingGateway annotation is not a part of Spring Integration Java DSL, it is an existing component in Spring Integration and serves the same purpose as the gateway component in XML based configuration. I like the fact that the transformation can be expressed using typesafe Java 8 lambda expressions rather than the Spring-EL expression. Note that the transformation expression could have coded in quite few alternate ways: ??.transform((String s) -> s.toUpperCase()) Or: ??.transform(s -> s.toUpperCase()) Or using method references: ??.transform(String::toUpperCase) Moving onto the more complicated Rube Goldberg flow to accomplish the same task, again starting with XML based configuration. There are two configurations to express this flow: rube-1.xml: This configuration takes care of steps 1, 2, 3, 6, 7, 8 : It takes in a message of this type - "hello from spring integ" splits it up into individual words(hello, from, spring, integ) sends each word to a ActiveMQ queue from the queue the word fragments are picked up by a enricher to capitalize each word placing the response back into a response queue It is picked up, resequenced based on the original sequence of the words aggregated back into a sentence("HELLO FROM SPRING INTEG") and returned back to the application. and rube-2.xml for steps 4, 5: It takes in a message of this type - "hello from spring integ" splits it up into individual words(hello, from, spring, integ) sends each word to a ActiveMQ queue from the queue the word fragments are picked up by a enricher to capitalize each word placing the response back into a response queue It is picked up, resequenced based on the original sequence of the words aggregated back into a sentence("HELLO FROM SPRING INTEG") and returned back to the application. Now, expressing this Rube Goldberg flow using Spring Integration Java DSL, the configuration looks like this, again in two parts: EchoFlowOutbound.java: @Bean public DirectChannel sequenceChannel() { return new DirectChannel(); } @Bean public DirectChannel requestChannel() { return new DirectChannel(); } @Bean public IntegrationFlow toOutboundQueueFlow() { return IntegrationFlows.from(requestChannel()) .split(s -> s.applySequence(true).get().getT2().setDelimiters("\\s")) .handle(jmsOutboundGateway()) .get(); } @Bean public IntegrationFlow flowOnReturnOfMessage() { return IntegrationFlows.from(sequenceChannel()) .resequence() .aggregate(aggregate -> aggregate.outputProcessor(g -> Joiner.on(" ").join(g.getMessages() .stream() .map(m -> (String) m.getPayload()).collect(toList()))) , null) .get(); } and EchoFlowInbound.java: @Bean public JmsMessageDrivenEndpoint jmsInbound() { return new JmsMessageDrivenEndpoint(listenerContainer(), messageListener()); } @Bean public IntegrationFlow inboundFlow() { return IntegrationFlows.from(enhanceMessageChannel()) .transform((String s) -> s.toUpperCase()) .get(); } Again here the code is completely typesafe and is checked for any errors at development time rather than at runtime as with the XML based configuration. Again I like the fact that transformation, aggregation statements can be expressed concisely using Java 8 lamda expressions as opposed to Spring-EL expressions. What I have not displayed here is some of the support code, to set up the activemq test infrastructure, this configuration continues to remain as xml and I have included this code in a sample github project. All in all, I am very excited to see this new way of expressing the Spring Integration messaging flow using pure Java and I am looking forward to seeing its continuing evolution and may be even try and participate in its evolution in small ways. Here is the entire working code in a github repo: https://github.com/bijukunjummen/rg-si References and Acknowledgement: Spring Integration Java DSL introduction blog article by Artem Bilan: https://spring.io/blog/2014/05/08/spring-integration-java-dsl-milestone-1-released Spring Integration Java DSL website and wiki: https://github.com/spring-projects/spring-integration-extensions/wiki/Spring-Integration-Java-DSL-Reference. A lot of code has been shamelessly copied over from this wiki by me :-). Also, a big thanks to Artem for guidance on a question that I had Webinar by Gary Russell on Spring Integration 4.0 in which Spring Integration Java DSL is covered in great detail.
June 3, 2014
by Biju Kunjummen
· 44,094 Views
article thumbnail
InterruptedException and Interrupting Threads Explained
Let's take a simple example of a thread that periodically does some clean up, but in between sleeps most of the time.
June 1, 2014
by Tomasz Nurkiewicz
· 18,753 Views · 1 Like
article thumbnail
Connecting to Cassandra from Java
In this post, I look at the basics of connecting to a Cassandra database from a Java client. I will use the DataStax Java Client JAR in order to do so.
May 30, 2014
by Dustin Marx
· 121,435 Views · 3 Likes
article thumbnail
SOAP Webservices Using Apache CXF: Adding Custom Object as Header in Outgoing Requests
What is CXF? Apache CXF is an open source services framework. CXF helps you build and develop services using frontend programming APIs, like JAX-WS and JAX-RS. These services can speak a variety of protocols such as SOAP, XML/HTTP, RESTful HTTP, or CORBA and work over a variety of transports such as HTTP, JMS etc. How CXF Works? As you can see here and here, how cxf service calls are processed,most of the functionality in the Apache CXF runtime is implemented by interceptors. Every endpoint created by the Apache CXF runtime has potential interceptor chains for processing messages. The interceptors in the these chains are responsible for transforming messages between the raw data transported across the wire and the Java objects handled by the endpoint’s implementation code. Interceptors in CXF When a CXF client invokes a CXF server, there is an outgoing interceptor chain for the client and an incoming chain for the server. When the server sends the response back to the client, there is an outgoing chain for the server and an incoming one for the client. Additionally, in the case of SOAPFaults, a CXF web service will create a separate outbound error handling chain and the client will create an inbound error handling chain. The interceptors are organized into phases to ensure that processing happens on the proper order.Various phases involved during the Interceptor chains are listed in CXF documentation here. Adding your custom Interceptor involves extending one of the Abstract Intereceptor classes that CXF provides, and providing a phase when that interceptor should be invoked. AbstractPhaseInterceptor class - This abstract class provides implementations for the phase management methods of the PhaseInterceptor interface. The AbstractPhaseInterceptor class also provides a default implementation of the handleFault() method. Developers need to provide an implementation of the handleMessage() method. They can also provide a different implementation for the handleFault() method. The developer-provided implementations can manipulate the message data using the methods provided by the generic org.apache.cxf.message.Message interface. For applications that work with SOAP messages, Apache CXF provides an AbstractSoapInterceptor class. Extending this class provides the handleMessage() method and the handleFault() method with access to the message data as an org.apache.cxf.binding.soap.SoapMessage object. SoapMessage objects have methods for retrieving the SOAP headers, the SOAP envelope, and other SOAP metadata from the message. Below piece of code will show, how we can add a Custom Object as Header to an outgoing request – Spring Configuration - Interceptor :- public class SoapHeaderInterceptor extends AbstractSoapInterceptor { public SoapHeaderInterceptor() { super(Phase.POST_LOGICAL); } @Override public void handleMessage(SoapMessage message) throws Fault { List headers = message.getHeaders(); TestHeader testHeader = new TestHeader(); JAXBElement testHeaders = new ObjectFactory() .createTestHeader(testHeader); try { Header header = new Header(testHeaders.getName(), testHeader, new JAXBDataBinding(TestHeader.class)); headers.add(header); message.put(Header.HEADER_LIST, headers); } catch (JAXBException e) { e.printStackTrace(); } }
May 29, 2014
by Saurabh Chhajed
· 15,530 Views · 1 Like
article thumbnail
Running the Maven Release Plugin with Jenkins
Learn more about using the Maven Release plugin on Jenkins, including subversion source control, artifactory, continuous integration, and more.
May 23, 2014
by $$anonymous$$
· 104,898 Views · 6 Likes
article thumbnail
Quick Note: SSL with SOAP and SOAPUI
For doing SSL with SOAP, there’s a few things you need to setup. C:\Program Files (x86)\SmartBear\soapUI-Pro-4.5.1\jre\lib\security Also did it at the main jre at C:\Program Files (x86)\Java\jre7\lib\security keytool -genkey -alias svs -keyalg RSA -keystore keystore.jks -keysize 2048 git config --global core.autocrlf true javax.net.ssl.trustStore=<> javax.net.ssl.trustStorePassword=<> If these properties are not set, the default ones will be picked up from your the default location.[$JAVA_HOME/lib/security/jssecacerts, $JAVA_HOME/lib/security/cacerts] To view the contents of keystore file, use: keytool -list -v -keystore file.keystore -storepass changeit To debug the ssl handshake process and view the certificates, set the VM parameter -Djavax.net.debug=all keytool -genkey -keyalg RSA -alias selfsigned -keystore keystore.jks -storepass changeit -validity 360 -keysize 2048 -Djava.net.preferIPv4Stack=true added to soapui.bat C:\Program Files (x86)\SmartBear\SoapUI-4.6.3\bin -Djavax.net.debug=ssl,trustmanager http://docs.oracle.com/cd/E19509-01/820-3503/ggfgo/index.html http://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html http://ianso.blogspot.com/2009/12/building-ws-security-enabled-soap.html http://javarevisited.blogspot.com/2012/09/difference-between-truststore-vs-keyStore-Java-SSL.html http://javarevisited.blogspot.com/2012/03/add-list-certficates-java-keystore.html http://www.ibm.com/developerworks/java/library/j-jws17/index.html http://www.coderanch.com/t/223027/Web-Services/java/SOAP-HTTPS-SSL http://ruchirawageesha.blogspot.in/2010/07/how-to-create-clientserver-keystores.html http://stackoverflow.com/questions/11001102/how-to-programmatically-set-the-sslcontext-of-a-jax-ws-client http://busylog.net/ssl-java-keytool-soap-and-eclipse/ http://www.sslshopper.com/article-how-to-create-a-self-signed-certificate-using-java-keytool.html openssl s_client -showcerts -host webservices-cert.storedvalue.com -port 443 keytool -keystore clientkeystore -genkey -alias client wsdl2java.bat -uri my.wsdl -o svsproj -p com.agilemobiledeveloper.service -d xmlbeans -t -ss -ssi -sd -g -ns2p System.setProperty("javax.net.ssl.keyStore", keystore.jks"); System.setProperty("javax.net.ssl.keyStorePassword", "changeit"); System.setProperty("javax.net.ssl.trustStore", "clientkeystore"); System.setProperty("javax.net.ssl.trustStorePassword", "changeit"); setx -m JAVA_HOME "C:\Program Files\Java\jdk1.7.0_51″ setx -m javax.net.ssl.keyStore "keystore.jks"); setx -m javax.net.ssl.keyStorePassword "changeit"); setx -m javax.net.ssl.trustStore "keystore.jks"); setx -m javax.net.ssl.trustStorePassword "passwordislong");
May 23, 2014
by Tim Spann DZone Core CORE
· 19,856 Views · 1 Like
article thumbnail
What's Wrong in Java 8, Part IV: Monads
Further exploring what's wrong with Java 8, we turn our discussion to monads.
May 21, 2014
by Pierre-Yves Saumont
· 112,516 Views · 36 Likes
article thumbnail
What's Wrong in Java 8, Part III: Streams and Parallel Streams
When the first early access versions of Java 8 were made available, what seemed the most important (r)evolution were lambdas. This is now changing and many developers seem to think now that streams are the most valuable Java 8 feature. And this is because they believe that by changing a single word in their programs (replacing stream with parallelStream) they will make these programs work in parallel. Many Java 8 evangelists have demonstrated amazing examples of this. Is there something wrong with this? No. Not something. Many things: Running in parallel may or may not be a benefit. It depends what you are using this feature for. Java 8 parallel streams may make your programs run faster. Or not. Or even slower. Thinking about streams as a way to achieve parallel processing at low cost will prevent developers to understand what is really happening. Streams are not directly linked to parallel processing. Most of the above problems are based upon a misunderstanding: parallel processing is not the same thing as concurrent processing. And most examples shown about “automatic parallelization” with Java 8 are in fact examples of concurrent processing. Thinking about map, filter and other operations as “internal iteration” is a complete nonsense (although this is not a problem with Java 8, but with the way we use it). So, what are streams According to Wikipedia: “a stream is a potentially infinite analog of a list, given by the inductive definition: data Stream a = Cons a (Stream a) Generating and computing with streams requires lazy evaluation, either implicitly in a lazily evaluated language or by creating and forcing thunks in an eager language.” One most important think to notice is that Java is what Wikipedia calls an “eager” language, which means Java is mostly strict (as opposed to lazy) in evaluating things. For example, if you create a List in Java, all elements are evaluated when the list is created. This may surprise you, since you may create an empty list and add elements after. This is only because either the list is mutable (and you are replacing a null reference with a reference to something) or you are creating a new list from the old one appended with the new element. Lists are created from something producing its elements. For example: List list = Arrays.asList(1, 2, 3, 4, 5); Here the producer is an array, and all elements of the array are strictly evaluated. It is also possible to create a list in a recursive way, for example the list starting with 1 and where all elements are equals to 1 plus the previous element and smaller than 6. In Java < 8, this translates into: List list = new ArrayList(); for(int i = 0; i < 6; i++) { list.add(i); } One may argue that the for loop is one of the rare example of lazy evaluation in Java, but the result is a list in which all elements are evaluated. What happens if we want to apply a function to all elements of this list? We may do this in a loop. For example, if with want to increase all elements by 2, we may do this: for(int i = 0; i < list.size(); i++) { list.set(i, list.get(i) * 2); } However, this does not allow using an operation that changes the type of the elements, for example increasing all elements by 10%. The following solution solves this problem: List list2 = new ArrayList(); for(int i = 0; i < list.size(); i++) { list2.add(list.get(i) * 1.2); } This form allows the use of a the Java 5 for each syntax: List list2 = new ArrayList<>(); for(Integer i : list) { list2.add(i * 1.2); } or the Java 8 syntax: List list2 = new ArrayList<>(); list.forEach(x -> list2.add(x * 1.2)); So far, so good. But what if we want to increase the value by 10% and then divide it by 3? The trivial answer would be to do: List list2 = new ArrayList<>(); list.forEach(x -> list2.add(x * 1.2)); List list3 = new ArrayList<>(); list2.forEach(x -> list3.add(x / 3)); This is far from optimal because we are iterating twice on the list. A much better solution is: List list2 = new ArrayList<>(); for(Integer i : list) { list2.add(i * 1.2 / 3); } Let aside the auto boxing/unboxing problem for now. In Java 8, this can be written as: List list2 = new ArrayList<>(); list.forEach(x -> list2.add(x * 1.2 / 3)); But wait... This is only possible because we see the internals of the Consumer bound to the list, so we are able to manually compose the operations. If we had: List list2 = new ArrayList<>(); list.forEach(consumer1); List list3 = new ArrayList<>(); list2.forEach(consumer2); How could we know how to compose them? No way. In Java 8, the Consumer interface has a default method andThen. We could be tempted to compose the consumers this way: list.forEach(consumer1.andThen(consumer2)); but this will result in an error, because andThen is defined as: default Consumer andThen(Consumer after) { Objects.requireNonNull(after); return (T t) -> { accept(t); after.accept(t); }; } This means that we can't use andThen to compose consumers of different types. In fact, we have it all wrong since the beginning. What we need is to bind the list to a function in order to get a new list, such as: Function function1 = x -> x * 1.2; Function function2 = x -> x / 3; list.bind(function1).bind(function2); where the bind method would be defined in a special FList class like: public class FList { final List list; public FList(List list) { this.list = list; } public FList bind(Function f) { List newList = new ArrayList(); for (T t : list) { newList.add(f.apply(t)); } return new FList(newList); } } and we would use it as in the following example: new Flist<>(list).bind(function1).bind(function2); The only trouble we have then is that binding twice would require iterating twice on the list. This is because bind is evaluated strictly. What we would need is a lazy evaluation, so that we could iterate only once. The problem here is that the bind method is not a real binding. It is in reality a composition of a real binding and a reduce. "Reducing" is applying an operation to each element of the list, resulting in the combination of this element and the result of the same operation applied to the previous element. As there is no previous element when we start from the first element, we start with an initial value. For example, applying (x) -> r + x, where r is the result of the operation on the previous element, or 0 for the first element, gives the sum of all elements of the list. Applying () -> r + 1 to each element, starting with r = 0 gives the length of the list. (This may not be the more efficient way to get the length of the list, but it is totally functional!) Here, the operation is add(element) and the initial value is an empty list. And this occurs only because the function application is strictly evaluated. What Java 8 streams give us is the same, but lazily evaluated, which means that when binding a function to a stream, no iteration is involved! Binding a Function to a Stream gives us a Stream with no iteration occurring. The resulting Stream is not evaluated, and this does not depend upon the fact that the initial stream was built with evaluated or non evaluated data. In functional languages, binding a Function to a Stream is itself a function. In Java 8, it is a method, which means it's arguments are strictly evaluated, but this has nothing to do with the evaluation of the resulting stream. To understand what is happening, we can imagine that the functions to bind are stored somewhere and they become part of the data producer for the new (non evaluated) resulting stream. In Java 8, the method binding a function T -> U to a Stream, resulting in a Stream is called map. The function binding a function T -> Stream to a Stream, resulting in a Stream is called flatMap. Where is flatten? Most functional languages also offer a flatten function converting a Stream> into a Stream, but this is missing in Java 8 streams. It may not look like a big trouble since it is so easy to define a method for doing this. For example, given the following function: Function> f = x -> Stream.iterate(1, y -> y + 1).limit(x); Stream stream = Stream.iterate(1, x -> x + 1); Stream stream2 = stream.limit(5).flatMap(f); System.out.println(stream2.collect(toList())) to produce: [1, 1, 2, 1, 2, 3, 1, 2, 3, 4, 1, 2, 3, 4, 5] Using map instead of flatMap: Stream stream = Stream.iterate(1, x -> x + 1); Stream stream2 = stream.limit(5).map(f); System.out.println(stream2.collect(toList())) will produce a stream of streams: [java.util.stream.SliceOps$1@12133b1, java.util.stream.SliceOps$1@ea2f77, java.util.stream.SliceOps$1@1c7353a, java.util.stream.SliceOps$1@1a9515, java.util.stream.SliceOps$1@f49f1c] Converting this stream of streams of integers to a stream of integers is very straightforward using the functional paradigm: one just need to flatMap the identity function to it: System.out.println(stream2.flatMap(x -> x).collect(toList())); It is however strange that a flatten method has not been added to the stream, knowing the strong relation that ties map, flatMap, unit and flatten, where unit is the function from T to Stream, represented by the method: Stream Stream.of(T... t) When are stream evaluated? Streams are evaluated when we apply to them some specific operations called terminal operation. This may be done only once. Once a terminal operation is applied to a stream, is is no longer usable. Terminal operations are: forEach forEachOrdered toArray reduce collect min max count anyMatch allMatch noneMatch findFirst findAny iterator spliterator Some of these methods are short circuiting. For example, findFirst will return as soon as the first element will be found. Non terminal operations are called intermediate and can be stateful (if evaluation of an element depends upon the evaluation of the previous) or stateless. Intermediate operations are: filter map mapTo... (Int, Long or Double) flatMap flatMapTo... (Int, Long or Double) distinct sorted peek limit skip sequential parallel unordered onClose Several intermediate operations may be applied to a stream, but only one terminal operation may be use. So what about parallel processing? One most advertised functionality of streams is that they allow automatic parallelization of processing. And one can find the amazing demonstrations on the web, mainly based of the same example of a program contacting a server to get the values corresponding to a list of stocks and finding the highest one not exceeding a given limit value. Such an example may show an increase of speed of 400 % and more. But this example as little to do with parallel processing. It is an example of concurrent processing, which means that the increase of speed will be observed also on a single processor computer. This is because the main part of each “parallel” task is waiting. Parallel processing is about running at the same time tasks that do no wait, such as intensive calculations. Automatic parallelization will generally not give the expected result for at least two reasons: The increase of speed is highly dependent upon the kind of task and the parallelization strategy. And over all things, the best strategy is dependent upon the type of task. The increase of speed in highly dependent upon the environment. In some environments, it is easy to obtain a decrease of speed by parallelizing. Whatever the kind of tasks to parallelize, the strategy applied by parallel streams will be the same, unless you devise this strategy yourself, which will remove much of the interest of parallel streams. Parallelization requires: A pool of threads to execute the subtasks, Dividing the initial task into subtasks, Distributing subtasks to threads, Collating the results. Without entering the details, all this implies some overhead. It will show amazing results when: Some tasks imply blocking for a long time, such as accessing a remote service, or There are not many threads running at the same time, and in particular no other parallel stream. If all subtasks imply intense calculation, the potential gain is limited by the number of available processors. Java 8 will by default use as many threads as they are processors on the computer, so, for intensive tasks, the result is highly dependent upon what other threads may be doing at the same time. Of course, if each subtask is essentially waiting, the gain may appear to be huge. The worst case is if the application runs in a server or a container alongside other applications, and subtasks do not imply waiting. In such a case, (for example running in a J2EE server), parallel streams will often be slower that serial ones. Imagine a server serving hundreds of requests each second. There are great chances that several streams might be evaluated at the same time, so the work is already parallelized. A new layer of parallelization at the business level will most probably make things slower. Worst: there are great chances that the business applications will see a speed increase in the development environment and a decrease in production. And that is the worst possible situation. Edit: for a better understanding of why parallel streams in Java 8 (and the Fork/Join pool in Java 7) are broken, refer to these excellent articles by Edward Harned: A Java Fork-Join Calamity A Java Parallel Calamity What streams are good for Stream are a useful tool because they allow lazy evaluation. This is very important in several aspect: They allow functional programming style using bindings. They allow for better performance by removing iteration. Iteration occurs with evaluation. With streams, we can bind dozens of functions without iterating. They allow easy parallelization for task including long waits. Streams may be infinite (since they are lazy). Functions may be bound to infinite streams without problem. Upon evaluation, there must be some way to make them finite. This is often done through a short circuiting operation. What streams are not good for Streams should be used with high caution when processing intensive computation tasks. In particular, by default, all streams will use the same ForkJoinPool, configured to use as many threads as there are cores in the computer on which the program is running. If evaluation of one parallel stream results in a very long running task, this may be split into as many long running sub-tasks that will be distributed to each thread in the pool. From there, no other parallel stream can be processed because all threads will be occupied. So, for computation intensive stream evaluation, one should always use a specific ForkJoinPool in order not to block other streams. To do this, one may create a Callable from the stream and submit it to the pool: List list = // A list of objects Stream stream = list.parallelStream().map(this::veryLongProcessing); Callable> task = () -> stream.collect(toList()); ForkJoinPool forkJoinPool = new ForkJoinPool(4); List newList = forkJoinPool.submit(task).get() This way, other parallel streams (using their own ForkJoinPool) will not be blocked by this one. In other words, we would need a pool of ForkJoinPool in order to avoid this problem. If a program is to be run inside a container, one must be very careful when using parallel streams. Never use the default pool in such a situation unless you know for sure that the container can handle it. In a Java EE container, do not use parallel streams. Previous articles What's Wrong with Java 8, Part I: Currying vs Closures What's Wrong in Java 8, Part II: Functions & Primitives
May 20, 2014
by Pierre-Yves Saumont
· 185,585 Views · 17 Likes
  • Previous
  • ...
  • 215
  • 216
  • 217
  • 218
  • 219
  • 220
  • 221
  • 222
  • 223
  • 224
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×