DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Languages Topics

article thumbnail
R: ggplot2 - When to Adjust the Group Aesthetic
Each group consists of only one observation. Do you need to adjust the group aesthetic? I’ve been playing around with some weather data over the last couple of days which I aggregated down to the average temperature per month over the last 4 years and stored in a CSV file. This is what the file looks like: $ cat /tmp/averageTemperatureByMonth.csv "month","aveTemperature" "January",6.02684563758389 "February",5.89380530973451 "March",7.54838709677419 "April",10.875 "May",13.3064516129032 "June",15.9666666666667 "July",18.8387096774194 "August",18.3709677419355 "September",16.2583333333333 "October",13.4596774193548 "November",9.19166666666667 "December",7.01612903225806 I wanted to create a simple line chart which would show the months of the year in ascending order with the appropriate temperature. My first attempt was the following: df = read.csv("/tmp/averageTemperatureByMonth.csv") df$month = factor(df$month, month.name) ggplot(aes(x = month, y = aveTemperature), data = df) + geom_line( ) + ggtitle("Temperature by month") which resulted in the following error: geom_path: Each group consist of only one observation. Do you need to adjust the group aesthetic? My understanding is that the points don’t get joined up by default because the variable on the x axis is not a continuous one but rather a factor variable. One way to work around this problem is to make it numeric, like so: ggplot(aes(x = as.numeric(month), y = aveTemperature), data = df) + geom_line( ) + ggtitle("Temperature by month") which results in the following chart: This isn’t bad but it’d be much nicer if we could have the month names along the bottom instead. It turns out we can but we need to specify a group that each point belongs to. ggplot will then connects points which belong to the same group. In this case we don’t really have one so we’ll define a dummy one instead: ggplot(aes(x = month, y = aveTemperature, group=1), data = df) + geom_line( ) + ggtitle("Temperature by month") And now we get the visualisation we want: Be Sociable, Share!
February 14, 2015
by Mark Needham
· 16,308 Views
article thumbnail
Converting an Application to JHipster
I've been intrigued by JHipster ever since I first tried it last September. I'd worked with AngularJS and Spring Boot quite a bit, and I liked the idea that someone had combined them, adding some nifty features along the way. When I spoke about AngularJS earlier this month, I included a few slides on JHipster near the end of the presentation. This week, I received an email from someone who attended that presentation. Hey Matt, We met a few weeks back when you presented at DOSUG. You were talking about JHipster which I had been eyeing for a few months and wanted your quick .02 cents. I have built a pretty heavy application over the last 6 months that is using mostly the same tech as JHipster. Java Spring JPA AngularJS Compass Grunt It's ridiculously close for most of the tech stack. So, I was debating rolling it over into a JHipster app to make it a more familiar stack for folks. My concern is that it I will spend months trying to shoehorn it in for not much ROI. Any thoughts on going down this path? What are the biggest issues you've seen in using JHipster? It seems pretty straightforward except for the entity generators. I'm concerned they are totally different than what I am using. The main difference in what I'm doing compared to JHipster is my almost complete use of groovy instead of old school Java in the app. I would have to be forced into going back to regular java beans... Thoughts? I replied with the following advice: JHipster is great for starting a project, but I don't know that it buys you much value after the first few months. I would stick with your current setup and consider JHipster for your next project. I've only prototyped with it, I haven't created any client apps or put anything in production. I have with Spring Boot and AngularJS though, so I like that JHipster combines them for me. JHipster doesn't generate Scala or Groovy code, but you could still use them in a project as long as you had Maven/Gradle configured properly. You might try generating a new app with JHipster and examine how they're doing this. At the very least, it can be a good learning tool, even if you're not using it directly. Java Hipsters: Do you agree with this advice? Have you tried migrating an existing app to JHipster? Are any of you using Scala or Groovy in your JHipster projects?
February 13, 2015
by Matt Raible
· 8,474 Views · 2 Likes
article thumbnail
Getting Started with Dropwizard: Authentication, Configuration and HTTPS
Basic Authentication is the simplest way to secure access to a resource.
February 10, 2015
by Dmitry Noranovich
· 48,822 Views · 1 Like
article thumbnail
The State of Java
I think living in a beautiful city in a fantastic climate has its advantages. Not just the obvious ones, but we find people unusually keen to come and visit us on the pretence of presenting at the Sevilla Java User Group (and please, DO come and present at our JUG, welove visitors). This week we were really lucky, we had Georges Saab and Aurelio Garcia-Ribeyro giving us an update on where Java is now and where it looks like it’s going in the future. I’m just starting to use Java 8 in real life, so this could not have been better timed - I got to ask the guys a bunch of questions about the intentions behind some of the Java 8 features, and the current vision for the future. 2015 Java update and roadmap, JUG sevilla from Georges Saab and Aurelio Garcia-Ribeyro My notes from the session: Lambdas could be just a syntax change, but they could be more than that - they could impact the language, the libraries and the JVM. They could have a positive impact on performance, and this work can continue to go on through small updates to Java that don’t impact the syntax. Streams are a pipeline of operations, made possible/easier/more readable by lambdas. The aim is to make operations on collections easier and more readable. In the Old World Order, you had to care about how to perform certain operations. With streams, you don’t need to tell the computer exactly how it’s done, you can simply say what operations you want performed. This makes it easier for developers Streams will take all the operations you pass in and perform them in a single pass of the data, so you don’t have to write multiple loops to perform multiple operations on the same data structure, or tie your brain in knots figuring out how to do it in one loop. There are also no intermediate data structures when you use streams. The implementation can be optimised under the covers (e.g. not performing the sort operation if the data is already ordered correctly), and the developer doesn’t have to worry about it. Java can introduce further optimisations in later releases without changing the API or impacting the code a developer has already written. These new features in Java have a focus on readability, since code is much more often read than written. The operations are easier to parallelise, because the developer is no longer dictating the how - multiple for loops might not be easy to parallelise, but a series of operations can be. Default methods and new support for static methods on interfacesare interesting. I’d forgotten you could put static methods on interfaces and I’m going to sneak them into my latest project. Nashorn is here to replace Rhino. Personally I haven’t worked in the sort of environment that would need server-side JavaScript so this whole area has passed me by somewhat, but seems it might be interesting for Node.js or creating a REPL in JavaScript that you want to run on the JVM. The additional annotation support in Java 8 will be useful for Java EE. As this is something I’m currently playing with (specifically web sockets) I’m interested in this, but it seems like it will be a while before this filters into the Java EE that’s used on a daily basis. Mission Control and Flight Recorder - look interesting. Feel like I should play with them. Many people are skipping straight from Java 6 to 8 - the new language features and improved performance are major driving factors End of public updates of Java 7 in April. Having libraries that, um…encourage… adoption of the latest version of Java makes life a lot easier for those who develop the Java language, as they can concentrate on moving the language forward and not be tied down supporting old versions. Either this is the first time I’ve heard of JavaDB, or my memory has completely discarded it. I had no idea what it was. Java 9 is well under way, check out the JEPs. (This is the JEP process) Jigsaw was explained, and actually I could see myself using it for the project I’m working on right now. I had a look to see if I could use it via the OpenJDK, but it looks like the groundwork is there, but not the actual modules themselves. The G1 Garbage Collector is “…the go forward GC”, it’s the one that’s being actively worked on. This is the first I’ve heard of Enhanced Volatiles, I’m so behind the times! Access to internal packages is going away in Java 9. So don’t use any sun.* packages. Use jdeps to identify any dependencies in your code that need to change. …and, looking further ahead than Java 9, we have value types andProject Valhalla …a REPL for Java …possibly a lightweight JSON API …and VarHandles were also mentioned. Finally, the guys mentioned a talk by Brian Goetz called “Stewardship: the Sobering Parts”, which has gone onto my to-watch list. Ideas It became clear throughout the talk there are plenty of ideas that we could explore in later presentations. If you want to see any of the following, add a comment or ping me or IsraKaos on twitter or Meetup and we’ll try and schedule it. Similarly, if you can present on any of these topics and want to come to a beautiful, sunny city with amazing food to do so, drop me a line: The OpenJDK The JCP, the purpose, the processes, the people Adopt a JSR, Adopt OpenJDK New Date/Time (JSR310) JavaFX Code optimisation vs Data optimisation (I honestly don’t know what this means, but I wrote it down in my notes) Java EE Further Reading I really liked Richard Warburton’s Lambdas and Streams book Java 8 in Action has details on other Java 8 features like Date and Time Java 8 for the Really Impatient covers JavaFX too, and highlights some Java 7 features you might like Brian Goetz did a great talk last year, “Lambdas in Java: A peek under the hood”. I had to watch it twice before even half of the info sank in, but it’s really interesting. Stephen Colebourne, the guy behind Joda time and the new Date and Time API, has this talk about the new API. Java 9 OpenJDK page
February 9, 2015
by Trisha Gee
· 8,331 Views · 2 Likes
article thumbnail
NetBeans in the Classroom: MySQL JDBC Connection Pool & JDBC Resource for GlassFish
This tutorial assumes that you have installed the Java EE version of NetBeans 8.02. It is further assumed that you have replaced the default GlassFish server instance in NetBeans with a new instance with its own folder for the domain. This is required for all platforms. See my previous article “Creating a New Instance of GlassFish in NetBeans IDE” The other day I presented to my students the steps necessary to make GlassFish responsible for JDBC connections. As I went through the steps I realized that I needed to record the steps for my students to reference. Here then are these steps. Steps 1 through 4 are required, Step 5 is optional. Step 1a: Manually Adding the MySQL driver to the GlassFish Domain Before we even start NetBeans we must do a bit of preliminary work. With the exception of Derby, GlassFish does not include the MySQL driver or any other driver in its distribution. Go to the MySql Connector/J download site at http://dev.mysql.com/downloads/connector/j/ and download the latest version. I recommend downloading the Platform Independent version. If your OS is Windows download the ZIP archive otherwise download the TAR archive. You are looking for the driver file named mysql-connector-java-5.1.34-bin.jar in the archive. Copy the driver file to the lib folder in the directory where you placed your domain. On my system the folder is located at C:\Users\Ken\personal_domain\lib. If GlassFish is already running then you will have to restart it so that it picks up the new library. Step 1b: Automatically Adding the MySQL driver to the GlassFish Domain NetBeans has a feature that deploys the database driver to the domain’s lib folder if that driver is in NetBeans’ folder of drivers. On my Windows 8.1 system the MySQL driver can be found in C:\Program Files\NetBeans 8.0.2\ide\modules\ext. Start NetBeans and go to the Services tab, expand Servers and right mouse click on your GlassFish Server. Click on Properties and the Servers dialog will appear. On this dialog you will see a check box labelled Enable JDBC Driver Deployment. By default it is checked. NetBeans determines the driver to copy to GlassFish from the file glassfish-resources.xml that we will create in Step 4 of this tutorial. Without this file and if you have not copied the driver into GlassFish manually then GlassFish will not be able to connect to the database. Any code in your web application will not work and all you will likely see are blank pages. Step 1a or Step 1b? I recommend Step 1a and manually add the driver. The reason I prefer this approach is that I can be certain that the most recent driver is in use. As of this writing NetBeans contains version 5.1.23 of the connector but the current version is 5.1.34. If you copy a driver into the lib folder then NetBeans will not replace it with an older driver even if the check box on the Server dialog is checked. NetBeans does not replace a driver if one is already in place. If you need a driver that NetBeans does have a copy of then Step 1b is your only choice. Step 2: Create a Database Connection in NetBeans One feature I have always liked in NetBeans is that it has an interface for working with databases. All that is required is that you create a connection to the database. It also has additional features for managing a MySQL server but we won’t need those. If you have not already started your MySQL DBMS then do that now. I assume that the database you wish to connect to already exists. Go to the Services tab and right mouse click on New Connection. In the next dialog you must choose the database driver you wish to use. It defaults to Java DB (Embedded). Pull down the combobox labeled Driver: and select MySQL (Connector/J driver). Click on Next and you will now see the Customize Connection dialog. Here you can enter the details of the connection. On my system the server is localhost and the database name is Aquarium. Here is what my dialog looks like. Notice the Test Connection button. I have clicked on mine and so I have the message Connection Succeeded. Click on Next. There is nothing to do on this dialog so click on Next. On this last dialog you have the option of assigning a name to the connection. By default it uses the URL but I prefer a more meaningful name. I have used AquariumMySQL. Click on Finish and the connection will appear under Databases. If the icon next to AquariumMySQL has what looks like a crack in it similar to the jdbc:derby connection then this means that a connection to the database could not be made. Verify that the database is running and is accessible. If it is then delete the connection and start over. Having a connection to the database in NetBeans is invaluable. You can interact with the database directly and issue SQL commands. As a MySQL user this means that I do not need to run the MySQL command line program to interact with the database. Step 3: Create a Web Application Project in NetBeans If you have not already done so create a New Project in NetBeans. I require my students to create a New Project in the Maven category of a Web Application project. Click on Next. In this dialog you can give the project a name and a location in your file system. The Artifact Id, Group Id and Version are used by Maven. The final dialog lets you select the application server that your application will use and the version of Java EE that your code must be compliant with. Here is my project ready for the next step. Step 4: Create the GlassFish JDBC Resource For GlassFish to manage your database connection you need to set up two resources, a JDBC Connection Pool and a JDBC Resource. You can create both in one step by creating a GlassFish JDBC Resource because you can create the Connection Pool as part of the same operation. Right mouse click on the project name and select New and then Other … Scroll down the Categories list and select GlassFish. In the File Types list select JDBC Resource. Click on Next. The next dialog is the General Attributes. Click on the radio button for Create New JDBC Connection Pool. In the text field JNDI Name enter a name that is unique for the project. JNDI names for connection resources always begin with jdbc/ followed by a name that starts with a lower case letter. I have used jdbc/myAquarium. Do not prefix the name with java:app/ as some tutorials suggest. An upcoming article will explain why. Click on Next. There is nothing for us to enter on the Properties dialog. Click on Next. On the Choose Database Connection dialog we will give our connection pool a name and select the database connection we created in Step 2. Notice that in the list of available connections you are shown the connection URL and not the name you assigned to it back in Step 2. Click on Next. On the Add Connection Pool Properties dialog you will see the connection URL and the user name and password. We do need to make one change. The resource type shows javax.sql.DataSource and we must change it to javax.sql.ConnectionPoolDataSource. Click on Next. There is nothing we need to change on Add Connection Pool Optional Properties so click on Finish. A new folder has appeared in the Projects view named Other Sources. It contains a sub folder named setup. In this folder is the file glassfish-resources.xml. The glassfish-resources.xml file will contain the following. I have reformatted the file for easier viewing. OPTIONAL Step 5: Configure GlassFish with glassfish-resources.xml The glassfish-resources.xml file, when included in the application’s WAR file in the WEB-INF folder, can configure the resource and pool for the application when it is deployed in GlassFish. When the application is un-deployed the resource and pool are removed. If you want to set up the resource and pool permanently in GlassFish then follow these steps. Go to the Services tab and select Servers and then right mouse click on GlassFish. If GlassFish is not running then click on Start. With the server started click on View Domain Admin Console. Your web browser will now open and show you the GlassFish console. If you assigned a user name and password to the server you will have to enter this information before you see the console. In the Common Tasks tree select Resources. You should now see in the panel adjacent to the tree the following: Click on Add Resources. You should now see: In the Location click on Choose File and locate your glassfish-resources.xml file. Mine is found at D:\NetBeansProjects\GlassFishTutorial\src\main\setup. You should now see: Click on OK. If everything has gone well you should see: The final task in this step is to test if the connection works. In the Common Tasks tree select Resources, JDBC, JDBC Connection Pools and aquariumPool. Click on Ping. You should see: The most common reason for the Ping to fail is that the database driver is not in the domain’s lib folder. Go to Step 1a and manually add the driver. The resources are now visible in NetBeans. Having the resource and pool add to GlassFish permanently will allow other applications to share this same resource and pool. You are now ready to code!
February 9, 2015
by Ken Fogel
· 52,613 Views · 3 Likes
article thumbnail
Groovy Goodness: Getting All But the Last Element in a Collection with Init Method
In Groovy we can use the head and tail methods for a long time on Collection objects. With head we get the first element and with tailthe remaining elements of a collection. Since Groovy 2.4 we have a new method init which returns all elements but the last in a collection. In the following example we have a simple list and apply the different methods: def gr8Tech = ['Groovy', 'Grails', 'Spock', 'Gradle', 'Griffon'] // Since Groovy 2.4 we can use the init method. assert gr8Tech.init() == ['Groovy', 'Grails', 'Spock', 'Gradle'] assert gr8Tech.last() == 'Griffon' assert gr8Tech.head() == 'Groovy' assert gr8Tech.tail() == ['Grails', 'Spock', 'Gradle', 'Griffon'] Code written with Groovy 2.4.
February 8, 2015
by Hubert Klein Ikkink
· 7,795 Views
article thumbnail
R: data.table/dplyr/lubridate - Error in wday
Error in wday(date, label = TRUE, abbr = FALSE) : unused arguments (label = TRUE, abbr = FALSE) I spent a couple of hours playing around with data.table this evening and tried changing some code written using a data frame to use a data table instead. I started off by building a data frame which contains all the weekends between 2010 and 2015… > library(lubridate) > library(dplyr) > dates = data.frame(date = seq( dmy("01-01-2010"), to=dmy("01-01-2015"), by="day" )) > dates = dates %>% filter(wday(date, label = TRUE, abbr = FALSE) %in% c("Saturday", "Sunday")) …which works fine: > dates %>% head() date 1: 2010-01-02 2: 2010-01-03 3: 2010-01-09 4: 2010-01-10 5: 2010-01-16 6: 2010-01-17 I then tried to change the code to use a data table instead which led to the following error: > library(data.table) > dates = data.table(date = seq( dmy("01-01-2010"), to=dmy("01-01-2015"), by="day" )) > dates = dates %>% filter(wday(date, label = TRUE, abbr = FALSE) %in% c("Saturday", "Sunday")) Error in wday(date, label = TRUE, abbr = FALSE) : unused arguments (label = TRUE, abbr = FALSE) I wasn’t sure what was going on so I went back to the data frame version to check if that still worked… > dates = data.frame(date = seq( dmy("01-01-2010"), to=dmy("01-01-2015"), by="day" )) > dates = dates %>% filter(wday(date, label = TRUE, abbr = FALSE) %in% c("Saturday", "Sunday")) Error in wday(c(1262304000, 1262390400, 1262476800, 1262563200, 1262649600, : unused arguments (label = TRUE, abbr = FALSE) …except it now didn’t work either! I decided to check what wday was referring to… Help on topic ‘wday’ was found in the following packages: Integer based date class (in package data.table in library /Library/Frameworks/R.framework/Versions/3.1/Resources/library) Get/set days component of a date-time. (in package lubridate in library /Library/Frameworks/R.framework/Versions/3.1/Resources/library) …and realised that data.table has its own wday function – I’d been caught out by R’s global scoping of all the things! We can probably work around that by the order in which we require the various libraries but for now I’m just prefixing the call to wday and all is well: dates = dates %>% filter(lubridate::wday(date, label = TRUE, abbr = FALSE) %in% c("Saturday", "Sunday"))
February 7, 2015
by Mark Needham
· 13,254 Views
article thumbnail
Java 8 Optional is Not Just for Replacing a null Value
In Java 8, you can return an Optional instead of return null; as you might do in Java 7. This may or may not make a big difference depending on whether you tend to forget to check for null or whether you use static code analysis to check to nullalbe references. However, there is a more compelling case which is to treat Optional like a Stream with 0 or 1 values. Simple Optional Use Case In the old days of Java 7 you would write something like String text = something(); if (text != null) { Note: Oracle Java 7 will be "End of Public Updates" in April 2015. With Optional you can instead write Optional text = something(); if (text.isPresent()) { String text2 = text.get(); However, if you are being paranoid you might write. Optional text = something(); if (text != null && text.isPresent()) { String text2 = text.get(); If you have NullPointerException errors often in your project Optional might help, but otherwise it's not looking like it helps much. A more complex example Lets instead consider this example static String getFirstSecondThird(Nested nested) { try { return ((Contained2) nested.first.second).get(0).third; } catch (NullPointerException | ClassCastException | IndexOutOfBoundsException ignored) { return null; } } This is really ugly. Instead of catching exceptions, you can build a long list of condition check but it becomes really hard to see what you are trying to do. Optional allows you to handle all the possible error conditions without an Exception or nested if/else logic. static Optional getFirstSecondThird(Optional nested) { return nested // could be non-present .map(x -> x.first) // could be null .map(x -> x.second) // could be null // could be another type .map(x -> x instanceof Contained2 ? (Contained2) x : null) .map(x -> x.list) // could be null .filter(x -> !x.isEmpty()) // could be empty .map(x -> x.get(0)) // could be null .map(x -> x.third); // could be null. } What we get is a series of mappings and filters which only progress if the value is non-null and present. If any value is null, or a filter is not true, the whole result is "not present". Conclusion Using Optional can be a powerful way to navigate a complex data structure in a safe way. The purpose of lambdas is to reduce boiler plate code, and in the case it avoids all the checks or errors you have. Additional For your interest, here is the classes I used in the example above. static class Nested { Contained first; } static class Contained { IContained2 second; } interface IContained2 { } static class Contained2 implements IContained2 { List list; } static class Data { String third; }
February 5, 2015
by Peter Lawrey
· 10,569 Views
article thumbnail
Resource Injection vs. Dependency Injection Explained!
Fellow geeks, the following article provides an overview of injection in Java EE and describes the two injection mechanisms provided by the platform: Resource Injection and Dependency Injection. Java EE provides injection mechanisms that enable our objects to obtain the references to resources and other dependencies without having to instantiate them directly (explicitly with ‘new’ keyword). We simply declare the needed resources & other dependencies in our classes by drawing fields or methods with annotations that denotes the injection point to the compiler. The container then provides the required instances at runtime. The advantage of Injection is that it simplifies our code and decouples it from the implementations of its dependencies. Note should be given for the fact that Dependency Injection is a specification (also a design pattern) and Context and Dependency Injection (CDI) is an implementation andJava standard for DI. The following topics are discussed here: · Resource Injection · Dependency Injection · Difference between Context and Dependency Injection 1. Resource Injection One of the simplification features of Java EE is the implementation of basic Resource Injection to simplify web and EJB components. Resource injection enables you to inject any resource available in the JNDI namespace into any container-managed object, such as a servlet, an enterprise bean, or a managed bean. For eg, we can use resource injection to inject data sources, connectors, or any other desired resources available in the JNDI namespace. The type we’ll use for the reference to the instance happen to be injected is usually an interface, which would decouple our code from the implementation of the resource. For better understanding of the above statement let’s take a look at the example. The resource injection can be performed in the following three ways: · Field Injection · Method Injection · Class injection Now, the javax.annotation.Resource annotation is used to declare a reference to a resource. So before proceeding, let’s learn few elements of @Resource annotation. @Resource has the following elements: · name: The JNDI name of the resource · type: The Java type of the resource · authenticationType: The authentication type to use for the resource · shareable: Indicates whether the resource can be shared · mappedName: A non-portable, implementation-specific name to which the resource should be mapped · description: The description of the resource Thenameelement is the JNDI name of the resource, and is optional for field- and method-based injection. For field injection, d defaultnameis the field name. For method-based injection, the defaultnameis the JavaBeans property name based on the method. The‘name’ and ‘type’element must be specified for class injection. Thedescriptionelement is the description of the resource (optional). Let’s hop on to the example now. Field Injection: To use field-based resource injection, declare a field and annotate it with the @Resource annotation. The container will refer the name and type of the resource if the name and type elements are not specified. If you do specify the type element, it must match the field’s type declaration. package com.example; public class SomeClass { @Resource private javax.sql.DataSource myDB; ... } In the code above, the container infers the name of the resource based on the class name and the field name: com.example.SomeClass/myDB. The inferred type isjavax.sql.DataSource.class. package com.example; public class SomeClass { @Resource(name="customerDB") private javax.sql.DataSource myDB; ... } In the code above, the JNDI name is customerDB, and the inferred type is javax.sql.DataSource.class. Method Injection: To use method injection, declare a setter method and preceding with the @Resource annotation. The container will itself refer the name and type of the resource if in case it is not specified by programmer. The setter method must follow the JavaBeans conventions for property names: the method name must begin with set, have a void return type, and only one parameter (needless to say :P). Anyways, if you do specify the return type, it must match the field’s type declaration. package com.example; public class SomeClass { private javax.sql.DataSource myDB; ... @Resource private void setMyDB(javax.sql.DataSource ds) { myDB = ds; } ... } In the code above, the container refers the name of the resource according to the class name and the field name: com.example.SomeClass/myDB. The type which is javax.sql.DataSource.class. package com.example; public class SomeClass { private javax.sql.DataSource myDB; ... @Resource (name="customerDB") private void setMyDB (javax.sql.DataSource ds) { myDB = ds; } ... } In the code above, the JNDI name is customerDB, and the inferred type is javax.sql.DataSource.class. Class Injection: To use class-based injection, decorate the class with a @Resource annotation, and set the requiredname and type elements. @Resource(name="myMessageQueue", type="javax.jms.ConnectionFactory") public class SomeMessageBean { ... } Declaring Multiple Resources The @Resources annotation is used to group together multiple @Resource declarations for class injection only. @Resources({ @Resource(name="myMessageQueue", type="javax.jms.ConnectionFactory"), @Resource(name="myMailSession", type="javax.mail.Session") }) public class SomeMessageBean { ... } The code above shows the @Resources annotation containing two @Resource declarations. One is a JMS (Java Messagin Service) message queue, and the other is a JavaMail session. 2. Dependency Injection Dependency injection enables us to turn regular Java classes into managed objects and to inject them into any other managed object (objects wich are managed by the container). Using DI, our code can declare dependencies on any managed object. The container automatically provides instances of these dependencies at the injection points at runtime, n it also manages the lifecycle of these instances right from class loading to releasing it for Garbage Collection. Dependency injection in Java EE defines scopes. For eg, a managed object that is only happen to respond to a single client request (such as a currency converter) has a different scope than a managed object that is needed to process multiple client requests within a session (such as a shopping cart). We can define managed objects (also called managed beans) so that we can later inject by assigning a scope to a needed class: @javax.enterprise.context.RequestScoped public class CurrencyConverter { ... } Use the javax.inject.Inject annotation to inject managed beans; for example: public class MyServlet extends HttpServlet { @Inject CurrencyConverter cc; ... } Umlike resource injection, dependency injection is typesafe because it resolves by type. To decouple our code from the implementation of the managed bean, we can reference the injected instances using an interface type and have our managed bean (regular class controlled by container) implement that interface. I wouldn’t like to discuss more on DI or better saying CDI since we already have a great article published on this. 3. Difference between Resource Injection and Dependency Injection The differences between the RI and DI are listed below. 1. Resource Injection can inject JNDI Resources directly whereas Dependency Injection cannot. 2. Dependency Injection can inject Regular Classes (managed bean) directly whereas Resource Injection cannot. 3. Resource Injection resolves by resource name whereas Dependency Injectin resolves by type. 4. Dependency Injection is typesafe whereas Resoiurce Injection is not. Conclusion: Thus we learnt concept on types on Injection in Java EE and the differences between them. Just a brief. There’s more to come
February 2, 2015
by Lalit Rao
· 69,091 Views · 10 Likes
article thumbnail
Getting Started with Dropwizard: First Steps
Dropwizard is a bunch of superb frameworks glued together to provide a fast way of building web applications including REST APIs. We'll talk about great frameworks which are parts of Dropwizard as we go over our examples. The full code for the examples can be obtained here . The simplest way to create a Dropwizard project is to use the Maven archetype called java-simple which is a part of Dropwizard. This can be accomplished either via the command line or using your favorite IDE. Let's stick to the former approach. The command below is formatted for clarity but it should be pasted as a single line to the terminal. mvn archetype:generate -DgroupId=com.javaeeeee -DartifactId=DWGettingStarted -Dname=DWGettingStarted -Dpackage=com.javaeeeee.dwstart -DarchetypeGroupId=io.dropwizard.archetypes -DarchetypeArtifactId=java-simple -DinteractiveMode=false After the project was created it can be modified using an editor or opened in an IDE. All the three IntelliJ, Eclipse and Netbeans have Maven support. An important parameter to note in the pom.xml file of the newly created project is dropwizard.version, which at the time of writing was automatically set to 0.8.0-rc2-SNAPSHOT and the latest stable version was 0.7.1. We'll discuss later the important consequences of the version for project development. The directory structure of the project is shown below. There are multiple sub-packages and two classes that were created for us in a package com.javaeeeee.dwstart. Now let's create a simple resource class that produces a string “Hello world!”, start the application and test the results using both browser and cURL. The snippet below shows the resource class. @Path("/hello") public class HelloResource { @GET @Produces(MediaType.TEXT_PLAIN) public String getGreeting() { return "Hello world!"; } } There is a special package, com.javaeeeee.dwstart.recources for placing resource classes or an appropriate folder can be found from the screenshot above if you use a text editor and not an IDE. One of the main tenets of the REST architecture style is that each resource is assigned a URL. In our case the URL of our resource would be http://localhost:8080/hello, that is we access it from our local machine, the default port which is used by Dropwizard is 8080 and the final part of the URL is enshrined in @Path annotation in our class, in other words, the aforementioned annotation helps to describe the URL used to access a resource. Moving on to method getGreeting() it can be seen that it is a simple method which returns the desired string “Hello world!”, although it is marked with two annotations: first, @GET prescribes that this method is called when the aforementioned URL is accessed using HTTP GET method, second, annotation @Produces specifies what media types can be produced when the method is accessed by the client, such as browser, cURL or some other program. In our case it is plain text. These annotations are part of Java API for RESTful Web Services (JAX-RS), and its reference implementation Jersey is the cornerstone of Dropwizard. Other annotations include @POST for HTTP POST method, @DELETE for DELETE and so on and media types include MediaType.APPLICATION_JSON and not so popular MediaType.APPLICATION_XML among others. Now, to see the result of our coding, we should do some configuration. Let's open one of the created by Maven files, DWGettingStartedApplication.java, and register our resource. public void run(final DWGettingStartedConfiguration configuration, final Environment environment) { environment.jersey().register(new HelloResource()); } We added a single line environment.jersey().register(new HelloResource()); to help Dropwizard find our resource class. Now we have to create a jar-file which contains an embedded Jersey web server to serve the incoming requests, as well as all the necessary libraries. Dropwizard applications are packaged as executable jar-files rather than war-files which are deployed to an application server. The kind of jar files used by Dropwizard applications are also called “fat” because they include all the .class files necessary to run the application as this leads to a situation that the versions of all libraries are the same in both development and production environments which leads to less problems. One shouldn't worry about creating such jar-files as Maven creates them for us using instructions in the generated pom.xml file of the project. The jar-file can be created using command-line command issued from the project's folder mvn package or using an IDE. After that we should start the application, which could be accomplished either from the command line by issuing a command java -jar target/DWGettingStarted-1.0-SNAPSHOT.jar server or from the IDE which can be instructed to pass a “server” argument to the executable jar file. To stop the application it is sufficient to press Ctrl+C in the terminal. Now it is time to check if it works. The simplest way to achieve this is to navigate your browser to the URL mentioned earlier. You should see the greeting string in the main window of a browser. If you cannot see the greeting, there is probably some problems and the program’s output is the place to look for clues for what went wrong. The same greeting can be seen in the terminal window with help of cURL. curl -w "\n" localhost:8080/hello Protocol HTTP and GET method are defaults, so it is not necessary that they be included explicitly in the command. As the result a message "Hello world!" without quotes should be printed. The w-option is used to add a trailing newline to prettify the output. It is a good idea to start using tests as early as possible, so let's write a test that checks the work of our resource class using in-memory Jersey. We'll create the test in test packages and as we are testing a resource an appropriate package to place our test is com.javaeeeee.dwstart.resources (or src/test/com/javaeeeee/dwstart/resources folder). It is necessary that the following dependency be added to the pom-file of the project. (It works without explicit jUnit dependency.) io.dropwizard dropwizard-testing ${dropwizard.version} test An example test for our greeting resource is shown below. public class HelloResourceTest { @Rule public ResourceTestRule resource = ResourceTestRule.builder() .addResource(new HelloResource()).build(); @Test public void testGetGreeting() { String expected = "Hello world!"; //Obtain client from @Rule. Client client = resource.client(); //Get WebTarget from client using URI of root resource. WebTarget helloTarget = client.target("http://localhost:8080/hello"); //To invoke response we use Invocation.Builder //and specify the media type of representation asked from resource. Invocation.Builder builder = helloTarget.request(MediaType.TEXT_PLAIN); //Obtain response. Response response = builder.get(); //Do assertions. assertEquals(Response.Status.OK, response.getStatusInfo()); String actual = response.readEntity(String.class); assertEquals(expected, actual); } } The annotation @Rule is from jUnit realm and could be placed on public non-static fields and methods that return a value. This annotation may perform some initial setup and clean-up like @Before and @After methods, but it is more powerful as it allows to share functionality between classes and even projects. This rule is needed to allow us to access our resources from the code of the test using in-memory Jersey. At this point of time we should pause and discuss the version of our product. There is a pom.xml file in the dropwizard folder on the GitHub, which can readily be accessed by cloning the project, and this file specifies the version of constituent products among other things. It can be seen from it that the version of Jersey in the snapshot version of Dropwizard is 2.13 and in the version 0.7.1 it is 1.18.1. The version of Jersey influences the code of the tests as 1.x and 2.x versions of Jersey have different client APIs, that is the code to access the service programmatically from Java depends on the version of Jersey framework. Let's stick to the latest 2.x version of Jersey as it is liable to be included in following 0.8.x releases of Dropwizard and if you are interested how to test using the older version of Jersey client, please check the project repository , there is a special branch v0.7.x. Another observation concerning testing is that Fest matchers were superseded by AssertJ counterparts. First, we obtain an instance of Jersey client to make requests to our service. Second, we create an instance of a class that implements WebTarget interface for the resource using its URL. After that we use Invocation.Builder that helps to build request and send it to the server. The Invocation.Builder interface extends SynchInvoker interface which defines a bunch of get(...) methods used to invoke HTTP GET method synchronously. Finally, we check the results, namely that the HTTP status code was 200 and the desired string was returned. The same result could be obtained by chaining the methods as in the snippet below. actual = resource.client() .target("http://localhost:8080/hello") .request(MediaType.TEXT_PLAIN) .get(String.class); assertEquals(expected, actual); The replacement of the two assertions by a single is possible due to the fact that the get(...) method we used throws a WebApplicationException if the HTTP status code is not in the 2xx range, in other words if there is a problem. Now let's improve our resource to take a name and return a bespoke greeting. It could be done in a couple of ways using Jersey's annotations. The first is to use so-called path parameters, that is you pass the name using a URL like http://localhost:8080/hello/path_param/your_name. The application should return “Hello your_name”. Let's see a code snippet. @Path("/path_param/{name}") @GET @Produces(MediaType.TEXT_PLAIN) public String getNamedGreeting(@PathParam(value = "name") String name) { return "Hello " + name; } Now we see a @Path annotation on our method. It adds its content to those produced by class level annotation as seen from the URL. Methods marked with this annotation are called sub-resource methods. The name of the parameter, name, is in curly braces. Something interesting happens inside the parenthesis of the getNamedGreeting method. There is an annotation which prescribes to extract the content of the url after “/path_param/” to the method as a value. It should be noted that if you find yourself marking each method with the same annotation @Produces(MediaType.TEXT_PLAIN), the latter can be removed from methods and used to mark the class, then all the methods produce the desired media types, although this can be overruled by an annotations with a different media type for some of the methods. The second way to pass a name is to use query parameters and the URL could look like http://localhost:8080/hello/query_param?name=your_name. The output should be “Hello your_name” without quotes. The snippet is shown below. @Path("/query_param") @GET @Produces(MediaType.TEXT_PLAIN) public String getNamedStringWithParam(@DefaultValue("world") @QueryParam("name") String name) { return "Hello " + name; } Once again, the @Path annotation adds something to our URL, then the default value of the parameter is set in case there is no question mark and following it symbols in the URL using @DefaultValue annotation. In other words, if the parameter is omitted in the URL and it looks like http://localhost:8080/hello/query_param, a phrase “Hello world” is printed. The last annotation injects a parameter from the URL to the method's parameter. By the way, all these cases can be tested as was shown above. To test a method using query parameters one can use a queryParam(...) method of WebTarget interface. There is a couple of points to pay attention to. Firstly, there are other @*Param annotations available and secondly, the @DefaultValue annotation will not work with path parameters and to process the absence of the parameter a special sub-resource method should be introduced. You were patient enough to bear all this plain text staff, but it is not what REST APIs are about. Let's create another sub-resource method which is preprogrammed to return JSON representation. The first step is to create a representation class, which should be placed to com.javaeeeee.dwstart.core package. Let's limit ourselves to extremely simple example which is shown below. public class Greeting { @JsonProperty private String greeting; public Greeting() { } public Greeting(String greeting) { this.greeting = greeting; } public String getGreeting() { return greeting; } } The class above relies on one more important part of the Dropwizard stack namely Jackson , which can turn Java objects to JSON and vice versa. A @JsonProperty annotation is used to do the job. The sub-resource method looks like pretty much the same as our previously discussed methods. @Path("/hello_json") @GET @Produces(MediaType.APPLICATION_JSON) public Greeting getJSONGreeting() { return new Greeting("Hello world!"); } The difference is that we changed the media type and the method returns an instance of the greeting class. If you navigate your browser to to a URL http://localhost:8080/hello/hello_json or use cURL, you should see a JSON representation of the object {"greeting":"Hello world!"}. What if we remove the @Path annotation from the aforementioned method altogether? Well, if you try to access the localhost:8080/hello resource you'll see the text representation. How one could access the JSON-producing method? There is a special HTTP header called Accept which is used to instruct the server what representation to return, the process called content negotiation. The Jersey inside the Dropwizard will choose for us what method to use based on the content of the Accept header. Let's try cURL to do this. curl -w "\n" -H 'Accept: application/json' localhost:8080/hello/ The H-option is used to send headers and we instructed cURL to ask the JSON response. Other possible types to try could include text/plain and application/xml. The former should return the plain-text representation and the letter should result in error as there is no method in our resource class that can produce the latter media type response. Another way to engage in content negotiation is to use a REST client for your browser. There are several clients for each of major browsers but we will use Postman extension for Chrome browser. There are special fields to input headers as the screenshot below shows. One can enter the resource URL, headers and press Send button and the response will show up in the lower part of the window. To sum up we created a simple REST-like API using Dropwizard, which can greet its users. We learned how to create a Dropwizard project using Maven archetype and created a simple resource class to accomplish the task of greeting and a representation class to produce JSON response. An important ingredient of REST architecture style, HATEOAS, was omitted for now for simplicity reasons. Also a problem of testing resource classes was touched as well as the ways to access resources both from browser and command line. There are some other ideas for creating sub-resources such as returning the current date and adding two numbers using query parameters which can be easily implemented using the information presented in this article. References Dropwizard on GitHub Maven Archetypes Dropwizard Getting Started JAX-RS Resources and Subresources Dropwizard Testing Jersey Client API 2.x Jersey Client API 1.x Jackson annotations
January 30, 2015
by Dmitry Noranovich
· 35,614 Views · 3 Likes
article thumbnail
Importing Big Tables With Large Indexes With Myloader MySQL Tool
originally written by david ducos mydumper is known as the faster (much faster) mysqldump alternative. so, if you take a logical backup you will choose mydumper instead of mysqldump. but what about the restore? well, who needs to restore a logical backup? it takes ages! even with myloader. but this could change just a bit if we are able to take advantage of fast index creation. as you probably know, mydumper and mysqldump export the struct of a table, with all the indexes and the constraints, and of course, the data. then, myloader and mysql import the struct of the table and import the data. the most important difference is that you can configure myloader to import the data using a certain amount of threads. the import steps are: create the complete struct of the table import the data when you execute myloader, internally it first creates the tables executing the “-schema.sql” files and then takes all the filenames without “schema.sql” and puts them in a task queue. every thread takes a filename from the queue, which actually is a chunk of the table, and executes it. when finished it takes another chunk from the queue, but if the queue is empty it just ends. this import procedure works fast for small tables, but with big tables with large indexes the inserts are getting slower caused by the overhead of insert the new values in secondary indexes. another way to import the data is: split the table structure into table creation with primary key, indexes creation and constraint creation create tables with primary key per table do: load the data create index create constraints this import procedure is implemented in a branch of myloader that can be downloaded from here or directly executing bzr with the repository: bzr branch lp:~david-ducos/mydumper/mydumper the tool reads the schema files and splits them into three separate statements which create the tables with the primary key, the indexes and the constraints. the primary key is kept in the table creation in order to avoid the recreation of the table when a primary key is added and the “key” and “constraint” lines are removed. these lines are added to the index and constraint statements, respectively. it processes tables according to their size starting with the largest because creating the indexes of a big table could take hours and is single-threaded. while we cannot process other indexes at the time, we are potentially able to create other tables with the remaining threads. it has a new thread (monitor_process) that decides which chunk of data will be put in the task queue and a communication queue which is used by the task processes to tell the monitor_process which chunk has been completed. i run multiple imports on an aws m1.xlarge machine with one table comparing myloader and this branch and i found that with large indexes the times were: as you can see, when you have less than 150m rows, import the data and then create the indexes is higher than import the table with the indexes all at once. but everything changes after 150m rows, import 200m takes 64 minutes more for myloader but just 24 minutes for the new branch. on a table of 200m rows with a integer primary key and 9 integer columns, you will see how the time increases as the index gets larger: where: 2-2-0: two 1-column and two 2-column index 2-2-1: two 1-column, two 2-column and one 3-column index 2-3-1: two 1-column, three 2-column and one 3-column index 2-3-2: two 1-column, three 2-column and two 3-column index conclusion this branch can only import all the tables with this same strategy, but with this new logic in myloader, in a future version it could be able to import each table with the best strategy reducing the time of the restore considerably.
January 27, 2015
by Peter Zaitsev
· 5,264 Views
article thumbnail
A very quick guide to deadlock diagnosis in SQL Server
Recently I was asked about diagnosing deadlocks in SQL Server – I’ve done a lot of work in this area way back in 2008, so I figure it’s time for a refresher. If there’s a lot of interest in exploring SQL Server and deadlocks further, I’m happy to write an extended article going into far more detail. Just let me know. Before we get into diagnosis and investigation, it’s a good time to pose the question: “what is a deadlock?”: From TechNet: A deadlock occurs when two or more tasks permanently block each other by each task having a lock on a resource which the other tasks are trying to lock. The following graph presents a high level view of a deadlock state where: Task T1 has a lock on resource R1 (indicated by the arrow from R1 to T1) and has requested a lock on resource R2 (indicated by the arrow from T1 to R2). Task T2 has a lock on resource R2 (indicated by the arrow from R2 to T2) and has requested a lock on resource R1 (indicated by the arrow from T2 to R1). Because neither task can continue until a resource is available and neither resource can be released until a task continues, a deadlock state exists. The SQL Server Database Engine automatically detects deadlock cycles within SQL Server. The Database Engine chooses one of the sessions as a deadlock victim and the current transaction is terminated with an error to break the deadlock. Basically, it’s a resource contention issue which blocks one process or transaction from performing actions on resources within SQL Server. This can be a serious condition, not just for SQL Server as processes become suspended, but for the applications which rely on SQL Server as well. The T-SQL Approach A fast way to respond is to execute a bit of T-SQL on SQL Server, making use of System Views. The following T-SQL will show you the “victim” processes, much like activity monitor does: select * from sys.sysprocesses where blocked > 0 Which is not particularly useful (but good to know, so you can see the blocked count). To get to the heart of the deadlock, this is what you want (courtesy of this SO question/answer): SELECT Blocker.text –, Blocker.*, * FROM sys.dm_exec_connections AS Conns INNER JOIN sys.dm_exec_requests AS BlockedReqs ON Conns.session_id = BlockedReqs.blocking_session_id INNER JOIN sys.dm_os_waiting_tasks AS w ON BlockedReqs.session_id = w.session_id CROSS APPLY sys.dm_exec_sql_text(Conns.most_recent_sql_handle) AS Blocker This will show you line and verse (the actual statement causing the resource block) – see the attached screenshot for an example. However, the generally accepted way to determine and diagnose deadlocks is through the use of SQL Server trace flags. SQL Trace Flags They are (usually) set temporarily, and they cause deadlocking information to be dumped to the SQL management logs. The flags that are useful are flags 1204 and 1222. From TechNet: https://technet.microsoft.com/en-us/library/ms178104%28v=sql.105%29.aspx Trace flags are set on or off by using either of the following methods: · Using the DBCC TRACEON and DBCC TRACEOFF commands. For example, DBCC TRACEON 2528: To enable the trace flag globally, use DBCC TRACEON with the -1 argument: DBCC TRACEON (2528, -1). To turn off a global trace flag, use DBCC TRACEOFF with the -1 argument. · Using the -T startup option to specify that the trace flag be set on during startup. The -T startup option enables a trace flag globally. You cannot enable a session-level trace flag by using a startup option. So to enable or disable deadlock trace flags globally, you’d use the following T-SQL: DBCC TRACEON (1204, -1) DBCC TRACEON (1222, -1) DBCC TRACEOFF (1204, -1) DBCC TRACEOFF (1222, -1) Due to the overhead, it’s best to enable the flag at runtime rather than on start up. Note that the scope of a non-startup trace flag can be global or session-level. Basic Deadlock Simulation By way of a very simple scenario, you can make use of SQL Management Studio (and breakpoints) to roughly simulate a deadlock scenario. Given the following basic table schema: CREATE TABLE [dbo].[UploadedFile]( [Id] [int] NOT NULL, [Filename] [nvarchar](50) NOT NULL, [DateCreated] [datetime] NOT NULL, [DateModified] [datetime] NULL, CONSTRAINT [PK_UploadedFile] PRIMARY KEY CLUSTERED ( [Id] ASC )WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF) ) With some basic test data in it: If you create two separate queries in SQL Management Studio, use the following transaction (Query #1) to lock rows in the table: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE BEGIN TRANSACTION SELECT [Id],[Filename],[DateCreated],[DateModified] FROM [dbo].[UploadedFile] WHERE DateCreated > ‘2015-01-01′ ROLLBACK TRANSACTION Now add a “victim” script (Query #2) in a separate query session: UPDATE [dbo].[UploadedFile] SET [DateModified] = ‘2014-12-31′ WHERE DateCreated > ‘2015-01-01′ As long as you set a breakpoint on the ROLLBACK TRANSACTION statement, you’ll block the second query due to the isolation level of the transaction which wraps query #1. Now you can use the diagnostic T-SQL to examine the victim and the blocking transaction. Enjoy!
January 27, 2015
by Rob Sanders
· 165,235 Views
article thumbnail
The Cost of Laziness
Recently I had a dispute with my colleagues regarding performance penalty of lazy vals in Scala. It resulted in a set of microbenchmarks which compare lazy and non-lazy vals performance. All the sources can be found at http://git.io/g3WMzA. But before going to the benchmark results let's try to understand what can cause the performance penalty. For my JMH benchmark I created a very simple Scala class with lazy val in it: @State(Scope.Benchmark) class LazyValCounterProvider { lazy val counter = SlowInitializer.createCounter() } Now let's take a look at what is hidden under the hood of lazy keyword. At first, we need to compile given code with scalac, and then it can be decompiled to correspondent Java code. For this sake I used JD decompiler. It produced the following code: @State(Scope.Benchmark) @ScalaSignature(bytes="...") public class LazyValCounterProvider { private SlowInitializer.Counter counter; private volatile boolean bitmap$0; private SlowInitializer.Counter counter$lzycompute() { synchronized (this) { if (!this.bitmap$0) { this.counter = SlowInitializer.createCounter(); this.bitmap$0 = true; } return this.counter; } } public SlowInitializer.Counter counter() { return this.bitmap$0 ? this.counter : counter$lzycompute(); } } As it's seen, the lazy keyword is translated to a classical double-checked locking idiom for delayed initialization. Thus, most of the time the only performance penalty may come from a single volatile read per lazy val read (except for the time it takes to initialize lazy val instance since its very first usage). Let's finally measure its impact in numbers. My JMH-based microbenchmark is as simple as: public class LazyValsBenchmarks { @Benchmark public long baseline(ValCounterProvider eagerProvider) { return eagerProvider.counter().incrementAndGet(); } @Benchmark public long lazyValCounter(LazyValCounterProvider provider) { return provider.counter().incrementAndGet(); } } A baseline method access a final counter object and increments an integer value by 1 by calling incrementAndGet . And as we've just found out, the main benchmark method - lazyValCounter - in addition to what baseline method does also does one volatile read. Note: all measurements are performed on MBA with Core i5 1.7GHz CPU. All results were obtained by running JMH in a throughput mode. Both score and score error columns show operations/second. Each JMH run made 10 iterations and took 50 seconds. I performed 6 measurements with the different JVM and JMH options: client VM, 1 thread Benchmark Score Score error baseline 412277751.619 8116731.382 lazyValCounter 352209296.485 6695318.185 client VM, 2 threads Benchmark Score Score error baseline 542605885.932 15340285.497 lazyValCounter 383013643.710 53639006.105 client VM, 4 threads Benchmark Score Score error baseline 551105008.767 5085834.663 lazyValCounter 394175424.898 3890422.327 server VM, 1 thread Benchmark Score Score error baseline 407010942.139 9004641.910 lazyValCounter 341478430.115 18183144.277 server VM, 2 threads Benchmark Score Score error baseline 531472448.578 22779859.685 lazyValCounter 428898429.124 24720626.198 server VM, 4 threads Benchmark Score Score error baseline 549568334.970 12690164.639 lazyValCounter 374460712.017 17742852.788 The numbers show that lazy vals performance penalty is quite small and can be ignored in practice. For further reading about the subject I would recommend SIP 20 - Improved Lazy Vals Initialization, which contains very interesting in-depth analysis of existing issues with lazy initialization implementation in Scala.
January 26, 2015
by Roman Gorodyshcher
· 11,792 Views · 1 Like
article thumbnail
Improving Lock Performance in Java
After we introduced locked thread detection to Plumbr couple of months ago, we have started to receive queries similar to “hey, great, now I understand what is causing my performance issues, but what I am supposed to do now?” We are working hard to build the solution instructions into our own product, but in this post I am going to share several common techniques you can apply independent of the tool used for detecting the lock. The methods include lock splitting, concurrent data structures, protecting the data instead of the code and lock scope reduction. Locking is not evil, lock contention is Whenever you face a performance problem with the threaded code there is a chance that you will start blaming locks. After all, common “knowledge” is that locks are slow and limit scalability. So if you are equipped with this “knowledge” and start to optimize the code and getting rid of locks there is a chance that you end up introducing nasty concurrency bugs that will surface later on. So it is important to understand the difference between contended and uncontended locks. Lock contention occurs when a thread is trying to enter the synchronized block/method currently executed by another thread. This second thread is now forced to wait until the first thread has completed executing the synchronized block and releases the monitor. When only one thread at a time is trying to execute the synchronized code, the lock stays uncontended. As a matter of fact, synchronization in JVM is optimized for the uncontended case and for the vast majority of the applications, uncontended locks pose next to no overhead during execution. So, it is not locks you should blame for performance, but contended locks. Equipped with this knowledge, lets see what we can do to reduce either the likelihood of contention or the length of the contention. Protect the data not the code A quick way to achieve thread-safety is to lock access to the whole method. For example, take look at the following example, illustrating a naive attempt to build an online poker server: class GameServer { public Map<> tables = new HashMap>(); public synchronized void join(Player player, Table table) { if (player.getAccountBalance() > table.getLimit()) { List tablePlayers = tables.get(table.getId()); if (tablePlayers.size() < 9) { tablePlayers.add(player); } } } public synchronized void leave(Player player, Table table) {/*body skipped for brevity*/} public synchronized void createTable() {/*body skipped for brevity*/} public synchronized void destroyTable(Table table) {/*body skipped for brevity*/} } The intentions of the author have been good - when new players join() the table, there must be a guarantee that the number of players seated at the table would not exceed the table capacity of nine. But whenever such a solution would actually be responsible for seating players to tables - even on a poker site with moderate traffic, the system would be doomed to constantly trigger contention events by threads waiting for the lock to be released. Locked block contains account balance and table limit checks which potentially can involve expensive operations both increasing the likelihood and length of the contention. First step towards solution would be making sure we are protecting the data, not the code by moving the synchronization from the method declaration to the method body. In the minimalistic example above, it might not change much at the first place. But lets consider the whole GameServerinterface, not just the single join() method: class GameServer { public Map> tables = new HashMap>(); public void join(Player player, Table table) { synchronized (tables) { if (player.getAccountBalance() > table.getLimit()) { List tablePlayers = tables.get(table.getId()); if (tablePlayers.size() < 9) { tablePlayers.add(player); } } } } public void leave(Player player, Table table) {/* body skipped for brevity */} public void createTable() {/* body skipped for brevity */} public void destroyTable(Table table) {/* body skipped for brevity */} } What originally seemed to be a minor change, now affects the behaviour of the whole class. Whenever players were joining tables, the previously synchronized methods locked on theGameServer instance (this) and introduced contention events to players trying to simultaneouslyleave() tables. Moving the lock from the method signature to the method body postpones the locking and reduces the contention likelihood. Reduce the lock scope Now, after making sure it is the data we actually protect, not the code, we should make sure our solution is locking only what is necessary - for example when the code above is rewritten as follows: public class GameServer { public Map> tables = new HashMap>(); public void join(Player player, Table table) { if (player.getAccountBalance() > table.getLimit()) { synchronized (tables) { List tablePlayers = tables.get(table.getId()); if (tablePlayers.size() < 9) { tablePlayers.add(player); } } } } //other methods skipped for brevity } then the potentially time-consuming operation of checking player account balance (which potentially can involve IO operations) is now outside the lock scope. Notice that the lock was introduced only to protect against exceeding the table capacity and the account balance check is not anyhow part of this protective measure. Split your locks When we look at the last code example, you can clearly notice that the whole data structure is protected by the same lock. Considering that we might hold thousands of poker tables in this structure, it still poses a high risk for contention events as we have to protect each table separately from overflowing in capacity. For this there is an easy way to introduce individual locks per table, such as in the following example: public class GameServer { public Map> tables = new HashMap>(); public void join(Player player, Table table) { if (player.getAccountBalance() > table.getLimit()) { List tablePlayers = tables.get(table.getId()); synchronized (tablePlayers) { if (tablePlayers.size() < 9) { tablePlayers.add(player); } } } } //other methods skipped for brevity } Now, if we synchronize the access only to the same table instead of all the tables, we have significantly reduced the likelihood of locks becoming contended. Having for example 100 tables in our data structure, the likelihood of the contention is now 100x smaller than before. Use concurrent data structures Another improvement is to drop the traditional single-threaded data structures and use data structures designed explicitly for concurrent usage. For example, when picking ConcurrentHashMapto store all your poker tables would result in code similar to following: public class GameServer { public Map> tables = new ConcurrentHashMap>(); public synchronized void join(Player player, Table table) {/*Method body skipped for brevity*/} public synchronized void leave(Player player, Table table) {/*Method body skipped for brevity*/} public synchronized void createTable() { Table table = new Table(); tables.put(table.getId(), table); } public synchronized void destroyTable(Table table) { tables.remove(table.getId()); } } The synchronization in join() and leave() methods is still behaving as in our previous example, as we need to protect the integrity of individual tables. So no help from ConcurrentHashMap in this regards. But as we are also creating new tables and destroying tables in createTable() and destroyTable()methods, all these operations to the ConcurrentHashMap are fully concurrent, permitting to increase or reduce the number of tables in parallel. Other tips and tricks Reduce the visibility of the lock. In the example above, the locks are declared public and are thus visible to the world, so there is there is a chance that someone else will ruin your work by also locking on your carefully picked monitors. Check out java.util.concurrent.locks to see whether any of the locking strategies implemented there will improve the solution. Use atomic operations. The simple counter increase we are actually conducting in example above does not actually require a lock. Replacing the Integer in count tracking withAtomicInteger would most suit this example just fine. Hope the article helped you to solve the lock contention issues, independent of whether you are using Plumbr automatic lock detection solution or manually extracting the information from thread dumps.
January 22, 2015
by Nikita Salnikov-Tarnovski
· 11,536 Views
article thumbnail
Implementing Ehcache using Spring context and Annotations
Part 1 – Configuring Ehcache with Spring Ehcache is most widely used open source cache for boosting performance, offloading your database, and simplifying scalability. It is very easy to configure Ehcache with Spring context xml. Assuming you have a working Spring configured application, here we discuss how to integrate Ehcache with Spring. In order to configure it, we need to add a CacheManager into the Spring context file. Code snippet added below In ‘ehcache’ bean, we are referring to ‘Ehcache.xml’ file which will contain all ehcache configurations. Sample ehcache file is added below. You can configure this file according to your project needs. Here you can two caches, defaultCache and one defined with name ‘TestCache’. DefaultCache configuration is applied to any cache that is not explicitly configured. Part 2 - Enabling Caching Annotations In order to used Spring provided java annotations , we need to declarative enable cache annotations in Spring config file. Code snippet added below. In order to use above annotation declaration, we need to add cache namespace into the spring file. Code snippet added below. Spring Annotations for caching @Cacheable triggers cache population @CacheEvict triggers cache eviction @CachePut updates the cache without interfering with the method execution I will give brief description and practical implementation of these annotations. For detailed explanations please refer spring documentation for caching . @Cacheable - This annotation means that the return value of the corresponding method can be cached and multiple invocation of this method with same arguments must populate the return value from cache without executing the method. Code snippet added below. @Cacheable("customer") public Customer findCustomer(String custId) {...} Each time the method is called, the cache with name ‘customer’ is checked to see whether the invocation has been already executed for the same ‘custId’. @CachePut - This annotation is used to update the cache without interfering the method execution. Method will always be executed and its result will be replaced in the cache. This annoatation is used for cache population. @CachePut("customer") public Customer updateBook(String custId){...} @CacheEvict – This annotation is used for cache eviction, used to remove stale or unused data from the cache. @CacheEvict(value="customer", allEntries=true) public void unloadCustomer(String newBatch) With above configurations, all entries in the ‘customer’ cache will be removed. Above description will give head start to configure Ehcachewith spring and use some of the caching annotations.
January 21, 2015
by Roshan Thomas
· 12,227 Views
article thumbnail
Avoiding MySQL ALTER Table Downtime
Originally Written byAndrew Moore MySQL table alterations can interrupt production traffic causing bad customer experience or in worst cases, loss of revenue. Not all DBAs, developers, syadmins know MySQL well enough to avoid this pitfall. DBAs usually encounter these kinds of production interruptions when working with upgrade scripts that touch both application and database or if an inexperienced admin/dev engineer perform the schema change without knowing how MySQL operates internally. Truths * Direct MySQL ALTER table locks for duration of change (pre-5.6) * Online DDL in MySQL 5.6 is not always online and may incurr locks * Even with Percona Toolkit‘s pt-online-schema-change there are several workloads that can experience blocking Here on the Percona MySQL Managed Services team we encourage our clients to work with us when planning and performing schema migrations. We aim to ensure that we are using the best method available in their given circumstance. Our intentions to avoid blocking when performing DDL on large tables ensures that business can continue as usual whilst we strive to improve response time or add application functionality. The bottom line is that a business relying on access to its data cannot afford to be down during core trading hours. Many of the installations we manage are still below MySQL 5.6, which requires us to seek workarounds to minimize the amount of disruption a migration can cause. This may entail slave promotion or changing the schema with an ‘online schema change’ tool. MySQL version 5.6 looks to address this issue by reducing the number of scenarios where a table is rebuilt and locked but it doesn’t yet cover all eventualities, for example when changing the data type of a column a full table rebuild is necessary. The topic of 5.6 Online Schema Change was discussed in great detail last year in the post, “Schema changes – what’s new in MySQL 5.6?” by Przemysław Malkowski With new functionality arriving in MySQL 5.7, we look forward to non-blocking DDL operations such as; OPTIMIZE TABLE and RENAME INDEX. (More info) The best advice for MySQL 5.6 users is to review the matrix to familiarize with situations where it might be best to look outside of MySQL to perform schema changes, the good news is that we’re on the right path to solving this natively. Truth be told, a blocking alter is usually going to go unnoticed on a 30MB table and we tend to use a direct alter in this situation, but on a 30GB or 300GB table we have some planning to do. If there is a period of time where activity is low and the this is permissive of locking the table then sometimes it is better execute within this window. Frequently though we are reactive to new SQL statements or a new performance issue and an emergency index is required to reduce load on the master in order to improve the response time. To pt-osc or not to pt-osc? As mentioned, pt-online-schema-change is a fixture in our workflow. It’s usually the right way to go but we still have occasions where pt-online-schema-change cannot be used, for example; when a table already uses triggers. It’s an important to remind ourselves of the the steps that pt-online-schema-change traverses to complete it’s job. Lets look at the source code to identify these; [moore@localhost]$ egrep 'Step' pt-online-schema-change # Step 1: Create the new table. # Step 2: Alter the new, empty table. This should be very quick, # Step 3: Create the triggers to capture changes on the original table and <--(metadata lock) # Step 4: Copy rows. # Step 5: Rename tables: orig -> old, new -> orig <--(metadata lock) # Step 6: Update foreign key constraints if there are child tables. # Step 7: Drop the old table. [moore@localhost]$egrep'Step'pt-online-schema-change # Step 1: Create the new table. # Step 2: Alter the new, empty table. This should be very quick, # Step 3: Create the triggers to capture changes on the original table and <--(metadata lock) # Step 4: Copy rows. # Step 5: Rename tables: orig -> old, new -> orig <--(metadata lock) # Step 6: Update foreign key constraints if there are child tables. # Step 7: Drop the old table. I pick out steps 3 and 5 from above to highlight a source of a source of potential downtime due to locks, but step 6 is also an area for concern since foreign keys can have nested actions and should be considered when planning these actions to avoid related tables from being rebuilt with a direct alter implicitly. There are several ways to approach a table with referential integrity constraints and they are detailed within the pt-osc documentation a good preparation step is to review the structure of your table including the constraints and how the ripples of the change can affect the tables around it. Recently we were alerted to an incident after a client with a highly concurrent and highly transactional workload ran a standard pt-online-schema-change script over a large table. This appeared normal to them and a few hours later our pager man was notified that this client was experiencing max_connections limit reached. So what was going on? When pt-online-schema-change reached step 5 it tried to acquire a metadata lock to rename the the original and the shadow table, however this wasn’t immediately granted due to open transactions and thus threads began to queue behind the RENAME command. The actual effect this had on the client’s application was downtime. No new connections could be made and all existing threads were waiting behind the RENAME command. Metadata locks Introduced in 5.5.3 at server level. When a transaction starts it will acquire a metadata lock (independent of storage engine) on all tables it uses and then releases them when it’s finished it’s work. This ensures that nothing can alter the table definition whilst a transaction is open. With some foresight and planning we can avoid these situations with non-default pt-osc options, namely –nodrop-new-table and –no-swap-tables. This combination leaves both the shadow table and the triggers inplace so that we can instigate an atomic RENAME when load permits. EDIT: as of percona-toolkit version 2.2 we have a new variable –tries which in conjunction with –set-vars has been deployed to cover this scenario where various pt-osc operations could block waiting for a metadata lock. The default behaviour of pt-osc (–set-vars) is to set the following session variables when it connects to the server; wait_timeout=10000 innodb_lock_wait_timeout=1 lock_wait_timeout=60 when using –tries we can granularly identify the operation, try count and the wait interval between tries. This combination will ensure that pt-osc will kill it’s own waiting session in good time to avoid the thread pileup and provide us with a loop to attempt to acquire our metadata lock for triggers|rename|fk management; –tries swap_tables:5:0.5,drop_triggers:5:0.5 The documentation is here http://www.percona.com/doc/percona-toolkit/2.2/pt-online-schema-change.html#cmdoption-pt-online-schema-change–tries This illustrates that even with a tool like pt-online-schema-change it is important to understand the caveats presented with the solution you think is most adequate. To help decide the direction to take use the flow chart to ensure you’re taking into account some of the caveats of the MySQL schema change. Be sure to read up on the recommended outcome though as there are uncharted areas such as disk space, IO load that are not featured on the diagram. Choosing the right DDL option Ensure you know what effect ALTER TABLE will have on your platform and pick the right method to suit your uptime. Sometimes that means delaying the change until a period of lighter use or utilising a tool that will avoid holding a table locked for the duration of the operation. A direct ALTER is sometimes the answer like when you have triggers installed on a table. – In most cases pt-osc is exactly what we need – In many cases pt-osc is needed but the way in which it’s used needs tweaking – In few cases pt-osc isn’t the right tool/method and we need to consider native blocking ALTER or using failovers to juggle the change into place on all hosts in the replica cluster. If you want to learn more about avoiding avoidable downtime please tune into my webinar Wednesday, November 19 at 10 a.m. PST. It’s titled “Tips from the Trenches: A Guide to Preventing Downtime for the Over-Extended DBA.” Register now!(If you miss it, don’t worry: Catch the recording and download the slides from that same registration page.)
January 20, 2015
by Peter Zaitsev
· 15,234 Views · 1 Like
article thumbnail
Angular JS: Conditional Enable/Disable Checkboxes
In this post you can see an approach for conditionally enabling/disabling a set of checkboxes. For this we can use the ng-disabled directive and some CSS clases of typeclassName-true and className-false: ENABLE/DISABLE CHECKBOXES USING ANGULAR JS Select the maximum prize money: Select one prize money{{item.prizemoney} {{item.name}
January 20, 2015
by Anghel Leonard DZone Core CORE
· 45,622 Views · 3 Likes
article thumbnail
Angular JS: Use an Angular Websocket Client with a Java Websocket Endpoint
In this tip you can see how to use the Angular Websocket module for connecting client applications to servers.
January 16, 2015
by Anghel Leonard DZone Core CORE
· 40,420 Views · 7 Likes
article thumbnail
How to Send SMS Messages in Java Using HTTP Requests
In this article I am going to present a solution about sending SMS messages in Java for those developers and marketers who think – like me – that SMS is not dead.
January 15, 2015
by Timothy Walker
· 286,965 Views · 5 Likes
article thumbnail
Fail-fast Validations Using Java 8 Streams
I’ve lost count of the number of times I’ve seen code which fail-fast validates the state of something, using an approach like public class PersonValidator { public boolean validate(Person person) { boolean valid = person != null; if (valid) valid = person.givenName != null; if (valid) valid = person.familyName != null; if (valid) valid = person.age != null; if (valid) valid = person.gender != null; // ...and many more } } It works, but it’s a brute force approach that’s filled with repetition due to the valid check. If your code style enforces braces for if statements (+1 for that), your method is also three times longer and growing every time a new check is added to the validator. Using Java 8’s new stream API, we can improve this by taking the guard condition of if (valid) and making a generic validator that handles the plumbing for you. import java.util.LinkedList; import java.util.List; import java.util.function.Predicate; public class GenericValidator implements Predicate { private final List> validators = new LinkedList<>(); public GenericValidator(List> validators) { this.validators.addAll(validators); } @Override public boolean test(final T toValidate) { return validators.parallelStream() .allMatch(predicate -> predicate.test(toValidate)); } } Using this, we can rewrite the Person validator to be a specification of the required validations. public class PersonValidator extends GenericValidator { private static final List> VALIDATORS = new LinkedList<>(); static { VALIDATORS.add(person -> person.givenName != null); VALIDATORS.add(person -> person.familyName != null); VALIDATORS.add(person -> person.age != null); VALIDATORS.add(person -> person.gender != null); // ...and many more } public PersonValidator() { super(VALIDATORS); } } PersonValidator, and all your other validators, can now focus completely on validation. The behaviour hasn’t changed – the validation still fails fast. There’s no boiler plate, which is A Good Thing. This one’s going in the toolbox.
January 15, 2015
by Steve Chaloner
· 20,415 Views · 2 Likes
  • Previous
  • ...
  • 403
  • 404
  • 405
  • 406
  • 407
  • 408
  • 409
  • 410
  • 411
  • 412
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×