Spring 3.1 now includes support for the long awaited environment aware feature called profiles. Now we can activate profiles in our application, which allows us to define beans by deployment regions, such as “dev”, “qa”, “production”, “cloud”, etc. We also can use this feature for other purposes: defining profiles for performance testing scenarios such as “cached” or “lazyload”. Essential Tokens Spring profiles are enabled using the case insensitive tokens spring.profiles.active or spring_profiles_active. This token can be set as: an Environment Variable a JVM Property Web Parameter Programmatic Spring also looks for the token, spring.profiles.default, which can be used to set the default profile(s) if none are specified with spring.profiles.active. Grouping Beans by Profile Spring 3.1 provides nested bean definitions, providing the ability to define beans for various environments: Nested must appear last in the file. Beans that are used in all profiles are declared in the outer as we always have, such as Service classes. If we put a single declaration at below any nested tags we will get the exception org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'bean'. Multiple beans can now share the same XML “id” In a typical scenario, we would want the DataSource bean to be called dataSource in both all profiles. Spring now allow us to create multiple beans within an XML file with the same ID providing they are defined in different sets. In other words, ID uniqueness is only enforced within each set. Automatic Profile Discovery (Programmatic) We can configure a class to set our profile(s) during application startup by implementing the appropriate interface. For example, we may configure an application to set different profiles based on where the application is deployed – in CloudFoundry or running as a local web application. In the web.xml file we can include an Servlet context parameter, contextInitializerClasses, to bootstrap this class: contextInitializerClasses com.gordondickens.springthreeone.services.CloudApplicationContextInitializer The Initializer class package com.gordondickens.springthreeone.services; import org.cloudfoundry.runtime.env.CloudEnvironment; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.context.ApplicationContextInitializer; import org.springframework.context.ConfigurableApplicationContext; public class CloudApplicationContextInitializer implements ApplicationContextInitializer { private static final Logger logger = LoggerFactory .getLogger(CloudApplicationContextInitializer.class); @Override public void initialize(ConfigurableApplicationContext applicationContext) { CloudEnvironment env = new CloudEnvironment(); if (env.getInstanceInfo() != null) { logger.info("Application running in cloud. API '{}'", env.getCloudApiUri()); applicationContext.getEnvironment().setActiveProfiles("cloud"); applicationContext.refresh(); } else { logger.info("Application running local"); applicationContext.getEnvironment().setActiveProfiles("dev"); } } } Annotation Support for JavaConfig If we are are using JavaConfig to define our beans, Spring 3.1 includes the @Profile annotation for enabling bean config files by profile(s). package com.gordondickens.springthreeone.configuration; import com.gordondickens.springthreeone.SimpleBean; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Profile; @Configuration @Profile("dev") public class AppConfig { @Bean public SimpleBean simpleBean() { SimpleBean simpleBean = new SimpleBean(); simpleBean.setMyString("Ripped Pants"); return simpleBean; } } Testing with XML Configuration With XML configuration we can simply add the annotation @ActiveProfiles to the JUnit test class. To include multiple profiles, use the format @ActiveProfiles(profiles = {"dev", "prod"}) package com.gordondickens.springthreeone; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.NoSuchBeanDefinitionException; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.ApplicationContext; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import static junit.framework.Assert.assertNotNull; import static junit.framework.Assert.assertNull; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration @ActiveProfiles(profiles = "dev") public class DevBeansTest { @Autowired ApplicationContext applicationContext; @Test public void testDevBeans() { SimpleBean simpleBean = applicationContext.getBean("constructorBean", SimpleBean.class); assertNotNull(simpleBean); } @Test(expected = NoSuchBeanDefinitionException.class) public void testProdBean() { SimpleBean prodBean = applicationContext.getBean("prodBean", SimpleBean.class); assertNull(prodBean); } } Testing with JavaConfig JavaConfig allows us to configure Spring with or without XML configuration. If we want to test beans that are defined in a Configuration class we configure our test with the loader and classes arguments of the @ContextConfiguration annotation. package com.gordondickens.springthreeone.configuration; import com.gordondickens.springthreeone.SimpleBean; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.test.context.ActiveProfiles; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; import org.springframework.test.context.support.AnnotationConfigContextLoader; import static org.junit.Assert.assertNotNull; @RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(classes = AppConfig.class, loader = AnnotationConfigContextLoader.class) @ActiveProfiles(profiles = "dev") public class BeanConfigTest { @Autowired SimpleBean simpleBean; @Test public void testBeanAvailablity() { assertNotNull(simpleBean); } } Declarative Configuration in WEB.XML If we desire to set the configuration in WEB.XML, this can be done with parameters on ContextLoaderListener. Application Context contextConfigLocation /WEB-INF/app-config.xml spring.profiles.active DOUBLEUPMINT Log Results DEBUG PropertySourcesPropertyResolver - Found key 'spring.profiles.active' in [servletContextInitParams] with type [String] and value 'DOUBLEUPMINT' Environment Variable/JVM Parameter Setting an environment variable can be done with either spring_profiles_default or spring_profiles_active. In Unix/Mac it would be export SPRING_PROFILES_DEFAULT=DEVELOPMENT for my local system. We can also use the JVM “-D” parameter which also works with Maven when using Tomcat or Jetty plugins. Note: Remember the tokens are NOT case sensitive and can use periods or underscores as separators. For Unix systems, you need to use the underscore, as above. Logging of system level properties DEBUG PropertySourcesPropertyResolver - Found key 'spring.profiles.default' in [systemProperties] with type [String] and value 'dev,default' Summary Now we are equipped to activate various Spring bean sets, based on profiles we define. We can use traditional, XML based configuration, or the features added to support JavaConfig originally introduced in Spring 3.0.

Check out an example of the volatile Java keyword.

CSS3 Fade slider Today I would like to show you how to create nice and smooth css3 slider. It uses fade effect to switch between slides. Plus, you can use custom promo text for each slide. We will use basic UL-LI unordered list to make this slider. We don’t need to click anywhere to switch slides – everything is automatically (css3 animation). Live Demo download result So, lets start Step 1. HTML Html markup is very easy. There are four slides. Each slide consists of image (as background) and promo text in DIV. If you need – you always can add more slides here. Promo text #1 Promo text #2 Promo text #3 Promo text #4 Step 2. CSS Now, it’s time to define css styles for our slider. Here are styles for main slider element, inner slides and for promo titles: /* fade slider */ .slides { height:300px; margin:50px auto; overflow:hidden; position:relative; width:900px; } .slides ul { list-style:none; position:relative; } /* keyframes #anim_slides */ @-webkit-keyframes anim_slides { 0% { opacity:0; } 6% { opacity:1; } 24% { opacity:1; } 30% { opacity:0; } 100% { opacity:0; } } @-moz-keyframes anim_slides { 0% { opacity:0; } 6% { opacity:1; } 24% { opacity:1; } 30% { opacity:0; } 100% { opacity:0; } } .slides ul li { opacity:0; position:absolute; top:0; /* css3 animation */ -webkit-animation-name: anim_slides; -webkit-animation-duration: 24.0s; -webkit-animation-timing-function: linear; -webkit-animation-iteration-count: infinite; -webkit-animation-direction: normal; -webkit-animation-delay: 0; -webkit-animation-play-state: running; -webkit-animation-fill-mode: forwards; -moz-animation-name: anim_slides; -moz-animation-duration: 24.0s; -moz-animation-timing-function: linear; -moz-animation-iteration-count: infinite; -moz-animation-direction: normal; -moz-animation-delay: 0; -moz-animation-play-state: running; -moz-animation-fill-mode: forwards; } /* css3 delays */ .slides ul li:nth-child(2), .slides ul li:nth-child(2) div { -webkit-animation-delay: 6.0s; -moz-animation-delay: 6.0s; } .slides ul li:nth-child(3), .slides ul li:nth-child(3) div { -webkit-animation-delay: 12.0s; -moz-animation-delay: 12.0s; } .slides ul li:nth-child(4), .slides ul li:nth-child(4) div { -webkit-animation-delay: 18.0s; -moz-animation-delay: 18.0s; } .slides ul li img { display:block; } /* keyframes #anim_titles */ @-webkit-keyframes anim_titles { 0% { left:100%; opacity:0; } 5% { left:10%; opacity:1; } 20% { left:10%; opacity:1; } 25% { left:100%; opacity:0; } 100% { left:100%; opacity:0; } } @-moz-keyframes anim_titles { 0% { left:100%; opacity:0; } 5% { left:10%; opacity:1; } 20% { left:10%; opacity:1; } 25% { left:100%; opacity:0; } 100% { left:100%; opacity:0; } } .slides ul li div { background-color:#000000; border-radius:10px 10px 10px 10px; box-shadow:0 0 5px #FFFFFF inset; color:#FFFFFF; font-size:26px; left:10%; margin:0 auto; padding:20px; position:absolute; top:50%; width:200px; /* css3 animation */ -webkit-animation-name: anim_titles; -webkit-animation-duration: 24.0s; -webkit-animation-timing-function: linear; -webkit-animation-iteration-count: infinite; -webkit-animation-direction: normal; -webkit-animation-delay: 0; -webkit-animation-play-state: running; -webkit-animation-fill-mode: forwards; -moz-animation-name: anim_titles; -moz-animation-duration: 24.0s; -moz-animation-timing-function: linear; -moz-animation-iteration-count: infinite; -moz-animation-direction: normal; -moz-animation-delay: 0; -moz-animation-play-state: running; -moz-animation-fill-mode: forwards; } You can see that I use two css3 animations here: anim_slides and anim_titles. First one is for separated slides, second one – for promo texts. In order to switch between slides – we change opacity of slides. For titles – we change Left position and opacity too. Live Demo download result Conclusion That is all for today. We have just created new cool pure CSS3-based slider with Fade effect. I hope that you like it. Good luck!

from timeit import default_timer class Timer(object): def __init__(self, verbose=False): self.verbose = verbose self.timer = default_timer def __enter__(self): self.start = self.timer() return self def __exit__(self, *args): end = self.timer() self.elapsed_secs = end - self.start self.elapsed = self.elapsed_secs * 1000 # millisecs if self.verbose: print 'elapsed time: %f ms' % self.elapsed To use the Timer (context manager object), invoke it using Python's `with` statement. The duration of the context (code inside your `with` block) will be timed. It uses the appropriate timer for your platform, via the `timeit` module. Timer is used like this: with Timer() as target: # block of code goes here. # result (elapsed time) is stored in `target` properties. Example script: timing a web request (HTTP GET), using the `requests` module. #!/usr/bin/env python import requests from timer import Timer url = 'https://github.com/timeline.json' with Timer() as t: r = requests.get(url) print 'fetched %r in %.2f millisecs' % (url, t.elapsed) Output: fetched 'https://github.com/timeline.json' in 458.76 millisecs `timer.py` in GitHub Gist form, with more examples: #!/usr/bin/env python # # Python Timer Class - Context Manager for Timing Code Blocks # Corey Goldberg - 2012 # from timeit import default_timer class Timer(object): def __init__(self, verbose=False): self.verbose = verbose self.timer = default_timer def __enter__(self): self.start = self.timer() return self def __exit__(self, *args): end = self.timer() self.elapsed_secs = end - self.start self.elapsed = self.elapsed_secs * 1000 # millisecs if self.verbose: print 'elapsed time: %f ms' % self.elapsed if __name__ == '__main__': # example: # 'HTTP GET' from requests module, inside timer blocks. # invoke the Timer context manager using the `with` statement. import requests url = 'https://github.com/timeline.json' # verbose (auto) timer output with Timer(verbose=True): r = requests.get(url) # print stored elapsed time in milliseconds with Timer() as t: r = requests.get(url) print 'response time (millisecs): %.2f' % t.elapsed # print stored elapsed time in seconds with Timer() as t: r = requests.get(url) print 'response time (secs): %.3f' % t.elapsed_secs # example output: # # $ python timer.py # elapsed time: 652.403831 ms # response time (millisecs): 635.49 # response time (secs): 0.624

Performance problems are one of the biggest challenges to expect when designing and implementing Java EE related technologies.

There are several cool ways to create PDFs with Python. In this article we will be focusing on a cool little tool called rst2pdf, which takes a text file that contains Restructured Text and converts it to a PDF. The rst2pdf package requires Reportlab to function. This won’t be a tutorial on Restructured Text, although we’ll have to discuss it to some degree just to understand what’s going on. Getting Started First off we’ll need to create a document with the required markup. Let’s do that first. Here’s some simple restructured text with a couple of directives mixed in. We’ll explain everything after you’ve had a chance to read the code: .. header:: Python Rules! - page ###Page### ===== Title ===== This is some blah blah blah .. raw:: pdf PageBreak oneColumn New section =========== yada yada yada import urllib import urllib2 import webbrowser url = "http://duckduckgo.com/html" data = urllib.urlencode({'q': 'Python'}) results = urllib2.urlopen(url, data) with open("results.html", "w") as f: f.write(results.read()) webbrowser.open("results.html") The first couple lines define the header that will be on every page. In this case, we’re going to have “Python Rules!” printed at the top of every page along with a page number. There are several other special hash-mark insert directives available. You should check out the official documentation for more information on those. Then we have a Title. Note that it is preceded and followed by a bunch of equal signs that are the same length as the text. This tells us that this text will be styled and centered. The following line is just a lame sentence for demonstration purposes. Next up is another special directive which tells rst2pdf to insert a page break. The second page contains a section header, a lame sentence and a code example that’s color-coded. To generate the PDF, you’ll need to do something like this on the command line: rst2pdf test.rst -o out.pdf You can also run rst2pdf against a config file to control some of the special PDF directives, like header and footer, etc. The information about how to make the config file get read is a little confusing though. It sounds like you have to place the file in a specific location: /etc/rst2pdf.conf and ~/.rst2pdf/config. There’s also a –config flag you can pass, but I’ve found various reports online that that doesn’t work, so your mileage may vary. There’s a sample config file in the project’s repo that you’ll find instructive. Wrapping Up I was hoping that rst2pdf would allow an easy way to specify absolute positions and create lines and boxes so I could replace an XSL / XML project I was working on with something much more simple. Alas, rst2pdf just doesn’t support the lines and boxes that reportlab itself does at the time of this writing. However if you need something easy to use to create your documents with and you already know restructured text, I think this is a very good way to go. You can also take your restructured text skills and use them with the Sphinx documentation project.

Innodb can indexes built by sort since Innodb Plugin for MySQL 5.1 which is a lot faster than building them through insertion, especially for tables much larger than memory and large uncorrelated indexes you might be looking at 10x difference or more. Yet for some reason Innodb team has chosen to use very small (just 1MB) and hard coded buffer for this operation, which means almost any such index build operation has to use excessive sort merge passes significantly slowing down index built process. Mark Callaghan and Facebook Team has fixed this in their tree back in early 2011 adding innodb_merge_sort_block_size variable and I was thinking this small patch will be merged to MySQL 5.5 promptly, yet it has not happen to date. Here is example of gains you can expect (courtesy of Alexey Kopytov), using 1Mil rows Sysbench table. Buffer Length | alter table sbtest add key(c) 1MB 34 sec 8MB 26 sec 100MB 21 sec 128MB 17 sec REBUILD 37 sec REBUILD in this table is using “fast_index_creation=0″ which allows to disable fast index creation in Percona Server and force complete table to be rebuilt instead. Looking at this data we can see even for such small table there is possible to improve index creation time 2x by using large buffer. Also we can see we can substantially improve performance even increasing it from 1MB to 8MB, which might be sensible as default as even small systems should be able to allocate 8MB to do alter table. You may be wondering why in this case table rebuild is so close in performance to building index by sort with small buffer – this comes from building index on long character field with very short length, Innodb would use fixed size records for sort space which results in a lot more work done than you would otherwise need. Having some optimization to better deal with this case also would be nice. The table also was fitting in buffer pool completely in this case which means table rebuild could have done fast too. Results are from Percona Server 5.5.24

Usually we have some required fields on our forms and it would be nice if ASP.NET MVC views can detect those fields automatically and display nice red asterisk after field label. As this functionality is not built in I built my own solution based on data annotations. In this posting I will show you how to show red asterisk after label of required fields. Here are the main information sources I used when working out my own solution: How can I modify LabelFor to display an asterisk on required fields? (stackoverflow) ASP.NET MVC – Display visual hints for the required fields in your model (Radu Enucă) Although my code was first written for completely different situation I needed it later and I modified it to work with models that use data annotations. If data member of model has Required attribute set then asterisk is rendered after field. If Required attribute is missing then there will be no asterisk. Here’s my code. You can take just LabelForRequired() methods and paste them to your own HTML extension class. public static class HtmlExtensions { [SuppressMessage("Microsoft.Design", "CA1006:DoNotNestGenericTypesInMemberSignatures", Justification = "This is an appropriate nesting of generic types")] public static MvcHtmlString LabelForRequired(this HtmlHelper html, Expression> expression, string labelText = "") { return LabelHelper(html, ModelMetadata.FromLambdaExpression(expression, html.ViewData), ExpressionHelper.GetExpressionText(expression), labelText); } private static MvcHtmlString LabelHelper(HtmlHelper html, ModelMetadata metadata, string htmlFieldName, string labelText) { if (string.IsNullOrEmpty(labelText)) { labelText = metadata.DisplayName ?? metadata.PropertyName ?? htmlFieldName.Split('.').Last(); } if (string.IsNullOrEmpty(labelText)) { return MvcHtmlString.Empty; } bool isRequired = false; if (metadata.ContainerType != null) { isRequired = metadata.ContainerType.GetProperty(metadata.PropertyName) .GetCustomAttributes(typeof(RequiredAttribute), false) .Length == 1; } TagBuilder tag = new TagBuilder("label"); tag.Attributes.Add( "for", TagBuilder.CreateSanitizedId( html.ViewContext.ViewData.TemplateInfo.GetFullHtmlFieldName(htmlFieldName) ) ); if (isRequired) tag.Attributes.Add("class", "label-required"); tag.SetInnerText(labelText); var output = tag.ToString(TagRenderMode.Normal); if (isRequired) { var asteriskTag = new TagBuilder("span"); asteriskTag.Attributes.Add("class", "required"); asteriskTag.SetInnerText("*"); output += asteriskTag.ToString(TagRenderMode.Normal); } return MvcHtmlString.Create(output); } } And here’s how to use LabelForRequired extension method in your view: @Html.LabelForRequired(m => m.Name) @Html.TextBoxFor(m => m.Name) @Html.ValidationMessageFor(m => m.Name) After playing with CSS style called .required my example form looks like this: These red asterisks are not part of original view mark-up. LabelForRequired method detected that these properties have Required attribute set and rendered out asterisks after field names. NB! By default asterisks are not red. You have to define CSS class called “required” to modify how asterisk looks like and how it is positioned.

Apache Camel has a very powerful bean injection framework which allows developers to focus only on solving business problems. However there are situations when you need to do a little bit more. Read below to see how easy it is to setup aspects (AspectJ) in Apache Camel. Use case In Qualitas I have an installation route which consists of 10 mandatory and 2 optional processors. Some processors like property resolvers or validators don't modify contents of message's body so I have to always copy body from the in message to out message. Also, all my processors require some headers to function properly. Finally, I would like to get a status updated after each of my processors either finishes processing successfully or fails. Setting up AspectJ This is just a plain Spring configuration frankly. All you have to do is: Apache Camel processor aspect To define aspect in AspectJ I used AspectJ-specific @Aspect and @Around annotations: @Aspect @Component @Order(Ordered.LOWEST) public class HeadersAndBodyCopierAspect { @Around("execution(* com.googlecode.qualitas.internal.installation..*.process(org.apache.camel.Exchange)) && args(exchange) && target(org.apache.camel.Processor)") public Object copyHeadersAndBody(ProceedingJoinPoint pjp, Exchange exchange) throws Throwable { Object retValue = pjp.proceed(); Message in = exchange.getIn(); Message out = exchange.getOut(); // always copy headers out.setHeaders(in.getHeaders()); // if output body is empty copy it from input if (out.getBody() == null) { out.setBody(in.getBody()); } return retValue; } } I also used @Order Spring-specific annotation to control the order of execution of my aspects and @Component for automatic context scanning. Now, the join point is defined as execution(* com.googlecode.qualitas.internal.installation..*.process(org.apache.camel.Exchange)) && args(exchange) && target(org.apache.camel.Processor) which basically means: apply this aspect to all process methods which are defined in all classes in com.googlecode.qualitas.internal.installation or subpackages and which take Exchange object as an argument there can be many custom methods whose names may be process and whose argument may be Exchange, so I added one more constraint, this class has to be an instance of Processor args(exchange) allows me to add Exchange object as an argument to my aspect More complex aspects and source code Of course in Spring you can inject other beans directly into your aspects. I used it in my ProcessStatusUpdaterAspect aspect which you can find in Qualitas repo on GoogleCode or GitHub. If you are interested in trying out the whole Qualitas system take a look at the following two links: BuildingTheProject and RunningTheProject. cheers, Łukasz

This article is part 2 of our NoClassDefFoundError troubleshooting series. It will focus and describe the simplest type of NoClassDefFoundError problem. This article is ideal for Java beginners and I highly recommend that you compile and run the sample Java program yourself. The following writing format will be used going forward and will provide you with: - Description of the problem case and type of NoClassDefFoundError - Sample Java program “simulating” the problem case - ClassLoader chain view - Recommendations and resolution strategies NoClassDefFoundError problem case 1 – missing JAR file The first problem case we will cover is related to a Java program packaging and / or classpath problem. A typical Java program can include one or many JAR files created at compile time. NoClassDefFoundError can often be observed when you forget to add JAR file(s) containing Java classes referenced by your Java or Java EE application. This type of problem is normally not hard to resolve once you analyze the Java Exception and missing Java class name. Sample Java program The following simple Java program is split as per below: - The main Java program NoClassDefFoundErrorSimulator - The caller Java class CallerClassA - The referencing Java class ReferencingClassA - A util class for ClassLoader and logging related facilities JavaEETrainingUtil This program is simple attempting to create a new instance and execute a method of the Java class CallerClassA which is referencing the class ReferencingClassA.It will demonstrate how a simple classpath problem can trigger NoClassDefFoundError. The program is also displaying detail on the current class loader chain at class loading time in order to help you keep track of this process. This will be especially useful for future and more complex problem cases when dealing with larger class loader chains. #### NoClassDefFoundErrorSimulator.java package org.ph.javaee.training1; import org.ph.javaee.training.util.JavaEETrainingUtil; /** * NoClassDefFoundErrorTraining1 * @author Pierre-Hugues Charbonneau * */ public class NoClassDefFoundErrorSimulator { /** * @param args */ public static void main(String[] args) { System.out.println("java.lang.NoClassDefFoundError Simulator - Training 1"); System.out.println("Author: Pierre-Hugues Charbonneau"); System.out.println("http://javaeesupportpatterns.blogspot.com"); // Print current Classloader context System.out.println("\nCurrent ClassLoader chain: "+JavaEETrainingUtil.getCurrentClassloaderDetail()); // 1. Create a new instance of CallerClassA CallerClassA caller = new CallerClassA(); // 2. Execute method of the caller caller.doSomething(); System.out.println("done!"); } } #### CallerClassA.java package org.ph.javaee.training1; import org.ph.javaee.training.util.JavaEETrainingUtil; /** * CallerClassA * @author Pierre-Hugues Charbonneau * */ public class CallerClassA { private final static String CLAZZ = CallerClassA.class.getName(); static { System.out.println("Classloading of "+CLAZZ+" in progress..."+JavaEETrainingUtil.getCurrentClassloaderDetail()); } public CallerClassA() { System.out.println("Creating a new instance of "+CallerClassA.class.getName()+"..."); } public void doSomething() { // Create a new instance of ReferencingClassA ReferencingClassA referencingClass = new ReferencingClassA(); } } #### ReferencingClassA.java package org.ph.javaee.training1; import org.ph.javaee.training.util.JavaEETrainingUtil; /** * ReferencingClassA * @author Pierre-Hugues Charbonneau * */ public class ReferencingClassA { private final static String CLAZZ = ReferencingClassA.class.getName(); static { System.out.println("Classloading of "+CLAZZ+" in progress..."+JavaEETrainingUtil.getCurrentClassloaderDetail()); } public ReferencingClassA() { System.out.println("Creating a new instance of "+ReferencingClassA.class.getName()+"..."); } public void doSomething() { //nothing to do... } } #### JavaEETrainingUtil.java package org.ph.javaee.training.util; import java.util.Stack; import java.lang.ClassLoader; /** * JavaEETrainingUtil * @author Pierre-Hugues Charbonneau * */ public class JavaEETrainingUtil { /** * getCurrentClassloaderDetail * @return */ public static String getCurrentClassloaderDetail() { StringBuffer classLoaderDetail = new StringBuffer(); Stack classLoaderStack = new Stack(); ClassLoader currentClassLoader = Thread.currentThread().getContextClassLoader(); classLoaderDetail.append("\n-----------------------------------------------------------------\n"); // Build a Stack of the current ClassLoader chain while (currentClassLoader != null) { classLoaderStack.push(currentClassLoader); currentClassLoader = currentClassLoader.getParent(); } // Print ClassLoader parent chain while(classLoaderStack.size() > 0) { ClassLoader classLoader = classLoaderStack.pop(); // Print current classLoaderDetail.append(classLoader); if (classLoaderStack.size() > 0) { classLoaderDetail.append("\n--- delegation ---\n"); } else { classLoaderDetail.append(" **Current ClassLoader**"); } } classLoaderDetail.append("\n-----------------------------------------------------------------\n"); return classLoaderDetail.toString(); } } Problem reproduction In order to replicate the problem, we will simply “voluntary” omit one of the JAR files from the classpath that contains the referencing Java class ReferencingClassA. The Java program is packaged as per below: - MainProgram.jar (contains NoClassDefFoundErrorSimulator.class and JavaEETrainingUtil.class) - CallerClassA.jar (contains CallerClassA.class) - ReferencingClassA.jar (contains ReferencingClassA.class) Now, let’s run the program as is: ## Baseline (normal execution) .\bin>java -classpath CallerClassA.jar;ReferencingClassA.jar;MainProgram.jar org.ph.javaee.training1.NoClassDefFoundErrorSimulator java.lang.NoClassDefFoundError Simulator - Training 1 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com Current ClassLoader chain: ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Creating a new instance of org.ph.javaee.training1.CallerClassA... Classloading of org.ph.javaee.training1.ReferencingClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Creating a new instance of org.ph.javaee.training1.ReferencingClassA... done! For the initial run (baseline), the main program was able to create a new instance of CallerClassA and execute its method successfully; including successful class loading of the referencing class ReferencingClassA. ## Problem reproduction run (with removal of ReferencingClassA.jar) ../bin>java -classpath CallerClassA.jar;MainProgram.jar org.ph.javaee.training1.NoClassDefFoundErrorSimulator java.lang.NoClassDefFoundError Simulator - Training 1 Author: Pierre-Hugues Charbonneau http://javaeesupportpatterns.blogspot.com Current ClassLoader chain: ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- Creating a new instance of org.ph.javaee.training1.CallerClassA... Exception in thread "main" java.lang.NoClassDefFoundError: org/ph/javaee/training1/ReferencingClassA at org.ph.javaee.training1.CallerClassA.doSomething(CallerClassA.java:25) at org.ph.javaee.training1.NoClassDefFoundErrorSimulator.main(NoClassDefFoundErrorSimulator.java:28) Caused by: java.lang.ClassNotFoundException: org.ph.javaee.training1.ReferencingClassA at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 2 more What happened? The removal of the ReferencingClassA.jar, containing ReferencingClassA, did prevent the current class loader to locate this referencing Java class at runtime leading to ClassNotFoundException and NoClassDefFoundError. This is the typical Exception that you will get if you omit JAR file(s) from your Java start-up classpath or within an EAR / WAR for Java EE related applications. ClassLoader view Now let’s review the ClassLoader chain so you can properly understand this problem case. As you saw from the Java program output logging, the following Java ClassLoaders were found: Classloading of org.ph.javaee.training1.CallerClassA in progress... ----------------------------------------------------------------- sun.misc.Launcher$ExtClassLoader@17c1e333 --- delegation --- sun.misc.Launcher$AppClassLoader@214c4ac9 **Current ClassLoader** ----------------------------------------------------------------- ** Please note that the Java bootstrap class loader is responsible to load the core JDK classes and is written in native code ** ## sun.misc.Launcher$AppClassLoader This is the system class loader responsible to load our application code found from the Java classpath specified at start-up. ##sun.misc.Launcher$ExtClassLoader This is the extension class loader responsible to load code in the extensions directories (/lib/ext, or any other directory specified by the java.ext.dirs system property). As you can see from the Java program logging output, the extension class loader is the actual super parent of the system class loader. Our sample Java program was loaded at the system class loader level. Please note that this class loader chain is very simple for this problem case since we did not create child class loaders at this point. This will be covered in future articles. Recommendations and resolution strategies Now find below my recommendations and resolution strategies for NoClassDefFoundError problem case 1: - Review the java.lang.NoClassDefFoundError error and identify the missing Java class - Verify and locate the missing Java class from your compile / build environment - Determine if the missing Java class is from your application code, third part API or even the Java EE container itself. Verify where the missing JAR file(s) is / are expected to be found - Once found, verify your runtime environment Java classpath for any typo or missing JAR file(s) - If the problem is triggered from a Java EE application, perform the same above steps but verify the packaging of your EAR / WAR file for missing JAR and other library file dependencies such as MANIFEST Please feel free to post any question or comment. The part 3 will be available shortly.

There are several code analysis tools for Python. The most well known is pylint. Then there’s pychecker and now we’re moving on to pyflakes. The pyflakes project is a part of something known as the Divmod Project. Pyflakes doesn’t actually execute the code it checks, unlike pychecker. Of course, pylint also doesn’t execute the code. Regardless, we’ll take a quick look at it and see how pyflakes works and if it’s better than the competition. Getting Started As you have probably guessed, pyflakes is not a part of the Python distribution. You will need to download it from PyPI or from the project’s launchpad page. Once you have it installed, you can run it against some of your own code. Or you can follow along and see how it works with our test script. Running pyflakes We’ll be using a super simple and pretty silly example script. In fact, it’s the same one we used for the pylint and pychecker articles. Here it is again for your viewing pleasure: import sys ######################################################################## class CarClass: """""" #---------------------------------------------------------------------- def __init__(self, color, make, model, year): """Constructor""" self.color = color self.make = make self.model = model self.year = year if "Windows" in platform.platform(): print "You're using Windows!" self.weight = self.getWeight(1, 2, 3) #---------------------------------------------------------------------- def getWeight(this): """""" return "2000 lbs" As was noted in the other articles, this dumb code has 4 issues, 3 of which would stop the programming from running. Let’s see what pyflakes can find! Try running the following command and you’ll see the following output: C:\Users\mdriscoll\Desktop>pyflakes crummy_code.py crummy_code.py:1: 'sys' imported but unused crummy_code.py:15: undefined name 'platform' While pyflakes was super fast at returning this output, it didn’t find all the errors. The getWeight method call is passing too many arguments and getWeight method itself is defined incorrectly as it doesn’t have a “self” argument. If you fixed your code according to what pyflakes told you, you’re code still wouldn’t work. Wrapping Up The pyflakes website claims that pyflakes is faster than pychecker and pylint. I didn’t test this, but anyone who wants to can do so pretty easily by just running it against some big files. Maybe grab the BeautifulSoup file or run it (and the others) against something complex like PySide or SQLAlchemy and see how they compare. I personally am disappointed that it didn’t catch all the issues I was looking for. I think for my purposes, I’ll be sticking with pylint. This might be a handy tool for a quick and dirty test or just to make you feel better after a particularly poor result from a pylint scan.

Here are the 10 important differences between WCF Services and ASP.NET Web Services.

Let’s assume that you’re writing code that’d track the price of mobile phones. Now, let’s say you have a collection of objects representing different Mobile phone vendors (MobileVendor), and each vendor has a collection of objects representing the PhoneModels they offer. To put it simple, there’s exists a one-to-many relationship between MobileVendor:PhoneModel. MobileVendor Class Class MobileVendor{ long vendor_id; PhoneModel[] phoneModels; ... } Okay, so you want to print out all the details of phone models. A naive O/R implementation would SELECT all mobile vendors and then do N additional SELECTs for getting the information of PhoneModel for each vendor. -- Get all Mobile Vendors SELECT * FROM MobileVendor; -- For each MobileVendor, get PhoneModel details SELECT * FROM PhoneModel WHERE MobileVendor.vendorId=? As you see, the N+1 problem can happen if the first query populates the primary object and the second query populates all the child objects for each of the unique primary objects returned. Resolve N+1 SELECTs problem (i) HQL fetch join "from MobileVendor mobileVendor join fetch mobileVendor.phoneModel PhoneModels" Corresponding SQL would be (assuming tables as follows: t_mobile_vendor for MobileVendor and t_phone_model for PhoneModel) SELECT * FROM t_mobile_vendor vendor LEFT OUTER JOIN t_phone_model model ON model.vendor_id=vendor.vendor_id (ii) Criteria query Criteria criteria = session.createCriteria(MobileVendor.class); criteria.setFetchMode("phoneModels", FetchMode.EAGER); In both cases, our query returns a list of MobileVendor objects with the phoneModels initialized. Only one query needs to be run to return all the PhoneModel and MobileVendor information required.

Today we’ll spend some time looking at three different ways to make Python submit a web form. In this case, we will be doing a web search with duckduckgo.com searching on the term “python” and saving the result as an HTML file. We will use Python’s included urllib modules and two 3rd party packages: requests and mechanize. We have three small scripts to cover, so let’s get cracking! Submitting a web form with urllib We will start with urllib and urllib2 since they are included in Python’s standard library. We’ll also import the webbrowser to open the search results for viewing. Here’s the code: import urllib import urllib2 import webbrowser url = "http://duckduckgo.com/html" data = urllib.urlencode({'q': 'Python'}) results = urllib2.urlopen(url, data) with open("results.html", "w") as f: f.write(results.read()) webbrowser.open("results.html") The first thing you have to do when you want to submit a web form is figure out what the form is called and what the url is that you will be posting to. If you go to duckduckgo’s website and view the source, you’ll notice that its action is pointing to a relative link, “/html”. So our url is “http://duckduckgo.com/html”. The input field is named “q”, so to pass duckduckgo a search term, we have to pass it to the “q” field. This is where the urllib.urlencode line comes in. It encodes our search term correctly and then we open the url and search. The results are read and written to disk. Finally, we open our saved results using the webbrowser module. Now let’s find out how this process differs when using the requests package. Submitting a web form with requests The requests package does form submissions a little bit more elegantly. Let’s take a look: import requests url = "http://duckduckgo.com/html" payload = {'q':'python'} r = requests.post(url, payload) with open("requests_results.html", "w") as f: f.write(r.content) With requests, you just need to create a dictionary with the field name as the key and the search term as the value. Then you use requests.post to do the search. Finally you use the resulting requests object, “r”, and access its content property which you save to disk. We skipped the webbrowser part in this example (and the next) for brevity. Now we should be ready to see how mechanize does its thing. Submitting a web form with mechanize The mechanize module has lots of fun features for browsing the internet with Python. Sadly it doesn’t support javascript. Anyway, let’s get on with the show! import mechanize url = "http://duckduckgo.com/html" br = mechanize.Browser() br.set_handle_robots(False) # ignore robots br.open(url) br.select_form(name="x") br["q"] = "python" res = br.submit() content = res.read() with open("mechanize_results.html", "w") as f: f.write(content) As you can see, mechanize is a little more verbose than the other two methods were. We also need to tell it to ignore the robots.txt directive or it will fail. Of course, if you want to be a good netizen, then you shouldn’t ignore it. Anyway, to start off, you need a Browser object. Then you open the url, select the form (in this case, “x”) and set up a dictionary with the search parameters as before. Note that in each method, the dict setup is a little different. Next you submit the query and read the result. Finally you save the result to disk and you’re done! Wrapping Up Of the three, requests was probably the simplest with urllib being a close second. Mechanize is made for doing a lot more then the other two though. It’s made for screen scraping and website testing, so it’s no surprise it’s a little more verbose. You can also do form submission with selenium, but you can read about that in this blog’s archives. I hope you found this article interesting and perhaps inspiring. See you next time!

One of the advantages of code generation is the ability to see how a specific language feature or framework is used. As I discussed in the post NetBeans 7.2 beta: Faster and More Helpful, NetBeans 7.2 beta provides TestNG integration. I did not elaborate further in that post other than a single reference to that feature because I wanted to devote this post to the subject. I use this post to demonstrate how NetBeans 7.2 can be used to help a developer new to TestNG start using this alternative (to JUnit) test framework. NetBeans 7.2's New File wizard makes it easier to create an empty TestNG test case. This is demonstrated in the following screen snapshots that are kicked off by using New File | Unit Tests (note that "New File" is available under the "File" drop-down menu or by right-clicking in the Projects window). Running the TestNG test case creation as shown above leads to the following generated test code. TestNGDemo.java (Generated by NetBeans 7.2) package dustin.examples; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeClass; import org.testng.annotations.Test; import org.testng.Assert; /** * * @author Dustin */ public class TestNGDemo { public TestNGDemo() { } @BeforeClass public void setUpClass() { } @AfterClass public void tearDownClass() { } @BeforeMethod public void setUp() { } @AfterMethod public void tearDown() { } // TODO add test methods here. // The methods must be annotated with annotation @Test. For example: // // @Test // public void hello() {} } The test generated by NetBeans 7.2 includes comments indicate how test methods are added and annotated (similar to modern versions of JUnit). The generated code also shows some annotations for overall test case set up and tear down and for per-test set up and tear down (annotations are similar to JUnit's). NetBeans identifies import statements that are not yet used at this point (import org.testng.annotations.Test; and import org.testng.Assert;), but are likely to be used and so have been included in the generated code. I can add a test method easily to this generated test case. The following code snippet is a test method using TestNG. testIntegerArithmeticMultiplyIntegers() @Test public void testIntegerArithmeticMultiplyIntegers() { final IntegerArithmetic instance = new IntegerArithmetic(); final int[] integers = {4, 5, 6}; final int expectedProduct = 2 * 3 * 4 * 5 * 6; final int product = instance.multiplyIntegers(2, 3, integers); assertEquals(product, expectedProduct); } This, of course, looks very similar to the JUnit equivalent I used against the same IntegerArithmetic class that I used for testing illustrations in the posts Improving On assertEquals with JUnit and Hamcrest and JUnit's Built-in Hamcrest Core Matcher Support. The following screen snapshot shows the output in NetBeans 7.2 beta from right-clicking on the test case class and selecting "Run File" (Shift+F6). The text output of the TestNG run provided in the NetBeans 7.2 beta is reproduced next. [TestNG] Running: Command line suite [VerboseTestNG] RUNNING: Suite: "Command line test" containing "1" Tests (config: null) [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @BeforeClass dustin.examples.TestNGDemo.setUpClass() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @BeforeClass dustin.examples.TestNGDemo.setUpClass() finished in 33 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @BeforeMethod dustin.examples.TestNGDemo.setUp() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @BeforeMethod dustin.examples.TestNGDemo.setUp() finished in 2 ms [VerboseTestNG] INVOKING: "Command line test" - dustin.examples.TestNGDemo.testIntegerArithmeticMultiplyIntegers() [VerboseTestNG] PASSED: "Command line test" - dustin.examples.TestNGDemo.testIntegerArithmeticMultiplyIntegers() finished in 12 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @AfterMethod dustin.examples.TestNGDemo.tearDown() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @AfterMethod dustin.examples.TestNGDemo.tearDown() finished in 1 ms [VerboseTestNG] INVOKING CONFIGURATION: "Command line test" - @AfterClass dustin.examples.TestNGDemo.tearDownClass() [VerboseTestNG] PASSED CONFIGURATION: "Command line test" - @AfterClass dustin.examples.TestNGDemo.tearDownClass() finished in 1 ms [VerboseTestNG] [VerboseTestNG] =============================================== [VerboseTestNG] Command line test [VerboseTestNG] Tests run: 1, Failures: 0, Skips: 0 [VerboseTestNG] =============================================== =============================================== Command line suite Total tests run: 1, Failures: 0, Skips: 0 =============================================== Deleting directory C:\Users\Dustin\AppData\Local\Temp\dustin.examples.TestNGDemo test: BUILD SUCCESSFUL (total time: 2 seconds) The above example shows how easy it is to start using TestNG, especially if one is moving to TestNG from JUnit and is using NetBeans 7.2 beta. Of course, there is much more to TestNG than this, but learning a new framework is typically most difficult at the very beginning and NetBeans 7.2 gets one off to a fast start.

A couple years ago I started a series of articles on XML parsing. I covered lxml’s etree and Python builtin minidom XML parsing library. For whatever reason I didn’t notice lxml’s objectify sub-package, but I saw it recently and decided to check it out. In my mind, the objectify module seems to be even more “Pythonic” than etree is. Let’s take some time and go over my old XML examples using objectify and see how it’s different! Let’s Get This Party Started! If you haven’t already, go out and download lxml or you won’t be able to follow along very well. Once you have it, we can continue. We’ll be using the following piece of XML for our parsing pleasure: 1181251680 040000008200E000 1181572063 1800 Bring pizza home 1234360800 1800 Check MS Office website for updates 604f4792-eb89-478b-a14f-dd34d3cc6c21-1234360800 dismissed Now we need to write some code that can parse and modify the XML. Let’s take a look at this little demo that shows a bunch of the neat abilities that objectify provides. from lxml import etree, objectify #---------------------------------------------------------------------- def parseXML(xmlFile): """""" with open(xmlFile) as f: xml = f.read() root = objectify.fromstring(xml) # returns attributes in element node as dict attrib = root.attrib # how to extract element data begin = root.appointment.begin uid = root.appointment.uid # loop over elements and print their tags and text for e in root.appointment.iterchildren(): print "%s => %s" % (e.tag, e.text) # how to change an element's text root.appointment.begin = "something else" print root.appointment.begin # how to add a new element root.appointment.new_element = "new data" # print the xml obj_xml = etree.tostring(root, pretty_print=True) print obj_xml # remove the py:pytype stuff #objectify.deannotate(root) etree.cleanup_namespaces(root) obj_xml = etree.tostring(root, pretty_print=True) print obj_xml # save your xml with open("new.xml", "w") as f: f.write(obj_xml) #---------------------------------------------------------------------- if __name__ == "__main__": f = r'path\to\sample.xml' parseXML(f) The code is pretty well commented, but we’ll spend a little time going over it anyway. First we pass it our sample XML file and objectify it. If you want to get access to a tag’s attributes, use the attrib property. It will return a dictionary of the attribute’s of the tag. To get to sub-tag elements, you just use dot notation. As you can see, to get to the begin tag’s value, we can just do something like this: begin = root.appointment.begin If you need to iterate over the children elements, you can use iterchildren. You may have to use a nested for loop structure to get everything. Changing an element’s value is as simple as just assigning it a new value. And if you need to create a new element, just add a period and the name of the new element (see below): root.appointment.new_element = "new data" When we add or change items using objectify, it will add some annotations to the XML, such as xmlns:py="http://codespeak.net/lxml/objectify/pytype" py:pytype="str". You may not want that included, so you'll have to call the following method to remove that stuff: [python] etree.cleanup_namespaces(root) You can also use “objectify.deannotate(root)” to do some deannotation chores, but I wasn’t able to get it to work for this example. To save the new XML, you actually seem to need lxml’s etree module to convert it to a string so you can save it. At this point, you should be able to parse most XML documents and edit them effectively with lxml’s objectify. I thought it was very intuitive and easy to pick up. Hopefully you will find it useful in your endeavors as well.

I often use IntelliJ to run applications, and on occasion things go wrong. For example, a thread that wont terminate can cause a running application to become unstoppable via the IntelliJ UI. Usually when this happens I end up running ps aux | grep java and following up with a kill -9 for each process that looks like it might be the one I'm looking for. On good days there's only a few processes; however, things are more complicated when I have several to look through. Last week I noticed that the command used to launch the process printed in the Console window, and, more importantly, the idea.launcher.port is part of that command: e.g. idea.launcher.port=7538. Assuming the port is unique, or even almost unique it's much easier to ps aux | grep for than java.

I.T. is full of complex things that should (and sometimes could) be simple. Getting the JQPL/SQL String representation for a JPA 2.0 CriteriaQuery is one of them. By now you all know the JPA 2.0 Criteria API : a type safe way to write a JQPL query. This API is clever in the way that you don’t use Strings to build your query, but is quite verbose… and sometimes you get lost in dozens of lines of Java code, just to write a simple query. You get lost in your CriteriaQuery, you don’t know why your query doesn’t work, and you would love to debug it. But how do you debug it ? Well, one way would be by just displaying the JPQL and/or SQL representation. Simple, isn’t it ? Yes, but JPA 2.0 javax.persistence.Query doesn’t have an API to do this. You then need to rely on the implementation… meaning, the code is different if you use EclipseLink, Hibernate or OpenJPA. The CriteriaQuery we want to debug Let’s say you have a simple Book entity and you want to retrieve all the books sorted by their id. Something like SELECT b FROM Book b ORDER BY b.id DESC. How would you write this with the CriteriaQuery ? Well, something like these 5 lines of Java code : CriteriaBuilder cb = em.getCriteriaBuilder(); CriteriaQuery q = cb.createQuery(Book.class); Root b = q.from(Book.class); q.select(b).orderBy(cb.desc(b.get("id"))); TypedQuery findAllBooks = em.createQuery(q); So imagine when you have more complex ones. Sometimes, you just get lost, it gets buggy and you would appreciate to have the JPQL and/or SQL String representation to find out what’s happening. You could then even unit test it. Getting the JPQL/SQL String Representations for a Criteria Query So let’s use an API to get the JPQL/SQL String representations of a CriteriaQuery (to be more precise, the TypedQuery created from a CriteriaQuery). The bad news is that there is no standard JPA 2.0 API to do this. You need to use the implementation API hoping the implementation allows it (thank god that’s (nearly) the case for the 3 main JPA ORM frameworks). The good news is that the Query interface (and therefore TypedQuery) has an unwrap method. This method returns the provider’s query API implementation. Let’s see how you can use it with EclipseLink, Hibernate and OpenJPA. EclipseLink EclipseLink‘s Query representation is the org.eclipse.persistence.jpa.JpaQuery interface and the org.eclipse.persistence.internal.jpa.EJBQueryImpl implementation. This interface gives you the wrapped native query (org.eclipse.persistence.queries.DatabaseQuery) with two very handy methods : getJPQLString() and getSQLString(). Unfortunatelly the getJPQLString() method will not translate a CriteriaQuery into JPQL, it only works for queries originally written in JPQL (dynamic or named query). The getSQLString() method relies on the query being “prepared”, meaning you have to run the query once before getting the SQL String representation. findAllBooks.unwrap(JpaQuery.class).getDatabaseQuery().getJPQLString(); // doesn't work for CriteriaQuery findAllBooks.unwrap(JpaQuery.class).getDatabaseQuery().getSQLString(); Hibernate Hibernate‘s Query representation is org.hibernate.Query. This interface has several implementations and the very useful method that returns the SQL query string : getQueryString(). I couldn’t find a method that returns the JPQL representation, if I’ve missed something, please let me know. findAllBooks.unwrap(org.hibernate.Query.class).getQueryString() OpenJPA OpenJPA‘s Query representation is org.apache.openjpa.persistence.QueryImpl and also has a getQueryString() method that returns the SQL (not the JPQL). It delegates the call to the internal org.apache.openjpa.kernel.Query interface. I couldn’t find a method that returns the JPQL representation, if I’ve missed something, please let me know. findAllBooks.unwrap(org.apache.openjpa.persistence.QueryImpl.class).getQueryString() Unit testing Once you get your SQL String, why not unit test it ? Hey, but I don’t want to test my ORM, why would I do that ? Well, it happens that I’ve discovered a but in the new releases of OpenJPA by unit testing a query… so, there is a use case for that. Anyway, this is how you could do it : assertEquals("SELECT b FROM Book b ORDER BY b.id DESC", findAllBooksCriteriaQuery.unwrap(org.apache.openjpa.persistence.QueryImpl.class).getQueryString()); Conclusion As you can see, it’s not that simple to get a String representation for a TypedQuery. Here is a digest of the three main ORMs : ORM Framework Query implementation How to get the JPQL String How to get the SPQL String EclipseLink JpaQuery getDatabaseQuery().getJPQLString()* getDatabaseQuery().getSQLString()** Hibernate Query N/A getQueryString() OpenJPA QueryImpl getQueryString() N/A (*) Only possible on a dynamic or named query. Not possible on a CriteriaQuery (**) You need to execute the query first, if not, the value is null To illustrate all that I’ve written simple test cases using EclipseLink, Hibernate and OpenJPA that you can download from GitHub. Give it a try and let me know. And what about having an API in JPA 2.1 ? For a developers’ point of view it would be great to have two methods in the javax.persistence.Query (and therefore javax.persistence.TypedQuery) interface that would be able to easily return the JPQL and SQL String representations, e.g : Query.getJPQLString() and Query.getSQLString(). Hey, that would be the perfect time to have it in JPA 2.1 that will be shipped in less than a year. Now, as an implementer, this might be tricky to do, I would love to ear your point of view on this. Anyway, I’m going to post an email to the JPA 2.1 Expert Group… just in case we can have this in the next version of JPA ;o) References http://efreedom.com/Question/1-6412774/Get-SQL-String-JPQLQuery http://old.nabble.com/Cannot-get-the-JPQL—SQL-String-of-a-CriteriaQuery-td33882629.html http://paddyweblog.blogspot.fr/2010/04/some-examples-of-criteria-api-jpa-20.html http://www.altuure.com/2010/09/23/jpa-criteria-api-by-samples-part-i/ http://www.altuure.com/2010/09/23/jpa-criteria-api-by-samples-%E2%80%93-part-ii/ http://www.jumpingbean.co.za/blogs/jpa2-criteria-api http://wiki.eclipse.org/EclipseLink/FAQ/JPA#How_to_get_the_SQL_for_a_Query.3F

In how many ways can you retrieve TeamCity artifacts? I say plenty to choose from! If you’re in a world of Java build tools then you can use plain HTTP request, Ant + Ivy, Gradle and Maven to download and use binaries produced by TeamCity build configurations. How? Read on. Build Configuration “id” Before you retrieve artifacts of any build configuration you need to know its "id" which can be seen in a browser when corresponding configuration is browsed. Let’s take IntelliJ IDEA Community Edition project hosted at teamcity.jetbrains.com as an example. Its “Community Dist” build configuration provides a number of artifacts which we’re going to play with. And as can be seen on the screenshot below, its "id" is "bt343". HTTP Anonymous HTTP access is probably the easiest way to fetch TeamCity artifacts, the URL to do so is: http://server/guestAuth/repository/download/// Fot this request to work 3 parameters need to be specified: btN Build configuration "id", as mentioned above. buildNumber Build number or one of predefined constants: "lastSuccessful", "lastPinned", or "lastFinished". For example, you can download periodic IDEA builds from last successful TeamCity execution. artifactName Name of artifact like "ideaIC-118.SNAPSHOT.win.zip". Can also take a form of "artifactName!archivePath" for reading archive’s content, like IDEA’s build file. You can get a list of all artifacts produced in a certain build by requesting a special "teamcity-ivy.xml" artifact generated by TeamCity. Ant + Ivy All artifacts published to TeamCity are accompanied by "teamcity-ivy.xml" Ivy descriptor, effectively making TeamCity an Ivy repository. The code below downloads "core/annotations.jar" from IDEA distribution to "download/ivy" directory: "ivyconf.xml" "ivy.xml" "build.xml" Gradle Identically to Ivy example above it is fairly easy to retrieve TeamCity artifacts with Gradle due to its built-in Ivy support. In addition to downloading the same jar file to "download/gradle" directory with a custom Gradle task let’s use it as "compile" dependency for our Java class, importing IDEA’s @NotNull annotation: "Test.java" import org.jetbrains.annotations.NotNull; public class Test { private final String data; public Test ( @NotNull String data ){ this.data = data; } } "build.gradle" apply plugin: 'java' repositories { ivy { ivyPattern 'http://teamcity.jetbrains.com/guestAuth/repository/download/[module]/[revision]/teamcity-ivy.xml' artifactPattern 'http://teamcity.jetbrains.com/guestAuth/repository/download/[module]/[revision]/[artifact](.[ext])' } } dependencies { compile ( 'org:bt343:lastSuccessful' ){ artifact { name = 'core/annotations' type = 'jar' } } } task copyJar( type: Copy ) { from configurations.compile into "${ project.projectDir }/download/gradle" } Maven The best way to use Maven with TeamCity is by setting up an Artifactory repository manager and its TeamCity plugin. This way artifacts produced by your builds are nicely deployed to Artifactory and can be served from there as from any other remote Maven repository. However, you can still use TeamCity artifacts in Maven without any additional setups. "ivy-maven-plugin" bridges two worlds allowing you to plug Ivy resolvers into Maven’s runtime environment, download dependencies required and add them to corresponding "compile" or "test" scopes. Let’s compile the same Java source from the Gradle example but using Maven this time. "pom.xml" 4.0.0 com.test maven jar 0.1-SNAPSHOT [${project.groupId}:${project.artifactId}:${project.version}] Ivy Maven plugin example com.github.goldin ivy-maven-plugin 0.2.5 get-ivy-artifacts ivy initialize ${project.basedir}/ivyconf.xml ${project.basedir}/ivy.xml ${project.basedir}/download/maven compile When this plugin runs it resolves IDEA annotations artifact using the same "ivyconf.xml" and "ivy.xml" files we’ve seen previously, copies it to "download/maven" directory and adds to "compile" scope so our Java sources can compile. GitHub Project All examples demonstrated are available in my GitHub project. Feel free to clone and run it: git clone git://github.com/evgeny-goldin/teamcity-download-examples.git cd teamcity-download-examples chmod +x run.sh dist/ant/bin/ant gradlew dist/maven/bin/mvn ./run.sh Resources The links below can provide you with more details: TeamCity – Patterns For Accessing Build Artifacts TeamCity – Accessing Server by HTTP TeamCity – Configuring Artifact Dependencies Using Ant Build Script Gradle – Ivy repositories "ivy-maven-plugin" That’s it, you’ve seen it – TeamCity artifacts are perfectly accessible using either of 4 ways: direct HTTP access, Ant + Ivy, Gradle or Maven. Which one do you use? Let me know!

I really like Spring, so I tend to use its features to the fullest. However, in some dark corners of its philosophy, I tend to disagree with some of its assumptions. One such assumption is the way database testing should work. In this article, I will explain how to configure your projects to make Spring Test and DBUnit play nice together in a multi-developers environment. Context My basic need is to be able to test some complex queries: before integration tests, I've to validate those queries get me the right results. These are not unit tests per se but let's assilimate them as such. In order to achieve this, I use since a while a framework named DBUnit. Although not maintained since late 2010, I haven't found yet a replacement (be my guest for proposals). I also have some constraints: I want to use TestNG for all my test classes, so that new developers wouldn't think about which test framework to use I want to be able to use Spring Test, so that I can inject my test dependencies directly into the test class I want to be able to see for myself the database state at the end of any of my test, so that if something goes wrong, I can execute my own queries to discover why I want every developer to have its own isolated database instance/schema Considering the last point, our organization let us benefit from a single Oracle schema per developer for those "unit-tests". Basic set up Spring provides the AbstractTestNGSpringContextTests class out-of-the-box. In turn, this means we can apply TestNG annotations as well as @Autowired on children classes. It also means we have access to the underlying applicationContext, but I prefer not to (and don't need to in any case). The structure of such a test would look like this: @ContextConfiguration(location = "classpath:persistence-beans.xml") public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Test public void whenXYZThenTUV() { ... } } Readers familiar with Spring and TestNG shouldn't be surprised here. Bringing in DBunit DbUnit is a JUnit extension targeted at database-driven projects that, among other things, puts your database into a known state between test runs. [...] DbUnit has the ability to export and import your database data to and from XML datasets. Since version 2.0, DbUnit can also work with very large datasets when used in streaming mode. DbUnit can also help you to verify that your database data match an expected set of values. DBunit being a JUnit extension, it's expected to extend the provided parent class org.dbunit.DBTestCase. In my context, I have to redefine some setup and teardown operation to use Spring inheritance hierarchy. Luckily, DBUnit developers thought about that and offer relevant documentation. Among the different strategies available, my tastes tend toward the CLEAN_INSERT and NONE operations respectively on setup and teardown. This way, I can check the database state directly if my test fails. This updates my test class like so: @ContextConfiguration(locations = {"classpath:persistence-beans.xml", "classpath:test-beans.xml"}) public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Autowired private IDatabaseTester databaseTester; @BeforeMethod protected void setUp() throws Exception { // Get the XML and set it on the databaseTester // Optional: get the DTD and set it on the databaseTester databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT); databaseTester.setTearDownOperation(DatabaseOperation.NONE); databaseTester.onSetup(); } @Test public void whenXYZThenTUV() { ... } } Per-user configuration with Spring Of course, we need to have a specific Spring configuration file to inject the databaseTester. As an example, here is one: However, there's more than meets the eye. Notice the databaseTester has to be fed a datasource. Since a requirement is to have a database per developer, there are basically two options: either use a in-memory database or use the same database as in production and provide one such database schema per developer. I tend toward the latter solution (when possible) since it tends to decrease differences between the testing environment and the production environment. Thus, in order for each developer to use its own schema, I use Spring's ability to replace Java system properties at runtime: each developer is characterized by a different user.name. Then, I configure a PlaceholderConfigurer that looks for {user.name}.database.properties file, that will look like so: db.username=myusername1 db.password=mypassword1 db.schema=myschema1 This let me achieve my goal of each developer using its own instance of Oracle. If you want to use this strategy, do not forget to provide a specific database.properties for the Continuous Integration server. Huh oh? Finally, the whole testing chain is configured up to the database tier. Yet, when the previous test is run, everything is fine (or not), but when checking the database, it looks untouched. Strangely enough, if you did load some XML dataset and assert it during the test, it does behaves accordingly: this bears all symptoms of a transaction issue. In fact, when you closely look at Spring's documentation, everything becomes clear. Spring's vision is that the database should be left untouched by running tests, in complete contradiction to DBUnit's. It's achieved by simply rollbacking all changes at the end of the test by default. In order to change this behavior, the only thing to do is annotate the test class with @TransactionConfiguration(defaultRollback=false). Note this doesn't prevent us from specifying specific methods that shouldn't affect the database state on a case-by-case basis with the @Rollback annotation. The test class becomes: @ContextConfiguration(locations = {classpath:persistence-beans.xml", "classpath:test-beans.xml"}) @TransactionConfiguration(defaultRollback=false) public class MyDaoTest extends AbstractTestNGSpringContextTests { @Autowired private MyDao myDao; @Autowired private IDatabaseTester databaseTester; @BeforeMethod protected void setUp() throws Exception { // Get the XML and set it on the databaseTester // Optional: get the DTD and set it on the databaseTester databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT); databaseTester.setTearDownOperation(DatabaseOperation.NONE); databaseTester.onSetup(); } @Test public void whenXYZThenTUV() { ... } } Conclusion Though Spring and DBUnit views on database testing are opposed, Spring's configuration versatility let us make it fit our needs (and benefits from DI). Of course, other improvements are possible: pushing up common code in a parent test class, etc. To go further: Spring Test documentation DBUnit site Database data verification Database testing best practices Generating DTD from your database schema