It’s a common pattern for command line tools to have multiple subcommands that run off of a single executable. For example, git fetch origin and git commit --amend both use the same executable /usr/bin/git to run. Each subcommand has its own set of required and optional parameters. This pattern is fairly easy to implement in your own Python command-line utilities using argparse. Here is a script that pretends to be git and provides the above two commands and arguments. #!/usr/bin/env python import argparse import sys class FakeGit(object): def __init__(self): parser = argparse.ArgumentParser( description='Pretends to be git', usage='''git [] The most commonly used git commands are: commit Record changes to the repository fetch Download objects and refs from another repository ''') parser.add_argument('command', help='Subcommand to run') # parse_args defaults to [1:] for args, but you need to # exclude the rest of the args too, or validation will fail args = parser.parse_args(sys.argv[1:2]) if not hasattr(self, args.command): print 'Unrecognized command' parser.print_help() exit(1) # use dispatch pattern to invoke method with same name getattr(self, args.command)() def commit(self): parser = argparse.ArgumentParser( description='Record changes to the repository') # prefixing the argument with -- means it's optional parser.add_argument('--amend', action='store_true') # now that we're inside a subcommand, ignore the first # TWO argvs, ie the command (git) and the subcommand (commit) args = parser.parse_args(sys.argv[2:]) print 'Running git commit, amend=%s' % args.amend def fetch(self): parser = argparse.ArgumentParser( description='Download objects and refs from another repository') # NOT prefixing the argument with -- means it's not optional parser.add_argument('repository') args = parser.parse_args(sys.argv[2:]) print 'Running git fetch, repository=%s' % args.repository if __name__ == '__main__': FakeGit() The argparse library gives you all kinds of great stuff. You can run ./git.py --help and get the following: usage: git [] The most commonly used git commands are: commit Record changes to the repository fetch Download objects and refs from another repository Pretends to be git positional arguments: command Subcommand to run optional arguments: -h, --help show this help message and exit You can get help on a particular subcommand with ./git.py commit --help. usage: git.py [-h] [--amend] Record changes to the repository optional arguments: -h, --help show this help message and exit --amend Want bash completion on your awesome new command line utlity? Try argcomplete, a drop in bash completion for Python + argparse.

I’ve just started looking at Docker. It’s a cool new technology that has the potential to make the management and deployment of distributed applications a great deal easier. I’d very much recommend checking it out. I’m especially interested in using it to deploy Mono applications because it promises to remove the hassle of deploying and maintaining the mono runtime on a multitude of Linux servers. I’ve been playing around creating new images and containers and debugging my Dockerfile, and I’ve wound up with lots of temporary containers and images. It’s really tedious repeatedly running ‘docker rm’ and ‘docker rmi’, so I’ve knocked up a couple of bash commands to bulk delete images and containers. Delete all containers: sudo docker ps -a -q | xargs -n 1 -I {} sudo docker rm {} Delete all un-tagged (or intermediate) images: sudo docker rmi $( sudo docker images | grep '' | tr -s ' ' | cut -d ' ' -f 3)

There is actually nothing very special about writing a Spring-boot web application purely using Scala, it just works! In this blog entry, I will slowly transform a Java based Spring-boot application completely to Scala - the Java based sample is available at this github location - https://github.com/bijukunjummen/spring-boot-mvc-test To start with, I had the option of going with either a maven based build or gradle based build - I opted to go with a gradle based build as gradle has a greatscala plugin, so for scala support the only changes to a build.gradle build script is the following: ... apply plugin: 'scala' ... jar { baseName = 'spring-boot-scala-web' version = '0.1.0' } dependencies { ... compile 'org.scala-lang:scala-library:2.10.2' ... } Essentially adding in the scala plugin and specifying the version of the scala-library. Now, I have one entity, a Hotel class, it transforms to the following with Scala: package mvctest.domain .... @Entity class Hotel { @Id @GeneratedValue @BeanProperty var id: Long = _ @BeanProperty var name: String = _ @BeanProperty var address: String = _ @BeanProperty var zip: String = _ } Every property is annotated with @BeanProperty annotation to instruct scala to generate the Java bean based getter and setter on the variables. With the entity in place a Spring-data repository for CRUD operations on this entity transforms from: import mvctest.domain.Hotel; import org.springframework.data.repository.CrudRepository; public interface HotelRepository extends CrudRepository { } to the following in Scala: import org.springframework.data.repository.CrudRepository import mvctest.domain.Hotel import java.lang.Long trait HotelRepository extends CrudRepository[Hotel, Long] And the Scala based controller which uses this repository to list the Hotels - vi... import org.springframework.web.bind.annotation.RequestMapping import org.springframework.stereotype.Controller import mvctest.service.HotelRepository import org.springframework.beans.factory.annotation.Autowired import org.springframework.ui.Model @Controller @RequestMapping(Array("/hotels")) class HotelController @Autowired() (private val hotelRepository: HotelRepository) { @RequestMapping(Array("/list")) def list(model: Model) = { val hotels = hotelRepository.findAll() model.addAttribute("hotels", hotels) "hotels/list" } } Here the constructor autowiring of the HotelRepository just works!. Do note the slightly awkward way of specifying the @Autowired annotation for constructor based injection. Finally, Spring-boot based application requires a main class to bootstrap the entire application, where this bootstrap class looks like this with Java: @Configuration @EnableAutoConfiguration @ComponentScan public class SampleWebApplication { public static void main(String[] args) { SpringApplication.run(SampleWebApplication.class, args); } } In scala, though I needed to provide two classes, one to specify the annotation and other to bootstrap the application, there may be better way to do this(blame it on my lack of Scala depth!) - package mvctest import org.springframework.context.annotation.Configuration import org.springframework.boot.autoconfigure.EnableAutoConfiguration import org.springframework.context.annotation.ComponentScan import org.springframework.boot.SpringApplication @Configuration @EnableAutoConfiguration @ComponentScan class SampleConfig object SampleWebApplication extends App { SpringApplication.run(classOf[SampleConfig]); } and that's it, with this set-up the entire application just works, the application can be started up with the following: ./gradlew build && java -jar build/libs/spring-boot-scala-web-0.1.0.jar and the sample endpoint listing the hotels accessed at this url: http://localhost:8080/hotels/list I have the entire git project available at this github location: https://github.com/bijukunjummen/spring-boot-scala-web In conclusion, Scala can be considered a first class citizen for a Spring-boot based application and there is no special configuration required to get a Scala based Spring-boot application to work. It just works!

So I finally made the decision on trying to learn Scala. Little did I know I was in for another round of IntelliJ integration hell. Let me rephrase that: IntelliJ with Gradle hell. I love Gradle. I love IntelliJ. However, the combination of the two is sometimes enough to drive me utterly crazy. Now take for example the Scala integration. I made the most simple Gradle build possible that compiles a standard Hello World application. apply plugin: 'scala' apply plugin: 'idea' repositories{ mavenCentral() mavenLocal() } dependencies{ compile 'org.slf4j:slf4j-api:1.7.5' compile "org.scala-lang:scala-library:2.10.4" compile "org.scala-lang:scala-compiler:2.10.4" testCompile "junit:junit:4.11" } task run(type: JavaExec, dependsOn: classes) { main = 'Main' classpath sourceSets.main.runtimeClasspath classpath configurations.runtime } First I stumbled upon the first issue: the Scala gradle plugin is incompatible with Java 8. Not a big issue, but this meant changing my java environment for this build, so it is a nuisance. Once this was fixed, the Gradle build succeeded and Hello World was printed out. I opened up IntelliJ and made sure the Scala plugin was installed. Then I imported the project using the Gradle build file. Everything looked okay, IntelliJ recognized the Scala source folder and provided the correct editor for the Scala source file. Then I tried to run the Main class. This resulted in a NoClassDefFoundException. IntelliJ didn’t want to compile my source classes. So I started digging. Apparently, the project was lacking a Scala facet. I’d expected IntelliJ to automatically add this once it saw I was using the scala plugin but it didn’t. So I tried manually adding the facet and there I got stuck. See, the facet requires you to state which scala compiler library you want to use. Luckily IntelliJ correctly added the jars to the classpath, so I was able to choose the correct jar. This, however, did not fix the issue as IntelliJ now complained it could not locate the scala runtime library (scala-library*.jar). This library was however included in the build. If you were to choose the runtime library as the scala library, it would complain it cannot find the compiler library. And this is where I am now: deadlocked. There is an issue in the bugtracker of IntelliJ here but it’s been eerily quiet at Jetbrains on this issue. As it is, it’s impossible to use IntelliJ with Gradle and Scala unless you’re willing to execute every bit of code including unit tests with Gradle instead of the IDE (which in effect defeats the purpose of an IDE). And I’ll die before adopting yet another build framework (SBT) that is supposed to work. Honestly, I really don’t know whether I want to learn Scala anymore. Just the fact that you can’t compile Scala in the most popular IDE at the moment when using the most popular build tool at the moment is something I cannot comprehend. Forcing me to adopt a Scala-specific build tool is unacceptable to me. If I were TypeSafe, I’d put an engineer on this and fix this as this would seriously aid in promoting the language. If it were easy to adopt Scala in an existing build cycle, it would pop up on more radars than it would right now. But it’s not just Scala and IntelliJ: most newer JVM languages struggle with IntelliJ. This is a real pity as this either forces me to change my IDE (i.e. Ceylon has its own IDE based on Eclipse) or not consider the language. As it is, the current viable option with IntelliJ is Java and Groovy (and Kotlin, but it’s not even near production ready quality). Wouldn’t it be nice to only need one IDE for all development? I couldn’t care less if it would cost $500, I just want things to work. I’d love to be able to write my AngularJS front-end that’s consuming my Scala/Java hydrid backend reading data from a MongoDB that’s feeded data from my Arduino sensors (for which I’ve written and uploaded the sketch from that same IDE).

Performance tuning SELECT statements can be a time consuming task which in my opinion follows Pareto principle’s. 20% effort is likely give you an 80% performance improvement. To get another 20% performance improvement you probably need to spend 80% of the time. Unless you work on the planet Venus where each day on Venus is equal to 243 Earth days, delivery deadlines are likely to mean you will not have enough time to put into tuning your SQL queries. After years writing and running SQL statements I began to develop a mental check-list of things I looked at when trying to improve query performance. These are the things I check before moving on to query plans and reading the sometimes complicated documentation of the database I am working on. My check-list is by no means comprehensive or scientific, more like a back of the envelope calculation but can I can say that most of the time I do get performance improvements following these simple steps. The check-list follows. Check Indexes There should be indexes on all fields used in the WHERE and JOIN portions of the SQL statement. Take the 3-Minute SQL performance test. Regardless of your score be sure to read through the answers as they are informative. Limit Size of Your Working Data Set Examine the tables used in the SELECT statement to see if you can apply filters in the WHERE clause of your statement. A classic example is when a query initially worked well when there were only a few thousand rows in the table. As the application grew the query slowed down. The solution may be as simple as restricting the query to looking at the current month’s data. When you have queries that have sub-selects, look to apply filtering to the inner statement of the sub-selects as opposed to the outer statements. Only Select Fields You Need Extra fields often increase the grain of the data returned and thus result in more (detailed) data being returned to the SQL client. Additionally: When using reporting and analytical applications, sometimes the slow report performance is because the reporting tool has to do the aggregation as data is received in detailed form. Occasionally the query may run quickly enough but your problem could be a network related issue as large amounts of detailed data are sent to the reporting server across the network. When using a column-oriented DBMS only the columns you have selected will be read from disk, the less columns you include in your query the less IO overhead. Remove Unnecessary Tables The reasons for removing unnecessary tables are the same as the reasons for removing fields not needed in the select statement. Writing SQL statements is a process that usually takes a number of iterations as you write and test your SQL statements. During development it is possible that you add tables to the query that may not have any impact on the data returned by the SQL code. Once the SQL is correct I find many people do not review their script and remove tables that do not have any impact or use in the final data returned. By removing the JOINS to these unnecessary tables you reduce the amount of processing the database has to do. Sometimes, much like removing columns you may find your reduce the data bring brought back by the database. Remove OUTER JOINS This can easier said than done and depends on how much influence you have in changing table content. One solution is to remove OUTER JOINS by placing placeholder rows in both tables. Say you have the following tables with an OUTER JOIN defined to ensure all data is returned: customer_id customer_name 1 John Doe 2 Mary Jane 3 Peter Pan 4 Joe Soap customer_id sales_person NULL Newbee Smith 2 Oldie Jones 1 Another Oldie NULL Greenhorn The solution is to add a placeholder row in the customer table and update all NULL values in the sales table to the placeholder key. customer_id customer_name 0 NO CUSTOMER 1 John Doe 2 Mary Jane 3 Peter Pan 4 Joe Soap customer_id sales_person 0 Newbee Smith 2 Oldie Jones 1 Another Oldie 0 Greenhorn Not only have you removed the need for an OUTER JOIN you have also standardised how sales people with no customers are represented. Other developers will not have to write statements such as ISNULL(customer_id, “No customer yet”). Remove Calculated Fields in JOIN and WHERE Clauses This is another one of those that may at times be easier said than done depending on your permissions to make changes to the schema. This can be done by creating a field with the calculated values used in the join on the table. Given the following SQL statement: FROM sales a JOIN budget b ON ((year(a.sale_date)* 100) + month(a.sale_date)) = b.budget_year_month Performance can be improved by adding a column with the year and month in the sales table. The updated SQL statement would be as follows: SELECT * FROM PRODUCTSFROM sales a JOIN budget b ON a.sale_year_month = b.budget_year_month Conclusion The recommendations boil down to a few short pointers check for indexes work with the smallest data set required remove unnecessary fields and tables and remove calculations in your JOIN and WHERE clauses. If all these recommendations fail to improve your SQL query performance my last suggestion is you move to Venus. All you will need is a single day to tune your SQL.

want to add java 8 support to kepler? java 8 has not yet landed in our standard download packages . but you can add it to your existing eclipse kepler package. i’ve got three different eclipse installations running java 8: a brand new kepler sr2 installation of the eclipse ide for java developers; a slightly used kepler sr1 installation of the eclipse for rcp/rap developers (with lots of other features already added); and a nightly build (dated march 24/2014) of eclipse 4.4 sdk. the jdt team recommends that you start from kepler sr2, the second and final service release for kepler (but using the exact same steps, i’ve installed it into kepler sr1 and sr2 packages). there are some detailed instructions for adding java 8 support by installing a feature patch in the eclipsepedia wiki . the short version is this: from kepler sr2, use the “help > install new software…” menu option to open the “available software” dialog; enter http://download.eclipse.org/eclipse/updates/4.3-p-builds/ into the “work with” field (highlighted below); put a checkbox next to “eclipse java 8 support (for kepler sr2)” (highlighted below); click “next”, click “next”, read and accept the license, and click “finish” watch the pretty progress bar move relatively quickly across the bottom of the window; and restart eclipse when prompted. select “help > install new software…” to open the available software dialog. voila! support for java 8 is installed. if you’ve already got the java 8 jdk installed and the corresponding jre is the default on your system, you’re done. if you’re not quite ready to make the leap to a java 8 jre, there’s still hope (my system is still configured with java 7 as the default). install the java 8 jdk; open the eclipse preferences, and navigate to “java > installed jres”; java runtime environment preferences click “add…”; select “standard vm”, click “next”; enter the path to the java 8 jre (note that this varies depending on platform, and how you obtain and install the bits); java 8 jre definition click “finish”. before closing the preferences window, you can set your workspace preference to use the newly-installed java 8 jre. or, if you’re just planning to experiment with java 8 for a while, you can configure this on a project-by-project basis. in the create a java project dialog, specify that your project will use a javase-1.8 jre. it’s probably better to do this on the project as this will become a project setting that will follow the project into your version control system. next step… learn how wrong my initial impressions of java 8 were (hint: it’s far better). the lambda is so choice. if you have the means, i highly recommend picking one up. about these ads

Spring WebApplicationInitializer provides a programatic way to configure the Spring DispatcherServlet and ContextLoaderListener in Servlet 3.0+ compliant servlet containers , rather than adding this configuration through a web.xml file. This is a quick note to show how implementation through WebApplicationInitializer interface internally works, given that this interface does not derive from any Servlet related interface! The answer is the ServletContainerInitializer interface introduced with Servlet 3.0 specification, implementors of this interface are notified during the context startup phase and can perform any programatic registration through the provided ServletContext. Spring implements the ServletContainerInitializer through SpringServletContainerInitializer class. Per the Servlet specs, this implementation must be declared in a META-INF/services/javax.servlet.ServletContainerInitializer file of the libraries jar file - Spring declares this in spring-web*.jar jar file and has an entry `org.springframework.web.SpringServletContainerInitializer` SpringServletContainerInitializer class has a @HandlerTypes annotation with a value of WebApplicationInitializer, this means that the Servlet container will scan for classes implementing the WebApplicationInitializer implementation and call the onStartUp method with these classes and that is where the WebApplicationInitializer fits in. A little convoluted, but the good thing is all these details are totally abstracted away within the spring-web framework and the developer only has to configure an implementation of WebApplicationInitializer and live in a web.xml free world.

Recently, I had to take some content in markdown, specifically markdown extra, and convert it to a series of PDFs styled with a specific branding. While some will argue that PDFs are dead and “long live the web”, many of us still need to produce PDFs for one reason or another. In this case I had to take markdown extra, with some html sprinkled in, clean it up, and convert it to a styled PDF. What follows is how I did that using QueryPath for the cleanup and DOMPDF to make the conversion. The Setup At the root of this little app was a PHP script with the dependencies managed through composer. The composer.json file looked like: { "name": "foo/bar", "description": "Convert markdown to PDF.", "type": "application", "require": { "php": ">=5.3.0", "michelf/php-markdown": "1.4.*", "dompdf/dompdf" : "0.6.*", "querypath/querypath": "3.*", "masterminds/html5": "1.*" } } Turning the Markdown into HTML Within the script I started with a file we’ll call $file. Form here it was easy using the official markdown extra conversion utility. $markdown = file_get_contents($file); $markdownParser = new \Michelf\MarkdownExtra(); $html = $markdownParser->transform($markdown); This produces the html needed to go inside the body of an html page. From here I wrapped it in a document because I could easily link to a CSS file for styling purposes. DOMPDF supports quite a bit of CSS 2.1. $html = '' . $html . '’; pdf.css is where you can style the PDF. If you know how to style web pages using CSS you can manage to style a PDF document. Cleaning Up The Content There were a number of places html had been injected into the markdown that was either broken, unwanted in a PDF, or an edge case that DOMPDF didn’t support. To make these changes I used QueryPath. For example, I needed to take relative links, normally used in generation of a website, and add a domain name to them: $dom = \HTML5::loadHTML($html); $links = htmlqp($dom, 'a'); foreach ($links as $link) { $href = $link->attr('href'); if (substr($href, 0, 1) == '/' && substr($href, 1, 1) != '/') { $link->attr('href', $domain_name . $href); } } $html = \HTML5::saveHTML($dom); Note, I used the HTML5 parser and writer rather than the built-in one designed for xhtml and HTML 4. This is because DOMPDF attempts to work with HTML5 and I wanted to keep that consistent from the beginning. Converting to PDF There is a little setup before using DOMPDF. It has a built in autoloader which should be disabled and needs a config file. In my case I used the default config file and handled this with: define('DOMPDF_ENABLE_AUTOLOAD', false); require_once __DIR__ . '/vendor/dompdf/dompdf/dompdf_config.inc.php'; The conversion was fairly straight forward. I used a snippet like: $dompdf = new DOMPDF(); $dompdf->load_html($html); $dompdf->render(); $output = $dompdf->output(); file_put_contents(‘path/to/file.pdf', $output); DOMPDF has a lot of options and some quirks. It wasn’t exactly designed for composer. For example, if you want to work with custom fonts you need to get the project from git and install submodules. Despite the quirks, needing to cleanup some of the html, and brand the documents, I was able to write a conversion script that handled dozens of documents quickly. Almost all of my time was on html cleanup and css styling.

In SQL Server management studio, using, View, Registered Servers (Ctrl+Alt+G) set up the servers that you want to execute the same query across all servers for, right click the group, select new query. Then when you execute the query, the results will come back with the first column showing you the database instance that that row came from.

over the last several months, i've been developing a rest api using spring boot . my client hired an outside company to develop a native ios app, and my development team was responsible for developing its api. our main task involved integrating with epic , a popular software system used in health care. we also developed a crowd -backed authentication system, based loosely on philip sorst's angular rest security . to document our api, we used spring mvc integration for swagger (a.k.a. swagger-springmvc). i briefly looked into swagger4spring-web , but gave up quickly when it didn't recognize spring's @restcontroller. we started with swagger-springmvc 0.6.5 and found it fairly easy to integrate. unfortunately, it didn't allow us to annotate our model objects and tell clients which fields were required. we were quite pleased when a new version (0.8.2) was released that supports swagger 1.3 and its @apimodelproperty. what is swagger? the goal of swagger is to define a standard, language-agnostic interface to rest apis which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection. to demonstrate how swagger works, i integrated it into josh long's x-auth-security project. if you have a boot-powered project, you should be able to use the same steps. 1. add swagger-springmvc dependency to your project. com.mangofactory swagger-springmvc 0.8.2 note: on my client's project, we had to exclude "org.slf4j:slf4j-log4j12" and add "jackson-module-scala_2.10:2.3.1" as a dependency. i did not need to do either of these in this project. 2. add a swaggerconfig class to configure swagger. the swagger-springmvc documentation has an example of this with a bit more xml. package example.config; import com.mangofactory.swagger.configuration.jacksonscalasupport; import com.mangofactory.swagger.configuration.springswaggerconfig; import com.mangofactory.swagger.configuration.springswaggermodelconfig; import com.mangofactory.swagger.configuration.swaggerglobalsettings; import com.mangofactory.swagger.core.defaultswaggerpathprovider; import com.mangofactory.swagger.core.swaggerapiresourcelisting; import com.mangofactory.swagger.core.swaggerpathprovider; import com.mangofactory.swagger.scanners.apilistingreferencescanner; import com.wordnik.swagger.model.*; import org.springframework.beans.factory.annotation.autowired; import org.springframework.beans.factory.annotation.value; import org.springframework.context.annotation.bean; import org.springframework.context.annotation.componentscan; import org.springframework.context.annotation.configuration; import java.util.arraylist; import java.util.arrays; import java.util.list; import static com.google.common.collect.lists.newarraylist; @configuration @componentscan(basepackages = "com.mangofactory.swagger") public class swaggerconfig { public static final list default_include_patterns = arrays.aslist("/news/.*"); public static final string swagger_group = "mobile-api"; @value("${app.docs}") private string docslocation; @autowired private springswaggerconfig springswaggerconfig; @autowired private springswaggermodelconfig springswaggermodelconfig; /** * adds the jackson scala module to the mappingjackson2httpmessageconverter registered with spring * swagger core models are scala so we need to be able to convert to json * also registers some custom serializers needed to transform swagger models to swagger-ui required json format */ @bean public jacksonscalasupport jacksonscalasupport() { jacksonscalasupport jacksonscalasupport = new jacksonscalasupport(); //set to false to disable jacksonscalasupport.setregisterscalamodule(true); return jacksonscalasupport; } /** * global swagger settings */ @bean public swaggerglobalsettings swaggerglobalsettings() { swaggerglobalsettings swaggerglobalsettings = new swaggerglobalsettings(); swaggerglobalsettings.setglobalresponsemessages(springswaggerconfig.defaultresponsemessages()); swaggerglobalsettings.setignorableparametertypes(springswaggerconfig.defaultignorableparametertypes()); swaggerglobalsettings.setparameterdatatypes(springswaggermodelconfig.defaultparameterdatatypes()); return swaggerglobalsettings; } /** * api info as it appears on the swagger-ui page */ private apiinfo apiinfo() { apiinfo apiinfo = new apiinfo( "news api", "mobile applications and beyond!", "https://helloreverb.com/terms/", "[email protected]", "apache 2.0", "http://www.apache.org/licenses/license-2.0.html" ); return apiinfo; } /** * configure a swaggerapiresourcelisting for each swagger instance within your app. e.g. 1. private 2. external apis * required to be a spring bean as spring will call the postconstruct method to bootstrap swagger scanning. * * @return */ @bean public swaggerapiresourcelisting swaggerapiresourcelisting() { //the group name is important and should match the group set on apilistingreferencescanner //note that swaggercache() is by defaultswaggercontroller to serve the swagger json swaggerapiresourcelisting swaggerapiresourcelisting = new swaggerapiresourcelisting(springswaggerconfig.swaggercache(), swagger_group); //set the required swagger settings swaggerapiresourcelisting.setswaggerglobalsettings(swaggerglobalsettings()); //use a custom path provider or springswaggerconfig.defaultswaggerpathprovider() swaggerapiresourcelisting.setswaggerpathprovider(apipathprovider()); //supply the api info as it should appear on swagger-ui web page swaggerapiresourcelisting.setapiinfo(apiinfo()); //global authorization - see the swagger documentation swaggerapiresourcelisting.setauthorizationtypes(authorizationtypes()); //every swaggerapiresourcelisting needs an apilistingreferencescanner to scan the spring request mappings swaggerapiresourcelisting.setapilistingreferencescanner(apilistingreferencescanner()); return swaggerapiresourcelisting; } @bean /** * the apilistingreferencescanner does most of the work. * scans the appropriate spring requestmappinghandlermappings * applies the correct absolute paths to the generated swagger resources */ public apilistingreferencescanner apilistingreferencescanner() { apilistingreferencescanner apilistingreferencescanner = new apilistingreferencescanner(); //picks up all of the registered spring requestmappinghandlermappings for scanning apilistingreferencescanner.setrequestmappinghandlermapping(springswaggerconfig.swaggerrequestmappinghandlermappings()); //excludes any controllers with the supplied annotations apilistingreferencescanner.setexcludeannotations(springswaggerconfig.defaultexcludeannotations()); // apilistingreferencescanner.setresourcegroupingstrategy(springswaggerconfig.defaultresourcegroupingstrategy()); //path provider used to generate the appropriate uri's apilistingreferencescanner.setswaggerpathprovider(apipathprovider()); //must match the swagger group set on the swaggerapiresourcelisting apilistingreferencescanner.setswaggergroup(swagger_group); //only include paths that match the supplied regular expressions apilistingreferencescanner.setincludepatterns(default_include_patterns); return apilistingreferencescanner; } /** * example of a custom path provider */ @bean public apipathprovider apipathprovider() { apipathprovider apipathprovider = new apipathprovider(docslocation); apipathprovider.setdefaultswaggerpathprovider(springswaggerconfig.defaultswaggerpathprovider()); return apipathprovider; } private list authorizationtypes() { arraylist authorizationtypes = new arraylist<>(); list authorizationscopelist = newarraylist(); authorizationscopelist.add(new authorizationscope("global", "access all")); list granttypes = newarraylist(); loginendpoint loginendpoint = new loginendpoint(apipathprovider().getappbasepath() + "/user/authenticate"); granttypes.add(new implicitgrant(loginendpoint, "access_token")); return authorizationtypes; } @bean public swaggerpathprovider relativeswaggerpathprovider() { return new apirelativeswaggerpathprovider(); } private class apirelativeswaggerpathprovider extends defaultswaggerpathprovider { @override public string getappbasepath() { return "/"; } @override public string getswaggerdocumentationbasepath() { return "/api-docs"; } } } the apipathprovider class referenced above is as follows: package example.config; import com.mangofactory.swagger.core.swaggerpathprovider; import org.springframework.beans.factory.annotation.autowired; import org.springframework.web.util.uricomponentsbuilder; import javax.servlet.servletcontext; public class apipathprovider implements swaggerpathprovider { private swaggerpathprovider defaultswaggerpathprovider; @autowired private servletcontext servletcontext; private string docslocation; public apipathprovider(string docslocation) { this.docslocation = docslocation; } @override public string getapiresourceprefix() { return defaultswaggerpathprovider.getapiresourceprefix(); } public string getappbasepath() { return uricomponentsbuilder .fromhttpurl(docslocation) .path(servletcontext.getcontextpath()) .build() .tostring(); } @override public string getswaggerdocumentationbasepath() { return uricomponentsbuilder .fromhttpurl(getappbasepath()) .pathsegment("api-docs/") .build() .tostring(); } @override public string getrequestmappingendpoint(string requestmappingpattern) { return defaultswaggerpathprovider.getrequestmappingendpoint(requestmappingpattern); } public void setdefaultswaggerpathprovider(swaggerpathprovider defaultswaggerpathprovider) { this.defaultswaggerpathprovider = defaultswaggerpathprovider; } } in src/main/resources/application.properties , add an "app.docs" property. this will need to be changed as you move your application from local -> test -> staging -> production. spring boot's externalized configuration makes this fairly simple. app.docs=http://localhost:8080 3. verify swagger produces json. after completing the above steps, you should be able to see the json swagger generates for your api. open http://localhost:8080/api-docs in your browser or curl http://localhost:8080/api-docs . { "apiversion": "1", "swaggerversion": "1.2", "apis": [ { "path": "http://localhost:8080/api-docs/mobile-api/example_newscontroller", "description": "example.newscontroller" } ], "info": { "title": "news api", "description": "mobile applications and beyond!", "termsofserviceurl": "https://helloreverb.com/terms/", "contact": "[email protected]", "license": "apache 2.0", "licenseurl": "http://www.apache.org/licenses/license-2.0.html" } } 4. copy swagger ui into your project. swagger ui is a good-looking javascript client for swagger's json. i integrated it using the following steps: git clone https://github.com/wordnik/swagger-ui cp -r swagger-ui/dist ~/dev/x-auth-security/src/main/resources/public/docs i modified docs/index.html, deleting its header () element, as well as made its url dynamic. ... $(function () { var apiurl = window.location.protocol + "//" + window.location.host; if (window.location.pathname.indexof('/api') > 0) { apiurl += window.location.pathname.substring(0, window.location.pathname.indexof('/api')) } apiurl += "/api-docs"; log('api url: ' + apiurl); window.swaggerui = new swaggerui({ url: apiurl, dom_id: "swagger-ui-container", ... after making these changes, i was able to open fire up the app with "mvn spring-boot:run" and view http://localhost:8080/docs/index.html in my browser. 5. annotate your api. there are two services in x-auth-security: one for authentication and one for news. to provide more information to the "news" service's documentation, add @api and @apioperation annotations. these annotations aren't necessary to get a service to show up in swagger ui, but if you don't specify the @api("user"), you'll end up with an ugly-looking class name instead (e.g. example_xauth_userxauthtokencontroller). @restcontroller @api(value = "news", description = "news api") class newscontroller { map entries = new concurrenthashmap(); @requestmapping(value = "/news", method = requestmethod.get) @apioperation(value = "get news", notes = "returns news items") collection entries() { return this.entries.values(); } @requestmapping(value = "/news/{id}", method = requestmethod.delete) @apioperation(value = "delete news item", notes = "deletes news item by id") newsentry remove(@pathvariable long id) { return this.entries.remove(id); } @requestmapping(value = "/news/{id}", method = requestmethod.get) @apioperation(value = "get a news item", notes = "returns a news item") newsentry entry(@pathvariable long id) { return this.entries.get(id); } @requestmapping(value = "/news/{id}", method = requestmethod.post) @apioperation(value = "update news", notes = "updates a news item") newsentry update(@requestbody newsentry news) { this.entries.put(news.getid(), news); return news; } ... } you might notice the screenshot above only shows news. this is because swaggerconfig.default_include_patterns only specifies news. the following will include all apis. public static final list default_include_patterns = arrays.aslist("/.*"); after adding these annotations and modifying swaggerconfig , you should see all available services. in swagger-springmvc 0.8.x, the ability to use @apimodel and @apimodelproperty annotations was added. this means you can annotate newsentry to specify which fields are required. @apimodel("news entry") public static class newsentry { @apimodelproperty(value = "the id of the item", required = true) private long id; @apimodelproperty(value = "content", required = true) private string content; // getters and setters } this results in the model's documentation showing up in swagger ui. if "required" isn't specified, a property shows up as optional . parting thoughts the qa engineers and 3rd party ios developers have been very pleased with our api documentation. i believe this is largely due to swagger and its nice-looking ui. the swagger ui also provides an interface to test the endpoints by entering parameters (or json) into html forms and clicking buttons. this could benefit those qa folks that prefer using selenium to test html (vs. raw rest endpoints). i've been quite pleased with swagger-springmvc, so kudos to its developers. they've been very responsive in fixing issues i've reported . the only thing i'd like is support for recognizing jsr303 annotations (e.g. @notnull) as required fields. to see everything running locally, checkout my modified x-auth-security project on github and the associated commits for this article.

Here's a great JDK 8 command-line static dependency checker.

A Mule application which uses the Jboss TX transaction manager needs a persistent Object Store to hold the objects and states of the transactions being processed (further information about different object stores can be found in the following page). By default Mule uses the ShadowNoFileLockStorem, which uses the file system to store the objects. As one can guess, if an application does not have permission to write the object store to the file system, the Jboss Transaction Manager will not be able to work properly and will throw an exception similar to the following: com.arjuna.ats.arjuna: ARJUNA12218: cant create new instance of {0} java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ... Caused by: com.arjuna.ats.arjuna.exceptions.ObjectStoreException: ARJUNA12225: FileSystemStore::setupStore - cannot access root of object store: //ObjectStore/ShadowNoFileLockStore/defaultStore/ at com.arjuna.ats.internal.arjuna.objectstore.FileSystemStore.(FileSystemStore.java:482) at com.arjuna.ats.internal.arjuna.objectstore.ShadowingStore.(ShadowingStore.java:619) at com.arjuna.ats.internal.arjuna.objectstore.ShadowNoFileLockStore.(ShadowNoFileLockStore.java:53) ... 36 more Since the object store is not created, the XA Transaction Manager is not initialised properly. This will throw a ‘Could not initialize class’ exception whenever the transaction manager is invoked. org.mule.exception.DefaultSystemExceptionStrategy: Caught exception in Exception Strategy: errorCode: 0 javax.resource.spi.work.WorkCompletedException: errorCode: 0 at org.mule.work.WorkerContext.run(WorkerContext.java:335) at java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:2025) ... Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.arjuna.ats.arjuna.coordinator.TxControl at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.begin(BaseTransaction.java:87) at org.mule.transaction.XaTransaction.doBegin(XaTransaction.java:63) at org.mule.transaction.AbstractTransaction.begin(AbstractTransaction.java:66) at org.mule.transaction.XaTransactionFactory.beginTransaction(XaTransactionFactory.java:32) at org.mule.execution.BeginAndResolveTransactionInterceptor.execute(BeginAndResolveTransactionInterceptor.java:51) at org.mule.execution.ResolvePreviousTransactionInterceptor.execute(ResolvePreviousTransactionInterceptor.java:48) at org.mule.execution.SuspendXaTransactionInterceptor.execute(SuspendXaTransactionInterceptor.java:54) at org.mule.execution.ValidateTransactionalStateInterceptor.execute(ValidateTransactionalStateInterceptor.java:44) at org.mule.execution.IsolateCurrentTransactionInterceptor.execute(IsolateCurrentTransactionInterceptor.java:44) at org.mule.execution.ExternalTransactionInterceptor.execute(ExternalTransactionInterceptor.java:52) at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:32) at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:17) at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:113) at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:34) at org.mule.transport.jms.XaTransactedJmsMessageReceiver.poll(XaTransactedJmsMessageReceiver.java:214) at org.mule.transport.AbstractPollingMessageReceiver.performPoll(AbstractPollingMessageReceiver.java:219) at org.mule.transport.PollingReceiverWorker.poll(PollingReceiverWorker.java:84) at org.mule.transport.PollingReceiverWorker.run(PollingReceiverWorker.java:53) at org.mule.work.WorkerContext.run(WorkerContext.java:311) ... 15 more Mule computes the default directory where to write the object store as follows : muleInternalDir = config.getWorkingDirectory(); (see the code for further analysis). If Mule is started from a directory where the user does not have write permissions, the problems mentioned above will be faced. The easiest way to fix this issue is to make sure that the user running Mule as full write permission to the working directory. If that cannot be achieved, fear not, there is a solution. On first analysis, one would be tempted to set the Object Store Directory by using Spring properties as follows: Unfortunately this will not work since the Jboss Transaction Manager is a Singleton and this property is used in the constructor of the object. Hence a behaviour similar to the following will be experienced: Caused by: com.arjuna.ats.arjuna.exceptions.ObjectStoreException: ARJUNA12225: FileSystemStore::setupStore - cannot access root of object store: PutObjectStoreDirHere/ShadowNoFileLockStore/defaultStore/ (Please note that “PutObjectStoreDirHere” is the default directory assigned by the JBoss TX transaction manager). One way to go around this issue is to be sure that these properties are set before the object is initialised. There are at least two ways to be sure that this is achieved: 1. Set the properties on start up as follows: ./mule -M-Dcom.arjuna.ats.arjuna.objectstore.objectStoreDir=/path/to/objectstoreDir -M-DObjectStoreEnvironmentBean.objectStoreDir=/path/to/objectstoreDir 2. Set the properties in the wrapper.config as follow: wrapper.java.additional.x=-Dcom.arjuna.ats.arjuna.objectstore.objectStoreDir=/path/to/objectstoreDir wrapper.java.additional.x=-DObjectStoreEnvironmentBean.objectStoreDir=/path/to/objectstoreDir (x is the next number available in the wrapper.config by default this is 4). Otherwise, take the easiest route and make sure that Mule can write to the start up directory.

Every GitHub repository comes with its own wiki. This is a great place to put the documentation for your project. What isn’t clear from the wiki documentation is how to add images to your wiki. Here’s my step-by-step guide. I’m going to add a logo to the main page of my WikiDemo repository’s wiki: https://github.com/mikehadlow/WikiDemo/wiki/Main-Page First clone the wiki. You grab the clone URL from the button at the top of the wiki page. $ git clone [email protected]:mikehadlow/WikiDemo.wiki.git Cloning into 'WikiDemo.wiki'... Enter passphrase for key '/home/mike.hadlow/.ssh/id_rsa': remote: Counting objects: 6, done. remote: Compressing objects: 100% (3/3), done. remote: Total 6 (delta 0), reused 0 (delta 0) Receiving objects: 100% (6/6), done. Create a new directory called ‘images’ (it doesn’t matter what you call it, this is just a convention I use): $ mkdir images Then copy your picture(s) into the images directory (I’ve copied my logo_design.png file to my images directory). $ ls -l -rwxr-xr-x 1 mike.hadlow Domain Users 12971 Sep 5 2013 logo_design.png Commit your changes and push back to GitHub: $ git add -A $ git status # On branch master # Changes to be committed: # (use "git reset HEAD ..." to unstage) # # new file: images/logo_design.png # $ git commit -m "Added logo_design.png" [master 23a1b4a] Added logo_design.png 1 files changed, 0 insertions(+), 0 deletions(-) create mode 100755 images/logo_design.png $ git push Enter passphrase for key '/home/mike.hadlow/.ssh/id_rsa': Counting objects: 5, done. Delta compression using up to 4 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (4/4), 9.05 KiB, done. Total 4 (delta 0), reused 0 (delta 0) To [email protected]:mikehadlow/WikiDemo.wiki.git 333a516..23a1b4a master -> master Now we can put a link to our image in ‘Main Page’: Save and there’s your image for all to see:

We all know we need to use connection pools where ever we connect to a database. All of the modern drivers using JDBC type 4 support it. In this post we will have look at an overview ofconnection pooling in spring applications and how to deal with same context in a non JEE enviorements (like tests). Most examples of connecting to database in spring is done using DriverManagerDataSource. If you don't read the documentation properly then you are going to miss a very important point. NOTE: This class is not an actual connection pool; it does not actually pool Connections. It just serves as simple replacement for a full-blown connection pool, implementing the same standard interface, but creating new Connections on every call. Useful for test or standalone environments outside of a J2EE container, either as a DataSource bean in a corresponding ApplicationContext or in conjunction with a simple JNDI environment. Pool-assuming Connection.close() calls will simply close the Connection, so any DataSource-aware persistence code should work. Yes, by default the spring applications does not use pooled connections. There are two ways to implement the connection pooling. Depending on who is managing the pool. If you are running in a JEE environment, then it is prefered use the container for it. In a non-JEE setup there are libraries which will help the application to manage the connection pools. Lets discuss them in bit detail below. 1. Server (Container) managed connection pool (Using JNDI) When the application connects to the database server, establishing the physical actual connection takes much more than the execution of the scripts. Connection pooling is a technique that was pioneered by database vendors to allow multiple clients to share a cached set of connection objects that provide access to a database resource. The JavaWorld article gives a good overview about this. In a J2EE container, it is recommended to use a JNDI DataSource provided by the container. Such a DataSource can be exposed as a DataSource bean in a Spring ApplicationContext via JndiObjectFactoryBean, for seamless switching to and from a local DataSource bean like this class. The below articles helped me in setting up the data source in JBoss AS. 1. DebaJava Post 2. JBoss Installation Guide 3. JBoss Wiki Next step is to use these connections created by the server from the application. As mentioned in the documentation you can use the JndiObjectFactoryBean for this. It is as simple as below If you want to write any tests using springs "SpringJUnit4ClassRunner" it can't load the context becuase the JNDI resource will not be available. For tests, you can then either set up a mock JNDI environment through Spring's SimpleNamingContextBuilder, or switch the bean definition to a local DataSource (which is simpler and thus recommended). As I was looking for a good solutions to this problem (I did not want a separate context for tests) this SO answer helped me. It sort of uses the various tips given in the Javadoc to good effect. The issue with the above solution is the repetition of code to create the JNDI connections. I have solved it using a customized runner SpringWithJNDIRunner. This class adds the JNDI capabilities to the SpringJUnit4ClassRunner. It reads the data source from "test-datasource.xml" file in the class path and binds it to the JNDI resource with name "java:/my-ds". After the execution of this code the JNDI resource is available for the spring container to consume. import javax.naming.NamingException; import org.junit.runners.model.InitializationError; import org.springframework.context.ApplicationContext; import org.springframework.context.support.ClassPathXmlApplicationContext; import org.springframework.mock.jndi.SimpleNamingContextBuilder; import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; /** * This class adds the JNDI capabilities to the SpringJUnit4ClassRunner. * @author mkadicha * */ public class SpringWithJNDIRunner extends SpringJUnit4ClassRunner { public static boolean isJNDIactive; /** * JNDI is activated with this constructor. * * @param klass * @throws InitializationError * @throws NamingException * @throws IllegalStateException */ public SpringWithJNDIRunner(Class klass) throws InitializationError, IllegalStateException, NamingException { super(klass); synchronized (SpringWithJNDIRunner.class) { if (!isJNDIactive) { ApplicationContext applicationContext = new ClassPathXmlApplicationContext( "test-datasource.xml"); SimpleNamingContextBuilder builder = new SimpleNamingContextBuilder(); builder.bind("java:/my-ds", applicationContext.getBean("dataSource")); builder.activate(); isJNDIactive = true; } } } } To use this runner you just need to use the annotation @RunWith(SpringWithJNDIRunner.class) in your test. This class extends SpringJUnit4ClassRunner beacuse a there can only be one class in the @RunWith annotation. The JNDI is created only once is a test cycle. This class provides a clean solution to the problem. 2. Application managed connection pool If you need a "real" connection pool outside of a J2EE container, consider Apache's Jakarta Commons DBCP or C3P0. Commons DBCP's BasicDataSource and C3P0's ComboPooledDataSource are full connection pool beans, supporting the same basic properties as this class plus specific settings (such as minimal/maximal pool size etc). Below user guides can help you configure this. 1. Spring Docs 2. C3P0 Userguide 3. DBCP Userguide The below articles speaks about the general guidelines and best practices in configuring the connection pools. 1. SO question on Spring JDBC Connection pools 2. Connection pool max size in MS SQL Server 2008 3. How to decide the max number of connections 4. Monitoring the number of active connections in SQL Server 2008 Note:- All the text in italics are copied from the spring documentation of the DriverManagerDataSource.

Postgres and Oracle compatibility with Hibernate There are situations your JEE application needs to support Postgres and Oracle as a Database. Hibernate should do the job here, however, there are some specifics worth mentioning. While enabling Postgres for application already running Oracle I came across following tricky parts: BLOBs support, CLOBs support, Oracle not knowing Boolean type (using Integer) instead and DUAL table. These were the tricks I had to apply to make the @Entity classes running on both of these. Please note I’ve used Postgres 9.3 with Hibernate 4.2.1.SP1. BLOBs support The problem with Postgres is that it offers 2 types of BLOB storage: bytea - data stored in table oid - table holds just identifier to data stored elsewhere I guess in the most of the situations you can live with the bytea as well as I did. The other one as far as I’ve read is to be used for some huge data (in gigabytes) as it supports streams for IO operations. Well, it sounds nice there is such a support, however using Hibernate in this case can make things quite problematic (due to need to use the specific annotations), especially if you try to achieve compatibility with Oracle. To see the trouble here, see StackOverflow: proper hibernate annotation for byte[] All- the combinations are described there: annotation postgres oracle works on ------------------------------------------------------------- byte[] + @Lob oid blob oracle byte[] bytea raw(255) postgresql byte[] + @Type(PBA) oid blob oracle byte[] + @Type(BT) bytea blob postgresql where @Type(PBA) stands for: @Type(type="org.hibernate.type.PrimitiveByteArrayBlobType") and @Type(BT) stands for: @Type(type="org.hibernate.type.BinaryType"). These result in all sorts of Postgres errors, like: ERROR: column “foo” is of type oid but expression is of type bytea or ERROR: column “foo” is of type bytea but expression is of type oid Well, there seems to be a solution, still it includes patching of Hibernate library (something I see as the last option when playing with 3.rd party library). There is also a reference to official blog post from the Hibernate guys on the topic: PostgreSQL and BLOBs. Still solution described in blog post seems not working for me and based on the comments, seems to be invalid for more people. BLOBs solved OK, so now the optimistic part. After quite some debugging I ended up with the Entity definition like this : @Lob private byte[] foo; Oracle has no trouble with that, moreover I had to customize the Postgres dialect in a way: public class PostgreSQLDialectCustom extends PostgreSQL82Dialect { @Override public SqlTypeDescriptor remapSqlTypeDescriptor(SqlTypeDescriptor sqlTypeDescriptor) { if (sqlTypeDescriptor.getSqlType() == java.sql.Types.BLOB) { return BinaryTypeDescriptor.INSTANCE; } return super.remapSqlTypeDescriptor(sqlTypeDescriptor); } } That’s it! Quite simple right? That works for persisting to bytea typed columns in Postgres (as that fits my usecase). CLOBs support The errors in misconfiguration looked something like this: org.postgresql.util.PSQLException: Bad value for type long : ... So first I’ve found (on String LOBs on PostgreSQL with Hibernate 3.6) following solution: @Lob @Type(type = "org.hibernate.type.TextType") private String foo; Well, that works, but for Postgres only. Then there was a suggestion (on StackOverflow: Postgres UTF-8 clobs with JDBC) from to go for: @Lob @Type(type="org.hibernate.type.StringClobType") private String foo; That pointed me the right direction (the funny part was that it was just a comment to some answers). It was quite close, but didn’t work for me in all cases, still resulted in errors in my tests. CLOBs solved The important was @deprecation javadocs in the org.hibernate.type.StringClobType that brought me to working one: @Lob @Type(type="org.hibernate.type.MaterializedClobType") private String foo; That works for both Postgres and Oracle, without any further hacking (on Hibernate side) needed. Boolean type Oracle knows no Boolean type and the trouble is that Postgres does. As there was also some plain SQL present, I ended up In Postgres with error: ERROR: column “foo” is of type boolean but expression is of type integer I decided to enable cast from Integer to Boolean in Postgres rather than fixing all the plain SQL places (in a way found in Forum: Automatically Casting From Integer to Boolean): update pg_cast set castcontext = 'i' where oid in ( select c.oid from pg_cast c inner join pg_type src on src.oid = c.castsource inner join pg_type tgt on tgt.oid = c.casttarget where src.typname like 'int%' and tgt.typname like 'bool%'); Please note you should run the SQL update by user with provileges to update catalogs (probably not your postgres user used for DB connection from your application), as I’ve learned on Stackoverflow: Postgres - permission denied on updating pg_catalog.pg_cast. DUAL table There is one more specific in the Oracle I came across. If you have plain SQL, in Oracle there is DUAL table provied (see more info on Wikipedia on that) that might harm you in Postgres. Still the solution is simple. In Postgres create a view that would fill the similar purpose. It can be created like this: create or replace view dual as select 1; Conclusion Well that should be it. Enjoy your cross DB compatible JEE apps.

Want to learn more about interface default methods in Java 8? Check out this tutorial to learn how using this new feature.

If you need the same functionality in two projects, you should reuse code between them, right? Or should you? For as long as there has been a profession of software engineering, we have tried to achieve more reuse. But reuse has both a benefit and a cost. Too often, the cost is forgotten. In this article, I examine the economics of reuse. True story: One of the earliest projects to embrace object-oriented programming in the 1990s did so with the goal of maximizing reuse. The team responsible for creating the company wide framework used the following formula for calculating the value of their work: [Value of reuse] = [numbers of uses of framework] * [value of the framework to reusers] – [cost of developing the framework] This formula is obviously correct, but this is where they went horribly wrong: The organization said [value of framework to reusers] = [cost of developing framework]. In other words: The more expensive it was to create, the more valuable it was to use. We have clearly progressed beyond this thinking. A more updated formula would say: [value of framework to reusers] = [cost of developing the feature in question]. But even this is too optimistic. No library comes for free to its users. At the very least, you have to discover the features and learn about the details. The cost of reusing depends on many factors, such as the quality of the framework and the documentation and also upon the type of feature. A complex algorithm with a simple interface is cheap to use, while most domain-specific frameworks require relatively much work to reuse. We can express this as a reuse value factor, likely between 90% and 50%. For most cases, my guess would be at about 75%. So we have: [value of reuse] = [number of users] * ([cost of feature] * [reuse cost factor]) – [cost of developing the reusable component] What about the other important factor: [cost of developing the reusable component]? It’s easy to assume that the cost of developing a feature in a framework is equal to that of developing the feature in an application, but on further analysis shows that this is far from true. A reusable component needs more documentation, it needs to handle more special cases and it has a slower feedback cycle. This cost is actually substantial and may mean that it costs between 150% to 300% or more to develop a feature for reuse. Personally, I think the reusability cost factor lies around 300%. And the lower this number, the higher the cost factor of reuse is likely to be, because that may mean we skimped on documentation etc. A revised number would be: [value of reuse] = [number of users] * ([cost of feature] * [reuse value factor]) – [reusability cost factor] * [cost of feature] Or [value of reuse] = [cost of feature] * ([number of users] * [reuse value factor] – [reusability cost factor]) The more complex formula actually lets us make a few predictions. Let’s say we assume a reuse value factor of 75 % (meaning that it requires 1/4 of the effort to reuse a library rather than creating the feature from scratch) and a reusability cost factor of 300 % (meaning that it requires three times the effort to create something that’s worth reusing). This means: [value of reuse] = [cost of feature] * ([number of users] * 75% – 300%) This equation breaks even when [number of users] = 4. That means that to get any value from your reused component, you better have five or more reusers or you have to find a way to substantially improve the [reuse value factor] or [reusability cost factor]. Very smart people have failed to do this. Improving the value: Increase the number of reusers: Simple enough, but when you do, you risk that the [reuse value factor] goes down as the framework doesn’t suit everybody equally well. Reduce the cost of reusing the library: This means investing in documentation, improving your design, improving testing to reduce the number of bugs, handle bug reports and feature requests faster from your reusers – all of which increase your cost reusability cost factor. Reduce the extra work in making the library reusable: The most important way to reduce the cost of developing for reuse is to choose the right kind of problem to solve. Problems with a small surface and big volume are best. That means: Easy to describe, hard to implement. Sadly, most of the juiciest fruit was picked years ago by the standard library in your programming language and by open source frameworks. On a global scale, reuse has saved the software industry tremendous amounts. In an organization, it can be hard to get the same effect. Reuse comes at a cost to the reuser and to the developer of the reusable library. How do you evaluate and improve your [reuse value factor] and your [reusability cost factor]?

Gradle, a versatile JVM build tool, effectively handles JavaScript and CSS tasks for web applications and server components.

In a previous post, I presented a method of loading initial data into a Django database by using a custom management command. An accompanying task is cleaning the database up. Here I want to discuss a few options for doing that. First, some general design notes on Django management commands. If you run manage.py help you’ll see a whole bunch of commands starting with sql. These all share a common idiom – print SQL statements to the standard output. Almost all DB engines have means to pipe commands from the standard input, so this plays great with the Unix philosophy of building pipes of single-task programs. Django even provides a convenient shortcut for us to access the actual DB that’s being used with a given project – the dbshell command. As an example, we have the sqlflush command, which returns a list of the SQL statements required to return all tables in the database to the state they were in just after they were installed. In a simple blog-like application with "post" and "tag" models, it may return something like: $ python manage.py sqlflush BEGIN; DELETE FROM "auth_permission"; DELETE FROM "auth_group"; DELETE FROM "django_content_type"; DELETE FROM "django_session"; DELETE FROM "blogapp_tag"; DELETE FROM "auth_user_groups"; DELETE FROM "auth_group_permissions"; DELETE FROM "auth_user_user_permissions"; DELETE FROM "blogapp_post"; DELETE FROM "blogapp_post_tags"; DELETE FROM "auth_user"; DELETE FROM "django_admin_log"; COMMIT; Note there’s a lot of tables here, because the project also installed the admin and auth applications from django.contrib. We can actually execute these SQL statements, and thus wipe out all the DB tables in our database, by running: $ python manage.py sqlflush | python manage.py dbshell For this particular sequence, since it’s so useful, Django has a special built-in command named flush. But there’s a problem with running flush that may or may not bother you, depending on what your goals are. It wipes out all tables, and this means authentication data as well. So if you’ve created a default admin user when jump-starting the application, you’ll have to re-create it now. Perhaps there’s a more gentle way to delete just your app’s data, without messing with the other apps? Yes. In fact, I’m going to show a number of ways. First, let’s see what other existing management commands have to offer. sqlclear will emit the commands needed to drop all tables in a given app. For example: $ python manage.py sqlclear blogapp BEGIN; DROP TABLE "blogapp_tag"; DROP TABLE "blogapp_post"; DROP TABLE "blogapp_post_tags"; COMMIT; So we can use it to target a specific app, rather than using the kill-all approach of flush. There’s a catch, though. While flush runs delete to wipe all data from the tables, sqlclear removes the actual tables. So in order to be able to work with the database, these tables have to be re-created. Worry not, there’s a command for that: $ python manage.py sql blogapp BEGIN; CREATE TABLE "blogapp_post_tags" ( "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "post_id" integer NOT NULL REFERENCES "blogapp_post" ("id"), "tag_id" varchar(50) NOT NULL REFERENCES "blogapp_tag" ("name"), UNIQUE ("post_id", "tag_id") ) ; CREATE TABLE "blogapp_post" ( "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, <.......> ) ; CREATE TABLE "blogapp_tag" ( <.......> ) ; COMMIT; So here’s a first way to do a DB cleanup: pipe sqlclear appname into dbshell. Then pipe sql appname to dbshell. An alternative way, which I like less, is to take the subset of DELETE statements generated by sqlflush, save them in a text file, and pipe it through to dbshell when needed. For example, for the blog app discussed above, these statements should do it: BEGIN; DELETE FROM "blogapp_tag"; DELETE FROM "blogapp_post"; DELETE FROM "blogapp_post_tags"; DELETE COMMIT; The reason I don’t like it is that it forces you to have explicit table names stored somewhere, which is a duplication of the existing models. If you happen to change some of your foreign keys, for example, tables will need changing so this file will have to be regenerated. The approach I like best is more programmatic. Django’s model API is flexible and convenient, and we can just use it in a custom management command: from django.core.management.base import BaseCommand from blogapp.models import Post, Tag class Command(BaseCommand): def handle(self, *args, **options): Tag.objects.all().delete() Post.objects.all().delete() Save this code as blogapp/management/commands/clear_models.py, and now it can be invoked with: $ python manage.py clear_models

If you need to embed and display Google Maps in your Java Desktop Swing application, then JxBrowser Java library is what you need.