Let me first define what we mean by circular dependency in OOAD terms vis-a-vis Java. Suppose we have a class called A which has class B’s Object. (in UML terms A HAS B). at the same time we class B is also composed of Object of class A (in UML terms B HAS A). obviously this represents circular dependency because while creating the object of A, the compiler must know the size of B... on the other hand while creating object of B, the compiler must know the size of A. this is something like egg vs. chicken problem... this may be possible in real life situation as well. for example suppose a multi storied building has a lift. so in the UML terms, the building HAS lift... but at the same time, suppose, while constructing the lift object, we need to give it the information about the building object to access various functionalities of the Building class... for example, suppose the speed of the lift is set depending on the number of floors of the Building... hence while constructing the Lift object it must access the functionalities of the Building object which will give the number of floors the building has got...hence in UML terms the lift HAS building... so this is a sure case of circular dependency... in real java code it will look something as follows: public class Building { private Lift lift; private int floor; public Building(){ lift = new Lift(); setFloor(15); } public int getFloor(){ return floor; } public void setFloor(int floor){ this.floor = floor; } }//end of class building //class Lift public class Lift { private Building building; private int Speed; public Lift(){ building = new Building(); setSpeed(); } public void setSpeed(){ if (building.getFloor()>20){ //one set of functionalities //may be the the speed of the lift will be more this.Speed = 10; } else { //different set of functionalities //may be the speed of the lift will be less this.Speed = 5; } } public int getSpeed(){ return Speed; } }//end of class Lift As it becomes clear from the above code, that while creating the Building object it will create the Lift object, and while creating the Lift object, it will try to create a Building object to access some of its functionalities... So, ultimately it will go out of memory and we get a StackOverflow runtime exception... So how do we handle this problem in Java? We actually tackle this problem by declaring an IBuildingProxy interface and by deriving our Building class from that... the lift class, instead of Having Building object, it Has IBuildingProxy... the source code of the solution looks like the following... public interface IBuildingProxy { int getFloor(); void setFloor(int floor); } public class Building implements IBuildingProxy{ private Lift lift; private int floor; public Building(){ lift = new Lift(this); setFloor(15); } public int getFloor(){ return floor; } public void setFloor(int floor){ this.floor = floor; } } public class Lift { private IBuildingProxy building; private int Speed; public Lift(Building b){ this.building = b; setSpeed(); } private void setSpeed(){ if (building.getFloor()>20){ / /one set of functionalities //may be the the speed of the lift will be more this.Speed = 10; } else { //different set of functionalities //may be the speed of the lift will be less this.Speed = 5; } } public int getSpeed(){ return Speed; } } public class CircularDependencyTest { public static void main(String[] args){ Building b = new Building(); Lift l = new Lift(b); } } So whats the principle behind such work around... It will be clear soon... As it becomes clear from the code that Building HAS Lift... That is not a problem... Now when it comes to solve the part that Lift HAS Building, instead of the Building object, we have created an IBuildingProxy interface and we pass it to the Lift class... what it essentially means, that the building class knows the memory requirement to initialize the Lift object, and as the Lift class HAS just a proxy interface of the Building, it does not have to care for the Building's memory requirement... and that solves the problem... Hope this discussion becomes helpful for Java learners...

A brief overview of dependency injection in Java EE, the difference between @Resource/@EJB and @Inject, and how does that all relate to Spring – mostly in the form of links. Context Dependency Injection (CDI, JSR 299) is a part of Java EE 6 Web Profile and itself builds on Dependency Injection for Java (JSR 330), which introduces @Inject, @Named etc. While JSR 330 is for DI only and is implemented e.g. by Guice and Spring, CDI adds various EE stuff such as @RequestScoped, interceptors/decorators, producers, eventing and a base for integration with JSF, EJBs etc. Java EE components such as EJBs have been redefined to build on top of CDI (=> @Stateless is now a CDI managed bean with additional services). A key part of CDI aside of its DI capabilities is its awarness of bean contexts and the management of bean lifecycle and dependencies within those contexts (such as @RequestScoped or @ConversationScoped). CDI is extensible – you can define new context scopes, drop-in interceptors and decorators, make other beans (e.g. from Spring) available for CDI,… . Resources to check: Contexts and Dependency Injection in Java EE 6 by Adam Bien – a very good explanation of the basics of CDI and how it differs from DI in Java EE 5 (hint: context awarness) Slideshow with a good overview of CDI and all it offers About CDI extensibility and SPIs (e.g. Seam 3 is basically a set of portable CDI extensions) Guice and Spring do not implement CDI (3/2011) – and Spring perhaps isn’t motivated to do so (it supports JSR 330, CDI would be too much work) DZone CDI Refcard may be handy CDI 1.0 vs. Spring 3.1 feature comparsion: bean definition & dependency injection: “in the area that I compared in this article [= DI], there is only little critical difference in the two technologies” (though Spring more fine-tunable) Java EE 6 (CDI / EJB 3.1) XOR Spring Core Reloaded: New projects should preferably start with pure Java EE including CDI and add Spring utilities such as JDBC/JMS when needed Oracle: CDI in the Java EE 6 Ecosystem – 62 pages slideshow, the stuff is explained more than in the previously mentioned slideshow Note: CDI 1.1 (JSR 346, Java EE 7) should have a standard way of bootstrapping it in non-EE environment (i.e. SE) From http://theholyjava.wordpress.com/2011/11/09/what-is-cdi-how-does-it-relate-to-ejb-and-spring/

So last time we helped out justice league to effectively manager their super heroes. Today we focus on how The Avengers will be using Hibernate's Detached Criteria to find out their enemies with respect to each superhero so as to protect their super heroes. You can download the working example from here. In this example we take only two entities into consideration. The Avenger & The Villain. We build a relationship between the two using a join table. Let us have a look at the domain mappings used in this example. package com.avengers.domain; import java.io.Serializable; import java.util.ArrayList; import java.util.List; import javax.persistence.CascadeType; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.FetchType; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.JoinColumn; import javax.persistence.JoinTable; import javax.persistence.OneToMany; import javax.persistence.Table; import org.hibernate.annotations.Type; /** * The domain class representing each member of the avengers * * @author Dinuka.Arseculeratne * */ @Entity @Table(name = "Avengers") public class Avenger implements Serializable { /** * The primary key of the Avenger table */ @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "avenger_id") private Long avengerId; /** * The name of the avenger member */ @Column(name = "avenger_name") private String avengerName; /** * A flag which holds whether the avenger's powers are awesome */ @Type(type = "yes_no") @Column(name = "is_awesome") private boolean isAwesome; /** * The list of enemies the avenger has */ @OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY) @JoinTable(name = "AVENGERS_AND_VILLAINS", joinColumns = { @JoinColumn(name = "avenger_id") }, inverseJoinColumns = { @JoinColumn(name = "villain_id") }) private List enemyList = new ArrayList(); public Long getAvengerId() { return avengerId; } public void setAvengerId(Long avengerId) { this.avengerId = avengerId; } public String getAvengerName() { return avengerName; } public void setAvengerName(String avengerName) { this.avengerName = avengerName; } public boolean isAwesome() { return isAwesome; } public void setAwesome(boolean isAwesome) { this.isAwesome = isAwesome; } public List getEnemyList() { return enemyList; } public void addEnemy(Villain enemy) { enemyList.add(enemy); } @Override public String toString() { return "Avenger [avengerId=" + avengerId + ", avengerName=" + avengerName + ", isAwesome=" + isAwesome + ", enemyList=" + enemyList + "]"; } } This class maps an avenger. I have used minimal fields in order to keep this example as simple and short as possible. And the Villain domain looks as follows; package com.avengers.domain; import java.io.Serializable; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.GenerationType; import javax.persistence.Id; import javax.persistence.Table; import org.hibernate.annotations.Type; /** * This class represents the Villain forces against the avengers * * @author Dinuka.Arseculeratne * */ @Entity @Table(name = "Villains") public class Villain implements Serializable { /** * The primary key of the Enemy table */ @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "villain_id") private Long villaiId; /** * The name of the enemy */ @Column(name = "villain_name") private String villainName; /** * A flag which checks whether the villain is super awesome */ @Type(type = "yes_no") @Column(name = "is_awesome") private boolean isAwesome; public Long getVillaidId() { return villaiId; } public void setVillaidId(Long villaidId) { this.villaiId = villaidId; } public String getVillainName() { return villainName; } public void setVillainName(String villainName) { this.villainName = villainName; } public boolean isAwesome() { return isAwesome; } public void setAwesome(boolean isAwesome) { this.isAwesome = isAwesome; } @Override public String toString() { return "Villain [villaiId=" + villaiId + ", villainName=" + villainName + ", isAwesome=" + isAwesome + "]"; } } Ok now that we have defined our domains, let us see how data retrieval happens with DetachedCriteria. I have used DetachedCriteria here because The Avengers were very specific and said they do not want anything to do with the Hibernate session, so hence i used DetachedCriteria which does not require a hibernate session to be present. Our primary objective is to retrieve The Avenger to whom a villain belongs to. Note that this assumes the same villain cannot be a villain to more than one superhero. So moving on i give below the method that retrieves an avenger based on the villain name passed. public Avenger retrieveAvengerByVillainName(String villainName) { Avenger avenger = null; /** * Selected a detached criteria so we do not need a session to run it * within. */ DetachedCriteria criteria = DetachedCriteria.forClass(Avenger.class); /** * Here we are doing an inner join with the Villain table in order to do * a name comparison with the villainName passed in as a method * parameter */ DetachedCriteria villainCriteria = criteria.createCriteria("enemyList"); villainCriteria.add(Restrictions.eq("villainName", villainName)); villainCriteria.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY); @SuppressWarnings("unchecked") List avengerList = getHibernateTemplate().findByCriteria( criteria); if (!avengerList.isEmpty()) { avenger = avengerList.get(0); getHibernateTemplate().initialize(avenger.getEnemyList()); } return avenger; } In this method what we do is first we create a criteria for our master class which in this case is Avenger.class. Then we need to do a join with the Villain table and hence we create a sub criteria from our main criteria with the name of the list we defined within the Avenger domain class. Then it is a matter of equaling the property of the Villain domain to the villain name passed in. The power of the Criteria API is such that you can create dynamic queries with ease which would be a hassle if we were to use pure HQL which would require substantial string concatenations in order to achieve the same. A sample test class called AvengerTest.java is given with the attachment which you can find at the top of the most. Note that you need to remove the comment on avenger-context.xml in order to create the tables needed for this example. So that is about it. The Avengers can be risk averse now that they have a system by which they can relate any super villain to a super hero within their league. As always your comments and suggestions are always welcome and appreciated. Thank you for taking the time to read!!!! From http://dinukaroshan.blogspot.com/2011/11/hibernate-by-example-part-2.html

1. Overview This is the third of a series of articles about setting up a secure RESTful Web Service using Spring 3.1 and Spring Security 3.1 with Java based configuration. This article will focus on the security configuration using Spring Security 3.1, assuming some understanding of Spring Security basics and focusing on the specifics of securing the RESTful web service. The REST with Spring series: Part 1 – Bootstrapping a web application with Spring 3.1 and Java based Configuration Part 2 – Building a RESTful Web Service with Spring 3.1 and Java based Configuration Part 4 – RESTful Web Service Discoverability Part 5 – REST Service Discoverability with Spring Part 6 – Basic and Digest authentication for a RESTful Service with Spring Security 3.1 Part 7 – REST Pagination in Spring Part 8 – Authentication against a RESTful Service with Spring Security Part 9 – ETags for REST with Spring 2. Introducing Spring Security in the web.xml The architecture of Spring Security is based entirely on servlet filters and, as such, comes before Spring MVC in regards to the processing of HTTP requests. Keeping this in mind, to begin with, a filter needs to be declared in the web.xml of the application: springSecurityFilterChain org.springframework.web.filter.DelegatingFilterProxy springSecurityFilterChain /* The filter must necessarily be named ‘springSecurityFilterChain’ to match the default bean created by Spring Security in the container. Note that the defined filter is not the actual class implementing the security logic but a DelegatingFilterProxy with the purpose of delegating the Filter’s methods to an internal bean. This is done so that the target bean can still benefit from the Spring context lifecycle and flexibility. The URL pattern used to configure the Filter is /* even though the entire web service is mapped to /api/* so that the security configuration has the option to secure other possible mappings as well, if required. 3. The security configuration Most of the configuration is done using the security namespace – for this to be enabled, the schema locations must be defined and pointed to the new 3.1 versions. The namespace is designed so that it expresses the common uses of Spring Security while still providing hooks to the underlying beans. 3.1. The basics The element is the main container element for HTTP security configuration. In the current implementation, it only secured a single mapping: /api/admin/**. Note that the mapping is relative to the root context of the web application, not to the rest servlet; this is because the entire security configuration lives in the root Spring context and not in the child context of the servlet. 3.2. The entry point In a standard web application, the authentication process may be automatically triggered when the client tries to access a secured resource without being authenticated – this is usually done by redirecting to a login page so that the user can enter credentials. However, for a RESTful Web Service this behavior doesn’t make much sense – authentication should only be done by a request to the correct URI and all other requests should simply fail with a 401 UNAUTHORIZED status code if the user is not authenticated. Spring Security handles this automatic triggering of the authentication process with the concept of an entry point; the entry point is a required part of the configuration, and can be injected via the entry-point-ref attribute of the element. Keeping in mind that this functionality doesn’t make sense in the context of the RESTful web service, the new custom entry point is defined: @Component( "restAuthenticationEntryPoint" ) public final class RestAuthenticationEntryPoint implements AuthenticationEntryPoint{ @Override public final void commence ( HttpServletRequest request, HttpServletResponse response, AuthenticationException authException ) throws IOException{ response.sendError( HttpServletResponse.SC_UNAUTHORIZED, "Unauthorized" ); } } 3.3. The login There are multiple ways to do authentication for a RESTful Web Service – one of the default Spring Security provides is form login – which uses an authentication processing filter – org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter. Since the element doesn’t automatically create this particular filter by default, it needs to be explicitly specified in the configuration, using the element at the position FORM_LOGIN_FILTER; the only required dependency for this bean is the authentication manager. Note that for a standard web application, the auto-configattribute of the element is shorthand syntax for some useful security configuration. While this may be appropriate for some very simple configurations, it doesn’t fit and should not be used for a REST API. 3.4. Authentication should return 200 instead of 301 By default, form login will answer a successful authentication request with a 301 MOVED PERMANENTLY status code; this makes sense in the context of an actual login form which needs to redirect after login. For a RESTful web service however, the desired response for a successful authentication should be 200 OK. This is done by injecting a custom authentication success handler in the form login filter, to replace the default one. The new handler implements the exact same login as the default org.springframework.security.web.authentication.SavedRequestAwareAuthenticationSuccessHandler with one notable difference – the redirect logic is removed: // Use the DefaultSavedRequest URL // final String targetUrl = savedRequest.getRedirectUrl(); // this.logger.debug( "Redirecting to DefaultSavedRequest Url: " + targetUrl ); // this.getRedirectStrategy().sendRedirect( request, response, targetUrl ); 3.5. The authentication manager and provider The authentication process uses an in-memory provider to perform authentication – this is meant to simplify the configuration as a production implementation of these artifacts is outside the scope of this post. 4. Maven and other trouble In addition to the pom.xml from the first post, as well as the one from the second post, the Spring Security maven dependencies need to be added: org.springframework.security spring-security-web ${spring-security.version} org.springframework.security spring-security-config ${spring-security.version} org.springframework spring-tx ${spring.version} org.springframework spring-aop ${spring.version} 3.2.2.RELEASE 3.1.3.RELEASE Notice that the version of the security dependencies is no longer 3.1.0.BUILD-SNAPSHOT as for the standard Spring framework, but 3.1.0.CI-SNAPSHOT. What is more, the security artifacts define dependencies to the 3.0.x versions of Spring, more specifically spring-security-web depends on spring-aop and on spring-tx version 3.0.x instead of the expected 3.1.x. To understand why this is a problem, we need to understand how the Maven conflict resolution algorithm works – in case of conflict, Maven will chose which jar to include based on the distance between the particular dependency and the root of the tree. In our case, the conflicts are the spring-aop and spring-tx jars, appearing once with version 3.0.6 and once with 3.1.0. In the case of spring-aop, it appears once as a level 1 dependency of both spring-security-web and spring-security-config with version 3.0.6 , and once as a level 2 dependency of spring-webmvc with the version 3.1.0; since the 3.0.6 versioned jar is closer to the root, it will be the one chosen by the conflict resolution mechanism. Now that we understand why it is that Maven will deploy the 3.0.6 version of the jars with the application and not the intended 3.1.0 version, we need to address the issue. The solution is to add the two dependencies, with the intended 3.1.0 versions, directly into the pom – this will shorten the distance between them and the root to 0 and will force Maven to use them first. 5. Conclusion This post covered the basic security configuration and implementation for a RESTful service using Spring Security 3.1, discussing the web.xml, the security configuration, the HTTP status codes for the authentication process and the Maven resolution of the security artifacts. In the next articles I will focus on a Java based configuration for Spring Security, integration testing of the secure API using the rest-assured library and HTTP basic authentication. In the meantime, check out the github project. If you read this far, you should follow me on twitter here. Original at Securing a RESTful Web Service with Spring Security 3.1 from the REST with Spring series.

1. Overview This is the second of a series of posts about setting up a RESTful web service using Spring 3.1 with Java based configuration. The first post of the series focused on bootstrapping the web application; this post will focus on setting up REST in Spring, the Controller and HTTP response codes, configuration of payload marshalling and content negotiation. 2. Understanding REST in Spring The Spring framework supports 2 ways of creating RESTful services: using MVC with ModelAndView using HTTP message converters The ModelAndView approach is older and much better documented, but also more verbose and configuration heavy. It tries to shoehorn the REST paradigm into the old model, which is not without problems. The Spring team understood this and provided first-class REST support starting with Spring 3.0. The new approach, based on HttpMessageConverter and annotations, is much more lightweight and easy to implement. Configuration is minimal and it provides sensible defaults for what you would expect from a RESTful service. It is however newer and a a bit on the light side concerning documentation; what’s more, the Spring reference doesn’t go out of it’s way to make the distinction and the tradeoffs between the two approaches as clear as they should be. Nevertheless, this is the way RESTful services should be build after Spring 3.0. 3. The Java configuration @Configuration @EnableWebMvc public class WebConfig{ // } The new @EnableWebMvc annotation does a number of useful things – specifically, in the case of REST, it detect the existence of Jackson and JAXB 2 on the classpath and automatically creates and registers default JSON and XML converters. The functionality of the annotation is equivalent to the XML version: This is a shortcut, and though it may be useful in many situations, it’s not perfect. When more complex configuration is needed, remove the annotation and extend WebMvcConfigurationSupport directly. 4. Testing the Spring context Starting with Spring 3.1, we get first-class testing support for @Configuration classes: @RunWith( SpringJUnit4ClassRunner.class ) @ContextConfiguration( classes = { ApplicationConfig.class, PersistenceConfig.class },loader = AnnotationConfigContextLoader.class ) public class SpringTest{ @Test public void whenSpringContextIsInstantiated_thenNoExceptions(){ // When } } The Java configuration classes are simply specified with the @ContextConfiguration annotation and the new AnnotationConfigContextLoader loads the bean definitions from the @Configuration classes. Notice that the WebConfig configuration class was not included in the test because it needs to run in a servlet context, which is not provided. 5. The Controller The @Controller is the central artifact in the entire Web Tier of the RESTful API. For the purpose of this post, the controller is modeling a simple REST resource – Foo: @Controller class FooController{ @Autowired IFooService service; @RequestMapping( value = "foo",method = RequestMethod.GET ) @ResponseBody public List< Foo > getAll(){ return this.service.getAll(); } @RequestMapping( value = "foo/{id}",method = RequestMethod.GET ) @ResponseBody public Foo get( @PathVariable( "id" ) Long id ){ return RestPreconditions.checkNotNull( this.service.getById( id ) ); } @RequestMapping( value = "foo",method = RequestMethod.POST ) @ResponseStatus( HttpStatus.CREATED ) @ResponseBody public Long create( @RequestBody Foo entity ){ RestPreconditions.checkNotNullFromRequest( entity ); return this.service.create( entity ); } @RequestMapping( value = "foo",method = RequestMethod.PUT ) @ResponseStatus( HttpStatus.OK ) public void update( @RequestBody Foo entity ){ RestPreconditions.checkNotNullFromRequest( entity ); RestPreconditions.checkNotNull( this.service.getById( entity.getId() ) ); this.service.update( entity ); } @RequestMapping( value = "foo/{id}",method = RequestMethod.DELETE ) @ResponseStatus( HttpStatus.OK ) public void delete( @PathVariable( "id" ) Long id ){ this.service.deleteById( id ); } } The Controller implementation is non-public – this is because there is no need for it to be. Usually the controller is the last in the chain of dependencies – it receives HTTP requests from the Spring front controller (the DispathcerServlet) and simply delegate them forward to a service layer. If there is no use case where the controller has to be injected or manipulated through a direct reference, then I prefer not to declare it as public. The request mappings are straightforward – as with any Spring controller, the actual value of the mapping as well as the HTTP method are used to determine the target method for the request. @RequestBody will bind the parameters of the method to the body of the HTTP request, whereas @ResponseBody does the same for the response and return type. They also ensure that the resource will be marshalled and unmarshalled using the correct HTTP converter. Content negotiation will take place to choose which one of the active converters will be used, based mostly on the Accept header, although other HTTP headers may be used to determine the representation as well. 6. Mapping the HTTP response codes The status codes of the HTTP response are one of the most important parts of the REST service, and the subject can quickly become very complex. Getting these right can be what makes or breaks the service. 6.1. Unmapped requests If Spring MVC receives a request which doesn’t have a mapping, it considers the request not to be allowed and returns a 405 METHOD NOT ALLOWED back to the client. It is also good practice to include the Allow HTTP header when returning a 405 to the client, in order to specify which operations are allowed. This is the standard behavior of Spring MVC and does not require any additional configuration. 6.2. Valid, mapped requests For any request that does have a mapping, Spring MVC considers the request valid and responds with 200 OK if no other status code is specified otherwise. It is because of this that controller declares different @ResponseStatus for the create, update and delete actions but not for get, which should indeed return the default 200 OK. 6.3. Client error In case of a client error, custom exceptions are defined and mapped to the appropriate error codes. Simply throwing these exceptions from any of the layers of the web tier will ensure Spring maps the corresponding status code on the HTTP response. @ResponseStatus( value = HttpStatus.BAD_REQUEST ) public class BadRequestException extends RuntimeException{ // } @ResponseStatus( value = HttpStatus.NOT_FOUND ) public class ResourceNotFoundException extends RuntimeException{ // } These exceptions are part of the REST API and, as such, should only be used in the appropriate layers corresponding to REST; if for instance a DAO/DAL layer exist, it should not use the exceptions directly. Note also that these are not checked exceptions but runtime exceptions – in line with Spring practices and idioms. 6.4. Using @ExceptionHandler Another option to map custom exceptions on specific status codes is to use the @ExceptionHandler annotation in the controller. The problem with that approach is that the annotation only applies to the controller in which it is defined, not to the entire Spring Container, which means that it needs to be declared in each controller individually. This quickly becomes cumbersome, especially in more complex applications which many controllers. There are a few JIRA issues opened with Spring at this time to handle this and other related limitations: SPR-8124, SPR-7278, SPR-8406. 7. Additional Maven dependencies In addition to the pom.xml from the first post, two dependencies need to be added: org.codehaus.jackson jackson-mapper-asl ${jackson-mapper-asl.version} runtime javax.xml.bind jaxb-api ${jaxb-api.version} runtime 1.9.12 2.2.4 These are the libraries used to convert the representation of the REST resource to either JSON or XML. 8. Conclusion This post covered the configuration and implementation of a RESTful service using Spring 3.1 and Java based configuration, discussing HTTP response codes, basic content negotiation and marshaling. In the next articles of the series I will focus on discoverability of the API, advanced content negotiation and working with additional representations of a resource. In the meantime, check out the github project.

What is JIT Compiler? The Just In Time Compiler (JIT) concept and more generally adaptive optimization is well known concept in many languages besides Java (.Net, Lua, JRuby). In order to explain what is JIT Compiler I want to start with a definition of compiler concept. According to wikipedia compiler is "a computer program that transforms the source language into another computer language (the target language)". We are all familiar with static java compiler (javac) that compiles human readable .java files to a byte code that can be interpreted by JVM - .class files. Then what does JIT compile? The answer will given a moment later after explanation of what is "Just in Time". According to most researches, 80% of execution time is spent in executing 20% of code. That would be great if there was a way to determine those 20% of code and to optimize them. That's exactly what JIT does - during runtime it gathers statistics, finds the "hot" code compiles it from JVM interpreted bytecode (that is stored in .class files) to a native code that is executed directly by Operating System and heavily optimizes it. Smallest compilation unit is single method. Compilation and statistics gathering is done in parallel to program execution by special threads. During statistics gathering the compiler makes hypotheses about code function and as the time passes tries to prove or to disprove them. If the hypothesis is dis-proven the code is deoptimized and recompiled again. The name "Hotspot" of Sun (Oracle) JVM is chosen because of the ability of this Virtual Machine to find "hot" spots in code. What optimizations does JIT? Let's look closely at more optimizations done by JIT. Inline methods - instead of calling method on an instance of the object it copies the method to caller code. The hot methods should be located as close to the caller as possible to prevent any overhead. Eliminate locks if monitor is not reachable from other threads Replace interface with direct method calls for method implemented only once to eliminate calling of virtual functions overhead Join adjacent synchronized blocks on the same object Eliminate dead code Drop memory write for non-volatile variables Remove prechecking NullPointerException and IndexOutOfBoundsException Et cetera When the Java VM invokes a Java method, it uses an invoker method as specified in the method block of the loaded class object. The Java VM has several invoker methods, for example, a different invoker is used if the method is synchronized or if it is a native method. The JIT compiler uses its own invoker. Sun production releases check the method access bit for value ACC_MACHINE_COMPILED to notify the interpreter that the code for this method has already been compiled and stored in the loaded class. JIT compiler compiles the method block into native code for this method and stores that in the code block for that method. Once the code has been compiled the ACC_MACHINE_COMPILED bit, which is used on the Sun platform, is set. How do we know what JIT is doing in our program and how can it be controlled? First of all to disable JIT Djava.compiler=NONE parameter can be used. There are 2 types of JIT compilers in Hotspot - one is used for client program and one for server (-server option in VM parameters). Program, running on server enjoys usually from more resources than program running on client and to server program top throughput is usually more important. Hence JIT in server is more resource consuming and gathering statistics takes more time to make the statistics more accurate. For client program gathering statics for a method lasts 1500 method calls, for server 15000. These default values can be changed by -XX:CompileThreshold=XXX VM parameter. In order to find out whether default value is good for you try enabling "XX:+PrintCompilation" and "-XX:-CITime" parameters that print JIT statistics and time CPU spent by JIT. Benchmarks Most of the benchmarks show that JITed code runs 10 to 20 times faster than interpreted code. There are many benchmarks done. Below given result graphs of two of them: Its worth to mention that programs that run in JIT mode, but are still in "learning mode" run much slower than non JITed programs. Drawbacks of JIT JIT Increases level of unpredictability and complexity in Java program. It adds another layer that developers don't really understand. Example of possible bugs - 'happens before relations" in concurrency. JIT can easily reorder code if the change is safe for a program running in single thread. To solve this problem developers make hints to JIT using "synchronized" word or explicit locking. Increases non heap memory footprint - JITed code is stored in "Code Cache" generation. Advanced JIT JIT and garbage collection. For GC to occur program must reach safe points. For this purpose JIT injects yieldpoints at regular intervals in native code. In addition to scanning of stack to find root references, registers must be scanned as they may hold objects created by JIT Comments are appresiated. The article can be found also at: artiomg.blogspot.com/2011/10/just-in-time-compiler-jit-in-hotspot.html

How often have we all heard that “batching” will increase latency? As someone with a passion for low-latency systems this surprises me. In my experience when batching is done correctly, not only does it increase throughput, it can also reduce average latency and keep it consistent. Well then, how can batching magically reduce latency? It comes down to what algorithm and data structures are employed. In a distributed environment we are often having to batch up messages/events into network packets to achieve greater throughput. We also employ similar techniques in buffering writes to storage to reduce the number of IOPS. That storage could be a block device backed file-system or a relational database. Most IO devices can only handle a modest number of IO operations per second, so it is best to fill those operations efficiently. Many approaches to batching involve waiting for a timeout to occur and this will by its very nature increase latency. The batch can also get filled before the timeout occurs making the latency even more unpredictable. Figure 1. Figure 1. above depicts decoupling the access to an IO device, and therefore the contention for access to it, by introducing a queue like structure to stage the messages/events to be sent and a thread doing the batching for writing to the device. The Algorithm An approach to batching uses the following algorithm in Java pseudo code: public final class NetworkBatcher implements Runnable { private final NetworkFacade network; private final Queue queue; private final ByteBuffer buffer; public NetworkBatcher(final NetworkFacade network, final int maxPacketSize, final Queue queue) { this.network = network; buffer = ByteBuffer.allocate(maxPacketSize); this.queue = queue; } @Override public void run() { while (!Thread.currentThread().isInterrupted()) { while (null == queue.peek()) { employWaitStrategy(); // block, spin, yield, etc. } Message msg; while (null != (msg = queue.poll())) { if (msg.size() > buffer.remaining()) { sendBuffer(); } buffer.put(msg.getBytes()); } sendBuffer(); } } private void sendBuffer() { buffer.flip(); network.send(buffer); buffer.clear(); } } Basically, wait for data to become available and as soon as it is, send it right away. While sending a previous message or waiting on new messages, a burst of traffic may arrive which can all be sent in a batch, up to the size of the buffer, to the underlying resource. This approach can use ConcurrentLinkedQueue which provides low-latency and avoid locks. However it has an issue in not creating back pressure to stall producing/publishing threads if they are outpacing the batcher whereby the queue could grow out of control because it is unbounded. I’ve often had to wrap ConcurrentLinkedQueue to track its size and thus create back pressure. This size tracking can add 50% to the processing cost of using this queue in my experience. This algorithm respects the single writer principle and can often be employed when writing to a network or storage device, and thus avoid lock contention in third party API libraries. By avoiding the contention we avoid the J-Curve latency profile normally associated with contention on resources, due to the queuing effect on locks. With this algorithm, as load increases, latency stays constant until the underlying device is saturated with traffic resulting in a more "bathtub" profile than the J-Curve. Let’s take a worked example of handling 10 messages that arrive as a burst of traffic. In most systems traffic comes in bursts and is seldom uniformly spaced out in time. One approach will assume no batching and the threads write to device API directly as in Figure 1. above. The other will use a lock free data structure to collect the messages plus a single thread consuming messages in a loop as per the algorithm above. For the example let’s assume it takes 100µs to write a single buffer to the network device as a synchronous operation and have it acknowledged. The buffer will ideally be less than the MTU of the network in size when latency is critical. Many network sub-systems are asynchronous and support pipelining but we will make the above assumption to clarify the example. If the network operation is using a protocol like HTTP under REST or Web Services then this assumption matches the underlying implementation. Best (µs) Average (µs) Worst (µs) Packets Sent Serial 100 500 1,000 10 Smart Batching 100 150 200 1-2 The absolute lowest latency will be achieved if a message is sent from the thread originating the data directly to the resource, if the resource is un-contended. The table above shows what happens when contention occurs and a queuing effect kicks in. With the serial approach 10 individual packets will have to be sent and these typically need to queue on a lock managing access to the resource, therefore they get processed sequentially. The above figures assume the locking strategy works perfectly with no perceivable overhead which is unlikely in a real application. For the batching solution it is likely all 10 packets will be picked up in first batch if the concurrent queue is efficient, thus giving the best case latency scenario. In the worst case only one message is sent in the first batch with the other nine following in the next. Therefore in the worst case scenario one message has a latency of 100µs and the following 9 have a latency of 200µs thus giving a worst case average of 190µs which is significantly better than the serial approach. This is one good example when the simplest solution is just a bit too simple because of the contention. The batching solution helps achieve consistent low-latency under burst conditions and is best for throughput. It also has a nice effect across the network on the receiving end in that the receiver has to process fewer packets and therefore makes the communication more efficient both ends. Most hardware handles data in buffers up to a fixed size for efficiency. For a storage device this will typically be a 4KB block. For networks this will be the MTU and is typically 1500 bytes for Ethernet. When batching, it is best to understand the underlying hardware and write batches down in ideal buffer size to be optimally efficient. However keep in mind that some devices need to envelope the data, e.g. the Ethernet and IP headers for network packets so the buffer needs to allow for this. There will always be an increased latency from a thread switch and the cost of exchange via the data structure. However there are a number of very good non-blocking structures available using lock-free techniques. For the Disruptor this type of exchange can be achieved in as little as 50-100ns thus making the choice of taking the smart batching approach a no brainer for low-latency or high-throughput distributed systems. This technique can be employed for many problems and not just IO. The core of the Disruptor uses this technique to help rebalance the system when the publishers burst and outpace the EventProcessors. The algorithm can be seen inside the BatchEventProcessor. Note: For this algorithm to work the queueing structure must handle the contention better than the underlying resource. Many queue implementations are extremely poor at managing contention. Use science and measure before coming to a conclusion. Batching with the Disruptor The code below shows the same algorithm in action using the Disruptor's EventHandler mechanism. In my experience, this is a very effective technique for handling any IO device efficiently and keeping latency low when dealing with load or burst traffic. public final class NetworkBatchHandler implements EventHander { private final NetworkFacade network; private final ByteBuffer buffer; public NetworkBatchHandler(final NetworkFacade network, final int maxPacketSize) { this.network = network; buffer = ByteBuffer.allocate(maxPacketSize); } public void onEvent(Message msg, long sequence, boolean endOfBatch) throws Exception { if (msg.size() > buffer.remaining()) { sendBuffer(); } buffer.put(msg.getBytes()); if (endOfBatch) { sendBuffer(); } } private void sendBuffer() { buffer.flip(); network.send(buffer); buffer.clear(); } } The endOfBatch parameter greatly simplifies the handling of the batch compared to the double loop in the algorithm above. I have simplified the examples to illustrate the algorithm. Clearly error handling and other edge conditions need to be considered. Separation of IO from Work Processing There is another very good reason to separate the IO from the threads doing the work processing. Handing off the IO to another thread means the worker thread, or threads, can continue processing without blocking in a nice cache friendly manner. I've found this to be critical in achieving high-performance throughput. If the underlying IO device or resource becomes briefly saturated then the messages can be queued for the batcher thread allowing the work processing threads to continue. The batching thread then feeds the messages to the IO device in the most efficient way possible allowing the data structure to handle the burst and if full apply the necessary back pressure, thus providing a good separation of concerns in the workflow. Conclusion So there you have it. Smart Batching can be employed in concert with the appropriate data structures to achieve consistent low-latency and maximum throughput. From http://mechanical-sympathy.blogspot.com/2011/10/smart-batching.html

as a user interface fan, i could not miss the development with html 5. so the goal of this post is to walk through a graphic application that uses javascript and html 5. we will see through examples one way (among others) to develop this kind of project. application overview tools the html 5 page data gathering cards loading & cache handling cards display mouse management state storage animations handling multi-devices conclusion to go further application overview we will produce an application that will let us display a magic the gathering ©(courtesy of www.wizards.com/magic ) cards collection. users will be able to scroll and zoom using the mouse (like bing maps, for example). you can see the final result here: http://bolaslenses.catuhe.com the project source files can be downloaded here: http://www.catuhe.com/msdn/bolaslenses.zip cards are stored on windows azure storage and use the azure content distribution network ( cdn : a service that deploys data near the final users) in order to achieve maximum performances. an asp.net service is used to return cards list (using json format). tools to write our application, we will use visual studio 2010 sp1 with web standards update . this extension adds intellisense support in html 5 page (which is a really important thing ). so, our solution will contain an html 5 page side by side with .js files (these files will contain javascript scripts). about debug, it is possible to set a breakpoint directly in the .js files under visual studio. it is also possible to use the developer bar of internet explorer 9 (use f12 key to display it). debug with visual studio 2010 debug with internet explorer 9 (f12/developer bar) so, we have a modern developer environment with intellisense and debug support. therefore, we are ready to start and first of all, we will write the html 5 page. the html 5 page our page will be built around an html 5 canvas which will be used to draw the cards: cards scanned by mwshq team magic the gathering official site : http://www.wizards.com/magic bolas lenses your browser does not support html5 canvas. loading data... if we dissect this page, we can note that it is divided into two parts: the header part with the title, the logo and the special mentions the main part (section) holds the canvas and the tooltips that will display the status of the application. there is also a hidden image ( backimage ) used as source for not yet loaded cards. to build the layout of the page, a style sheet ( full.css ) is applied. style sheets are a mechanism used to change the tags styles (in html, a style defines the entire display options for a tag): html, body { height: 100%; } body { background-color: #888888; font-size: .85em; font-family: "segoe ui, trebuchet ms" , verdana, helvetica, sans-serif; margin: 0; padding: 0; color: #696969; } a:link { color: #034af3; text-decoration: underline; } a:visited { color: #505abc; } a:hover { color: #1d60ff; text-decoration: none; } a:active { color: #12eb87; } header, footer, nav, section { display: block; } table { width: 100%; } header, #header { position: relative; margin-bottom: 0px; color: #000; padding: 0; } #title { font-weight: bold; color: #fff; border: none; font-size: 60px !important; vertical-align: middle; margin-left: 70px } #legal { text-align: right; color: white; font-size: 14px; width: 50%; position: absolute; top: 15px; right: 10px } #leftheader { width: 50%; vertical-align: middle; } section { margin: 20px 20px 20px 20px; } #maincanvas{ border: 4px solid #000000; } #cardscount { font-weight: bolder; font-size: 1.1em; } .tooltip { position: absolute; bottom: 5px; color: black; background-color: white; margin-right: auto; margin-left: auto; left: 35%; right: 35%; padding: 5px; width: 30%; text-align: center; border-radius: 10px; -webkit-border-radius: 10px; -moz-border-radius: 10px; box-shadow: 2px 2px 2px #333333; } #bolaslogo { width: 64px; height: 64px; } #picturecell { float: left; width: 64px; margin: 5px 5px 5px 5px; vertical-align: middle; } thus, this sheet is responsible for setting up the following display: style sheets are powerful tools that allow an infinite number of displays. however, they are sometimes complicated to setup (for example if a tag is affected by a class, an identifier and its container). to simplify this setup, the development bar of internet explorer 9 is particularly useful because we can use it to see styles hierarchy that is applied to a tag. for example let’s take a look at the waittext tooltip with the development bar. to do this, you must press f12 in internet explorer and use the selector to choose the tooltip: once the selection is done, we can see the styles hierarchy: thus, we can see that our div received its styles from the body tag and the . tooltip entry of the style sheet. with this tool, it becomes possible to see the effect of each style (which can be disabled). it is also possible to add new style on the fly. another important point of this window is the ability to change the rendering mode of internet explorer 9. indeed, we can test how, for example, internet explorer 8 will handle the same page. to do this, go to the [ browser mode ] menu and select the engine of internet explorer 8. this change will especially impact our tooltip as it uses border-radius (rounded edge) and box-shadow that are features of css 3: internet explorer 9 internet explorer 8 our page provides a graceful degradation as it still works (with no annoying visual difference) when the browser does not support all the required technologies. now that our interface is ready, we will take a look at the data source to retrieve the cards to display. the server provides the cards list using json format on this url: http://bolaslenses.catuhe.com/ home/listofcards/?colorstring=0 it takes one parameter ( colorstring ) to select a specific color (0 = all). when developing with javascript, there is a good reflex to have (reflex also good in other languages too, but really important in javascript): one must ask whether what we want to develop has not been already done in an existing framework. indeed, there is a multitude of open source projects around javascript. one of them is jquery which provides a plethora of convenient services. thus, in our case to connect to the url of our server and get the cards list, we could go through a xmlhttprequest and have fun to parse the returned json. or we can use jquery . so we will use the getjson function which will take care of everything for us: function getlistofcards() { var url = "http://bolaslenses.catuhe.com/home/listofcards/?jsoncallback=?"; $.getjson(url, { colorstring: "0" }, function (data) { listofcards = data; $("#cardscount").text(listofcards.length + " cards displayed"); $("#waittext").slidetoggle("fast"); }); } as we can see, our function stores the cards list in the listofcards variable and calls two jquery functions: text that change the text of a tag slidetoggle that hides (or shows) a tag by animating its height the listofcards list contains objects whose format is: id : unique identifier of the card path : relative path of the card (without the extension) it should be noted that the url of the server is called with the “ ?jsoncallback=? ” suffix. indeed, ajax calls are constrained in terms of security to connect only to the same address as the calling script. however, there is a solution called jsonp that will allow us to make a concerted call to the server (which of course must be aware of the operation). and fortunately, jquery can handle it all alone by just adding the right suffix. once we have our cards list, we can set up the pictures loading and caching. cards loading & cache handling the main trick of our application is to draw only the cards effectively visible on the screen. the display window is defined by a zoom level and an offset (x, y) in the overall system. var visucontrol = { zoom : 0.25, offsetx : 0, offsety : 0 }; the overall system is defined by 14819 cards that are spread over 200 columns and 75 rows. also, we must be aware that each card is available in three versions: high definition: 480x680 without compression (.jpg suffix) medium definition: 240x340 with standard compression (.50.jpg suffix) low definition: 120x170 with strong compression (.25.jpg suffix) thus, depending on the zoom level, we will load the correct version to optimize networks transfer. to do this we will develop a function that will give an image for a given card. this function will be configured to download a certain level of quality. in addition it will be linked with lower quality level to return it if the card for the current level is not yet uploaded: function imagecache(substr, replacementcache) { var extension = substr; var backimage = document.getelementbyid("backimage"); this.load = function (card) { var localcache = this; if (this[card.id] != undefined) return; var img = new image(); localcache[card.id] = { image: img, isloaded: false }; currentdownloads++; img.onload = function () { localcache[card.id].isloaded = true; currentdownloads--; }; img.onerror = function() { currentdownloads--; }; img.src = "http://az30809.vo.msecnd.net/" + card.path + extension; }; this.getreplacementfromlowercache = function (card) { if (replacementcache == undefined) return backimage; return replacementcache.getimageforcard(card); }; this.getimageforcard = function(card) { var img; if (this[card.id] == undefined) { this.load(card); img = this.getreplacementfromlowercache(card); } else { if (this[card.id].isloaded) img = this[card.id].image; else img = this.getreplacementfromlowercache(card); } return img; }; } an imagecache is built by giving the associated suffix and the underlying cache. here you can see two important functions: load : this function will load the right picture and will store it in a cache (the msecnd.net url is the azure cdn address of the cards) getimageforcard : this function returns the card picture from the cache if already loaded. otherwise it requests the underlying cache to return its version (and so on) so to handle our 3 levels of caches, we have to declare three variables: var imagescache25 = new imagecache(".25.jpg"); var imagescache50 = new imagecache(".50.jpg", imagescache25); var imagescachefull = new imagecache(".jpg", imagescache50); selecting the right cover is only depending on zoom: function getcorrectimagecache() { if (visucontrol.zoom <= 0.25) return imagescache25; if (visucontrol.zoom <= 0.8) return imagescache50; return imagescachefull; } to give a feedback to the user, we will add a timer that will manage a tooltip that indicates the number of images currently loaded: function updatestats() { var stats = $("#stats"); stats.html(currentdownloads + " card(s) currently downloaded."); if (currentdownloads == 0 && statsvisible) { statsvisible = false; stats.slidetoggle("fast"); } else if (currentdownloads > 1 && !statsvisible) { statsvisible = true; stats.slidetoggle("fast"); } } setinterval(updatestats, 200); again we note the use of jquery to simplify animations. we will now discuss the display of cards. cards display to draw our cards, we need to actually fill the canvas using its 2d context (which exists only if the browser supports html 5 canvas): var maincanvas = document.getelementbyid("maincanvas"); var drawingcontext = maincanvas.getcontext('2d'); the drawing will be made by processlistofcards function (called 60 times per second): function processlistofcards() { if (listofcards == undefined) { drawwaitmessage(); return; } maincanvas.width = document.getelementbyid("center").clientwidth; maincanvas.height = document.getelementbyid("center").clientheight; totalcards = listofcards.length; var localcardwidth = cardwidth * visucontrol.zoom; var localcardheight = cardheight * visucontrol.zoom; var effectivetotalcardsinwidth = colscount * localcardwidth; var rowscount = math.ceil(totalcards / colscount); var effectivetotalcardsinheight = rowscount * localcardheight; initialx = (maincanvas.width - effectivetotalcardsinwidth) / 2.0 - localcardwidth / 2.0; initialy = (maincanvas.height - effectivetotalcardsinheight) / 2.0 - localcardheight / 2.0; // clear clearcanvas(); // computing of the viewing area var initialoffsetx = initialx + visucontrol.offsetx * visucontrol.zoom; var initialoffsety = initialy + visucontrol.offsety * visucontrol.zoom; var startx = math.max(math.floor(-initialoffsetx / localcardwidth) - 1, 0); var starty = math.max(math.floor(-initialoffsety / localcardheight) - 1, 0); var endx = math.min(startx + math.floor((maincanvas.width - initialoffsetx - startx * localcardwidth) / localcardwidth) + 1, colscount); var endy = math.min(starty + math.floor((maincanvas.height - initialoffsety - starty * localcardheight) / localcardheight) + 1, rowscount); // getting current cache var imagecache = getcorrectimagecache(); // render for (var y = starty; y < endy; y++) { for (var x = startx; x < endx; x++) { var localx = x * localcardwidth + initialoffsetx; var localy = y * localcardheight + initialoffsety; // clip if (localx > maincanvas.width) continue; if (localy > maincanvas.height) continue; if (localx + localcardwidth < 0) continue; if (localy + localcardheight < 0) continue; var card = listofcards[x + y * colscount]; if (card == undefined) continue; // get from cache var img = imagecache.getimageforcard(card); // render try { if (img != undefined) drawingcontext.drawimage(img, localx, localy, localcardwidth, localcardheight); } catch (e) { $.grep(listofcards, function (item) { return item.image != img; }); } } }; // scroll bars drawscrollbars(effectivetotalcardsinwidth, effectivetotalcardsinheight, initialoffsetx, initialoffsety); // fps computefps(); } this function is built around many key points: if the cards list is not yet loaded, we display a tooltip indicating that download is in progress: var pointcount = 0; function drawwaitmessage() { pointcount++; if (pointcount > 200) pointcount = 0; var points = ""; for (var index = 0; index < pointcount / 10; index++) points += "."; $("#waittext").html("loading...please wait" + points); subsequently, we define the position of the display window (in terms of cards and coordinates), then we proceed to clean the canvas: function clearcanvas() { maincanvas.width = document.body.clientwidth - 50; maincanvas.height = document.body.clientheight - 140; drawingcontext.fillstyle = "rgb(0, 0, 0)"; drawingcontext.fillrect(0, 0, maincanvas.width, maincanvas.height); } then we browse the cards list and call the drawimage function of the canvas context. the current image is provided by the active cache (depending on the zoom): // get from cache var img = imagecache.getimageforcard(card); // render try { if (img != undefined) drawingcontext.drawimage(img, localx, localy, localcardwidth, localcardheight); } catch (e) { $.grep(listofcards, function (item) { return item.image != img; }); we also have to draw the scroll bar with the roundedrectangle function that uses quadratic curves: function roundedrectangle(x, y, width, height, radius) { drawingcontext.beginpath(); drawingcontext.moveto(x + radius, y); drawingcontext.lineto(x + width - radius, y); drawingcontext.quadraticcurveto(x + width, y, x + width, y + radius); drawingcontext.lineto(x + width, y + height - radius); drawingcontext.quadraticcurveto(x + width, y + height, x + width - radius, y + height); drawingcontext.lineto(x + radius, y + height); drawingcontext.quadraticcurveto(x, y + height, x, y + height - radius); drawingcontext.lineto(x, y + radius); drawingcontext.quadraticcurveto(x, y, x + radius, y); drawingcontext.closepath(); drawingcontext.stroke(); drawingcontext.fill(); } function drawscrollbars(effectivetotalcardsinwidth, effectivetotalcardsinheight, initialoffsetx, initialoffsety) { drawingcontext.fillstyle = "rgba(255, 255, 255, 0.6)"; drawingcontext.linewidth = 2; // vertical var totalscrollheight = effectivetotalcardsinheight + maincanvas.height; var scaleheight = maincanvas.height - 20; var scrollheight = maincanvas.height / totalscrollheight; var scrollstarty = (-initialoffsety + maincanvas.height * 0.5) / totalscrollheight; roundedrectangle(maincanvas.width - 8, scrollstarty * scaleheight + 10, 5, scrollheight * scaleheight, 4); // horizontal var totalscrollwidth = effectivetotalcardsinwidth + maincanvas.width; var scalewidth = maincanvas.width - 20; var scrollwidth = maincanvas.width / totalscrollwidth; var scrollstartx = (-initialoffsetx + maincanvas.width * 0.5) / totalscrollwidth; roundedrectangle(scrollstartx * scalewidth + 10, maincanvas.height - 8, scrollwidth * scalewidth, 5, 4); } and finally, we need to compute the number of frames per second: function computefps() { if (previous.length > 60) { previous.splice(0, 1); } var start = (new date).gettime(); previous.push(start); var sum = 0; for (var id = 0; id < previous.length - 1; id++) { sum += previous[id + 1] - previous[id]; } var diff = 1000.0 / (sum / previous.length); $("#cardscount").text(diff.tofixed() + " fps. " + listofcards.length + " cards displayed"); } drawing cards relies heavily on the browser's ability to speed up canvas rendering. for the record, here are the performances on my machine with the minimum zoom level (0.05): browser fps internet explorer 9 30 firefox 5 30 chrome 12 17 ipad (with a zoom level of 0.8) 7 windows phone mango (with a zoom level of 0.8) 20 (!!) the site even works on mobile phones and tablets as long as they support html 5. here we can see the inner power of html 5 browsers that can handle a full screen of cards more than 30 times per second! mouse management to browse our cards collection, we have to manage the mouse (including its wheel). for the scrolling, we'll just handle the onmouvemove , onmouseup and onmousedown events. onmouseup and onmousedown events will be used to detect if the mouse is clicked or not: var mousedown = 0; document.body.onmousedown = function (e) { mousedown = 1; getmouseposition(e); previousx = posx; previousy = posy; }; document.body.onmouseup = function () { mousedown = 0; }; the onmousemove event is connected to the canvas and used to move the view: var previousx = 0; var previousy = 0; var posx = 0; var posy = 0; function getmouseposition(eventargs) { var e; if (!eventargs) e = window.event; else { e = eventargs; } if (e.offsetx || e.offsety) { posx = e.offsetx; posy = e.offsety; } else if (e.clientx || e.clienty) { posx = e.clientx; posy = e.clienty; } } function onmousemove(e) { if (!mousedown) return; getmouseposition(e); mousemovefunc(posx, posy, previousx, previousy); previousx = posx; previousy = posy; } this function (onmousemove) calculates the current position and provides also the previous value in order to move the offset of the display window: function move(posx, posy, previousx, previousy) { currentaddx = (posx - previousx) / visucontrol.zoom; currentaddy = (posy - previousy) / visucontrol.zoom; } mousehelper.registermousemove(maincanvas, move); note that jquery also provides tools to manage mouse events. for the management of the wheel, we will have to adapt to different browsers that do not behave the same way on this point: function wheel(event) { var delta = 0; if (event.wheeldelta) { delta = event.wheeldelta / 120; if (window.opera) delta = -delta; } else if (event.detail) { /** mozilla case. */ delta = -event.detail / 3; } if (delta) { wheelfunc(delta); } if (event.preventdefault) event.preventdefault(); event.returnvalue = false; } we can see that everyone does what he wants :). the function to register with this event is: mousehelper.registerwheel = function (func) { wheelfunc = func; if (window.addeventlistener) window.addeventlistener('dommousescroll', wheel, false); window.onmousewheel = document.onmousewheel = wheel; }; and we will use this function to change the zoom with the wheel: // mouse mousehelper.registerwheel(function (delta) { currentaddzoom += delta / 500.0; }); finally we will add a bit of inertia when moving the mouse (and the zoom) to give some kind of smoothness: // inertia var inertia = 0.92; var currentaddx = 0; var currentaddy = 0; var currentaddzoom = 0; function doinertia() { visucontrol.offsetx += currentaddx; visucontrol.offsety += currentaddy; visucontrol.zoom += currentaddzoom; var effectivetotalcardsinwidth = colscount * cardwidth; var rowscount = math.ceil(totalcards / colscount); var effectivetotalcardsinheight = rowscount * cardheight var maxoffsetx = effectivetotalcardsinwidth / 2.0; var maxoffsety = effectivetotalcardsinheight / 2.0; if (visucontrol.offsetx < -maxoffsetx + cardwidth) visucontrol.offsetx = -maxoffsetx + cardwidth; else if (visucontrol.offsetx > maxoffsetx) visucontrol.offsetx = maxoffsetx; if (visucontrol.offsety < -maxoffsety + cardheight) visucontrol.offsety = -maxoffsety + cardheight; else if (visucontrol.offsety > maxoffsety) visucontrol.offsety = maxoffsety; if (visucontrol.zoom < 0.05) visucontrol.zoom = 0.05; else if (visucontrol.zoom > 1) visucontrol.zoom = 1; processlistofcards(); currentaddx *= inertia; currentaddy *= inertia; currentaddzoom *= inertia; // epsilon if (math.abs(currentaddx) < 0.001) currentaddx = 0; if (math.abs(currentaddy) < 0.001) currentaddy = 0; } this kind of small function does not cost a lot to implement, but adds a lot to the quality of user experience. state storage also to provide a better user experience, we will save the display window’s position and zoom. to do this, we will use the service of localstorage (which saves pairs of keys / values for the long term (the data is retained after the browser is closed) and only accessible by the current window object): function saveconfig() { if (window.localstorage == undefined) return; // zoom window.localstorage["zoom"] = visucontrol.zoom; // offsets window.localstorage["offsetx"] = visucontrol.offsetx; window.localstorage["offsety"] = visucontrol.offsety; } // restore data if (window.localstorage != undefined) { var storedzoom = window.localstorage["zoom"]; if (storedzoom != undefined) visucontrol.zoom = parsefloat(storedzoom); var storedoffsetx = window.localstorage["offsetx"]; if (storedoffsetx != undefined) visucontrol.offsetx = parsefloat(storedoffsetx); var storedoffsety = window.localstorage["offsety"]; if (storedoffsety != undefined) visucontrol.offsety = parsefloat(storedoffsety); } animations to add even more dynamism to our application we will allow our users to double-click on a card to zoom and focus on it. our system should animate three values: the two offsets (x, y) and the zoom. to do this, we will use a function that will be responsible of animating a variable from a source value to a destination value with a given duration: var animationhelper = function (root, name) { var paramname = name; this.animate = function (current, to, duration) { var offset = (to - current); var ticks = math.floor(duration / 16); var offsetpart = offset / ticks; var tickscount = 0; var intervalid = setinterval(function () { current += offsetpart; root[paramname] = current; tickscount++; if (tickscount == ticks) { clearinterval(intervalid); root[paramname] = to; } }, 16); }; }; the use of this function is: // prepare animations parameters var zoomanimationhelper = new animationhelper(visucontrol, "zoom"); var offsetxanimationhelper = new animationhelper(visucontrol, "offsetx"); var offsetyanimationhelper = new animationhelper(visucontrol, "offsety"); var speed = 1.1 - visucontrol.zoom; zoomanimationhelper.animate(visucontrol.zoom, 1.0, 1000 * speed); offsetxanimationhelper.animate(visucontrol.offsetx, targetoffsetx, 1000 * speed); offsetyanimationhelper.animate(visucontrol.offsety, targetoffsety, 1000 * speed); the advantage of the animationhelper function is that it is able to animate as many parameters as you wish (and that only with the settimer function!) handling multi-devices finally we will ensure that our page can also be seen on tablets pc and even on phones. to do this, we will use a feature of css 3: the media-queries . with this technology, we can apply style sheets according to some queries such as a specific display size: here we see that if the screen width is less than 480 pixels, the following style sheet will be added: #legal { font-size: 8px; } #title { font-size: 30px !important; } #waittext { font-size: 12px; } #bolaslogo { width: 48px; height: 48px; } #picturecell { width: 48px; } finally we will ensure that our page can also be seen on tablets pc and even on phones. to do this, we will use a feature of css 3: #legal { font-size: 8px; } #title { font-size: 30px !important; } #waittext { font-size: 12px; } #bolaslogo { width: 48px; height: 48px; } #picturecell { width: 48px; } conclusion html 5 / css 3 / javascript and visual studio 2010 allow to develop portable and efficient solutions (within the limits of browsers that support html 5 of course) with some great features such as hardware accelerated rendering. this kind of development is also simplified by the use of frameworks like jquery. also, i am especially fan of javascript that turns out to be a very powerful dynamic language. of course, c# or vb.net developers have to change theirs reflexes but for the development of web pages it's worth. in conclusion, i think that the best to be convinced is to try! to go further internet explorer test drive: http://ie.microsoft.com/testdrive/ internet explorer 9 guide for developer : http://msdn.microsoft.com/en-us/ie/ff468705 w3c site for html 5 : http://dev.w3.org/html5/spec/overview.html internet explorer site : http://msdn.microsoft.com/en-us/ie/aa740469 about the author david catuhe is a developer evangelist for microsoft france in charge of user experience development tools (from xaml to directx/xna and html5). he defines himself as a geek and likes coding all that refer to graphics. before working for microsoft, he founded a company that developed a realtime 3d engine written with directx ( www.vertice.fr ). source: http://blogs.msdn.com/b/eternalcoding/archive/2011/07/25/feedback-of-a-graphic-development-using-html5-amp-javascript.aspx

This tutorial will walk you throughout how to save and load an image from database (MySQL) using Hibernate. Requirements For this sampel project, we are going to use: Eclipse IDE (you can use your favorite IDE); MySQL (you can use any other database, make sure to change the column type if required); Hibernate jars and dependencies (you can download the sample project with all required jars); JUnit - for testing (jar also included in the sample project). PrintScreen When we finish implementing this sample projeto, it should look like this: Database Model Before we get started with the sample projet, we have to run this sql script into MySQL: DROP SCHEMA IF EXISTS `blog` ; CREATE SCHEMA IF NOT EXISTS `blog` DEFAULT CHARACTER SET latin1 COLLATE latin1_swedish_ci ; USE `blog` ; -- ----------------------------------------------------- -- Table `blog`.`BOOK` -- ----------------------------------------------------- DROP TABLE IF EXISTS `blog`.`BOOK` ; CREATE TABLE IF NOT EXISTS `blog`.`BOOK` ( `BOOK_ID` INT NOT NULL AUTO_INCREMENT , `BOOK_NAME` VARCHAR(45) NOT NULL , `BOOK_IMAGE` MEDIUMBLOB NOT NULL , PRIMARY KEY (`BOOK_ID`) ) ENGINE = InnoDB; This script will create a table BOOK, which we are going to use in this tutorial. Book POJO We are going to use a simple POJO in this project. A Book has an ID, a name and an image, which is represented by an array of bytes. As we are going to persist an image into the database, we have to use the BLOB type. MySQLhas some variations of BLOBs, you can check the difference between them here. In this example, we are going to use the Medium Blob, which can store L + 3 bytes, where L < 2^24. Make sure you do not forget to add the column definition on the Column annotation. package com.loiane.model; import javax.persistence.Column; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.Id; import javax.persistence.Lob; import javax.persistence.Table; @Entity @Table(name="BOOK") public class Book { @Id @GeneratedValue @Column(name="BOOK_ID") private long id; @Column(name="BOOK_NAME", nullable=false) private String name; @Lob @Column(name="BOOK_IMAGE", nullable=false, columnDefinition="mediumblob") private byte[] image; public long getId() { return id; } public void setId(long id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public byte[] getImage() { return image; } public void setImage(byte[] image) { this.image = image; } } Hibernate Config This configuration file contains the required info used to connect to the database. com.mysql.jdbc.Driver jdbc:mysql://localhost/blog root root org.hibernate.dialect.MySQLDialect 1 true Hibernate Util The HibernateUtil class helps in creating the SessionFactory from the Hibernate configuration file. package com.loiane.hibernate; import org.hibernate.SessionFactory; import org.hibernate.cfg.AnnotationConfiguration; import com.loiane.model.Book; public class HibernateUtil { private static final SessionFactory sessionFactory; static { try { sessionFactory = new AnnotationConfiguration() .configure() .addPackage("com.loiane.model") //the fully qualified package name .addAnnotatedClass(Book.class) .buildSessionFactory(); } catch (Throwable ex) { System.err.println("Initial SessionFactory creation failed." + ex); throw new ExceptionInInitializerError(ex); } } public static SessionFactory getSessionFactory() { return sessionFactory; } } DAO In this class, we created two methods: one to save a Book instance into the database and another one to load a Book instance from the database. package com.loiane.dao; import org.hibernate.HibernateException; import org.hibernate.Session; import org.hibernate.Transaction; import com.loiane.hibernate.HibernateUtil; import com.loiane.model.Book; public class BookDAOImpl { /** * Inserts a row in the BOOK table. * Do not need to pass the id, it will be generated. * @param book * @return an instance of the object Book */ public Book saveBook(Book book) { Session session = HibernateUtil.getSessionFactory().openSession(); Transaction transaction = null; try { transaction = session.beginTransaction(); session.save(book); transaction.commit(); } catch (HibernateException e) { transaction.rollback(); e.printStackTrace(); } finally { session.close(); } return book; } /** * Delete a book from database * @param bookId id of the book to be retrieved */ public Book getBook(Long bookId) { Session session = HibernateUtil.getSessionFactory().openSession(); try { Book book = (Book) session.get(Book.class, bookId); return book; } catch (HibernateException e) { e.printStackTrace(); } finally { session.close(); } return null; } } Test To test it, first we need to create a Book instance and set an image to the image attribute. To do so, we need to load an image from the hard drive, and we are going to use the one located in the images folder. Then we can call the DAO class and save into the database. Then we can try to load the image. Just to make sure it is the same image we loaded, we are going to save it in the hard drive. package com.loiane.test; import static org.junit.Assert.assertNotNull; import java.io.File; import java.io.FileInputStream; import java.io.FileOutputStream; import org.junit.AfterClass; import org.junit.BeforeClass; import org.junit.Test; import com.loiane.dao.BookDAOImpl; import com.loiane.model.Book; public class TestBookDAO { private static BookDAOImpl bookDAO; @BeforeClass public static void runBeforeClass() { bookDAO = new BookDAOImpl(); } @AfterClass public static void runAfterClass() { bookDAO = null; } /** * Test method for {@link com.loiane.dao.BookDAOImpl#saveBook()}. */ @Test public void testSaveBook() { //File file = new File("images\\extjsfirstlook.jpg"); //windows File file = new File("images/extjsfirstlook.jpg"); byte[] bFile = new byte[(int) file.length()]; try { FileInputStream fileInputStream = new FileInputStream(file); fileInputStream.read(bFile); fileInputStream.close(); } catch (Exception e) { e.printStackTrace(); } Book book = new Book(); book.setName("Ext JS 4 First Look"); book.setImage(bFile); bookDAO.saveBook(book); assertNotNull(book.getId()); } /** * Test method for {@link com.loiane.dao.BookDAOImpl#getBook()}. */ @Test public void testGetBook() { Book book = bookDAO.getBook((long) 1); assertNotNull(book); try{ //FileOutputStream fos = new FileOutputStream("images\\output.jpg"); //windows FileOutputStream fos = new FileOutputStream("images/output.jpg"); fos.write(book.getImage()); fos.close(); }catch(Exception e){ e.printStackTrace(); } } } To verify if it was really saved, let’s check the table Book: and if we right click… and choose to see the image we just saved, we will see it: Source Code Download You can download the complete source code (or fork/clone the project – git) from: Github: https://github.com/loiane/hibernate-image-example BitBucket: https://bitbucket.org/loiane/hibernate-image-example/downloads Happy Coding! From http://loianegroner.com/2011/10/how-to-load-or-save-image-using-hibernate-mysql/

Version 2.3 of the Java servlet spec introduced the concept of filters. According to the documentation from Oracle’s site: “A filter dynamically intercepts requests and responses to transform or use the information contained in the requests or responses”. Today I’ll show you how to build a simple filter to intercept the response HTTP response code using annotations introduced in the Servlet 3.0 specification. With NetBeans IDE 7 create a new Maven Java Web Application called: Intercept Delete the index.jsp file under the Web Pages folder. Right-click on the project and add a new servlet called: MainServlet Since we are using the new Servlet 3 annotations we don’t need to set a whole lot of properties. Maven generates a decent MainServlet.java file for us, I just removed the comments for the output. My file looks like this: package com.giantflyingsaucer.intercept; import java.io.IOException; import java.io.PrintWriter; import javax.servlet.ServletException; import javax.servlet.annotation.WebServlet; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; @WebServlet(name = "MainServlet", urlPatterns = {"/"}) public class MainServlet extends HttpServlet { protected void processRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { response.setContentType("text/html;charset=UTF-8"); PrintWriter out = response.getWriter(); try { out.println(""); out.println(""); out.println(""); out.println(""); out.println(""); out.println("Servlet MainServlet"); out.println(""); out.println(""); } finally { out.close(); } } // /** * Handles the HTTP GET method. * @param request servlet request * @param response servlet response * @throws ServletException if a servlet-specific error occurs * @throws IOException if an I/O error occurs */ @Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { processRequest(request, response); } /** * Handles the HTTP POST method. * @param request servlet request * @param response servlet response * @throws ServletException if a servlet-specific error occurs * @throws IOException if an I/O error occurs */ @Override protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { processRequest(request, response); } /** * Returns a short description of the servlet. * @return a String containing servlet description */ @Override public String getServletInfo() { return "Short description"; }// } Right-click on the project and add a Filter called: InterceptFilter We will add the following two lines to the doFilter method. HttpServletResponse hsr = (HttpServletResponse) response; System.out.println("HTTP Status: " + hsr.getStatus()); My doFilter method looks like this: @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { if (debug) { log("InterceptFilter:doFilter()"); } doBeforeProcessing(request, response); HttpServletResponse hsr = (HttpServletResponse) response; System.out.println("HTTP Status: " + hsr.getStatus()); Throwable problem = null; try { chain.doFilter(request, response); } catch (Throwable t) { problem = t; t.printStackTrace(); } doAfterProcessing(request, response); if (problem != null) { if (problem instanceof ServletException) { throw (ServletException) problem; } if (problem instanceof IOException) { throw (IOException) problem; } sendProcessingError(problem, response); } } Clean and Build the project and deploy it to Apache Tomcat. Access the URL with a browser and take a look at your catalina.out file and you should see the HTTP response code. Note: You shouldn’t need to do any changes to the web.xml file for this project to work. From http://www.giantflyingsaucer.com/blog/?p=3279

this article is the fourth in a series about my adventures developing a fitness tracking application with html5, play scala, coffeescript and jade. previous articles can be found at: integrating scalate and jade with play 1.2.3 trying to make coffeescript work with scalate and play integrating html5 boilerplate with scalate and play developing features after getting my desired infrastructure setup, i started coding like a madman. the first feature i needed was a stopwatch to track the duration of a workout, so i started writing one with coffeescript. after spending 20 minutes playing with dates and settimeout, i searched and found a stopwatch jquery plug-in . i added this to my app, deployed it to heroku , brought up the app on my iphone 3g, clicked start and started riding my bike to work. when i arrived, i unlocked my phone and discovered that the time had stopped. at first, i thought this was a major setback. my disappointed disappeared when i found a super neat javascript stopwatch and kåre byberg's version that worked just fine. this stopwatch used settimeout, so by keeping the start time, the app on the phone would catch up as soon as you unlocked it. i ported kåre's script to coffeescript and rejoiced in my working stopwatch. # created by kåre byberg © 21.01.2005. please acknowledge if used # on other domains than http://www.timpelen.com. # ported to coffeescript by matt raible. also added hours support. flagclock = 0 flagstop = 0 stoptime = 0 refresh = null clock = null start = (button, display) -> clock = display startdate = new date() starttime = startdate.gettime() if flagclock == 0 $(button).html("stop") flagclock = 1 counter starttime, display else $(button).html("start") flagclock = 0 flagstop = 1 counter = (starttime) -> currenttime = new date() timediff = currenttime.gettime() - starttime timediff = timediff + stoptime if flagstop == 1 if flagclock == 1 $(clock).val formattime timediff, "" callback = -> counter starttime refresh = settimeout callback, 10 else window.cleartimeout refresh stoptime = timediff formattime = (rawtime, roundtype) -> if roundtype == "round" ds = math.round(rawtime / 100) + "" else ds = math.floor(rawtime / 100) + "" sec = math.floor(rawtime / 1000) min = math.floor(rawtime / 60000) hour = math.floor(rawtime / 3600000) ds = ds.charat(ds.length - 1) start() if hour >= 24 sec = sec - 60 * min + "" sec = prependzerocheck sec min = min - 60 * hour + "" min = prependzerocheck min hour = prependzerocheck hour hour + ":" + min + ":" + sec + "." + ds prependzerocheck = (time) -> time = time + "" # convert from int to string unless time.charat(time.length - 2) == "" time = time.charat(time.length - 2) + time.charat(time.length - 1) else time = 0 + time.charat(time.length - 1) reset = -> flagstop = 0 stoptime = 0 window.cleartimeout refresh if flagclock == 1 resetdate = new date() resettime = resetdate.gettime() counter resettime else $(clock).val "00:00:00.0" @stopwatch = { start: start reset: reset } the scalate/jade template to render this stopwatch looks as follows: script(type="text/javascript" src={uri("/public/javascripts/stopwatch.coffee")}) #display input(id="clock" class="xlarge" type="text" value="00:00:00.0" readonly="readonly") #controls button(id="start" type="button" class="btn primary") start button(id="reset" type="button" class="btn :disabled") reset :plain next, i wanted to create a map that would show your location. for this, i used merge design's html 5 geolocation demo as a guide. the html5 geo api is pretty simple, containing only three methods: // gets the users current position navigator.geolocation.getcurrentposition(successcallback, errorcallback, options); // request repeated updates of position watchid = navigator.geolocation.watchposition(successcallback, errorcallback); // cancel the updates navigator.geolocation.clearwatch(watchid); after rewriting the geolocation example in coffeescript, i ended up with the following code in my map.coffee script. you'll notice it uses google maps javascript api to show an actual map with a marker. # geolocation with html 5 and google maps api based on example from maxheapsize: # http://maxheapsize.com/2009/04/11/getting-the-browsers-geolocation-with-html-5/ # this script is by merge database and design, http://merged.ca/ -- if you use some, # all, or any of this code, please offer a return link. map = null mapcenter = null geocoder = null latlng = null timeoutid = null initialize = -> if modernizr.geolocation navigator.geolocation.getcurrentposition showmap showmap = (position) -> latitude = position.coords.latitude longitude = position.coords.longitude mapoptions = { zoom: 15, maptypeid: google.maps.maptypeid.roadmap } map = new google.maps.map(document.getelementbyid("map"), mapoptions) latlng = new google.maps.latlng(latitude, longitude) map.setcenter(latlng) geocoder = new google.maps.geocoder() geocoder.geocode({'latlng': latlng}, addaddresstomap) addaddresstomap = (results, status) -> if (status == google.maps.geocoderstatus.ok) if (results[1]) marker = new google.maps.marker({ position: latlng, map: map }) $('#location').html('your location: ' + results[0].formatted_address) else alert "sorry, we were unable to geocode that address." start = -> timeoutid = settimeout initialize, 500 reset = -> if (timeoutid) cleartimeout timeoutid @map = { start: start reset: reset } the template to show the map is a mere 20 lines of jade: script(type="text/javascript" src="http://www.google.com/jsapi") script(type="text/javascript" src="http://maps.googleapis.com/maps/api/js?sensor=false") :css .demo-map { border: 1px solid silver; height: 200px; margin: 10px auto; width: 280px; } #map(class="demo-map") p(id="location") span(class="label success") new | fetching your location with html 5 geolocation... script(type="text/javascript" src={uri("/public/javascripts/map.coffee")}) :javascript map.start(); the last two features i wanted were 1) distance traveled and 2) drawing the route taken on the map. for this i learned from a simple trip meter using the geolocation api . as i was beginning to port the js to coffeescript, i thought, "there's got to be a better way." i searched and found js2coffee to do most of the conversion for me. if you know javascript and you're learning coffeescript, this is an invaluable tool. i tried out the trip meter that night evening on a bike ride and noticed it said i'd traveled 3 miles when i'd really gone 6. i quickly figured out it was only calculating start point to end point and not taking into account all the turns in between. to view what was happening, i integrated my odometer.coffee with my map using google maps polylines . upon finishing the integration, i discovered two things, 1) html5 geolocation was highly inaccurate and 2) geolocation doesn't run in the background . i was able to solve the first problem by passing in {enablehighaccuracy: true} to navigator.geolocation.watchposition(). below are two screenshots showing before high accuracy and after. both screenshots are from the same two-block walk. the second issue is a slight show-stopper. phonegap might be able to solve the problem, but i'm currently using a workaround → turning off auto-lock and keeping safari in the foreground. making it look good after i got all my desired features developed, i moved onto making the app look good. i started by using sass for my css and installed play's sass module . i then switched to less when i discovered and added twitter's bootstrap to my project. at first i used play's less module (version 0.3), but ran into compilation issues . i then tried play's greenscript module , but gave up on it when i found it was incompatible with the coffeescript module . switching back to the less module and using the "0.3.compatibility" version solved all remaining issues. you might remember that i integrated html5 boilerplate and wondering why i have both bootstrap and boilerplate in my project. at this point, i don't think boilerplate is needed, but i've kept it just in case it's doing something for html5 cross-browser compatibility. i've renamed its style.css to style.less and added the following so it has access to bootstrap's variables. /* variables from bootstrap */ @import "libs/variables.less"; then i made my app look a lot better with layouts, stylish forms , a fixed topbar and alerts . for example, here's the coffeescript i wrote to display geolocation errors: geolocationerror = (error) -> msg = 'unable to locate position. ' switch error.code when error.timeout then msg += 'timeout.' when error.position_unavailable then msg += 'position unavailable.' when error.permission_denied then msg += 'please turn on location services.' when error.unknown_error then msg += error.code $('.alert-message').remove() alert = $('') alert.html('×' + msg); alert.insertbefore($('.span10')) then i set about styling up the app so it looked good on a smartphone with css3 media queries . below is the less code i used to hide elements and squish the widths for smaller devices. @media all and (max-device-width: 480px) { /* hide scrollbar on mobile */ html { overflow-y:hidden } /* hide sidebar on mobile */ .home .span4, .home .page-header, .topbar form { display: none } .home .container { width: 320px; } .about { .container, .span10 { width: 280px; } .span10 { padding-top: 0px; } } tools in the process of developing a stopwatch, odometer, displaying routes and making everything look good, i used a number of tools. i started out primarily with textmate and its bundles for less , coffeescript and jade . when i started writing more scala, i installed the scala textmate bundle . when i needed some debugging, i switched to intellij and installed its scala plugin. coffeescript, less and haml plugins (for jade) were already installed by default. i also used james ward's setup play framework with scala in intellij . issues i think it's obvious that my biggest issue so far is the fact that a webapp can't multitask in the background like a native app can. beyond that, there's accuracy issues with html5's geolocation that i haven't seen in native apps. i also ran into a caching issue when calling getcurrentposition(). it only worked the first time and i had to refresh my browser to get it to work again. strangely enough, this only happened on my desktop (in safari and firefox) and worked fine on my iphone. unfortunately, it looks like phonegap has issues similar to this. my workaround for no webapp multitasking is turning off auto-lock and leaving the browser in the foreground while i exercise. the downside to this is it really drains the battery quickly (~ 3 hours). i constantly have to charge my phone if i'm testing it throughout the day. the testing is a real pain too. i have to deploy to heroku (which is easy enough), then go on a walk or bike ride. if something's broke, i have to return home, tweak some things, redeploy and go again. also, there's been a few times where safari crashes halfway through and i lose all the tracking data. this happens with native apps too, but seemingly not as often. if you'd like to try the app on your mobile phone and see if you experience these issues, checkout play-more.com . summary going forward, there's still more html5 features i'd like to use. in particular, i'd like to play music while the fitness tracker is running. i'd love it if cloud music services (e.g. pandora or spotify) had an api i could use to play music in a webapp. soundcloud might be an option, but i've also thought of just uploading some mp3s and playing them with the tag. i've really enjoyed developing with all these technologies and haven't experienced much frustration so far. the majority has come from integrating scalate into play, but i've resolved most problems. next, i'll talk about how i've improved play's scalate support and my experience working with anorm . source: http://raibledesigns.com/rd/entry/developing_with_html5_coffeescript_and

Subtyping Issues in Java Behavior customization is a common requirement for large software systems. A software product cannot fully satisfy requirements of all customers; there are always differences that must be implemented, which are specific to each customer. Thus, software products must be flexible enough to allow these customizations in behavior even without changes in product's source code. In Java-based software products, a solution for such problems is usually based on a principle called subtyping. Subtyping can be defined on an example of two types A and B: if type B is a subtype of type A, then all code that operates correctly on objects of type A will operate correctly on objects of type B. Naturally, a software product behavior can be customized by replacing some default class A with custom class B where B is a subtype of A. If subtyping is based on an interface, then implementations of the interface can be independent of each other, which decreases complexity of the system and benefits modularity. At the same time such independence limits code reusability. A custom implementation can use Delegation pattern to call methods, but it is impossible to override a method in the base class. Another approach for subtyping is direct subclassing of the implementation class. Subclassing allows method overriding in the base class and also introduces design-time dependencies on that class. These dependencies limit subclass reusability. As you can see standard Java subtyping techniques limit reusability of either a base class or a subclass. Dynamic subtyping The approach described below combines the advantages of standard Java subtyping techniques. It allows creation of a new class, which can override methods in an original class, but at the same time, is dependent only on the interface of the original. Let's start with the following example: Suppose we have an interface Action as described below and its implementation in form of class ActionImpl. //This is an interface of Action public interface Action{ void doAction1(); void doAction2(); void doAll(); } //This is a default implementation of the interface Action class ActionImpl implements Action{ public void doAction1(){ ... } public void doAction2(){ ... } public void doAll(){ doAction1(); doAction2(); } } Class ActionImpl implements all methods of the interface and is used as a default implementation. Now let's suppose we want to override method doAction2 in the ActionImpl in such a way that the new class won't depend on ActionImpl. Let's create a new class ActionExtension as the following: //This is a custom implementation that overrides one method. abstract class ActionExtension extends ImplementationOf implements Action{ public void doAction2(){ … Super.doAction2(); } } This class extends a simple base class which looks like this: //This is base class for dynamic subclasses public abstract class ImplementationOf{ /** * A field for calling super class methods. * Should be used only in expressions where super keyword can be used. * */ protected final T Super = null; } Note that ActionExtension class is an abstract class, so we don't have to define all methods, just those that we want to override in ActionImpl. That's all we need to do at design time. Now let's look how it works at run-time. At the point of code where instance of Action is expected we need to create a new class: //Fragment of code that creates dynamic subclass of ActionImpl Class dynamicSubclass = DynamicClassExtender.extend(ActionImpl.class, Action.class, ActionExtension.class); The created class is a subclass of ActionImpl and contains all methods copied from ActionExtension class and modified in such a way that all calls to methods of Super field are directed to ActionImpl class. This class can be used to create an instance which can be used instead of instance of ActionImpl class: Action action = (Action) dynamicSubclass.newInstance(); In the real product the code above should be integrated into a dependency injection framework of your choice, and actual mapping of extension classes to implementations should be configurable. Dynamic class extender From the previous chapter we saw that all the complexities of dynamic subclassing are hidden in DynamicClassExtender class, which uses a byte code manipulation library to create a direct subclass of ActionImpl from ActionExtension class. In order to see what kind of byte code manipulation is involved let's start with ActionExtender class decompiled using standard javap utility: public abstract class ActionExtension extends ImplementationOf implements Action{ public ActionExtension(); Code: 0: aload_0 1: invokespecial #10; //Method ImplementationOf."":()V 4: return public void doAction2(); Code: 0: aload_0 1: getfield #17; //Field Super:Ljava/lang/Object; 4: checkcast #5; //class Action 7: invokeinterface #21, 1; //InterfaceMethodAction.doAction2:()V 12: return } Now if we manually create a class that extends ActionImpl and overrides doAction2 method and then decompile it with javap we'll see the following byte code: public class ActionExtension$ActionImpl extends ActionImpl{ public ActionExtension$ActionImpl(); Code: 0: aload_0 1: invokespecial #8; //Method ActionImpl."":()V 4: return public void doAction2(); Code: 0: aload_0 1: invokespecial #15; //Method ActionImpl.doAction2:()V 4: return } After comparing byte codes from both classes we can figure out a list of manipulations that DynamicClassExtender must apply on ActionExtension class in order to create a dynamic subclass of ActionImpl: change class name to ActionExtension$ActionImpl change access attribute to public, remove abstract attribute change super class name to ActionImpl replace references to ImplementationOf with references to ActionImpl replace method invocations on Super field (instructions 1,4,7 in ActionExtension.doAction2) with method invocations in super class (instruction 1 in ActionExtension$ActionImpl.doAction2) You can find a proof-of-concept implementation of DynamicClassExtender in the zip file attached to this article. The source code is distributed under the terms of BSD License. Conclusion The dynamic approach for subtyping, presented above, may help to create customizable Java applications without sharing implementation source code and without sacrificing code reusability.

i guess, i’m writing this post after a long time. this time, i’m writing about apache tika api that a friend of mine and i tried out to extract/retrieve metadata information from audio files supported by it – .mp3, .aiff, .au, .midi, .wav. to make it clear, here’s a screenshot of the information shown by windows vista about an audio file: we wanted to extract this using java and with googling, found that apache tika would help. we needed this metadata to index audio files for it to be searchable in a search application that we’re building using apache lucene . here’s a sample java program that extracts metadata from an mp3 file: package singz.samples.search.audio.metadata; import java.io.file; import java.io.fileinputstream; import java.io.filenotfoundexception; import java.io.ioexception; import java.io.inputstream; import org.apache.tika.exception.tikaexception; import org.apache.tika.metadata.metadata; import org.apache.tika.parser.parsecontext; import org.apache.tika.parser.parser; import org.apache.tika.parser.mp3.mp3parser; import org.xml.sax.contenthandler; import org.xml.sax.saxexception; import org.xml.sax.helpers.defaulthandler; /** * @author singaram subramanian * extract metadata of an audio file using apache tika api * */ public class audiometadataextractordemo { public static void main(string[] args) { // this audio file has metadata embedded in xmp (extensible metadata platform) standard // created by adobe systems inc. xmp standardizes the definition, creation, and // processing of extensible metadata. string audiofileloc = "c:\\pop\\backstreetboys_showmethemeaningofbeinglonely.mp3"; try { inputstream input = new fileinputstream(new file(audiofileloc)); contenthandler handler = new defaulthandler(); metadata metadata = new metadata(); parser parser = new mp3parser(); parsecontext parsectx = new parsecontext(); parser.parse(input, handler, metadata, parsectx); input.close(); // list all metadata string[] metadatanames = metadata.names(); for(string name : metadatanames){ system.out.println(name + ": " + metadata.get(name)); } // retrieve the necessary info from metadata // names - title, xmpdm:artist etc. - mentioned below may differ based // on the standard used for processing and storing standardized and/or // proprietary information relating to the contents of a file. system.out.println("title: " + metadata.get("title")); system.out.println("artists: " + metadata.get("xmpdm:artist")); system.out.println("genre: " + metadata.get("xmpdm:genre")); } catch (filenotfoundexception e) { e.printstacktrace(); } catch (ioexception e) { e.printstacktrace(); } catch (saxexception e) { e.printstacktrace(); } catch (tikaexception e) { e.printstacktrace(); } } } maven pom xml 4.0.0 singz.samples.search.audio audiometadataextractor 0.0.1 jar audiometadataextractor http://maven.apache.org utf-8 org.apache.tika tika-core 0.10 org.apache.tika tika-parsers 0.10 output xmpdm:releasedate: 2001 xmpdm:audiochanneltype: stereo xmpdm:album: top 100 pop author: backstreet boys xmpdm:artist: backstreet boys channels: 2 xmpdm:audiosamplerate: 44100 xmpdm:logcomment: eng xmpdm:tracknumber: 04 version: mpeg 3 layer iii version 1 xmpdm:composer: null xmpdm:audiocompressor: mp3 title: show me the meaning of being lonely samplerate: 44100 xmpdm:genre: pop content-type: audio/mpeg title: show me the meaning of being lonely artists: backstreet boys genre: pop about apache tika http://tika.apache.org/index.html “the apache tika™ toolkit detects and extracts metadata and structured text content from various documents using existing parser libraries.” http://www.lucidimagination.com/devzone/technical-articles/content-extraction-tika#article.tika “apache tika is a content type detection and content extraction framework. tika provides a general application programming interface that can be used to detect the content type of a document and also parse textual content and metadata from several document formats. tika does not try to understand the full variety of different document formats by itself but instead delegates the real work to various existing parser libraries such as apache poi for microsoft formats, pdfbox for adobe pdf, neko html for html etc. the grand idea behind tika is that it offers a generic interface for parsing multiple formats. the tika api hides the technical differences of the various parser implementations. this means that you don’t have to learn and consume one api for every format you use but can instead use a single api – the tika api. internally tika usually delegates the parsing work to existing parsing libraries and adapts the parse result so that client applications can easily manage variety of formats. tika aims to be efficient in using available resources (mainly ram) while parsing. the tika api is stream oriented so that the parsed source document does not need to be loaded into memory all at once but only as it is needed. ultimately, however, the amount of resources consumed is mandated by the parser libraries that tika uses. at the time of writing this, tika supports directly around 30 document formats. see list of supported document formats . the list of supported document formats is not limited by tika in any way. in the simplest case you can add support for new document formats by implementing a thin adapter that that implements the parser interface for the new document format.” about xmp standard http://en.wikipedia.org/wiki/extensible_metadata_platform “the adobe extensible metadata platform ( xmp ) is a standard, created by adobe systems inc. , for processing and storing standardized and proprietary information relating to the contents of a file. xmp standardizes the definition, creation, and processing of extensible metadata . serialized xmp can be embedded into a significant number of popular file formats, without breaking their readability by non-xmp-aware applications. embedding metadata avoids many problems that occur when metadata is stored separately. xmp is used in pdf , photography and photo editing applications. xmp can be used in several file formats such as pdf , jpeg , jpeg 2000 , jpeg xr , gif , png , html , tiff , adobe illustrator , psd , mp3 , mp4 , audio video interleave , wav , rf64 , audio interchange file format , postscript , encapsulated postscript , and proposed for djvu . in a typical edited jpeg file, xmp information is typically included alongside exif and iptc information interchange model data.” from http://singztechmusings.wordpress.com/2011/10/17/how-to-retrieveextract-metadata-information-from-audio-files-using-java-and-apache-tika-api/

our development team wanted to get notified as soon as something goes wrong in our production system, a critical java web application serving thousands of customers daily. the idea was to let it send us an email when there are too many errors, indicating usually a problem with a database, an external web service, or something really bad with the application itself. in this post i want to present a simple solution we have implemented using a custom log4j appender based on stats4j and an smtpappender (which is more difficult to configure and troubleshoot than you might expect) and in the following post i explore how to achieve the same effect with the open-source hyperic hq monitoring sw. the challenge we faced the following challenges with the logs: it’s unfortunately normal to have certain number of exceptions (customers select search criteria yielding no results, temporary, unimportant outages of external services etc.) and we certainly don’t want to be spammed because of that. so the solution must have a configurable threshold and only send an alert when it is exceeded. the failure rate should be computed for a configurable period (long enough not to trigger an alert because of few-minutes outages yet short enough for the team to be informed asap when something serious happens). once an alert is send, no further alerts should be send again for some time (ideally until the original problem is fixed), we don’t want to be spammed because of a problem we already know about. the solution we’ve based our solution on lara d’abreo’s stat4j , which provides a custom log4j appender that uses the logs to compute configurable measures and triggers alerts when they exceed their warning or critical thresholds. it is couple of years old, alpha-quality (regarding its generality and flexibility) open-source library, which is fortunately simple enough to be modified easily for one’s needs. so we have tweaked stat4j to produce alerts when the number of alerts exceeds thresholds and keep quiet thereafter and combined that with a log4j smtpappender that listens for the alerts and sends them via e-mail to the team. stat4j tweaking the key components of stat4j are the stat4jappender for log4j itself, calculators (measures) that aggregate the individual logs (e.g. by counting them or extracting some number form them), statistics that define which logs to consider via regular expressions and how to process them by referencing a calculator, and finally alerts that log a warning when the value of a statistics exceeds its limits. you can learn more in an article that introduces stat4j . we have implemented a custom measure calculator, runningrate (to count the number of failures in the last n minutes) and modified stat4j as follows: we’ve enhanced alert to support a new attribute, quietperiod , so that once triggered, subsequent alerts will be ignored for that duration (unless the previous alert was just a warning while the new one is a critical one) we’ve modified the appender to include the log’s throwable together with the log message, which is then passed to the individual statistics calcualtors, so that we could filter more precisely what we want to count finally we’ve modified alert to log alerts as errors instead of warnings so that the smtpappender wouldn’t ignore them get our modified stat4j from github (sources or a compiled jar ). disclaimer: it is one day’s hack and i’m not proud of the code. stat4j configuration take the example stat4j.properties and put it on the classpath. it is already configured with the correct calculator, statistics, and alert. see this part: ... ### jakub holy - my config calculator.minuterate.classname=net.sourceforge.stat4j.calculators.runningrate # period is in [ms] 1000 * 60 * 10 = 10 min: calculator.minuterate.period=600000 statistic.runningerrorrate.description=errors per 10 minutes statistic.runningerrorrate.calculator=minuterate # regular expression to match " <- " statistic.runningerrorrate.first.match=.*exception.* # error rate alert.toomanyerrorsrecently.description=too many errors in the log alert.toomanyerrorsrecently.statistic=runningerrorrate alert.toomanyerrorsrecently.warn= >=3 alert.toomanyerrorsrecently.critical= >=10 alert.toomanyerrorsrecently.category=alerts # ignore following warnings (or criticals, after the first critical) for the given amount of time: # 1000 * 60 * 100 = 100 min alert.toomanyerrorsrecently.quietperiod=6000000 the important config params are calculator.minuterate.period (in ms) – count errors over this period, reset the count at its end; a reasonable value may be 10 minutes alert.toomanyerrorsrecently.warn and .critical – trigger the alert when so many errors in the period has been encountered; reasonable values depend on your application’s normal error rate alert.toomanyerrorsrecently.quietperiod (in ms) – don’t send further alerts for this period not to spam in a persistent failure situation; the reasonable value depends on how quickly you usually fix problems, 1 hour would seem ok to me notice that statistic.runningerrorrate.first.match is a regular expression defining which logs to count; “.*” would include any log, “your\.package\..*exception” any exception in the package and so on, you can even specify logs to exclude using a negative lookahead ( (?! x )) log4j configuration now we need to tell log4j to use the stat4j appender to count error occurences and to send alerts via email: log4j.rootcategory=debug, console, fileappender, stat4jappender ... ### stat4jappender & emailalertsappender ### # collects statistics about logs and sends alerts when there # were too many failures in cooperation with the emailalertsappender ## stat4jappender log4j.appender.stat4jappender=net.sourceforge.stat4j.log4j.stat4jappender log4j.appender.stat4jappender.threshold=error # for configuration see stat4j.properties ## emailalertsappender # beware: smtpappender ignores its thresholds and only evers sends error or higher messages log4j.category.alerts=error, emailalertsappender log4j.appender.emailalertsappender=org.apache.log4j.net.smtpappender [email protected] # beware: the address below must have a valid domain or some receivers will reject it (e.g. gmail) [email protected] log4j.appender.emailalertsappender.smtphost=172.20.20.70 log4j.appender.emailalertsappender.buffersize=1 log4j.appender.emailalertsappender.subject=[stat4j] too many exceptions in log log4j.appender.emailalertsappender.layout=org.apache.log4j.patternlayout log4j.appender.emailalertsappender.layout.conversionpattern=%d{iso8601} %-5p %x{clientidentifier} %c %x - %m%n comments #8 specify the stat4j appender #9 only send errors to stat4j, we are not interested in less serious exceptions #14 “alerts” is the log category used by stat4jappender to log alerts (the same you would create via logger.getlogger(“alerts”)); as mentioned, smtpappender will without respect to the configuration only process errors and higher issues with the smtpappender it is quite tricky to get the smtpappender working. some pitfall: smtpappender ignores all logs that are not error or higher without respect to how you set its threshold if you specify a non-existing from domain then some recipient’s mail servers can just delete the email as spam (e.g. gmail) to send emails, you of course need mail.jar (and for older jvms also activation.jar), here are instructions for tomcat and one $100 tip: to debug it, run your application in the debug mode and set a method breakpoint on javax.mail.transport#send (you don’t need the source code) and when there, set this.session.debug to true to get a very detailed log of the following smtp communication in the server log. sidenote the fact that this article is based on log4j doesn’t mean i’d personally choose it, it just came with the project. i’d at least consider using the newer and shiny logback instead . conclusion stat4j + smtpappender are a very good base for a rather flexible do-it-yourself alerting system based on logs and e-mail. you can achieve the same thing out-out-the-box with hyperic hq plus much much more (provided that you get your admins to open two ports for it), which i will describe in the next blog post. links an alternative for preventing the smtpappender from spamming in persisten failure situations (aside of its built-in buffer size): log4j-email-throttle eventconsolidatingappender – announced via mailing list in 2/2011 – “the purpose of this appender is to consolidate multiple events that are received by a single logger within a specified number of seconds into a single event; this single consolidated event is then forwarded to a ‘downstream’ appender” from http://theholyjava.wordpress.com/2011/10/15/aggregating-error-logs-to-send-a-warning-email-when-too-many-of-them-log4j-stat4j-smtpappender/

If you've looked at my recent posts, you know I'm working on a plugin for VisualVM, a very useful tool supplied with the JDK. In one example, I showed how to attach to a waiting Java application using a socket-based AttachingConnector. At that time I said that there were two primary ways of attaching to a process with JDI -- via shared memory, and with a socket. It turns out there is a "third way". Following is an example of why this way is useful, and why it was provided. When I last wrote JDI programs (in Java 5), I would notice that my target application would start up and print (to stdout) the port on which it was listening, as in the following: Listening for transport dt_socket at address: 55779 In Java 5, if you detached your debugger from this process, you would get another line to stdout in the target's console, like this: Listening for transport dt_socket at address: 55779 and this would go on for as long as you chose to attach and detach, etc. At some point (and I don't know when this started happening), the port on which the target is listening started changing on each detach of an external debugger. If in Java 6 (I'm using u20), you repeatedly attach and detach from the target process, you'll see the following out in the target's console: Listening for transport dt_socket at address: 55837 ERROR: transport error 202: recv error: Connection reset by peer Listening for transport dt_socket at address: 55844 ERROR: transport error 202: recv error: Connection reset by peer Listening for transport dt_socket at address: 55846 ERROR: transport error 202: recv error: Connection reset by peer Listening for transport dt_socket at address: 55911 If you're writing an application that attaches using the debug port, each time you attach you need to find out what port the target is using. This information is not available from the process itself; in other words, you have to play the usual unpleasant game of capturing console output to know what the port is. Even if you specify a port at target start, you still need to get your hands on the value. You can still find the original request for a feature to attach to a process by its process ID if you search around the old Java bug reports. The long and short of it: a new AttachingConnector was created, one which attaches by PID. As you know, sometimes it isn't much fun finding a process's PID either. In my case, however, I am writing a plugin for VisualVM, and one thing you get for free when you do that is Visual VM's API, which as you might expect includes calls to get the PID. My goal, then, is to use this new connector in my VisualVM plugin, and I thought it might be appreciated if I shared the details. I've adapted my test program from an earlier post so that it now outputs the details of each AttachingConnector; the changed code fragment is shown here: List attachingConnectors = vmMgr.attachingConnectors(); for (AttachingConnector ac: attachingConnectors) { Map paramsMap = ac.defaultArguments(); Iterator keyIter = paramsMap.keySet().iterator(); System.out.println("AttachingConnector: '" + ac.getClass().getName() + "'"); System.out.println(" name: '" + ac.name() + "'"); System.out.println(" description: '" + ac.description() + "'"); System.out.println(" transport name: '" + ac.transport().name() + "'"); System.out.println(" default arguments:"); while (keyIter.hasNext()) { String nextKey = keyIter.next(); System.out.println(" key: '" + nextKey + "'; value: '" + paramsMap.get(nextKey) + "'"); } } The output from this code is shown below: AttachingConnector: 'com.sun.tools.jdi.SocketAttachingConnector' name: 'com.sun.jdi.SocketAttach' description: 'Attaches by socket to other VMs' transport name: 'dt_socket' default arguments: key: 'timeout'; value: 'timeout=' key: 'hostname'; value: 'hostname=AdamsResearch' key: 'port'; value: 'port=' AttachingConnector: 'com.sun.tools.jdi.SharedMemoryAttachingConnector' name: 'com.sun.jdi.SharedMemoryAttach' description: 'Attaches by shared memory to other VMs' transport name: 'dt_shmem' default arguments: key: 'timeout'; value: 'timeout=' key: 'name'; value: 'name=' AttachingConnector: 'com.sun.tools.jdi.ProcessAttachingConnector' name: 'com.sun.jdi.ProcessAttach' description: 'Attaches to debuggee by process-id (pid)' transport name: 'local' default arguments: key: 'pid'; value: 'pid=' key: 'timeout'; value: 'timeout=' A couple of things I hadn't noticed before is that the socket-based connector comes with the hostname argument pre-set to my machine's hostname, and that all three connectors have a timeout default argument. The first observation brings up an interesting point: if you use the local, PID-based connector, remember that you'll only be attaching to processes on your debugger's host. I changed my test program to use the local connector and it works as before! Well, no, actually, it does not. Here's what I now get: java.lang.UnsatisfiedLinkError: no attach in java.library.path Exception in thread "main" java.io.IOException: no providers installed at com.sun.tools.jdi.ProcessAttachingConnector.attach(ProcessAttachingConnector.java:86) at com.adamsresearch.jdiDemo.JDIDemo.main(JDIDemo.java:70) Does this mean the local connector isn't exactly ready for use? No, but I have been burned by the same issue that has plagued a number of others (scroll down in that page -- the issue was found by a reader of that post and was solved, partially, by another reader of that post). I'm working on a Windows platform, and when you do that you have to be a little careful ;-> . In this case, the problem is caused by 1) using the java interpreter as found on the system path, and 2) not making sure that path points directly to your JDK or JRE directory. The executable will look in a path relative to itself for the needed libraries, and when Windows copies the java executable to C:\Windows\system32 (or similar) -- and if you use that executable -- that relative path is broken. I believe this is the true issue, unlike described in the comments on the above post, where the distinction is made between using the JRE java and the JDK java. I don't think that's the issue. For example, below are the results of my attach test in 3 different scenarios: Using java from my path, the first hit of which comes from C:\Windows\system32: java -cp c:\jdk1.6.0_20\lib\tools.jar;. com.adamsresearch.jdiDemo.JDIDemo 10816 863 fileName ... java.lang.UnsatisfiedLinkError: no attach in java.library.path Exception in thread "main" java.io.IOException: no providers installed at com.sun.tools.jdi.ProcessAttachingConnector.attach(ProcessAttachingConnector.java:86) at com.adamsresearch.jdiDemo.JDIDemo.main(JDIDemo.java:70) Using the full path to the JRE bin java: c:\jdk1.6.0_20\jre\bin\java -cp c:\jdk1.6.0_20\lib\tools.jar;. com.adamsresearch.jdiDemo.JDIDemo 10816 863 fileName ... Attached to process 'Java HotSpot(TM) 64-Bit Server VM' Using the full path to the JDK bin java: c:\jdk1.6.0_20\bin\java -cp c:\jdk1.6.0_20\lib\tools.jar;. com.adamsresearch.jdiDemo.JDIDemo 10816 863 fileName ... Attached to process 'Java HotSpot(TM) 64-Bit Server VM' As you can see, the above seems to support my theory that it's not the JRE vs the JDK, but rather the context-poor placement of the java executable in the "usual" Windows binaries directory, that caused the problem. That posting is several years old, so it is possible that at that time, the needed JDI libraries actually were not included in the JRE, but it is clear that today, you will see the same exception if you use the java executable found in Windows' default binaries directory. Now, if I run my JDI application against my JarView utility, searching for AttachingConnector in the JDK installation directory, I get the following output: Breakpoint at line 863: fileName = 'AttachingConnector.class' Breakpoint at line 863: fileName = 'GenericAttachingConnector$1.class' Breakpoint at line 863: fileName = 'GenericAttachingConnector.class' Breakpoint at line 863: fileName = 'ProcessAttachingConnector$1.class' Breakpoint at line 863: fileName = 'ProcessAttachingConnector$2.class' Breakpoint at line 863: fileName = 'ProcessAttachingConnector.class' Breakpoint at line 863: fileName = 'SharedMemoryAttachingConnector$1.class' Breakpoint at line 863: fileName = 'SharedMemoryAttachingConnector.class' Breakpoint at line 863: fileName = 'SocketAttachingConnector$1.class' Breakpoint at line 863: fileName = 'SocketAttachingConnector.class' and so have done what I set out to do, which is 1) debug-attach by process ID, and 2) thrash through the inevitable hiccups and share the solutions. Hopefully this will be useful to you, too. Note: actually, there are even more ways to attach to a Java process. JPDA Connection and Invocation is the definitive guide, from Oracle. If you're going to be writing debuggers, you can't go wrong reading this page first. From http://wayne-adams.blogspot.com/2011/10/jdi-three-ways-to-attach-to-java.html

Let’s say you are using some external components in your web page that renders a button as below: The above code will call the delete() method when user clicks on the button. But, what if you want to show a confirmation box before calling the delete()? Since the code is rendered by a external component, most probably you won’t be having control over how its code generated. So, you have to intercept the onclick event and block it until your method is called and then invoke the original method. Lets do it.. var btn = $("#btn"); btn.data("funcToCall", btn.attr("onclick")); $("#btn").removeAttr("onclick"); $("#btn").bind("click", function(e){ if(confirm("Are you sure?")){ var func = $(this).data("funcToCall"); eval(func); } }); As you can see, the onclick function is first stored into that element and then the onclick attribute is removed. Finally when the button in clicked, our custom method is called first and based on the user decision, the delete() method is either called or not called. From http://veerasundar.com/blog/2011/10/intercepting-onclick-event-in-javascript-using-jquery/

I had to answer the following question: Shall we use a Portal and if yes, should it be Liferay Portal or Oracle Portal? Or shall we use just one or more Java web frameworks? This article shows my result. I had to look especially at Liferay and Oracle products, nevertheless the result can be used for other products, too. The short answer: A Portal makes sense only in a few use cases, in the majority of cases you should not use one. In my case, we will not use one. What is a Portal? It is important to know that we are talking about an Enterprise Portal. Wikipedia has a good definition: „An enterprise portal [...] is a framework for integrating information, people and processes across organizational boundaries. It provides a secure unified access point, often in the form of a web-based user interface, and is designed to aggregate and personalize information through application-specific portlets.“ Several Portal server are available in the Java / JVM environment. Liferay Portal and GateIn Portal (former JBoss Portal) are examples for open source products while Oracle Portal or IBM WebSphere Portal are proprietary products. You develop Portlets („simple“ web applications) and deploy them in your portal. If you need to know more about a Portal or the Portlet JSR standards, ask Wikipedia: http://en.wikipedia.org/wiki/Portlet. Should we use a Portal or not? I found several pros and cons for using a Portal instead of just web applications. Disadvantages of using a Portal: Higher complexity Additional configuration (e.g. portlet.xml, Portal server) Communication between Portlets using Events is not trivial (it is also not trivial if two applications communicate without portlets, of course) Several restrictions when developing a web application within a Portlet Additional testing efforts (test your web applications and test it within a Portal and all its Portal features) Additional costs Open source usually offers enterprise editions which include support (e.g. Liferay) Proprietary products have very high initial costs. Besides, you need support, too (e.g. Oracle) You still have to customize the portal and integrate applications. A portal product does not give you corporate identity or systems integration for free. Software licensing often is only ten percent of the total price. Developers need additional skills besides using a web framework Several restrictions must be considered choosing a web-framework and implement the web application Rethinking about web application design is necessary Portlets use other concepts such as events or an action and render phase instead of only one phase Frameworks (also called bridges) help to solve this problem (but these are standardized for JSF only, a few other plugins are available, e.g. for GWT or Grails) Actually, IMO you have to use JSF if you want to realize Portlets in a stable, relatively „easy“ and future-proof way. There is no standard bridge for other frameworks. There are no books, best practices, articles or conference sessions about Portlets without JSF, right? Advantages of using a Portal Important: Many of the pros can be realized by oneself with relatively low efforts (see the "BUT" notes after each bullet point). Single Sign On (SSO) BUT: Several Java frameworks are available, e.g OpenSSO (powerful, but complicated) or JOSSO (not so powerful, but easy to use). Good products are available, e.g. Atlassian Crowd (I love Atlassian products such as Crowd, Jira or Confluence, because they are very intuitive and easy to use). Integration of several applications within one GUI A portal gives you layout and sequence of the applications for free (including stuff such as drag & drop, minimizing windows, and so on) Communication between Portlets (i.e. between different applications) BUT: This is required without a portal, too. Several solutions can be used, such as a database, messaging, web services, events, and so on. Even „push“ is possible for some time now (using specific web framework features or HTML 5 websockets). Uniform appearence BUT: CSS can solve this problem (the keyword „corporate identity“ exists in almost every company). Create a HTML template and include your applications within this template. Done. Personalization Regarding content, structure or graphical presentation Based on individual preferences or metadata BUT: Some of these features can be realized very easily by oneself (e.g. a simple role concept). Nevertheless, GUI features such as drag & drop are more effort (although component libraries can help you a lot). Many addons are included Search Content management Document management Web 2.0 tools (e.g. blogs or wikis) Collaboration suites (e.g. team pages) Analytics and reporting Development platforms BUT: A) Do you really need these Features? B) Is the offered functionality sufficent? Portals only offer „basic“ versions of stand-alone products. For instance, the content management system or search engine of a Portal is less powerful than other „real“ products offering this functionality. Thus, you have to think about the following central question: Do we really need all those features offered by a portal? Conclusion: The total cost of ownership (TCO) is much higher when using a portal. You have to be sure, that you really need the offered features. In some situations, you can defer your decision. Create your web applications as before. You can still integrate them in a Portal later, if you really need one. For instance, the following Oracle blog describes how you can use iFrames to do this: http://blogs.oracle.com/jheadstart/entry/integrating_a_jsf_application If you decide to use a Portal, you have to choose a Portal product. Should we use an Open Source or Proprietary Portal Product? Both, open source and proprietary Portal products have pros and cons. I especially looked at Oracle Portal and Liferay Portal, but probably most aspects can be considered when evaluating other products, too. Advantages of Oracle Portal Oracle offers a full-stack suite for development (including JSF and Portlets): Oracle Application Development Framework (ADF) Oracle JDeveloper offers good support for ADF. Everything from one product line increases efficiency (database, application server, ESB, IDE, Portal, …) – at least in theory :-) Disadvantages of Oracle Portal: High initial costs (I heard something about 200K in our company) Complex, heavyweight product (compared to Liferay Portal) Proprietary Communication between Portlets is not implemented using the standard JSR-286, but a custom proprietary solution (Source: http://www.contribute.be/web/contribute/news/-/journal_content/56_INSTANCE_pdF5/10234/21893) Advantages of Liferay Portal: Open source Drastically lower initial costs Lightwight product (1-Click-Install, etc.) Disadvantages of Liferay Portal: Not everything is from one product line (this cannot be considered as disadvantage always, but in our case the customer preferred very few different vendors (keyword “IT consolidation”) Portlets are still Portlets. Although Liferay is lightweight, realizing Portlets still sucks as it does with a proprietary product When to use a Portal? Well, the conclusion is difficult. In my opinion, it does make sense only in a few use cases. If you really need many or all of those Portal features, and they are also sufficient, then use a Portal product. Though, usually it is much easier to create a simple web application which integrates your applications. Use a SSO framework, create a template, and you are done. Your developers will appreciate not to work with Portlets and its increased complexity and restrictions. Did I miss any pros or cons? Do you have another opinion (probably, many people do???), then please write a comment and let’s discuss… Best regards, Kai Wähner (Twitter: @KaiWaehner) [Content from my Blog: Kai Wähner's Blog: Pros and Cons - When to use a Portal and Portlets instead of just Java Web-Frameworks]

If you need to rename the Java package of a 3rd party library, e.g. to include it directly in your project while avoiding possible conflicts, you can use one of the following Maven plugins (and they may be more) in the package lifecycle phase: Uberize plugin (latest – org.fusesource.mvnplugins:maven-uberize-plugin:1.20) – originally inspired by the Shade plugin, intended to overcome some of its limitations. Intended primarily to merge your code and dependencies into one jar. Shade plugin package-rename-task, Ant-based Maven plugin – I’m not sure whether this is further maintained From http://theholyjava.wordpress.com/2011/10/06/tools-for-renaming-the-package-of-a-dependency-with-maven/

This may not be strictly true, but it seems to me that the Guy’s at Spring are always banging on about AspectJ and Aspect Oriented Programming (AOP), to the point where I suspect that it’s used widely under the hood and is an integral part of Spring and I say “widely used under the hood”, because I haven’t come across too many projects that do use AspectJ or AOP in general. I suspect that part of the reason for this is possibly down to the fact that AOP different is a concept to Object Oriented Programming (OOP). AOP, they say, is all to do with an application’s cross-cutting concerns, which translates to mean stuff that’s common to all classes within your application. The usual example given here is logging and example code usually demonstrates logging all method entry and exit details, which is something that I’ve never found that useful. The other reason that I’ve not seen it used is that the AspectJ documentation is a bit ropey. Today’s blog is a demonstration of how to implement AspectJ’s @AfterThrowing advice in a Spring application. The idea of the after throwing advice is that you intercept an exception after it’s thrown, but before it’s caught - as shown in this rather simplistic diagram: I said above that the AOP sample code usually demonstrates logging, and in that respect this blog is no different. The idea in this contrived scenario is to log to a simple Apache Commons log any exceptions thrown. The class that actually does all this is IncidentThrowsAdvice: @Aspect public class IncidentThrowsAdvice { // Obtain a suitable logger. private static Log logger = LogFactory.getLog(IncidentThrowsAdvice.class); /** * Called between the throw and the catch */ @AfterThrowing(pointcut = "execution(* *(String, ..))", throwing = "e") public void myAfterThrowing(JoinPoint joinPoint, Throwable e) { System.out.println("Okay - we're in the handler..."); Signature signature = joinPoint.getSignature(); String methodName = signature.getName(); String stuff = signature.toString(); String arguments = Arrays.toString(joinPoint.getArgs()); logger.info("Write something in the log... We have caught exception in method: " + methodName + " with arguments " + arguments + "\nand the full toString: " + stuff + "\nthe exception is: " + e.getMessage(), e); } } The class itself is fairly straight forward. It has one method myAfterThrowing(...), which takes two arguments of type: JoinPoint and Throwable. Taking each of these in turn, JoinPoint just an class that holds information that describes a point within your code. Applying it to this particular after throwing advice means that it’s the point in your code where the exception occurred. The second argument is more straight forward as it’s the actual exception that’s currently being thrown There are two annotations applied to this class: @Aspect and @AfterThrowing. @Aspect marks the AnyOldExampleBean class as an AspectJ class, whilst the second annotation: @AfterThrowing is more interesting. This annotation tells AspectJ to call the myAfterThrowing() method when an exception occurs. It has two attributes, pointcut and throwing. pointcut defines the circumstances in which the myAfterThrowing() method is called as defined by the expression * *.*(..). This, like the rest of AspectJ seems remarkably badly documented, however you can determine that this signifies a method signature. In this case we’re checking every method in every class. This expression breaks down as follows: * - The first star is method visibility and or the method return type. The following will work in this example: * public void void whereas public on its own throws a BeanCreationException exception when loading the Spring config. *.* - this represents the package and method, again, using the same wild card formatting; hence, the following are all valid: example_10_annotations.afterthrowing_annotation.Sneeze.sneeze *.* *.sneeze example_10_annotations.*.Sneeze.sneeze ...however, example_10_annotations.*.sneeze won’t match for some reason... (..) - Defines the method arguments. In this example, the following will match: (..) (String, String, int) (String, ..) Having defined an AspectJ after throwing advice, the next step is to integrate it into the Spring application and this is a matter of adding one line to your Spring config file together with the appropriate schema reference in your : XML element: The important line in this file is: ...as it switches AOP on. The bean definitions that follow it demonstrate how the after throws advice works. The Sneeze class simply throws an exception when its called and the AnyOldExampleBean is a simple test class that calls Sneeze.sneeze(...) as shown below: public class Sneeze { /** * Throw an exception */ public void sneeze(String arg0, String arg1, int i) throws Exception { throw new Exception("Simulate an error"); } } public class AnyOldExampleBean { private Sneeze sneeze; /** * @return */ public Sneeze getSneeze() { return sneeze; } /** * @param sneeze */ public void setSneeze(Sneeze sneeze) { this.sneeze = sneeze; } /** * Do something * * @return */ public void run() { try { sneeze.sneeze("arg0", "arg1", 42); } catch (Exception e) { System.out.println("Caught e"); } finally { System.out.println("the end..."); } } } The code to run the above is: ApplicationContext ctx = new ClassPathXmlApplicationContext("example10_throwsadvice.xml"); AnyOldExampleBean myExampleBean = ctx.getBean(AnyOldExampleBean.class); myExampleBean.run(); ...and when running this code, you should get the following output: This is the after throwing exception handler 13:41:46,399 INFO ClassPathXmlApplicationContext:456 - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@4ce2cb55: startup date [Sun Aug 14 13:41:46 BST 2011]; root of context hierarchy 13:41:46,462 INFO XmlBeanDefinitionReader:315 - Loading XML bean definitions from class path resource [example10_throwsadvice.xml] 13:41:46,863 INFO DefaultListableBeanFactory:555 - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@663257b8: defining beans [org.springframework.aop.config.internalAutoProxyCreator,exceptionHandler,example_10_annotations.afterthrowing_annotation.AfterThrowingBean#0,sneeze]; root of factory hierarchy Okay - we're in the handler... 13:41:47,171 INFO IncidentThrowsAdvice:42 - Write something in the log... We have caught exception in method: sneeze with arguments [arg0, arg1, 42] and the full toString: void example_10_annotations.afterthrowing_annotation.Sneeze.sneeze(String,String,int) the exception is: Simulate an error java.lang.Exception: Simulate an error at example_10_annotations.afterthrowing_annotation.Sneeze.sneeze(Sneeze.java:20) at example_10_annotations.afterthrowing_annotation.Sneeze$$FastClassByCGLIB$$5c47789.invoke() at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149) at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:55) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at ...THE REST HAS BEEN REMOVED FOR CLARITY... Caught e the end... As I said above, AspectJ seems very badly documented as can be borne out by the JavaDocs, which if you look through means that the code contains very little documentation, which in turn means that I’m glad I’m not working on it... Finally, in covering the after throwing advice, this blog only really touches on AspectJ and there are other useful AspectJ advice annotations that I will probably be covering in the near future. From http://www.captaindebug.com/2011/09/using-aspectjs-afterthrowing-advice-in.html

Everyone in the Java world seems to use various opensource libraries and frameworks... and why not, there are hundreds available covering virtually every type of programming problem you’re likely to come across in today’s programming landscape. This blog takes a quick look at the reasons for using opensource artifacts and examines what could go wrong... The first reason for using them is reduced cost as it’s cheaper for your project to grab hold of an opensource library than it is for your to write the same thing yourself. The second reason for using opensource artifacts is reduced cost: you get free support from a bunch of capable and enthusiastic developers, usually in the form of copious amounts of documentation and forums. The third reason is reduced cost: you get free updates and enhancements from the opensource community and free bug fixes, although you don’t get to choose which enhancements are added to the project. Some projects, such as Tomcat, have a mechanism for voting on what enhancements are made, but at the end of the day it’s down to what really interests the developers. There are also a couple of unspoken reasons for using popular opensource libraries and frameworks: firstly, they make your CV look good. If opensource X is popular and you put that on your CV then your chances of getting a pay rise or a better job will improve. Secondly, if you work on one of the opensource projects, then you’ll earn some kudos, which, again, makes your CV look good improves the chances of you increasing the size of your pay-packet. There is an obvious downside to using opensource artifacts and that is all projects have an natural life-cycle. New versions of libraries are released, old libraries are deprecated, falling out of use because the technology’s too old, the developers have lost interest or moved on, or the rest of the community found something else that’s better and jumped on that bandwagon deserting yours. So, the problems of finding yourself saddled with retired and deprecated opensource libraries are firstly extra cost: there’s no support, no forum and no bug fixes. You’re on your own. You can often manage to download the source code to retired projects and support it yourself, but that’s not guaranteed and that costs money. The second problem of using deprecated code is extra cost: old code usually encompasses obsolete architecture and patterns, which contain known flaws and problems - after all, that’s why they’re obsolete. Using obsolete patterns and architecture encourages and in some cases forces developers to write bad code, not because your developers are bad, but that’s just the way it is... For example, there are some very obsolete JSP tags that blatantly mix database calls with business and presentation logic, which is a well know way of producing crumby, unmaintainable, spaghetti code. The third problem is, believe it or not, extra cost: I’ve recently come across a project where the code is so old that there are JAR file clashes, with different JARs containing different versions of the same API being dragged into the classpath. Certain bits of the code use one version of the API whilst other bits use the other version. eclipse didn’t know what to make of it all. There are also hidden costs: no one in there right mind wants to work on obsolete spaghetti code - it damages moral and saps the will to live, whilst damaging your ability to find that next, more highly paid, job. Plus, when people do leave, you’ve got the extra cost of finding and training their replacements. Never forget that the best people will be the first to leave, leaving you with the less experienced developers, again driving up your cost So, what can you do when faced with obsolete opensource libraries and frameworks? 1) Do nothing, continue using the obsolete library and hope everything will be alright. 2) Scrap the whole project and start again from scratch - the Big Bang Theory. 3) Refactor vigorously to remove the obsolete opensource code. This could also be seen as a way of changing the architecture of an application, updating the programming practices of the team and improving the code and whole build process. From the above I guess that you can figure out that in my opinion I prefer option 3. Option 1 is very risky, but then again, so is option 2: starting from scratch wastes time simply re-inventing the wheel, and whilst you do that, you don’t have a product, plus you may also end up with a big a mess as you started with. Option 3 is evolution and not revolution, quite the most sensible way to go. Having said all this, I definitely won’t stop using opensource code... From http://www.captaindebug.com/2011/09/benefits-and-dangers-of-using.html