DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

The Latest Languages Topics

article thumbnail
Android Tutorial: How to Parse/Read JSON Data Into a Android ListView
Today we get on with our series that will connect our Android applications to internet webservices! Next up in line: from JSON to a Listview. A lot of this project is identical to the previous post in this series so try to look there first if you have any problems. On the bottom of the post ill add the Eclipse project with the source. For this example i made use of an already existing JSON webservice located here. This is a piece of the JSON array that gets returned: {"earthquakes": [ { "eqid": "c0001xgp", "magnitude": 8.8, "lng": 142.369, "src": "us", "datetime": "2011-03-11 04:46:23", "depth": 24.4, "lat": 38.322 }, { "eqid": "2007hear", "magnitude": 8.4, "lng": 101.3815, "src": "us", "datetime": "2007-09-12 09:10:26", "depth": 30, "lat": -4.5172 }<--more -->]} So how do we get this data into our application! Behold our getJSON class! getJSON(String url) public static JSONObject getJSONfromURL(String url){//initializeInputStream is = null;String result = "";JSONObject jArray = null;//http posttry{HttpClient httpclient = new DefaultHttpClient();HttpPost httppost = new HttpPost(url);HttpResponse response = httpclient.execute(httppost);HttpEntity entity = response.getEntity();is = entity.getContent();}catch(Exception e){Log.e("log_tag", "Error in http connection "+e.toString());}//convert response to stringtry{BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8);StringBuilder sb = new StringBuilder();String line = null;while ((line = reader.readLine()) != null) {sb.append(line + "\n");}is.close();result=sb.toString();}catch(Exception e){Log.e("log_tag", "Error converting result "+e.toString());}//try parse the string to a JSON objecttry{ jArray = new JSONObject(result);}catch(JSONException e){Log.e("log_tag", "Error parsing data "+e.toString());}return jArray;} The code above can be divided in 3 parts. the first part makes the HTTP call the second part converts the stream into a String the third part converts the string to a JSPNObject Now we only have to implement this into out ListView. We can use the same method as in the XML tutorial. We make a HashMap that stores our data and we put JSON values in the HashMap. After that we will bind that HashMap to a SimpleAdapter. Here is how its done: Implementation ArrayList> mylist = new ArrayList>();//Get the data (see above)JSONObject json =JSONfunctions.getJSONfromURL("http://api.geonames.org/postalCodeSearchJSON?formatted=true&postalcode=9791&maxRows=10&username=demo&style=full"); try{//Get the element that holds the earthquakes ( JSONArray )JSONArray earthquakes = json.getJSONArray("earthquakes"); //Loop the Array for(int i=0;i < earthquakes.length();i++){ HashMap map = new HashMap(); JSONObject e = earthquakes.getJSONObject(i); map.put("id", String.valueOf(i)); map.put("name", "Earthquake name:" + e.getString("eqid")); map.put("magnitude", "Magnitude: " + e.getString("magnitude")); mylist.add(map);} }catch(JSONException e) { Log.e("log_tag", "Error parsing data "+e.toString()); } After this we only need to make up the Simple Adapter ListAdapter adapter = new SimpleAdapter(this, mylist , R.layout.main, new String[] { "name", "magnitude" }, new int[] { R.id.item_title, R.id.item_subtitle }); setListAdapter(adapter); final ListView lv = getListView(); lv.setTextFilterEnabled(true); lv.setOnItemClickListener(new OnItemClickListener() { public void onItemClick(AdapterView parent, View view, int position, long id) { @SuppressWarnings("unchecked") Toast.makeText(Main.this, "ID '" + o.get("id") + "' was clicked.", Toast.LENGTH_SHORT).show(); }); Now we have a ListView filled with JSON data! Here is the Eclipse project: source code Have fun playing around with it.
June 8, 2011
by Mark Mooibroek
· 260,353 Views
article thumbnail
CDI AOP Tutorial: Java Standard Method Interception Tutorial - Java EE
This article discusses CDI based AOP in a tutorial format. CDI is the Java standard for dependency injection (DI) and interception (AOP). It is evident from the popularity of DI and AOP that Java needs to address DI and AOP so that it can build other standards on top of it. DI and AOP are already the foundation of many Java frameworks. CDI is a foundational aspect of Java EE 6. It is or will be shortly supported by Caucho's Resin Java Application Server, Java EE WebProfile certified, IBM's WebSphere, Oracle's Glassfish, Red Hat's JBoss and many more application servers. CDI is similar to core Spring and Guice frameworks. Like JPA did for ORM, CDI simplifies and sanitizes the API for DI and AOP. If you have worked with Spring or Guice, you will find CDI easy to use and easy to learn. If you are new to AOP, then CDI is an easy on ramp for picking up AOP quickly, as it uses a small subset of what AOP provides. CDI based AOP is simpler to use and learn. One can argue that CDI only implements a small part of AOP—method interception. While this is a small part of what AOP has to offer, it is also the part that most developers use. CDI can be used standalone and can be embedded into any application. Here is the source code for this tutorial, and instructions for use. It is no accident that this tutorial follows many of the same examples in the written three years ago. It will be interesting to compare and contrast the examples in this tutorial with the one written three years ago for Spring based AOP. Design goals of this tutorial This tutorial is meant to be a description and explanation of AOP in CDI without the clutter of EJB 3.1 or JSF. There are already plenty of tutorials that cover EJB 3.1 and JSF (and CDI). We believe that CDI has merit on its own outside of the EJB and JSF space. This tutorial only covers CDI. Repeat there is no JSF 2 or EJB 3.1 in this tutorial. There are plenty of articles and tutorials that cover using CDI as part of a larger JEE 6 application. This tutorial is not that. This tutorial series is CDI and only CDI. This tutorial only has full, complete code examples with source code you can download and try out on your own. There are no code snippets where you can't figure out where in the code you are suppose to be. So far these tutorials have been well recieved and we got a lot of feedback. There appears to be a lot of interest in the CDI standard. Thanks for reading and thanks for your comments and participation so far. AOP Basics For some, AOP seems like voodoo magic. For others, AOP seems like a cure-all. For now, let's just say that AOP is a tool that you want in your developer toolbox. It can make seemingly impossible things easy. Aagin, when we talk about AOP in CDI, we are really talking about interception which is a small but very useful part of AOP. For brevity, I am going to refer to interception as AOP. The first time that I used AOP was with Spring's transaction management support. I did not realize I was using AOP. I just knew Spring could apply EJB-style declarative transaction management to POJOs. It was probably three to six months before I realized that I was using was Spring's AOP support. The Spring framework truly brought AOP out of the esoteric closet into the main stream light of day. CDI brings these concepts into the JSR standards where other Java standards can build on top of CDI. You can think of AOP as a way to apply services (called cross-cutting concerns) to objects. AOP encompasses more than this, but this is where it gets used mostly in the main stream. I've using AOP to apply caching services, transaction management, resource management, etc. to any number of objects in an application. I am currently working with a team of folks on the CDI implementation for the revived JSR-107 JCache. AOP is not a panacea, but it certainly fits a lot of otherwise difficult use cases. You can think of AOP as a dynamic decorator design pattern. The decorator pattern allows additional behavior to be added to an existing class by wrapping the original class and duplicating its interface and then delegating to the original. See this article decorator pattern for more detail about the decorator design pattern. (Notice in addition to supporting AOP style interception CDI also supports actual decorators, which are not covered in this article.) Sample application revisited For this introduction to AOP, let's take a simple example, let's apply security services to our Automated Teller Machine example from the first the first in this series. Let's say when a user logs into a system that a SecurityToken is created that carries the user's credentials and before methods on objects get invoked, we want to check to see if the user has credentials to invoke these methods. For review, let's look at the AutomatedTellerMachine interface. Code Listing: AutomatedTellerMachine interface package org.cdi.advocacy; import java.math.BigDecimal; public interface AutomatedTellerMachine { public abstract void deposit(BigDecimal bd); public abstract void withdraw(BigDecimal bd); } In a web application, you could write a ServletFilter, that stored this SecurityToken in HttpSession and then on every request retrieved the token from Session and put it into a ThreadLocal variable where it could be accessed from a SecurityService that you could implement. Perhaps the objects that needed the SecurityService could access it as follows: Code Listing: AutomatedTellerMachineImpl implementing security without AOP public void deposit(BigDecimal bd) { /* If the user is not logged in, don't let them use this method */ if(!securityManager.isLoggedIn()){ throw new SecurityViolationException(); } /* Only proceed if the current user is allowed. */ if (!securityManager.isAllowed("AutomatedTellerMachine", operationName)){ throw new SecurityViolationException(); } ... transport.communicateWithBank(...); } In our ATM example, the above might work out well, but imagine a system with thousands of classes that needed security. Now imagine, the way we check to see if a user is "logged in" changed. If we put this code into every method that needed security, then we could possibly have to change this a thousand times if we changed the way we checked to see if a user was logged in. What we want to do instead is to use CDI to create a decorated version of the AutomateTellerMachineImpl bean. The decorated version would add the additional behavior to the AutomateTellerMachineImpl object without changing the actual implementation of the AutomateTellerMachineImpl. In AOP speak, this concept is called a cross-cutting concern. A cross-cutting concern is a concern that crosses the boundry of many objects. CDI does this by creating what is called an AOP proxy. An AOP proxy is like a dynamic decorator. Underneath the covers CDI can generate a class at runtime (the AOP proxy) that has the same interface as our AutomatedTellerMachine. The AOP proxy wraps our existing atm object and provides additional behavior by delegating to a list of method interceptors. The method interceptors provide the additional behavior and are similar to ServletFilters but for methods instead of requests. Diagrams of CDI AOP support Thus before we added CDI AOP, our atm example was like Figure 1. Figure 1: Before AOP advice After we added AOP support, we now get an AOP proxy that applies the securityAdvice to the atm as show in figure 2. Figure 2: After AOP advice You can see that the AOP proxy implements the AutomatedTellerMachine interface. When the client object looks up the atm and starts invoking methods instead of executing the methods directly, it executes the method on the proxy, which then delegates the call to a series of method interceptor called advice, which eventually invoke the actual atm instance (now called atmTarget). Let's actually look at the code for this example. For this example, we will use a simplified SecurityToken that gets stored into a ThreadLocal variable, but one could imagine one that was populated with data from a database or an LDAP server or some other source of authentication and authorization. Here is the SecurityToken, which gets stored into a ThreadLocal variable, for this example: SecurityToken.java Gets stored in ThreadLocal package org.cdi.advocacy.security; /** * @author Richard Hightower * */ public class SecurityToken { private boolean allowed; private String userName; public SecurityToken() { } public SecurityToken(boolean allowed, String userName) { super(); this.allowed = allowed; this.userName = userName; } public boolean isAllowed(String object, String methodName) { return allowed; } /** * @return Returns the allowed. */ public boolean isAllowed() { return allowed; } /** * @param allowed The allowed to set. */ public void setAllowed(boolean allowed) { this.allowed = allowed; } /** * @return Returns the userName. */ public String getUserName() { return userName; } /** * @param userName The userName to set. */ public void setUserName(String userName) { this.userName = userName; } } The SecurityService stores the SecurityToken into the ThreadLocal variable, and then delegates to it to see if the current user has access to perform the current operation on the current object as follows: SecurityService.java Service package org.cdi.advocacy.security; public class SecurityService { private static ThreadLocal currentToken = new ThreadLocal(); public static void placeSecurityToken(SecurityToken token){ currentToken.set(token); } public static void clearSecuirtyToken(){ currentToken.set(null); } public boolean isLoggedIn(){ SecurityToken token = currentToken.get(); return token!=null; } public boolean isAllowed(String object, String method){ SecurityToken token = currentToken.get(); return token.isAllowed(); } public String getCurrentUserName(){ SecurityToken token = currentToken.get(); if (token!=null){ return token.getUserName(); }else { return "Unknown"; } } } The SecurityService will throw a SecurityViolationException if a user is not allowed to access a resource. SecurityViolationException is just a simple exception for this example. SecurityViolationException.java Exception package com.arcmind.springquickstart.security; /** * @author Richard Hightower * */ public class SecurityViolationException extends RuntimeException { /** * */ private static final long serialVersionUID = 1L; } To remove the security code out of the AutomatedTellerMachineImpl class and any other class that needs security, we will write an Aspect in CDI to intercept calls and perform security checks before the method call. To do this we will create a method interceptor (known is AOP speak as an advice) and intercept method calls on the atm object. Here is the SecurityAdvice class which will intercept calls on the AutomatedTellerMachineImpl class. SecurityAdvice package org.cdi.advocacy.security; import javax.inject.Inject; import javax.interceptor.AroundInvoke; import javax.interceptor.Interceptor; import javax.interceptor.InvocationContext; /** * @author Richard Hightower */ @Secure @Interceptor public class SecurityAdvice { @Inject private SecurityService securityManager; @AroundInvoke public Object checkSecurity(InvocationContext joinPoint) throws Exception { System.out.println("In SecurityAdvice"); /* If the user is not logged in, don't let them use this method */ if(!securityManager.isLoggedIn()){ throw new SecurityViolationException(); } /* Get the name of the method being invoked. */ String operationName = joinPoint.getMethod().getName(); /* Get the name of the object being invoked. */ String objectName = joinPoint.getTarget().getClass().getName(); /* * Invoke the method or next Interceptor in the list, * if the current user is allowed. */ if (!securityManager.isAllowed(objectName, operationName)){ throw new SecurityViolationException(); } return joinPoint.proceed(); } } Notice that we annotate the SecuirtyAdvice class with an @Secure annotation. The @Secure annotation is an @InterceptorBinding. We use it to denote both the interceptor and the classes it intercepts. More on this later. Notice that we use @Inject to inject the securityManager. Also we mark the method that implements that around advice with and @AroundInvoke annotation. This essentially says this is the method that does the dynamic decoration. Thus, the checkSecurity method of SecurityAdvice is the method that implements the advice. You can think of advice as the decoration that we want to apply to other objects. The objects getting the decoration are called advised objects. Notice that the SecurityService gets injected into the SecurityAdvice and the checkSecurity method uses the SecurityService* to see if the user is logged in and the user has the rights to execute the method. An instance of InvocationContext, namely joinPoint, is passed as an argument to checkSecurity. The InvocationContext has information about the method that is being called and provides control that determines if the method on the advised object's methods gets invoked (e.g., AutomatedTellerMachineImpl.withdraw and AutomatedTellerMachineImpl.deposit). If *`joinPoint.proceed()`* is not called then the wrapped method of the advised object (withdraw or deposit) is not called. (The proceed method causes the actual decorated method to be invoked or the next interceptor in the chain to get invoked.) In Spring, to apply an Advice like SecurityAdvice to an advised object, you need a pointcut. A pointcut is like a filter that picks the objects and methods that get decorated. In CDI, you just mark the class or methods of the class that you want decorated with an interceptor binding annotation. There is no complex pointcut language. You could implement one as a CDI extention, but it does not come with CDI by default. CDI uses the most common way developer apply interceptors, i.e., with annotations. CDI scans each class in each jar (and other classpath locations) that has a META-INF/beans.xml. The SecurityAdvice get installed in the CDI beans.xml. META-INF/beans.xml org.cdi.advocacy.security.SecurityAdvice You can install interceptors in the order you want them called. In order to associate a interceptor with the classes and methods it decorates, you have to define an InterceptorBinding annotation. An example of such a binding is defined below in the @Secure annotation. Secure.java annotation package org.cdi.advocacy.security; import java.lang.annotation.Retention; import java.lang.annotation.Target; import static java.lang.annotation.ElementType.*; import static java.lang.annotation.RetentionPolicy.*; import javax.interceptor.InterceptorBinding; @InterceptorBinding @Retention(RUNTIME) @Target({TYPE, METHOD}) public @interface Secure { } Notice that we annotated the @Secure annotation with the @InterceptorBinding annotation. InterceptorBindings follow a lot of the same rules as Qualifiers as discussed in the first two articles in this series. InterceptorBindings are like qaulifiers for injection in that they can have members which can further qualify the injection. You can also disable InterceptorBinding annotation members from qualifying an interception by using the @NonBinding just like you can in Qualifiers. To finish our example, we need to annotate our AutomatedTellerMachine with the same @Secure annotation; thus, associating the AutomatedTellerMachine with our SecurityAdvice. AutomatedTellerMachine class using @Secure package org.cdi.advocacy; ... import javax.inject.Inject; import org.cdi.advocacy.security.Secure; @Secure public class AutomatedTellerMachineImpl implements AutomatedTellerMachine { @Inject @Json private ATMTransport transport; public void deposit(BigDecimal bd) { System.out.println("deposit called"); transport.communicateWithBank(null); } public void withdraw(BigDecimal bd) { System.out.println("withdraw called"); transport.communicateWithBank(null); } } You have the option of use @Secure on the methods or at the class level. In this example, we annotated the class itself, which then applies the interceptor to every method. Let's complete our example by reviewing the AtmMain main method that looks up the atm out of CDI's beanContainer. Let's review AtmMain as follows: AtmMain.java package org.cdi.advocacy; import java.math.BigDecimal; import org.cdi.advocacy.security.SecurityToken; import org.cdiadvocate.beancontainer.BeanContainer; import org.cdiadvocate.beancontainer.BeanContainerManager; import org.cdi.advocacy.security.SecurityService; public class AtmMain { public static void simulateLogin() { SecurityService.placeSecurityToken(new SecurityToken(true, "Rick Hightower")); } public static void simulateNoAccess() { SecurityService.placeSecurityToken(new SecurityToken(false, "Tricky Lowtower")); } public static BeanContainer beanContainer = BeanContainerManager .getInstance(); static { beanContainer.start(); } public static void main(String[] args) throws Exception { simulateLogin(); //simulateNoAccess(); AutomatedTellerMachine atm = beanContainer .getBeanByType(AutomatedTellerMachine.class); atm.deposit(new BigDecimal("1.00")); } } Continue reading... Click on the navigation links below the author bio to read the other pages of this article. Be sure to check out part I of this series as well! Although not a fan of EJB 3, Rick is a big fan of the potential of CDI and thinks that EJB 3.1 has come a lot closer to the mark. CDI Implementations - Resin Candi - Seam Weld - Apache OpenWebBeans Before we added AOP support when we looked up the atm, we looked up the object directly as shown in figure 1, now that we applied AOP when we look up the object we get what is in figure 2. When we look up the atm in the application context, we get the AOP proxy that applies the decoration (advice, method interceptor) to the atm target by wrapping the target and delegating to it after it invokes the series of method interceptors. Victroy lap The last code listing works just like you think. If you use simulateLogin, atm.deposit does not throw a SecurityException. If you use simulateNoAccess, it does throw a SecurityException. Now let's weave in a few more "Aspects" to the mix to drive some points home and to show how interception works with multiple interceptors. I will go quicker this time. LoggingInterceptor package org.cdi.advocacy; import java.util.Arrays; import java.util.logging.Logger; import javax.interceptor.AroundInvoke; import javax.interceptor.Interceptor; import javax.interceptor.InvocationContext; @Logable @Interceptor public class LoggingInterceptor { @AroundInvoke public Object log(InvocationContext ctx) throws Exception { System.out.println("In LoggingInterceptor"); Logger logger = Logger.getLogger(ctx.getTarget().getClass().getName()); logger.info("before call to " + ctx.getMethod() + " with args " + Arrays.toString(ctx.getParameters())); Object returnMe = ctx.proceed(); logger.info("after call to " + ctx.getMethod() + " returned " + returnMe); return returnMe; } } Now we need to define the Logable interceptor binding annotation as follows: package org.cdi.advocacy; import java.lang.annotation.Retention; import java.lang.annotation.Target; import static java.lang.annotation.ElementType.*; import static java.lang.annotation.RetentionPolicy.*; import javax.interceptor.InterceptorBinding; @InterceptorBinding @Retention(RUNTIME) @Target({TYPE, METHOD}) public @interface Logable { } Now to use it we just mark the methods where we want this logging. AutomatedTellerMachineImpl.java using Logable package org.cdi.advocacy; ... @Secure public class AutomatedTellerMachineImpl implements AutomatedTellerMachine { ... @Logable public void deposit(BigDecimal bd) { System.out.println("deposit called"); transport.communicateWithBank(null); } public void withdraw(BigDecimal bd) { System.out.println("withdraw called"); transport.communicateWithBank(null); } } Notice that we use the @Secure at the class level which will applies the security interceptor to every mehtod in the AutomatedTellerMachineImpl. But, we use @Logable only on the deposit method which applies it, you guessed it, only on the deposit method. Now you have to add this interceptor to the beans.xml: META-INF/beans.xml org.cdi.advocacy.LoggingInterceptor org.cdi.advocacy.security.SecurityAdvice When we run this again, we get something like this in our console output: May 15, 2011 6:46:22 PM org.cdi.advocacy.LoggingInterceptor log INFO: before call to public void org.cdi.advocacy.AutomatedTellerMachineImpl.deposit(java.math.BigDecimal) with args [1.00] May 15, 2011 6:46:22 PM org.cdi.advocacy.LoggingInterceptor log INFO: after call to public void org.cdi.advocacy.AutomatedTellerMachineImpl.deposit(java.math.BigDecimal) returned null Notice that the order of interceptors in the beans.xml file determines the order of execution in the code. (I added a println to each interceptor just to show the ordering.) When we run this, we get the following output. Output: In LoggingInterceptor In SecurityAdvice If we switch the order in the beans.xml file, we will get a different order in the console output. META-INF/beans.xml org.cdi.advocacy.security.SecurityAdvice org.cdi.advocacy.LoggingInterceptor In SecurityAdvice In LoggingInterceptor This is important as many interceptors can be applied. You have one place to set the order. Conclusion AOP is neither a cure all or voodoo magic, but a powerful tool that needs to be in your bag of tricks. The Spring framework has brought AOP to the main stream masses and Spring 2.5/3.x has simplified using AOP. CDI brings AOP and DI into the standard's bodies where it can get further mainstreamed, refined and become part of future Java standards like JCache, Java EE 6 and Java EE 7. You can use Spring CDI to apply services (called cross-cutting concerns) to objects using AOP's interception model. AOP need not seem like a foreign concept as it is merely a more flexible version of the decorator design pattern. With AOP you can add additional behavior to an existing class without writing a lot of wrapper code. This can be a real time saver when you have a use case where you need to apply a cross cutting concern to a slew of classes. To reiterate... CDI is the Java standard for dependency injection and interception (AOP). It is evident from the popularity of DI and AOP that Java needs to address DI and AOP so that it can build other standards on top of it. DI and AOP are the foundation of many Java frameworks. I hope you share my excitement of CDI as a basis for other JSRs, Java frameworks and standards. CDI is a foundational aspect of Java EE 6. It is or will be shortly supported by Caucho's Resin, IBM's WebSphere, Oracle's Glassfish, Red Hat's JBoss and many more application servers. CDI is similar to core Spring and Guice frameworks. However CDI is a general purpose framework that can be used outside of JEE 6. CDI simplifies and sanitizes the API for DI and AOP. I find that working with CDI based AOP is easier and covers the most common use cases. CDI is a rethink on how to do dependency injection and AOP (interception really). It simplifies it. It reduces it. It gets rid of legacy, outdated ideas. CDI is to Spring and Guice what JPA is to Hibernate, and Toplink. CDI will co-exist with Spring and Guice. There are plugins to make them interoperate nicely (more on these shortly). This is just a brief taste. There is more to come. Resources CDI Source CDI advocacy group CDI advocacy blog CDI advocacy google code project Google group for CDI advocacy Manisfesto version 1 Weld reference documentation CDI JSR299 Resin fast and light CDI and Java EE 6 Web Profile implementation CDI & JSF Part 1 Intro by Andy Gibson CDI & JSF Part 2 Intro by Andy Gibson CDI & JSF Part 3 Intro by Andy Gibson About the Author This article was written with CDI advocacy in mind by Rick Hightower with some collaboration from others. Rick Hightower has worked as a CTO, Director of Development and a Developer for the last 20 years. He has been involved with J2EE since its inception. He worked at an EJB container company in 1999. He has been working with Java since 1996, and writing code professionally since 1990. Rick was an early Spring enthusiast. Rick enjoys bouncing back and forth between C, Python, Groovy and Java development. Although not a fan of EJB 3, Rick is a big fan of the potential of CDI and thinks that EJB 3.1 has come a lot closer to the mark. Rick Hightower is CTO of Mammatus and is an expert on Java and Cloud Computing. There are 18 code listings in this article
May 25, 2011
by Rick Hightower
· 83,674 Views · 10 Likes
article thumbnail
How to Iterate ArrayList in Struts2
We will discuss how to iterate over a collection of String objects in Struts2 tag libraries and then a List of custom class objects. It looks as if iterating a list of string objects is easier than iterating over a list of custom class objects in Struts 2. But the reality is that iterating a list of custom class objects is also equally easier. By custom class we mean the User, Employee, Department, Products, Vehicles classes that are created in any web application. Download Working Sample Here Usually it happens that one needs to fetch a list of records from database/files and then display it in the JSP. The module requiring this functionality could be Search, Listing users/departments/products etc. The basic flow of struts2 web application goes like: The user initiates the request from one page. This request is received by the interceptor which further invokes the Struts2 action. The action class fetches the records and stores in a list. This list is available to the next JSP using the public getter method. Please note that the public getter method for the List is mandatory. Once the List has been populated by Struts2 action class, the JSP then iterates over this List and displays the corresponding information. In the days gone by, one would store the List as a session attribute and then access the list in JSP using the scriptlets to display appropriate output to the users. Here is a Struts2 sample application to iterate one String and one custom class objects List. Though we are using the Struts2 tag library to iterate the list but JSTL can also be used for iteration. Also if you are going to use the code examples given below, use the following URL's to access the application: http://localhost:8080//index.action Iterate a Custom class ArrayList in Struts2 web.xml struts2 org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter struts2 *.action struts.xml /home.jsp /success.jsp /failure.jsp home.jsp Enter a user name to get the documents uploaded by that user. Username success.jsp Documents uploaded by the user are: failure.jsp FileAction.java package com.example; import java.util.ArrayList; import java.util.List; public class FetchAction { private String username; private String message; private List documents = new ArrayList(); public List getDocuments() { return documents; } public String getMessage() { return message; } public void setMessage(String message) { this.message = message; } public String getUsername() { return username; } public void setUsername(String username) { this.username = username; } public String execute() { if( username != null) { //logic to fetch the document list (say from database) Document d1 = new Document(); d1.setName("user.doc"); Document d2 = new Document(); d2.setName("office.doc"); Document d3 = new Document(); d3.setName("transactions.doc"); documents.add(d1); documents.add(d2); documents.add(d3); return "success"; } else { message="Unable to fetch"; return "failure"; } } } Document.java package com.example; public class Document { private String name; public String getName() { return name; } public void setName(String name) { this.name = name; } } Iterate String List in Struts2 The way to iterate the a String list is similar with the only difference that the action class FetchAction.java now populates the name of documents into an ArrayList of String objects. The code zip file containing the iteration over an ArrayList of custom class object or bean can be downloaded at: http://www.fileserve.com/file/QmrsJ7k The URL to access this application will be: http://localhost:8080/IteratorExample/index.action The code zip file containing the iteration over an ArrayList of string class object or bean can be downloaded at: http://www.fileserve.com/file/V2kXkfx The URL to access this application will be: http://localhost:8080/StringIteratorExample/index.action From http://extreme-java.blogspot.com/2011/05/how-to-iterate-arraylist-in-struts2.html
May 17, 2011
by Sandeep Bhandari
· 71,003 Views
article thumbnail
Java Web Application Security - Part I: Java EE 6 Login Demo
Back in February, I wrote about my upcoming conferences: In addition to Vegas and Poland, there's a couple other events I might speak at in the next few months: the Utah Java Users Group (possibly in April), Jazoon and ÜberConf (if my proposals are accepted). For these events, I'm hoping to present the following talk: Webapp Security: Develop. Penetrate. Protect. Relax. In this session, you'll learn how to implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol' Java EE Container Managed Authentication. You'll also learn how to secure your REST API with OAuth and lock it down with SSL. After learning how to develop authentication, I'll introduce you to OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. From there, I'll discuss using WebGoat to verify your app is secure and commercial tools like webapp firewalls and accelerators. Fast forward a couple months and I'm happy to say that I've completed my talk at the Utah JUG and it's been accepted at Jazoon and Über Conf. For this talk, I created a presentation that primarily consists of demos implementing basic, form and Ajax authentication using Java EE 6, Spring Security and Apache Shiro. In the process of creating the demos, I learned (or re-educated myself) how to do a number of things in all 3 frameworks: Implement Basic Authentication Implement Form-based Authentication Implement Ajax HTTP -> HTTPS Authentication (with programmatic APIs) Force SSL for certain URLs Implement a file-based store of users and passwords (in Jetty/Maven and Tomcat standalone) Implement a database store of users and passwords (in Jetty/Maven and Tomcat standalone) Encrypt Passwords Secure methods with annotations For the demos, I showed the audience how to do almost all of these, but skipped Tomcat standalone and securing methods in the interest of time. In July, when I do this talk at ÜberConf, I plan on adding 1) hacking the app (to show security holes) and 2) fixing it to protect it against vulnerabilities. I told the audience at UJUG that I would post the presentation and was planning on recording screencasts of the various demos so the online version of the presentation would make more sense. Today, I've finished the first screencast showing how to implement security with Java EE 6. Below is the presentation (with the screencast embedded on slide 10) as well as a step-by-step tutorial. * You can also watch the screencast on YouTube or download the presentation PDF. Java EE 6 Login Tutorial Download and Run the Application Implement Basic Authentication Implement Form-based Authentication Force SSL Store Users in a Database Summary Download and Run the Application To begin, download the application you'll be implementing security in. This app is a stripped-down version of the Ajax Login application I wrote for my article on Implementing Ajax Authentication using jQuery, Spring Security and HTTPS. You'll need Java 6 and Maven installed to run the app. Run it using mvn jetty:run and open http://localhost:8080 in your browser. You'll see it's a simple CRUD application for users and there's no login required to add or delete users. Implement Basic Authentication The first step is to protect the list screen so people have to login to view users. To do this, add the following to the bottom of src/main/webapp/WEB-INF/web.xml: users /users GET POST ROLE_ADMIN BASIC Java EE Login ROLE_ADMIN At this point, if you restart Jetty (Ctrl+C and jetty:run again), you'll get an error about a missing LoginService. This happens because Jetty doesn't know where the "Java EE Login" realm is located. Add the following to pom.xml, just after in the Jetty plugin's configuration. Java EE Login ${basedir}/src/test/resources/realm.properties The realm.properties file already exists in the project and contains user names and passwords. Start the app again using mvn jetty:run and you should be prompted to login when you click on the "Users" tab. Enter admin/admin to login. After logging in, you can try to logout by clicking the "Logout" link in the top-right corner. This calls a LogoutController with the following code that logs the user out. public void logout(HttpServletResponse response) throws ServletException, IOException { request.getSession().invalidate(); response.sendRedirect(request.getContextPath()); } You'll notice that clicking this link doesn't log you out, even though the session is invalidated. The only way to logout with basic authentication is to close the browser. In order to get the ability to logout, as well as to have more control over the look-and-feel of the login, you can implement form-based authentication. Implement Form-based Authentication To change from basic to form-based authentication, you simply have to replace the in your web.xml with the following: FORM /login.jsp /login.jsp?error=true The login.jsp page already exists in the project, in the src/main/webapp directory. This JSP has 3 important elements: 1) a form that submits to "${contextPath}/j_security_check", 2) an input element named "j_username" and 3) an input element named "j_password". If you restart Jetty, you'll now be prompted to login with this JSP instead of the basic authentication dialog. Force SSL Another thing you might want to implement to secure your application is forcing SSL for certain URLs. To do this on the same you already have in web.xml, add the following after : CONFIDENTIAL To configure Jetty to listen on an SSL port, add the following just after in your pom.xml: true 8080 true 8443 60000 ${project.build.directory}/ssl.keystore appfuse appfuse The keystore must be generated for Jetty to start successfully, so add the keytool-maven-plugin just above the jetty-maven-plugin in pom.xml. org.codehaus.mojo keytool-maven-plugin 1.0 generate-resources clean clean generate-resources genkey genkey ${project.build.directory}/ssl.keystore cn=localhost appfuse appfuse appfuse RSA Now if you restart Jetty, go to http://localhost:8080 and click on the "Users" tab, you'll get a 403. What the heck?! When this first happened to me, it took me a while to figure out. It turns out that Jetty doesn't redirect to HTTPS when using Java EE authentication, so you have to manually type in https://localhost:8443/ (or add a filter to redirect for you). If you deployed this same application on Tomcat (after enabling SSL), it would redirect for you. Store Users in a Database Finally, to store your users in a database instead of file, you'll need to change the in the Jetty plugin's configuration. Replace the existing element with the following: Java EE Login ${basedir}/src/test/resources/jdbc-realm.properties The jdbc-realm.properties file already exists in the project and contains the database settings and table/column names for the user and role information. jdbcdriver = com.mysql.jdbc.Driver url = jdbc:mysql://localhost/appfuse username = root password = usertable = app_user usertablekey = id usertableuserfield = username usertablepasswordfield = password roletable = role roletablekey = id roletablerolefield = name userroletable = user_role userroletableuserkey = user_id userroletablerolekey = role_id cachetime = 300 Of course, you'll need to install MySQL for this to work. After installing it, you should be able to create an "appfuse" database and populate it using the following commands: mysql -u root -p -e 'create database appfuse' curl https://gist.github.com/raw/958091/ceecb4a6ae31c31429d5639d0d1e6bfd93e2ea42/create-appfuse.sql > create-appfuse.sql mysql -u root -p appfuse < create-appfuse.sql Next you'll need to configure Jetty so it has MySQL's JDBC Driver in its classpath. To do this, add the following dependency just after the element (before ) in pom.xml: mysql mysql-connector-java 5.1.14 Now run the jetty-password.sh file in the root directory of the project to generate a password of your choosing. For example: $ sh jetty-password.sh javaeelogin javaeelogin OBF:1vuj1t2v1wum1u9d1ugo1t331uh21ua51wts1t3b1vur MD5:53b176e6ce1b5183bc970ef1ebaffd44 The last two lines are obfuscated and MD5 versions of the password. Update the admin user's password to this new value. You can do this with the following SQL statement. UPDATE app_user SET password='MD5:53b176e6ce1b5183bc970ef1ebaffd44' WHERE username = 'admin'; Now if you restart Jetty, you should be able to login with admin/javaeelogin and view the list of users. Summary In this tutorial, you learned how to implement authentication using standard Java EE 6. In addition to the basic XML configuration, there's also some new methods in HttpServletRequest for Java EE 6 and Servlet 3.0: authenticate(response) login(user, pass) logout() This tutorial doesn't show you how to use them, but I did play with them a bit as part of my UJUG demo when implementing Ajax authentication. I found that login() did work, but it didn't persist the authentication for the users session. I also found that after calling logout(), I still needed to invalidate the session to completely logout the user. There are some additional limitations I found with Java EE authentication, namely: No error messages for failed logins No Remember Me No auto-redirect from HTTP to HTTPS Container has to be configured Doesn’t support regular expressions for URLs Of course, no error messages indicating why login failed is probably a good thing (you don't want to tell users why their credentials failed). However, when you're trying to figure out if your container is configured properly, the lack of container logging can be a pain. In the next couple weeks, I'll post Part II of this series, where I'll show you how to implement this same set of features using Spring Security. In the meantime, please let me know if you have any questions. From http://raibledesigns.com/rd/entry/java_web_application_security_part
May 15, 2011
by Matt Raible
· 42,002 Views · 3 Likes
article thumbnail
Real time monitoring PHP applications with websockets and node.js
The inspection of the error logs is a common way to detect errors and bugs. We also can show errors on-screen within our developement server, or we even can use great tools like firePHP to show our PHP errors and warnings inside our firebug console. That’s cool, but we only can see our session errors/warnings. If we want to see another’s errors we need to inspect the error log. tail -f is our friend, but we need to surf against all the warnings of all sessions to see our desired ones. Because of that I want to build a tool to monitor my PHP applications in real-time. Let’s start: What’s the idea? The idea is catch all PHP’s errors and warnings at run time and send them to a node.js HTTP server. This server will work similar than a chat server but our clients will only be able to read the server’s logs. Basically the applications have three parts: the node.js server, the web client (html5) and the server part (PHP). Let me explain a bit each part: The node Server Basically it has two parts: a http server to handle the PHP errors/warnings and a websocket server to manage the realtime communications with the browser. When I say that I’m using websockets that’s means the web client will only work with a browser with websocket support like chrome. Anyway it’s pretty straightforward swap from a websocket sever to a socket.io server to use it with every browser. But websockets seems to be the future, so I will use websockets in this example. The http server: http.createServer(function (req, res) { var remoteAdrress = req.socket.remoteAddress; if (allowedIP.indexOf(remoteAdrress) >= 0) { res.writeHead(200, { 'Content-Type': 'text/plain' }); res.end('Ok\n'); try { var parsedUrl = url.parse(req.url, true); var type = parsedUrl.query.type; var logString = parsedUrl.query.logString; var ip = eval(parsedUrl.query.logString)[0]; if (inspectingUrl == "" || inspectingUrl == ip) { clients.forEach(function(client) { client.write(logString); }); } } catch(err) { console.log("500 to " + remoteAdrress); res.writeHead(500, { 'Content-Type': 'text/plain' }); res.end('System Error\n'); } } else { console.log("401 to " + remoteAdrress); res.writeHead(401, { 'Content-Type': 'text/plain' }); res.end('Not Authorized\n'); } }).listen(httpConf.port, httpConf.host); and the web socket server: var inspectingUrl = undefined; ws.createServer(function(websocket) { websocket.on('connect', function(resource) { var parsedUrl = url.parse(resource, true); inspectingUrl = parsedUrl.query.ip; clients.push(websocket); }); websocket.on('close', function() { var pos = clients.indexOf(websocket); if (pos >= 0) { clients.splice(pos, 1); } }); }).listen(wsConf.port, wsConf.host); If you want to know more about node.js and see more examples, have a look to the great site: http://nodetuts.com/. In this site Pedro Teixeira will show examples and node.js tutorials. In fact my node.js http + websoket server is a mix of two tutorials from this site. The web client. The web client is a simple websockets application. We will handle the websockets connection, reconnect if it dies and a bit more. I’s based on node.js chat demo Real time monitor Socket status: Conecting ...IP: [all]" ?>count: 0 And the javascript magic var timeout = 5000; var wsServer = '192.168.2.2:8880'; var unread = 0; var focus = false; var count = 0; function updateCount() { count++; $("#count").text(count); } function cleanString(string) { return string.replace(/&/g,"&").replace(//g,">"); } function updateUptime () { var now = new Date(); $("#uptime").text(now.toRelativeTime()); } function updateTitle(){ if (unread) { document.title = "(" + unread.toString() + ") Real time " + selectedIp + " monitor"; } else { document.title = "Real time " + selectedIp + " monitor"; } } function pad(n) { return ("0" + n).slice(-2); } function startWs(ip) { try { ws = new WebSocket("ws://" + wsServer + "?ip=" + ip); $('#toolbar').css('background', '#65A33F'); $('#socketStatus').html('Connected to ' + wsServer); //console.log("startWs:" + ip); //listen for browser events so we know to update the document title $(window).bind("blur", function() { focus = false; updateTitle(); }); $(window).bind("focus", function() { focus = true; unread = 0; updateTitle(); }); } catch (err) { //console.log(err); setTimeout(startWs, timeout); } ws.onmessage = function(event) { unread++; updateTitle(); var now = new Date(); var hh = pad(now.getHours()); var mm = pad(now.getMinutes()); var ss = pad(now.getSeconds()); var timeMark = '[' + hh + ':' + mm + ':' + ss + '] '; logString = eval(event.data); var host = logString[0]; var line = "" + timeMark + "" + host + ""; line += "" + logString[1]; + ""; if (logString[2]) { line += " " + logString[2] + ""; } $('#log').append(line); updateCount(); window.scrollBy(0, 100000000000000000); }; ws.onclose = function(){ //console.log("ws.onclose"); $('#toolbar').css('background', '#933'); $('#socketStatus').html('Disconected'); setTimeout(function() {startWs(selectedIp)}, timeout); } } $(document).ready(function() { startWs(selectedIp); }); The server part: The server part will handle silently all PHP warnings and errors and it will send them to the node server. The idea is to place a minimal PHP line of code at the beginning of the application that we want to monitor. Imagine the following piece of PHP code $a = $var[1]; $a = 1/0; class Dummy { static function err() { throw new Exception("error"); } } Dummy1::err(); it will throw: A notice: Undefined variable: var A warning: Division by zero An Uncaught exception ‘Exception’ with message ‘error’ So we will add our small library to catch those errors and send them to the node server include('client/NodeLog.php'); NodeLog::init('192.168.2.2'); $a = $var[1]; $a = 1/0; class Dummy { static function err() { throw new Exception("error"); } } Dummy1::err(); The script will work in the same way than the fist version but if we start our node.js server in a console: $ node server.js HTTP server started at 192.168.2.2::5672 Web Socket server started at 192.168.2.2::8880 We will see those errors/warnings in real-time when we start our browser Here we can see a small screencast with the working application: This is the server side library: class NodeLog { const NODE_DEF_HOST = '127.0.0.1'; const NODE_DEF_PORT = 5672; private $_host; private $_port; /** * @param String $host * @param Integer $port * @return NodeLog */ static function connect($host = null, $port = null) { return new self(is_null($host) ? self::$_defHost : $host, is_null($port) ? self::$_defPort : $port); } function __construct($host, $port) { $this->_host = $host; $this->_port = $port; } /** * @param String $log * @return Array array($status, $response) */ public function log($log) { list($status, $response) = $this->send(json_encode($log)); return array($status, $response); } private function send($log) { $url = "http://{$this->_host}:{$this->_port}?logString=" . urlencode($log); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_NOBODY, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $response = curl_exec($ch); $status = curl_getinfo($ch, CURLINFO_HTTP_CODE); curl_close($ch); return array($status, $response); } static function getip() { $realip = '0.0.0.0'; if ($_SERVER) { if ( isset($_SERVER['HTTP_X_FORWARDED_FOR']) && $_SERVER['HTTP_X_FORWARDED_FOR'] ) { $realip = $_SERVER["HTTP_X_FORWARDED_FOR"]; } elseif ( isset($_SERVER['HTTP_CLIENT_IP']) && $_SERVER["HTTP_CLIENT_IP"] ) { $realip = $_SERVER["HTTP_CLIENT_IP"]; } else { $realip = $_SERVER["REMOTE_ADDR"]; } } else { if ( getenv('HTTP_X_FORWARDED_FOR') ) { $realip = getenv('HTTP_X_FORWARDED_FOR'); } elseif ( getenv('HTTP_CLIENT_IP') ) { $realip = getenv('HTTP_CLIENT_IP'); } else { $realip = getenv('REMOTE_ADDR'); } } return $realip; } public static function getErrorName($err) { $errors = array( E_ERROR => 'ERROR', E_RECOVERABLE_ERROR => 'RECOVERABLE_ERROR', E_WARNING => 'WARNING', E_PARSE => 'PARSE', E_NOTICE => 'NOTICE', E_STRICT => 'STRICT', E_DEPRECATED => 'DEPRECATED', E_CORE_ERROR => 'CORE_ERROR', E_CORE_WARNING => 'CORE_WARNING', E_COMPILE_ERROR => 'COMPILE_ERROR', E_COMPILE_WARNING => 'COMPILE_WARNING', E_USER_ERROR => 'USER_ERROR', E_USER_WARNING => 'USER_WARNING', E_USER_NOTICE => 'USER_NOTICE', E_USER_DEPRECATED => 'USER_DEPRECATED', ); return $errors[$err]; } private static function set_error_handler($nodeHost, $nodePort) { set_error_handler(function ($errno, $errstr, $errfile, $errline) use($nodeHost, $nodePort) { $err = NodeLog::getErrorName($errno); /* if (!(error_reporting() & $errno)) { // This error code is not included in error_reporting return; } */ $log = array( NodeLog::getip(), "{$err} {$errfile}:{$errline}", nl2br($errstr) ); NodeLog::connect($nodeHost, $nodePort)->log($log); return false; }); } private static function register_exceptionHandler($nodeHost, $nodePort) { set_exception_handler(function($exception) use($nodeHost, $nodePort) { $exceptionName = get_class($exception); $message = $exception->getMessage(); $file = $exception->getFile(); $line = $exception->getLine(); $trace = $exception->getTraceAsString(); $msg = count($trace) > 0 ? "Stack trace:\n{$trace}" : null; $log = array( NodeLog::getip(), nl2br("Uncaught exception '{$exceptionName}' with message '{$message}' in {$file}:{$line}"), nl2br($msg) ); NodeLog::connect($nodeHost, $nodePort)->log($log); return false; }); } private static function register_shutdown_function($nodeHost, $nodePort) { register_shutdown_function(function() use($nodeHost, $nodePort) { $error = error_get_last(); if ($error['type'] == E_ERROR) { $err = NodeLog::getErrorName($error['type']); $log = array( NodeLog::getip(), "{$err} {$error['file']}:{$error['line']}", nl2br($error['message']) ); NodeLog::connect($nodeHost, $nodePort)->log($log); } echo NodeLog::connect($nodeHost, $nodePort)->end(); }); } private static $_defHost = self::NODE_DEF_HOST; private static $_defPort = self::NODE_DEF_PORT; /** * @param String $host * @param Integer $port * @return NodeLog */ public static function init($host = self::NODE_DEF_HOST, $port = self::NODE_DEF_PORT) { self::$_defHost = $host; self::$_defPort = $port; self::register_exceptionHandler($host, $port); self::set_error_handler($host, $port); self::register_shutdown_function($host, $port); $node = self::connect($host, $port); $node->start(); return $node; } private static $time; private static $mem; public function start() { self::$time = microtime(TRUE); self::$mem = memory_get_usage(); $log = array(NodeLog::getip(), "Start >>>> {$_SERVER['REQUEST_URI']}"); $this->log($log); } public function end() { $mem = (memory_get_usage() - self::$mem) / (1024 * 1024); $time = microtime(TRUE) - self::$time; $log = array(NodeLog::getip(), "End <<<< mem: {$mem} time {$time}"); $this->log($log); } } And of course the full code on gitHub: RealTimeMonitor
May 15, 2011
by Gonzalo Ayuso
· 29,292 Views
article thumbnail
Reset MySQL Root Password On Linux
Five easy steps to reset MySQL root password. Stop the MySQL server. Start the MySQL server with the --skip-grant-tables option. (it will not prompt for password) Connect to MySQL server as the root user. Setup new MySQL root password. Exit and restart the MySQL server. ### Shell Commands ### /etc/init.d/mysql stop mysqld_safe --skip-grant-tables & mysql -u root ### SQL Commands ### use mysql; UPDATE user SET password=PASSWORD("new-password") WHERE user='root'; flush privileges; exit ### Shell Commands ### /etc/init.d/mysql stop /etc/init.d/mysql start mysql -u root -pnew-password
May 13, 2011
by Artur Mkrtchyan
· 5,779 Views
article thumbnail
Practical PHP Testing Patterns: Transaction Rollback Teardown
Maintaining isolation of tests when they have a database as Shared Fixture is not a trivial task. An important constraint is not having the headache of keeping track what manipulations on the database has your code done; in that case the rollback may not even be performed in case of a regression. An alternative way to resetting the database via DELETE and TRUNCATE queries is to roll back a transaction which has been started in the setup phase during the teardown. Implementation The phases of a database test involving Transaction Rollback Teardown are roughly the following: begin transaction, usually in setUp(). arrange, act, assert actions in the various Test Methods. rollback of the transaction in teardown(). The active transaction is never committed. An issue with using this pattern is that code that already uses transaction is prone to generate errors, and ultimately should never be tested with this technique. The rules for your safety are simple: the SUT should never start a transaction or committing it. Some databases support nested transaction levels, but it's very brittle to use them for testing purposes, and in case of any failure the whole suite will blow up as test executes teardowns at the wrong level of nesting. This pattern safety is also difficult to ensure, as DDL statements like CREATE/DELETE or other commands may commit the current transaction automatically. Check the documentation of your testing database. The advantage of this pattern is great performance: rollback is faster than every other command, including TRUNCATE. Moreover, if you encapsulate transactions well in your production code, most of it won't commit them (typically leaving the control over the transaction to an upper layer). Doctrine 2 In a sense, we already use this pattern with an UnitOfWork ORM such as Doctrine 2 when we do not flush() the ORM in our code. The flow is: The database is ready by setup. Exercise code. Check results as persisted or removed entities. Instead of calling flush() over the Entity Manager, call clear(). In this case, the database never sees a transaction, as Doctrine 2 keeps everything in the Unit Of Work until you say to flush it. Even when your code is calling flush(), you can explicitly use beginTransaction() and rollback() over the connection object: in this other scenario, the testing database sees an open transaction, but it's never committed and can be discarded in teardown() like the pattern prescribes. Example The code sample is the same test case shown in the Table Truncation Teardown article, which now uses transactions to encapsulate the single tests. The various tests check the tables content is restored, along with the AUTOINCREMENT next value. exec('CREATE TABLE users ( id INTEGER PRIMARY KEY AUTOINCREMENT, name VARCHAR(255) )'); } $this->connection = self::$sharedConnection; $this->connection->beginTransaction(); } public function teardown() { $this->connection->rollback(); } public function testTableCanBePopulated() { $this->connection->exec('INSERT INTO users (name) VALUES ("Giorgio")'); $this->assertEquals(1, $this->howManyUsers()); } public function testTableRestartsFrom1() { $this->assertEquals(0, $this->howManyUsers()); $this->connection->exec('INSERT INTO users (name) VALUES ("Isaac")'); $stmt = $this->connection->query('SELECT name FROM users WHERE id=1'); $result = $stmt->fetch(); $this->assertEquals('Isaac', $result['name']); } public function testTableIsEmpty() { $this->assertEquals(0, $this->howManyUsers()); } private function howManyUsers() { $stmt = $this->connection->query('SELECT COUNT(*) AS number FROM users'); $result = $stmt->fetch(); return $result['number']; } }
May 11, 2011
by Giorgio Sironi
· 7,769 Views
article thumbnail
10 Tricky Java Interview Questions
Here are some Java interview questions which are un-common What is the performance effect of a large number of import statements which are not used? Answer: They are ignored if the corresponding class is not used. Give a scenario where hotspot will optimize your code? Answer: If we have defined a variable as static and then initialized this variable in a static block then the Hotspot will merge the variable and the initialization in a single statement and hence reduce the code. What will happen if an exception is thrown from the finally block? Answer: The program will exit if the exception is not catched in the finally block. How does decorator design pattern works in I/O classes? Answer: The various classes like BufferedReader , BufferedWriter workk on the underlying stream classes. Thus Buffered* class will provide a Buffer for Reader/Writer classes. If I give you an assignment to design Shopping cart web application, how will you define the architecture of this application. You are free to choose any framework, tool or server? Answer: Usually I will choose a MVC framework which will make me use other design patterns like Front Controller, Business Delegate, Service Locater, DAO, DTO, Loose Coupling etc. Struts 2 is very easy to configure and comes with other plugins like Tiles, Velocity and Validator etc. The architecture of Struts becomes the architecture of my application with various actions and corresponding JSP pages in place. What is a deadlock in Java? How will you detect and get rid of deadlocks? Answer: Deadlock exists when two threads try to get hold of a object which is already held by another object. Why is it better to use hibernate than JDBC for database interaction in various Java applications? Answer: Hibernate provides an OO view of the database by mapping the various classes to the database tables. This helps in thinking in terms of the OO language then in RDBMS terms and hence increases productivity. How can one call one constructor from another constructor in a class? Answer: Use the this() method to refer to constructors. What is the purpose of intern() method in the String class? Answer: It helps in moving the normal string objects to move to the String literal pool How will you make your web application to use the https protocol? Answer: This has more to do with the particular server being used than the application itself. Here is how it can be done on tomcat: http://tomcat.apache.org/tomcat-4.1-doc/ssl-howto.html From http://extreme-java.blogspot.com/2011/05/10-tricky-java-interview-questions.html
May 10, 2011
by Sandeep Bhandari
· 101,769 Views · 1 Like
article thumbnail
Practical PHP Testing Patterns: Stored Procedure Test
It happened in the day before the advent of DDD and the Hexagonal architecture, that you had code that lived inside the database, such as Stored Procedures, constraints, and triggers. Back in the day, the relational database was considered the single source of truth instead of a Domain Model written in a language like PHP or Java. Today the picture is different - but there are still scenarios where pushing code in the database make sense. One of the reasons for having logic expressed as SQL and in other database languages is their power, and their performance. SQL operators, especially when augmented by proprietary extensions, let you declare pieces of logic that you would instead have to code by hand. SQL that is executed directly on the database can accomplish operations too onerous to perform over a reconstituted object graph with a subsequent saving. In fact, every decent ORM include a language for batch updates that translates to SQL, like Doctrine with DQL; and also a mechanism for providing hints for the underlying database, like indexes definitions. The problem with SQL derivates and other database-specific embedded logic is that we cannot execute it and test it in isolation - we need a real copy of the database to perform our tests. Thus the Stored Procedure Test is an umbrella term for tests that encompasses database code, even when they're not actually stored procedures. When I'll use the term stored procedure in this article, it will be to signify any database-specific code, such as complex queries, triggers and so on. Implementation The pattern prescribes to write unit tests for the stored procedure, to test it in isolation from the rest of application a first simplification. These tests cover nontrivial logic in database code - probably you don't need them for indexes definition, but more for queries with aggregate functions. In the PHP world, Sqlite often suffices for testing queries - as long as you have an intermediate layer like Doctrine DBAL (part of Doctrine 2) which smooths out the differences between vendors. You use MySQL in production, Sqlite in the test suite, and you can write queries in Doctrine's DQL being confident that it will translate them to the right SQL dialect. These tests should be executed in a sandbox - a database with just enough structure and data to test the stored procedure at hand. This sandbox should run by definition on the production dbms. The most difficult aspect of the pattern is integrating with the dbms: it should be running and listening on the right port. A sandbox should be created in setUp() or setUpBeforeClass(), and destroyed during teardown. In case the database is not available, the tests should be marked as skipped or incomplete. Variations In In-Database Stored Procedure Tests, the test is written in the same language as the database code. I cannot imagine something more boring for a PHP programmer. In Remoted Stored Procedure Tests, which is the variation of interest for us, the tests are written in PHPUnit and integrated with the suite (slowing it down a bit). The logic is that whatever SQL logic you're going to add to your application, is already encapsulated in some PHP class: for example, complex queries are encapsulated in Repositories or DAO. So it's going to be feasible to build a sandbox via PHP code, and test the stored procedure as a black box. It will be encapsulated for a unique execution, like schema creation, or for executing multiple times in case of queries. Example The example shows you how to test a query with a real database - supposing a surrogate database does not support all the needed functions - from inside a test suite. I thought it would be difficult to write this test, but instead it required less than a Pomodoro. connection = new PDO("mysql:host=localhost;dbname=sandbox", 'root', ''); $this->connection->exec("CREATE TABLE users (name VARCHAR(255) NOT NULL PRIMARY KEY, year YEAR)"); $this->repository = new UserRepository($this->connection, 2011); } public function testAverageAgeIsCalculated() { $this->insertUser('Giorgio', 1942); $this->insertUser('Isaac', 1920); $this->assertEquals(80, $this->repository->getAverageAge()); } private function insertUser($name, $year) { $stmt = $this->connection->prepare("INSERT INTO users (name, year) VALUES (:name, :year)"); $stmt->bindValue('name', $name, PDO::PARAM_STR); $stmt->bindValue('year', $year, PDO::PARAM_INT); return $stmt->execute(); } public function tearDown() { $this->connection->exec('DROP TABLE users'); } } class UserRepository { private $connection; private $currentYear; public function __construct(PDO $connection, $currentYear) { $this->connection = $connection; $this->currentYear = $currentYear; } /** * We suppose AVG() cannot be correctly implemented by Sqlite or * another surrogate database (substitute another vendor feature * for the same effect). * We also suppose reconstituting millions of User objects to calculate * their average age isn't feasible: that's why we used SQL directly. */ public function getAverageAge() { $stmt = $this->connection->prepare('SELECT AVG(:year - year) AS average_age FROM users'); $stmt->bindValue('year', $this->currentYear, PDO::PARAM_INT); $stmt->execute(); $row = $stmt->fetch(); return $row['average_age']; } }
May 4, 2011
by Giorgio Sironi
· 2,813 Views
article thumbnail
Gant 1.9.5 released
There was a regression in the 1.9.4 release of Gant that could not be ignored. The problem has been corrected, and a new release candidate checked by the people who found the regression (thanks due to Jeff Brown and the Grails folk). There is therefore a shiny new Gant 1.9.5 release. Everything should be in place at Codehaus already and Maven will get updated as soon as the Codehaus -> Maven sync happens. The regression whilst essentially trivial, was sufficiently fundamental that I have taken the drastic step of removing the 1.9.4 release from everywhere. This should have no effect on people using Groovy 1.7 or 1.8, they should use Gant 1.9.5. People still using Groovy 1.6 will not see a Gant 1.9.5 and will have to stay weith Gant 1.9.3. Of course people still using Groovy 1.6 should upgrade to 1.8.0 immediately and therefore not have a problem with Gant :-)
May 3, 2011
by Russel Winder
· 340 Views
article thumbnail
Why does Void class exist in JDK
I always try to bring some thing new and useful on this blog. This time we will understand the Void.class (which in itself looks something tricky) present in rt.jar. One can consider the java.lang.Void class as a wrapper for the keyword void. Some developers draw the analogy with the primitive data types int, long, short and byte etc. which have the wrapper classes as Integer, Long, Short and Byte receptively. But it should be kept in mind that unlike those wrappers Void class doesn't store a value of type void in itself and hence is not a wrapper in true essence. Purpose: The Void class according to javadoc exists because of the fact that some time we may need to represent the void keyword as an object. But at the same point we cannot create an instance of the Void class using the new operator. This is because the constructor in Void has been declared as private. Moreover the Void class is a final class which means that there is no way we can inherit this class. So the only purpose that remains for the existence of the Void class is reflection, where we can get the return type of a method as void. The following piece of code will demonstrate this purpose: public class Test { public static void main(String[] args) throws SecurityException, NoSuchMethodException { Class c1 = Test1.class.getMethod("Testt",null).getReturnType(); System.out.println(c1 == Void.TYPE); System.out.println(c1 == Void.class); } } class Test1{ public void Testt(){} } One can also use Void class in Generics to specify that you don't care about the specific type of object being used. For example: List list1; From http://extreme-java.blogspot.com/2011/04/void-class-java.html
May 3, 2011
by Sandeep Bhandari
· 23,746 Views · 2 Likes
article thumbnail
Reasons for Slow Database Performance
Usually there are scenarios where the application does not perform as expected. A simple web page which fetches data from database and displays optimizes it for mobiles should be fast and turnaround times should be less than 30 seconds on a good network connection. But still there are cases where these kinds of applications suffer the most performance issues. This is because the database in these cases is not designed by giving proper attention to the application requirements. You can change one application design even after delivery but changing the database design once a number of application have been integrated with it is like explosion. Here I am giving some points which should be kept in mind while designing application and database. Only basic idea is being provided. For details one can search each topic on the web as each point expands well to multiple articles. 1) Bind Variables: When a SQL query is sent to the database engine for processing and sending the result, it is compiled by the database compiler to get the tokens of the query. This involves parsing, optimizing and identifying the query. After a number of steps, the SQL query is passed to the database engine for processing. In a small application with a user base of less than 500, it is usually the same query which is executed more often than others. The use of bind variables helps in storing the compiled query once and executing it with different data at different times. For using bind variables, one needs to use PreparedStatement objects in Java. 2) Query is not well formed: Usually the same SQL query can be written in multiple ways. There are ways by which a query can be optimized to give the best performance. The corresponding SQL construct should be chosen depending upon requirement. I have scenarios where people have used WHERE clause instead of GROUP BY and are complaining of poor response times. Similarly Sub queries and Joins complement each other. 3) Database structure is not well defined/normalized: This is probably known to everybody that the database tables should be properly normalized as this is part of every DBMS course at graduation level. If the tables are not properly designed and normalized, anomalies set in. 4) Proper caching is not in place: Many applications make use of temporary caches on the application server to store the reference data or frequently accessed data as memory is less of an issue than the time with new generation servers. 5) Number of rows in the table too large: If the table itself has too much of data then the queries will take time to execute. Partitioning a table into multiple tables is recommended in these situations. For example: If a table has employee records of 1000000 employees then it could be split into 5 small tables each having 200000 rows. The advantage is we know beforehand in which smaller table to look for a particular employee code as the division of large table can be done on the employee id column. 6) Connections are not being pooled: If connections are not pooled then the each time a new connection is requested for a request to database. Maintaining a connection pool is much better than creating and destroying the connection for executing every SQL query. Of course, there are frameworks like Hibernate which take care of creating the connection pools and also allow the customization of these pools 7) Connections not closed/returned to pool in case of exceptions: When an exception occurs while performing database operations, it ought to be caught. Usually catching the exception is not the issue because SQLException is a checked exception but closing the connection is something that most of the times is left out. If the connection is not released, the same connection cannot be used for any other purpose till the connection is timed out. 8) Stored procedures for complex computations on database: Stored procedures are a good way to perform database intensive operations. This is because they are already compiled and there is less network trips for getting the same results as compared to SQL queries. From http://extreme-java.blogspot.com/2011/04/reasons-for-slow-database-performance.html
April 30, 2011
by Sandeep Bhandari
· 59,826 Views · 1 Like
article thumbnail
Modules and namespaces in JavaScript
JavaScript does not come with support for modules. This blog post examines patterns and APIs that provide such support. It is split into the following parts: Patterns for structuring modules. APIs for loading modules asynchronously. Related reading, background and sources. 1. Patterns for structuring modules A module fulfills two purposes: First, it holds content, by mapping identifiers to values. Second, it provides a namespace for those identifiers, to prevent them from clashing with identifiers in other modules. In JavaScript, modules are implemented via objects. Namespacing: A top-level module is put into a global variable. That variable is the namespace of the module content. Holding content: Each property of the module holds a value. Nesting modules: One achieves nesting by putting a module inside another one. Filling a module with content Approach 1: Object literal. var namespace = { func: function() { ... }, value: 123 }; Approach 2: Assigning to properties. var namespace = {}; namespace.func = function() { ... }; namespace.value = 123; Accessing the content in either approach: namespace.func(); console.log(namespace.value + 44); Assessment: Object literal. Pro: Elegant syntax. Con: As a single, sometimes very long syntactic construct, it imposes constraints on its contents. One must maintain the opening brace before the content and the closing brace after the content. And one must remember to not add a comma after the last property value. This makes it harder to move content around. Assigning to properties. Con: Redundant repetitions of the namespace identifier. The Module pattern: private data and initialization In the module pattern, one uses an Immediately-Invoked Function Expression (IIFE, [1]) to attach an environment to the module data. The bindings inside that environment can be accessed from the module, but not from outside. Another advantage is that the IIFE gives you a place to perform initializations. var namespace = function() { // set up private data var arr = []; // not visible outside for(var i=0; i<4; i++) { arr.push(i); } return { // read-only access via getter get values() { return arr; } }; }(); console.log(namespace.values); // [0,1,2,3] Comments: Con: Harder to read and harder to figure out what is going on. Con: Harder to patch. Every now and then, you can reuse existing code by patching it just a little. Yes, this breaks encapsulation, but it can also be very useful for temporary solutions. The module pattern makes such patching impossible (which may be a feature, depending on your taste). Alternative for private data: use a naming convention for private properties, e.g. all properties whose names start with an underscore are private. Variation: Namespace is a function parameter. var namespace = {}; (function(ns) { // (set up private data here) ns.func = function() { ... }; ns.value = 123; }(namespace)); Variation: this as the namespace identifier (cannot accidentally be assigned to). var namespace = {}; (function() { // (set up private data here) this.func = function() { ... }; this.value = 123; }).call(namespace); // hand in implicit parameter "this" Referring to sibling properties Use this. Con: hidden if you nest functions (which includes methods in nested objects). var namespace = { _value: 123; // private via naming convention getValue: function() { return this._value; } anObject: { aMethod: function() { // "this" does not point to the module here } } } Global access. Cons: makes it harder to rename the namespace, verbose for nested namespaces. var namespace = { _value: 123; // private via naming convention getValue: function() { return namespace._value; } } Custom identifier: The module pattern (see above) enables one to use a custom local identifier to refer to the current module. Module pattern with object literal: assign the object to a local variable before returning it. Module pattern with parameter: the parameter is the custom identifier. Private data and initialization for properties An IFEE can be used to attach private data and initialization code to an object. It can do the same for a single property. var ns = { getValue: function() { var arr = []; // not visible outside for(var i=0; i<4; i++) { arr.push(i); } return function() { // actual property value return arr; }; }() }; Read on for an application of this pattern. Types in object literals Problem: A JavaScript type is defined in two steps. First, define the constructor. Second, set up the prototype of the constructor. These two steps cannot be performed in object literals. There are two solutions: Use an inheritance API where constructor and prototype can be defined simultaneously [4]. Wrap the two parts of the type in an IIFE: var ns = { Type: function() { var constructor = function() { // ... }; constructor.prototype = { // ... }; return constructor; // value of Type }() }; Managing namespaces Use the same namespace in several files: You can spread out a module definition across several files. Each file contributes features to the module. If you create the namespace variable as follows then the order in which the files are loaded does not matter. Note that this pattern does not work with object literals. var namespace = namespace || {}; Nested namespaces: With multiple modules, one can avoid a proliferation of global names by creating a single global namespace and adding sub-modules to it. Further nesting is not advisable, because it adds complexity and is slower. You can use longer names if name clashes are an issue. var topns = topns || {}; topns.module1 = { // content } topns.module2 = { // content } YUI2 uses the following pattern to create nested namespaces. YAHOO.namespace("foo.bar"); YAHOO.foo.bar.doSomething = function() { ... }; 2. APIs for loading modules asynchronously Avoiding blocking: The content of a web page is processed sequentially. When a script tag is encountered that refers to a file, two steps happen: The file is downloaded. The file is interpreted. All browsers block the processing of subsequent content until (2) is finished, because everything is single-threaded and must be processed in order. Newer browsers perform some downloads in parallel, but rendering is still blocked [2]. This unnecessarily delays the initial display of a page. Modern module APIs provide a way around this by supporting asynchronous loading of modules. There are usually two parts to using such APIs: First one specifies what modules one would like to use. Second, one provides a callback that is invoked once all modules are ready. The goal of this section is not to be a comprehensive introduction, but rather to give you an overview of what is possible in the design space of JavaScript modules. 2.1. RequireJS RequireJS has been created as a standard for modules that work both on servers and in browsers. The RequireJS website explains the relationship between RequireJS and the earlier CommonJS standard for server-side modules [3]: CommonJS defines a module format. Unfortunately, it was defined without giving browsers equal footing to other JavaScript environments. Because of that, there are CommonJS spec proposals for Transport formats and an asynchronous require. RequireJS tries to keep with the spirit of CommonJS, with using string names to refer to dependencies, and to avoid modules defining global objects, but still allow coding a module format that works well natively in the browser. RequireJS implements the Asynchronous Module Definition (formerly Transport/C) proposal. If you have modules that are in the traditional CommonJS module format, then you can easily convert them to work with RequireJS. RequireJS projects have the following file structure: project-directory/ project.html legacy.js scripts/ main.js require.js helper/ util.js project.html: My Sample Project main.js: helper/util is resolved relative to data-main. legacy.js ends with .js and is assumed to not be in module format. The consequences are that its path is resolved relative to project.html and that there isn’t a function parameter to access its (module) contents. require(["helper/util", "legacy.js"], function(util) { //This function is called when scripts/helper/util.js is loaded. require.ready(function() { //This function is called when the page is loaded //(the DOMContentLoaded event) and when all required //scripts are loaded. }); }); Other features of RequireJS: Specify and use internationalization data. Load text files (e.g. to be used for HTML templating) Use JSONP service results for initial application setup. 2.2. YUI3 Version 3 of the YUI JavaScript framework brings its own module infrastructure. YUI3 modules are loaded asynchronously. The general pattern for using them is as follows. YUI().use('dd', 'anim', function(Y) { // Y.DD is available // Y.Anim is available }); Steps: Provide IDs “dd” and “anim” of the modules you want to load. Provide a callback to be invoked once all modules have been loaded. The parameter Y of the callback is the YUI namespace. This namespace contains the sub-namespaces DD and Anim for the modules. As you can see, the ID of a module and its namespace are usually different. Method YUI.add() allows you to register your own modules. YUI.add('mymodules-mod1', function(Y) { Y.namespace('mynamespace'); Y.mynamespace.Mod1 = function() { // expose an API }; }, '0.1.1' // module version ); YUI includes a loader for retrieving modules from external files. It is configured via a parameter to the API. The following example loads two modules: The built-in YUI module dd and the external module yui_flot that is available online. YUI({ modules: { yui2_yde_datasource: { // not used below fullpath: 'http://yui.yahooapis.com/datasource-min.js' }, yui_flot: { fullpath: 'http://bluesmoon.github.com/yui-flot/yui.flot.js' } } }).use('dd', 'yui_flot', function(Y) { // do stuff }); 2.3. Script loaders Similarly to RequireJS, script loaders are replacements for script tags that allow one to load JavaScript code asynchronously and in parallel. But they are usually simpler than RequireJS. Examples: LABjs: a relatively simple script loader. Use it instead of RequireJS if you need to load scripts in a precise order and you don't need to manage module dependencies. Background: “LABjs & RequireJS: Loading JavaScript Resources the Fun Way” describes the differences between LABjs and RequireJS. yepnope: A fast script loader that allows you to make the loading of some scripts contingent on the capabilities of the web browser. 3. Related reading, background and sources Related reading: A first look at the upcoming JavaScript modules Background: JavaScript variable scoping and its pitfalls Loading Scripts Without Blocking CommonJS Modules Lightweight JavaScript inheritance APIs Main sources of this post: Namespacing in JavaScript YUI2: A JavaScript Module Pattern YUI3: YUI Global Object How to get started with RequireJS From http://www.2ality.com/2011/04/modules-and-namespaces-in-javascript.html
April 29, 2011
by Axel Rauschmayer
· 18,520 Views
article thumbnail
Bootstrapping CDI in several environments
i feel like writing some posts about cdi (contexts and dependency injection). so this is the first one of a series of x posts ( 0 javax.enterprise cdi-api 1.0 provided an empty beans.xml will do to enable cdi you must have a beans.xml file in your project (under the meta-inf or web-inf). that’s because cdi needs to identify the beans in your classpath (this is called bean discovery) and build its internal metamodel. with the beans.xml file cdi knows it has beans to discover. so, for all the following examples i’ll make it simple and will leave this file completely empty. java ee 6 containers let’s start with the easiest possible environment : java ee 6 containers . why is it the simplest ? well, because you don’t have to do anything : cdi is part of java ee 6 as well as the web profile 1.0 so you don’t need to manually bootstrap it. let’s see how to inject a cdi bean within an ejb 3.1 and a servlet 3.0 . ejb 3.1 since ejb 3.1 you can use the ejbcontainer api to get an in-memory embedded ejb container and you can easily unit test your ejbs. so let’s write an ejb and a test class. first let’s have a look at the code of the ejb. as you can see, with version 3.1 an ejb is just a pojo : no inheritance, no interface, just one @stateless annotation. it gets a reference of the hello bean buy using the @inject annotation and uses it in the saysomething() method. @stateless public class mainejb31 { @inject hello hello; public string saysomething() { return hello.sayhelloworld(); } } you can now package the mainejb31, hello and world classes with the empty beans.xml file into a jar, deploy it to glassfish 3.x , and it will work. but if you don’t want to bother deploying it to glassfish and just unit test it, this is what you need to do : public class mainejbtest { private static ejbcontainer ec; private static context ctx; @beforeclass public static void initcontainer() throws exception { map properties = new hashmap(); properties.put(ejbcontainer.modules, new file("target/classes")); ec = ejbcontainer.createejbcontainer(properties); ctx = ec.getcontext(); } @afterclass public static void closecontainer() throws exception { if (ec != null) ec.close(); } @test public void shoulddisplayhelloworld() throws exception { // looks up the ejb mainejb31 mainejb = (mainejb31) ctx.lookup("java:global/classes/mainejb!org.antoniogoncalves.cdi.helloworld.mainejb"); assertequals("should say hello world !!!", "hello world !!!", mainejb.saysomething()); } } in the code above the method initcontainer() initializes the ejbcontainer. the shoulddisplayhelloworld() looks up the ejb (using the new portable jndi name ), invokes it and makes sure the saysomething() method returns hello world !!!. green test. that was pretty easy too. servlet 3.0 servlet 3.0 is part of java ee 6, so again, there is no needed configuration to bootstrap cdi. let’s use the new @webservlet annotation and write a very simple one that injects a reference of hello and displays an html page with hello world !!!. this is what the servlet looks like : @webservlet(urlpatterns = "/mainservlet") public class mainservlet30 extends httpservlet { @inject hello hello; @override protected void service(httpservletrequest req, httpservletresponse resp) throws servletexception, ioexception { resp.setcontenttype("text/html"); printwriter out = resp.getwriter(); out.println(""); out.println(""); out.println(""); out.println(saysomething()); out.println(""); out.println(""); out.close(); } public string saysomething() { return hello.sayhelloworld(); } } thanks to the @webservlet i don’t need any web.xml (it’s optional in servlet 3.0) to map the mainservlet30 to the /mainservlet url. you can now package the mainservlet30, hello and world classes with the empty beans.xml and no web.xml into a war, deploy it to glassfish 3.x , go to http://localhost:8080/bootstrapping-servlet30-1.0/mainservlet and it will work. unfortunately servlet 3.0 doesn’t have an api for the container (such as ejbcontainer). there is no servletcontainer api that would let you use an embedded servlet container in a standard way and, why not, easily unit test it. application client container not many people know it, but java ee (or even older j2ee versions) comes with an application client container (acc). it’s like an ejb or servlet container but for plain pojos. for example you can develop a swing application (yes, i’m sure that some of you still use swing), run it into the acc and get some extra services given by the container (security, naming, certain annotations…). glassfish v3 has an acc that you can launch in a command line : appclient -jar . so i thought, great, i can use cdi with acc the same way i use it within ejb or servlet container, no need to bootstrap anything, it’s all out of the box. i was wrong . as per the cdi specification (section 12.1), cdi is not required to support application client bean archives. so the glassfish application client container doesn’t support it. i haven’t tried the jboss acc , maybe it works. other containers the beauty of cdi is that it doesn’t require java ee 6 . you can use cdi with simple pojos in a java se environment, as well as some servlet 2.5 containers. of course it’s not as easy to bootstrap because you need a bit of configuration. but it then works fine (not always but). java se 6 ok, so until now there was nothing to do to bootstrap cdi. it is already bundled with the ejb 3.1 and servlet 3.0 containers of java ee 6 (and web profile). so the idea here is to use cdi in a simple java se environment. coming back to our hello and world classes, we need a pojo with an entry point that will bootstrap cdi so we can use injection to get those classes. in standard java se when we say entry point , we think of a public static void main(string[] args) method. well, we need something similar… but different. weld is the reference implementation of cdi. that means it implements the specification, the standard apis (mostly found in javax.inject and javax.enterprise.context packages) but also some proprietary code (in org.jboss.weld package). bootstrapping cdi in java se is not specified so you will need to use specific weld features. you can do that in two different flavors: by observing the containerinitialized event or using the programatic bootstrap api consisting of the weld and weldcontainer classes. the following code uses the containerinitialized event. as you can see, it uses the @observes annotation that i’ll explain in a future post. but the idea is that this class is listening to the event and processes the code once the event is triggered. import org.jboss.weld.environment.se.events.containerinitialized; import javax.enterprise.event.observes; import javax.inject.inject; public class mainjavase6 { @inject hello hello; public void saysomething(@observes containerinitialized event) { system.out.println(hello.sayhelloworld()); } } but who trigers the containerinitialized event ? well, it’s the org.jboss.weld.environment.se.startmain class. i’m using maven so a nice trick is to use the exec-maven-plugin to run the startmain class. download the code , have a look at the pom.xml and give it a try. the other possibility is to programmatically bootstrap the weld container. this can be handy in unit testing. the code below initializes the weld container (with new weld().initialize()) and then looks for the hello class (using weld.instance().select(hello.class).get()). import org.jboss.weld.environment.se.weld; import org.jboss.weld.environment.se.weldcontainer; import org.junit.beforeclass; import org.junit.test; import static junit.framework.assert.assertequals; public class hellotest { @test public void shoulddisplayhelloworld() { weldcontainer weld = new weld().initialize(); hello hello = weld.instance().select(hello.class).get(); assertequals("should say hello world !!!", "hello world !!!", hello.sayhelloworld()); } } execute the test with mvn test and it should be green. as you can see, there is a bit more work using cdi in a java se environment, but it’s not that complicated. tomcat 6.x ok, and what about your legacy servlet 2.5 containers ? the first one that comes in mind is tomcat 6.x ( note that tomcat 7.x will implement servlet 3.0 but is still in beta version at the time of writing this post ). weld provides support for tomcat but you need to configure it a bit to make cdi work. first of all, this is a servlet 2.5, not a 3.0. so the code of the servlet is slightly different from the one seen before (no annotation allowed) and of course, you need your good old web.xml file : public class mainservlet25 extends httpservlet { @inject hello hello; @override protected void service(httpservletrequest req, httpservletresponse resp) throws servletexception, ioexception { resp.setcontenttype("text/html"); printwriter out = resp.getwriter(); out.println(""); out.println(""); out.println(""); out.println(saysomething()); out.println(""); out.println(""); out.close(); } public string saysomething() { return hello.sayhelloworld(); } } because we don’t have a @webservlet annotation in servlet 2.5, we need to declare and map it in the web.xml (using the servlet and servlet-mapping tags). then, you need to explicitly specify the servlet listener to boot weld and control its interaction with requests (org.jboss.weld.environment.servlet.listener). tomcat has a read-only jndi, so weld can’t automatically bind the beanmanager extension spi. to bind the beanmanager into jndi, you should populate meta-inf/context.xml and make the beanmanager available to your deployment by adding it to your web.xml: mainservlet25 org.antoniogoncalves.cdi.bootstrapping.servlet.mainservlet25 mainservlet25 /mainservlet org.jboss.weld.environment.servlet.listener beanmanager javax.enterprise.inject.spi.beanmanager the meta-inf/context.xml file is an optional file which contains a context for a single tomcat web application. this can be used to define certain behaviours for your application, jndi resources and other settings. package all the files (mainservlet25, hello, world, meta-inf/context.xml, beans.xml and web.xml) into a war and deploy it into tomcat 6.x. go to http://localhost:8080/bootstrapping-servlet25-tomcat-1.0/mainservlet and you will see your hello world page. jetty 6.x another famous servlet 2.5 containers is jetty 6.x (at codehaus) and jetty 7.x ( note that jetty 8.x will implement servlet 3.0 but it’s still in experimental stage at the time of writing this post ). if you look at the weld documentation, there is actually support for jetty 6.x and 7.x . the code is the same one as tomcat (because it’s a servlet 2.5 container), but the configuration changes. with jetty you need to add two files under web-inf : jetty-env.xml and jetty-web.xml : beanmanager javax.enterprise.inject.spi.beanmanager org.jboss.weld.resources.managerobjectfactory true package all the files (mainservlet25, hello, world, web-inf/jetty-env.xml, web-inf/jetty-web.xml, beans.xml and web.xml) into a war and deploy it into jetty 6.x. go to http://localhost:8080/bootstrapping-servlet25-jetty6/mainservlet and you will see your hello world page. there was a mistake in the weld documentation so i couldn’t make it work. i started a thread on the weld forum and thanks to dan allen , pete muir and all the weld team, this was fixed and i managed to make it work. simple as posting an email to the forum . thanks for your help guys. spring 3.x here is the tricky part. spring 3.x implements the jsr 330 : dependency injection for java , which means that @inject works out of the box. but i didn’t find a way to integrate cdi with spring 3.x . the weld documentation mentions that because of its extension points, “ integration with third-party frameworks such as spring (…) was envisaged by the designers of cdi “. i did find this blog that simulates cdi features by enabling spring ones. what i didn’t find is a clear statement or roadmap on springsource about supporting cdi or not in future releases. the last trace of this topic is a comment on a long tss flaming thread . at that time (16 december 2009), juergen huller said “ with respect to implementing cdi on top of spring (…) trying to hammer it into the semantic frame of another framework such as cdi would be an exercise that is certainly achievable (…) but ultimately pointless “. but if you have any fresh news about it, let me know. conclusion as i said, this post is not about explaining cdi, i’ll do that in future posts. i just wanted to focus on how to bootstrap it in several environments so you can try by yourself. as you saw, it’s much simpler to use cdi within an ejb 3.1 or servlet 3.0 container in java ee 6. i’ve used glassfish 3.x but it should also work with other java ee 6 or web profile containers such as jboss 6 or resin . when you don’t use java ee 6, there is a bit more work to do. depending on your environment or servlet container you need some configuration to bootstrap weld. by the way, i’ve used weld because it’s the reference implementation, the one bunddled with glassfish and jboss. but you could also use openwebbeans , another cdi implementation. download the code , give it a try, and give me some feedback. from http://agoncal.wordpress.com/2011/01/12/bootstrapping-cdi-in-several-environments/
April 28, 2011
by Antonio Goncalves
· 31,407 Views
article thumbnail
Just say no to swapping!
Imagine you love to cook; it's an intense hobby of yours. Over time, you've accumulated many fun spices, but your pantry is too small, so, you rent an off-site storage facility, and move the less frequently used spice racks there. Problem solved! Suddenly you decide to cook this great new recipe. You head to the pantry to retrieve your Saffron, but it's not there! It was moved out to the storage facility and must now be retrieved (this is a hard page fault). No problem -- your neighbor volunteers to go fetch it for you. Unfortunately, the facility is ~2,900 miles away, all the way across the US, so it takes your friend 6 days to retrieve it! This assumes you normally take 7 seconds to retrieve a spice from the pantry; that your data was in main memory (~100 nanoseconds access time), not in the CPU's caches (which'd be maybe 10 nanoseconds); that your swap file is on a fast (say, WD Raptor) spinning-magnets hard drive with 5 millisecond average access time; and that your neighbor drives non-stop at 60 mph to the facility and back. Even worse, your neighbor drives a motorcycle, and so he can only retrieve one spice rack at a time. So, after waiting 6 days for the Saffron to come back, when you next go to the pantry to get some Paprika, it's also "swapped out" and you must wait another 6 days! It's possible that first spice rack also happened to have the Paprika but it's also likely it did not; that depends on your spice locality. Also, with each trip, your neighbor must pick a spice rack to move out to the facility, so that the returned spice rack has a place to go (it is a "swap", after all), so the Paprika could have just been swapped out! Sadly, it might easily be many weeks until you succeed in cooking your dish. Maybe in the olden days, when memory itself was a core of little magnets, swapping cost wasn't so extreme, but these days, as memory access time has improved drastically while hard drive access time hasn't budged, the disparity is now unacceptable. Swapping has become a badly leaking abstraction. When a typical process (say, your e-mail reader) has to "swap back in" after not being used for a while, it can hit 100s of such page faults, before finishing redrawing its window. It's an awful experience, though it has the fun side effect of letting you see, in slow motion, just what precise steps your email reader goes through when redrawing its window. Swapping is especially disastrous with JVM processes. See, the JVM generally won't do a full GC cycle until it has run out of its allowed heap, so most of your heap is likely occupied by not-yet-collected garbage. Since these pages aren't being touched (because they are garbage and thus unreferenced), the OS happily swaps them out. When GC finally runs, you have a ridiculous swap storm, pulling in all these pages only to then discover that they are in fact filled with garbage and should be discarded; this can easily make your GC cycle take many minutes! It'd be better if the JVM could work more closely with the OS so that GC would somehow run on-demand whenever the OS wants to start swapping so that, at least, we never swap out garbage. Until then, make sure you don't set your JVM's heap size too large! Just use an SSD... These days, many machines ship with solid state disks, which are an astounding (though still costly) improvement over spinning magnets; once you've used an SSD you can never go back; it's just one of life's many one-way doors. You might be tempted to declare that this problem is solved, since SSDs are so blazingly fast, right? Indeed, they are orders of magnitudes faster than spinning magnets, but they are still 2-3 orders of magnitude slower than main memory or CPU cache. The typical SSD might have 50 microsends access time, which equates to ~58 total miles of driving at 60 mph. Certainly a huge improvement, but still unacceptable if you want to cook your dish on time! Just add RAM... Another common workaround is to put lots of RAM in your machine, but this can easily back-fire: operating systems will happily swap out memory pages in favor of caching IO pages, so if you have any processes accessing lots of bytes (say, mencoder encoding a 50 GB bluray movie, maybe a virus checker or backup program, or even Lucene searching against a large index or doing a large merge), the OS will swap your pages out. This then means that the more RAM you have, the more swapping you get, and the problem only gets worse! Fortunately, some OS's let you control this behavior: on Linux, you can tune swappiness down to 0 (most Linux distros default this to a highish number); Windows also has a checkbox, under My Computer -> Properties -> Advanced -> Performance Settings -> Advanced -> Memory Usage, that lets you favor Programs or System Cache, that's likely doing something similar. There are low-level IO flags that these programs are supposed to use so that the OS knows not to cache the pages they access, but sometimes the processes fail to use them or cannot use them (for example, they are not yet exposed to Java), and even if they do, sometimes the OS ignores them! When swapping is OK If your computer never runs any interactive processes, ie, a process where a human is blocked (waiting) on the other end for something to happen, and only runs batch processes which tend to be active at different times, then swapping can be an overall win since it allows that process which is active to make nearly-full use of the available RAM. Net/net, over time, this will give greater overall throughput for the batch processes on the machine. But, remember that the server running your web-site is an interactive process; if your server processes (web/app server, database, search server, etc.) are stuck swapping, your site has for all intents and purposes become unusable to your users. This is a fixable problem Most processes have known data structures that consume substantial RAM, and in many cases these processes could easily discard and later regenerate their data structures in much less time than even a single page fault. Caches can simply be pruned or discarded since they will self-regenerate over time. These data structures should never be swapped out, since regeneration is far cheaper. Somehow the OS should ask each RAM-intensive and least-recently-accessed process to discard its data structures to free up RAM, instead of swapping out the pages occupied by the data structure. Of course, this would require a tighter interaction between the OS and processes than exists today; Java's SoftReference is close, except this only works within a single JVM, and does not interact with the OS. What can you do? Until this problem is solved for real, the simplest workaround is to disable swapping entirely, and stuff as much RAM as you can into the machine. RAM is cheap, memory modules are dense, and modern motherboards accept many modules. This is what I do. Of course, with this approach, when you run out of RAM stuff will start failing. If the software is well written, it'll fail gracefully: your browser will tell you it cannot open a new window or visit a new page. If it's poorly written it will simply crash, thus quickly freeing up RAM and hopefully not losing any data or corrupting any files in the process. Linux takes the simple draconian approach of picking a memory hogging process and SIGKILL'ing it. If you don't want to disable swapping you should at least tell the OS not to swap pages out for IO caching. Just say no to swapping!
April 27, 2011
by Michael Mccandless
· 26,037 Views
article thumbnail
Check if PHP is running in safe mode or not
Lately, I have been playing around with PHP configuration. And I realized that the runtime configurations do not change using ini_set function, if you are running PHP in the safe mode. A simple check in PHP can tell you this. Notice the code block below and which allows to check this. Next, time when you are going to change any configuration at runtime, I am sure you want to check this, before you could see that change working. Hope that helps. Stay digified !!
April 26, 2011
by Sachin Khosla
· 15,012 Views
article thumbnail
Turning Off Session Fixation Protection in Tomcat 7
Session fixation protection is a new feature that was introduced as part of the Apache Tomcat 7 release process, and has been backported and turned on by default in all versions from version 6.0.21 on. Session fixation happens when an attacker sends a link to the victim with a session ID included. The minute the victim authenticates the session in the link by logging in, the attacker is able to just click the session link and have full access to the intended victims account. According to Mark Thomas, release manager for Tomcat 7 and member of the Apache Security Committee, in an article on TomcatExpert.com today": Essentially, when a user authenticates their session, Tomcat will change the session ID. It does not destroy the previous session, rather it renames it so it is no longer found by that ID. So in our example above, Bob would try and log on with that session, and he would not be able to find it. Turning Off Session Fixation Protection Since it is turned on by default, the configuration is implicit inside of Tomcat. To turn it off, you will need to explicitly add the configuration and turn it off. First you will need to find the details of the specific type of authentication valve you are using in the Apache Tomcat Valve Configuration Documentation. Then you would need to navigate to the context.xml file for your application (not $CATALINA_BASE/conf/context.xml - that is the global context.xml that provides defaults for all web applications). Add the valve with the appropriate class for your authentication type (e.g., for basic authentication add a valve using class="org.apache.catalina.authenticator.BasicAuthenticator". Finally, set the parameter changeSessionIdOnAuthentication="false". When to Turn Off Session Fixation Protection Generally, this is not recommended. The Apache Security Team deemed this feature a basic enough protection that it should be afforded to all users implicitly. However, in some cases application specific behavior could break and the feature would need to be turned off until the functionality could be restored using the protection. For more information on the feature and when to use it, see the full article from Mark Thomas on TomcatExpert.com.
April 26, 2011
by Stacey Schneider
· 15,072 Views
article thumbnail
Exploring TDD in JavaScript with a small kata
A code kata is an exercise where you focus on your technique instead of on the final product of your mind and fingers. But a kata can also be used as a constant parameter, while other variables change, like in scientific experiments. For example, when learning a new programming language or framework, you can execute an old kata in order to explore it. I decided to perform a small and famous Kata that we used also during interviews to separate programmers from not programmers: the FizzBuzz kata. My goal was to learn how to setup a platform for Test-Driven Development in JavaScript, following the advice of the Test-Driven JavaScript Development book. The parameters that change from my habits are the tools for running tests and the programming language, but my IDE (Unix&Vim) remained fixed along with the Kata: Write a function that returns its numerical argument. But for multiples of three return Fizz instead of the number and for the multiples of five return Buzz. For numbers which are multiples of both three and five return FizzBuzz. Additional requirement: when passed a multiple of 7, return Bang; when passed a multiple of 5 and 7, return BuzzBang; and so on for all the combinations. As my tools for running the tests, I used JsTestDriver and Firefox, as suggested by the book Test-Driven JavaScript Development which I'm currently reading. JsTestDriver JsTestDriver will make you feel the joy of a green bar again. Download its jar, put it somewhere and add an alias in your .bashrc: export JSTESTDRIVER_HOME=~/bin alias jstestdriver="java -jar $JSTESTDRIVER_HOME/JsTestDriver-1.3.2.jar" Start the server: jstestdriver --port 4224 Point an open browser (I used Firefox) to localhost:4224. The browser will ping it via Ajax requests undefinitely to gather tests to run. Now we can use the command line to run tests, like you'll do with PHPUnit if you are a PHPer: jstestdriver --tests all The Kata I started with a simple function, fizzbuzz(), and a single test case. I never wrote a test with JsTestDriver before so I needed to gain some confidence and be sure the configuration file was correct. server: http://localhost:4224 load: - src/*.js - test/*.js In JsTestDriver, a Test case is created by passing to TestCase (global function provided by JsTestDriver) a map containing anonymous functions. TestCase("FizzBuzzTest", { "test should return Fizz when passed 3" : function () { assertEquals("Fizz", fizzbuzz(3)); } }); The functions whose names start with test will be executed; there are some reserved keywords like setUp which are used as hooks for fixture creation. Running the test with the alias command is really simple: jstestdriver --tests all I made the first test pass with fizzbuzz.js, a file containing a first version of the function (with a fake implementation): function fizzbuzz() { return 'Fizz'; } The result? A green bar (metaphorically green; all tests pass.) . Total 1 tests (Passed: 1; Fails: 0; Errors: 0) (0,00 ms) Firefox 4.0 Linux: Run 1 tests (Passed: 1; Fails: 0; Errors 0) (0,00 ms) You can capture more than one browser if you want to run test simultaneously in all of them, but it will probably slow down the TDD basic cycle. You can leave cross-browser testing for later. Going on After this first test, I went on adding new ones and making them pass, until I even converted the function to an object, for the sake of easy configuration (a function returning a function would be the same). Since I also needed to create the object in just one place, I started using setUp for the fixture creation: TestCase("FizzBuzzTest", { setUp : function () { this.fizzbuzz = new FizzBuzz({ 3 : 'Fizz', 5 : 'Buzz', 7 : 'Bang' }); }, "test should return the number when passed 1 or 2" : function () { assertEquals(1, this.fizzbuzz.accept(1)); assertEquals(2, this.fizzbuzz.accept(2)); }, "test should return Fizz when passed 3 or a multiple" : function () { assertEquals("Fizz", this.fizzbuzz.accept(3)); assertEquals("Fizz", this.fizzbuzz.accept(6)); }, "test should return Buzz when passed 5 or a multiple" : function () { assertEquals("Buzz", this.fizzbuzz.accept(5)); assertEquals("Buzz", this.fizzbuzz.accept(10)); }, "test should return FizzBuzz when passed a multiple of both 3 and 5" : function () { assertEquals("FizzBuzz", this.fizzbuzz.accept(15)); assertEquals("FizzBuzz", this.fizzbuzz.accept(30)); }, "test should return Bang when passed a multiple of 7" : function () { assertEquals("Bang", this.fizzbuzz.accept(7)); assertEquals("Bang", this.fizzbuzz.accept(14)); }, "test should return FizzBuzzBang when it is the case" : function () { assertEquals("FizzBuzzBang", this.fizzbuzz.accept(3*5*7)); } }); You can use this to share fixtures between the setUp and the different test methods: the test does not look different from JUnit and PHPUnit ones. Like in all xUnit testing frameworks, the setUp is executed on a brand new object for each test, to preserve isolation. I like a bit the way in which in JavaScript you can tear and put together objects: after all, it's called object-oriented programming, not class-oriented programming. I decided to use a small function constructor as you may infer from the test: function FizzBuzz(correspondences) { this.correspondences = correspondences; this.accept = function (number) { var result = ''; for (var divisor in this.correspondences) { if (number % divisor == 0) { result = result + this.correspondences[divisor]; } } if (result) { return result; } else { return number; } } } All the code is on Github, to see the intermediate steps of the Kata if you need them. You can also use the repository to try out your installation of JsTestDriver: a git pull followed by running the tests will confirm that it's working. Sometimes we don't test code in alien environments like JavaScript console or database queries because we don't know how; but a Kata which takes just two Pomodoros can solve the issue and let you enjoy a green bar even when working with a browser's interpreter.
April 21, 2011
by Giorgio Sironi
· 13,135 Views
article thumbnail
HTML encoding/escaping with StringTemplate and Spring MVC
Last week my colleague T.C. and I had to work out how to HTML encode the values entered by the user when redisplaying those onto the page to prevent a cross site scripting attack on the website. I wrote a blog post a couple of years ago describing how to do this in ASP.NET MVC and the general idea is that we need to have a custom renderer which HTML encodes any strings that pass through it. In our case this means that we needed to write a custom renderer for String Template and hook that into Spring MVC. We already had a view class StringTemplateView so we needed to add to that class and add our custom renderer. The viewResolver was defined like so: @Bean public ViewResolver viewResolver() { InternalResourceViewResolver viewResolver = new InternalResourceViewResolver(); viewResolver.setPrefix("/WEB-INF/templates/"); viewResolver.setViewClass(StringTemplateView.class); viewResolver.setSuffix(".st"); return viewResolver; } And after some guidance from Jim we changed StringTemplateView to look like this: public class StringTemplateView extends InternalResourceView { @Override protected void renderMergedOutputModel(Map model, HttpServletRequest request, HttpServletResponse response) throws Exception { String templateRootDir = format("%s/WEB-INF/templates", getServletContext().getRealPath("/")); StringTemplateGroup group = new StringTemplateGroup("view", templateRootDir); StringTemplate template = group.getInstanceOf(getBeanName()); AttributeRenderer htmlEncodedRenderer = new HtmlEncodedRenderer(); template.registerRenderer(String.class, htmlEncodedRenderer); ... } private class HtmlEncodedRenderer implements AttributeRenderer { @Override public String toString(Object o) { return HtmlUtils.htmlEscape(o.toString()); } @Override public String toString(Object o, String formatName) { return HtmlUtils.htmlEscape(o.toString()); } } } At the moment we want to HTML encode everything that we render through StringTemplate but if that changes then we could make use of the formatName parameter which we’re currently ignoring. In retrospect this looks pretty simple to do but my Googling skills were pretty much failing me at the time so I thought it’d be good to document. From http://www.markhneedham.com/blog/2011/04/09/html-encodingescaping-with-stringtemplate-and-spring-mvc
April 19, 2011
by Mark Needham
· 24,230 Views · 15 Likes
article thumbnail
Hammurabi - A Scala Rule Engine
One of the most common reasons why software projects fail, or suffer unbearable delays, is the misunderstandings between the analysts who define the business rules of the domain for which the software is going to be written and the developers who have to code these rules. The latter write those rules in a language that is completely obscure for the first ones. In this way the business analysts don't have a chance to read, understand and validate what the programmers developed and then they can only empirically test the final software behavior, hardly covering all the possible corner cases and often recognizing mistakes only when it is too late. What Hammurabi is Hammurabi is a rule engine written in Scala that tries to leverage the features of this language making it particularly suitable to implement extremely readable internal Domain Specific Languages. Indeed, what actually makes Hammurabi different from all other rule engines is that it is possible to write and compile its rules directly in the host language. Anyway the Hammurabi's rules also have the important property of being readable even by non technical person. As usual a practical example worth more than a thousand words. The golfers problem This logical puzzle has been taken from the first chapter of the Jess in Action book written by Ernest Friedman-Hill and published by Manning. It is described there as it follows: A foursome of golfers is standing at a tee, in a line from left to right. Each golfer wears different colored pants; one is wearing red pants. The golfer to Fred’s immediate right is wearing blue pants. Joe is second in line. Bob is wearing plaid pants. Tom isn’t in position one or four, and he isn’t wearing the hideous orange pants. In what order will the four golfers tee off, and what color are each golfer’s pants?” The Jess solution Jess is written in Java and is one of the most popular rule engine on the market. The solution to the golfers problems presented in the book mentioned above is the following: first it is necessary to define the data structures representing the problem (deftemplate pants-color (slot of) (slot is)) (deftemplate position (slot of) (slot is)) A deftemplate is a bit like a class declaration in Java and in this case is used to write a first rule that in turns creates the facts representing each of the possible combinations of golfers, pants-color and positions: (defrule generate-possibilities => (foreach ?name (create$ Fred Joe Bob Tom) (foreach ?color (create$ red blue plaid orange) (assert (pants-color (of ?name)(is ?color))) ) (foreach ?position (create$ 1 2 3 4) (assert (position (of ?name)(is ?position))) ) ) ) After that it is possible to translate the sentences of the problem in the corresponding Jess rule: (defrule find-solution ;; There is a golfer named Fred, whose position is ?p1 ;; and pants color is ?c1 (position (of Fred) (is ?p1)) (pants-color (of Fred) (is ?c1)) ;; The golfer to immediate right of Fred ;; is wearing blue pants. (position (of ?n&~Fred)(is ?p&:(eq ?p (+ ?p1 1)))) (pants-color (of ?n&~Fred)(is blue&~?c1)) ;; Joe is in position #2 (position (of Joe) (is ?p2&2&~?p1)) (pants-color (of Joe) (is ?c2&~?c1)) ;; Bob is wearing the plaid pants (position (of Bob)(is ?p3&~?p1&~?p&~?p2)) (pants-color (of Bob&~?n)(is plaid&?c3&~?c1&~?c2)) ;; Tom is not in position 1 or 4 ;; and is not wearing orange (position (of Tom&~?n)(is ?p4&~1&~4&~?p1&~?p2&~?p3)) (pants-color (of Tom)(is ?c4&~orange&~blue&~?c1&~?c2&~?c3)) => (printout t Fred " " ?p1 " " ?c1 crlf) (printout t Joe " " ?p2 " " ?c2 crlf) (printout t Bob " " ?p3 " " ?c3 crlf) (printout t Tom " " ?p4 " " ?c4 crlf) ) where the rows starting with ;; are just comments. In this way if you enter the code for the problem into Jess and then run it, you get the answer directly: Fred 1 orange Joe 2 blue Bob 4 plaid Tom 3 red Note that the facts that the golfers are in different positions and wear pants of different colors is not expressed in an explicit rule but need to be spread and repeated in many statements. This solution is clearly difficult to be maintained and doesn't scale well as underlined by the last condition statement stating that the position ?p4 of Tom is ?p4&~1&~4&~?p1&~?p2&~?p3 where ~ means not in the Jess language. In other words it says that the Tom's position is not only different from the position 1 and 4 but it is also different from the positions of all the other golfers (named one by one) formerly defined. Actually the needs to describe a golfer's position also as the negation of the positions of all the other golfers implies something even worse: it is not possible to translate each sentence of the problem in a different rule, but they have to be combined together in a single big rule. After this huge if part, its then section (the one after the => symbol) prints out a table containing the set of variables ?p1…?p4 and ?c1…?c4 that solves the problem. The Hammurabi solution As done while presenting the Jess solution, also with Hammurabi the first thing to do is to define the domain of the problem. In order to do that, since the Hammurabi rules are valid Scala statements, it is sufficient to create a plain Scala Person class having as attributes the name, the position and the color of the pants of the golfer that it represents: class Person(n: String) { val name = n var pos: Int = _ var color: String = _ } Then we can model the fact that all the golfers must have different position and pants color by putting them in 2 different Set: var availablePos = (1 to 4).toSet var availableColors = Set("blue", "plaid", "red", "orange") and write two small methods that pull off them from the corresponding set once they have been assigned to a specific golfer: val assign = new { def color(color: String) = new { def to(person: Person) = { person.color = color availableColors = availableColors - color } } def position(pos: Int) = new { def to(person: Person) = { person.pos = pos availablePos = availablePos - pos } } } Those methods are written in a quite weird way just to make even more readable the DSL that will be used to define the rules and to stress the idea that it is possible to write the rules in a valid Scala that could be easily understood by a non-technical person. Of course it is easy to go even further by encapsulating other concepts in some convenient methods as it has been done above. Now everything is ready to write the set of rules describing the golfers problem: val ruleSet = Set( rule ("Unique positions") let { val p = any(kindOf[Person]) when { (availablePos.size equals 1) and (p.pos equals 0) } then { assign position availablePos.head to p } }, rule ("Unique colors") let { val p = any(kindOf[Person]) when { (availableColors.size equals 1) and (p.color == null) } then { assign color availableColors.head to p } }, rule ("Joe is in position 2") let { val p = any(kindOf[Person]) when { p.name equals "Joe" } then { assign position 2 to p } }, rule ("Person to Fred’s immediate right is wearing blue pants") let { val p1 = any(kindOf[Person]) val p2 = any(kindOf[Person]) when { (p1.name equals "Fred") and (p2.pos equals p1.pos + 1) } then { assign color "blue" to p2 } }, rule ("Fred isn't in position 4") let { val possibleFredPos = availablePos - 4 val p = any(kindOf[Person]) when { (p.name equals "Fred") and (possibleFredPos.size == 1) } then { assign position possibleFredPos.head to p } }, rule ("Tom isn't in position 1 or 4") let { val possibleTomPos = availablePos - 1 - 4 val p = any(kindOf[Person]) when { (p.name equals "Tom") and (possibleTomPos.size equals 1) } then { assign position possibleTomPos.head to p } }, rule ("Bob is wearing plaid pants") let { val p = any(kindOf[Person]) when { p.name equals "Bob" } then { assign color "plaid" to p } }, rule ("Tom isn't wearing orange pants") let { val possibleTomColors = availableColors - "orange" val p = any(kindOf[Person]) when { (p.name equals "Tom") and (possibleTomColors.size equals 1) } then { assign color possibleTomColors.head to p } } ) Here the first 2 rules explicitly leverage the uniqueness of positions and pants colors by assigning the last available of them to the only person who still doesn't have one. The other rules just match one by one the sentences of the problem as it has been defined. Now it is possible to make Hammurabi solve the problem by creating the four golfers: val tom = new Person("Tom") val joe = new Person("Joe") val fred = new Person("Fred") val bob = new Person("Bob") add them to a working memory (the set of objects against which the rule engine will evaluate and fire the rules): val workingMemory = WorkingMemory(tom, joe, fred, bob) and letting the rule engine, initialized with the formerly defined set of rules, to work on it: RuleEngine(ruleSet) execOn workingMemory Working with immutable data structures Immutability is probably not something that should be enforced at all costs in Hammurabi, for a very simple reason: the largest part of the execution time is spent by Hammurabi, like all other rule engines, looking for rules that can actually be executed (fired), i.e. the ones for which the when condition is true. During this phase data are only read and never written, so immutability doesn't matter at all, and all the rules that can be fired are put in an agenda. During the subsequent phase the rules in the agenda MUST be fired one by one, since the execution of one of them could make false the when condition of another one. It means that lack of immutability shouldn't prevent the rule engine to safely run in parallel during the discovery phase. That said, it is also possible to obtain the same result working with immutable data structures. For example having an immutable person: case class Person(name: String, pos: Int = 0, color: String = null) it's enough to rewrite the methods that assign the position and pants color to the golfers as it follows: val assign = new { def color(color: String) = new { def to(person: Person) = { remove(person) produce(person.copy(color = color)) availableColors = availableColors - color } } def position(pos: Int) = new { def to(person: Person) = { remove(person) produce(person.copy(pos = pos)) availablePos = availablePos - pos } } } leaving all the rules unchanged. In this way the old version of the Person is removed from the working memory and a brand new one, with its position or color set accordingly to the fired rule, is produced and then added to the working memory itself. The methods remove and produce can be indeed used respectively to remove objects from the working memory and produce new objects, that could be then used to evaluate and fire other rules. Hammurabi internals In Hammurabi all the rules are evaluated (but not fired) in parallel basically by assigning each rule to a different Scala actor. The replacement of this actor implementation with one based on the upcoming Scala parallel collections is currently under evaluation but I decided to wait until this technology will be stable. As anticipated, the evaluation of the rule in parallel is safe because is a read only process. While the actors discover set of variables that can fire the rules they are evaluating, they add them to the rule engine's agenda. At the end of this evaluation process the rules are fired sequentially one by one after a re-evaluation of their application condition, because the execution of one of them could change the result of that condition for a subsequent one. All the rules are fired in no particular order unless a different priority has been specified for some of them. Indeed since sometimes there could be the need to treat some rules as special cases they have an optional property called salience that acts as a priority setting for that rule in order to allow activated rules with the highest salience to always fire first, followed by rules of lower salience. By default all rules have salience 0 but you can alter it in the rule definition as it follows: rule ("Important rule") withSalience 10 let { ... } rule ("Negligible rule") withSalience -5 let { ... } Each actor also records the set of values on which the rule it is responsible for has been already executed, in order to don't fire again the same rule on the same values. The evaluation phase and the firing one are then executed again and again until either there is no rule that can be fired or one of the rule during its firing phase invokes one of the methods exitWith, making the rule engine gracefully finishing returning a value representing the result of the whole evaluation process or failWith that cause the rule engine to terminate by throwing an exception. Of course if the rule engine stops just because there are no longer rules that can be fired, the result of the evaluation is represented by the whole working set (as in the former example) and you can read from there the values you are interested in. Further implementations and improvements At the moment it is only possible to take from the working memory the object(s) against which a given rule will be evaluated only selecting them by type, as shown in the statement: val p = any(kindOf[Person]) Further mechanisms to categorize and select those objects in a more precise way are under evaluation, even if it is already possible to limit them with a Boolean function as in the following example: val p = kindOf[Person] having (_.name == "Joe") This is useful also under a performance point of view because it dramatically lowers the number of combination that the rule engine needs to check before to find a rule that can be fired. For example the "Person to Fred’s immediate right is wearing blue pants" rule could be rewritten as it follows bringing the number of combination that has to be tried from 16 to 4: rule ("Person to Fred’s immediate right is wearing blue pants") let { val p1 = kindOf[Person] having (_.name == "Fred") val p2 = any(kindOf[Person]) when { p2.pos equals p1.pos + 1 } then { assign color "blue" to p2 } } I am also evaluating of directly feeding the working memory with a NoSQL database. In other words, with this solution, the data present in the db could represent the working memory itself. At the moment I am experimenting with MongoDB since is the one I know best, but if somebody has some other good idea or even better wants to collaborate with this project I'd be very glad of it. The version 0.1 of the Hammurabi rule engine has been just released and it is available here.
April 18, 2011
by Mario Fusco
· 27,846 Views
  • Previous
  • ...
  • 445
  • 446
  • 447
  • 448
  • 449
  • 450
  • 451
  • 452
  • 453
  • 454
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×