DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Latest Articles - DZone

article thumbnail
ASP.NET - Client Side State Management - Hidden Fields
What are Hidden Fields? Hidden fields technique is widely used in ASP.NET programing. Hidden fields are html input control with hidden type that store hidden data in the html. An example for a hidden field can look like this: In the example above, I chose to show the event target hidden field in order to indicate that even in postback mechanism hidden fields are being used. The data stored in a hidden field is available when the form is processed on the server or when we use javascript. Hidden Fields Values Hidden fields store only one value in their value property. The value is saved as a string and therefore in order to use it for other types you need to perform casting. Hidden fields' data is submitted to the server only in HTTP post operation. You can see the stored data easily by using the View Source operation of the browser. You can see it by clicking the right mouse button and then choosing View Source from the menu (if the operation is available). Be aware not to use hidden fields to store confidential data! The values has page context and therefore when you leave a page the data stored in the hidden fields is disposed. Server Control Hidden Fields There are two types of server control hidden fields - System.Web.UI.WebControls.HiddenField and System.Web.UI.HtmlControls.HtmlInputHidden. Both types has the same primary Value property to hold the value of the hidden field. You should choose between these types whenever you need to use a server side hidden field (A note - The difference between HtmlControls and WebControls isn't in the context of this post). When you don't use server controls you can use the Request.Form NameValueCollection to get the hidden field value by providing the client id of the hidden field. For example the code bellow will return the string value of the __EVENTTARGET hidden field: string eventTarget = Request.Form["__EVENTTARGET"]; Hidden Fields Example I got some requests for an example of how to use the hidden fields inside web forms. The example is very easy to understand. I have an html input with type hidden. In the first request I fill the input with a message value with javascript function. I use a button control to do a post back to the server and in the post back the value of the hidden field is inserted into a label control. Pay attention that because I use an html hidden field that isn’t a server control after a second post back it’s value will be empty. The only reason that the label will still show a message is because the label’s default value of the EnableViewState is true. The web page’s code: On the server side: public partial class _Default : System.Web.UI.Page { #region Consts private const string SCRIPT_KEY = "HtmlHiddenFieldScript"; #endregion #region Page Events protected void Page_Load(object sender, EventArgs e) { // Insert message to the label of the html control hidden // field if there is a value in the html hidden field string message = Request.Form["HTMLHiddenField"]; if (!string.IsNullOrEmpty(message)) { lblHTMLHiddenField.Text = message; } } protected void Page_PreRender(object sender, EventArgs e) { // Register a startup script in order to fill the html // hidden field with a message value if (!ClientScript.IsClientScriptBlockRegistered("HtmlHiddenFieldScript") && !IsPostBack) { ClientScript.RegisterStartupScript(typeof(Page), SCRIPT_KEY, "PutHtmlHiddenFieldValue('Html hidden hello world');", true); } } #endregion } Summary To sum up the post, today I drilled down into the hidden fields technique. This technique is very popular and is used widely in ASP.NET. In the next post I'll continue the tour in the client side state management techniques.
July 14, 2008
by Gil Fink
· 138,739 Views
article thumbnail
First Release of Nuxeo WebEngine
WebEngine relies on the Nuxeo content infrastructure (OSGi runtime, component architecture, document repository, ECM services, etc.) to provide a component-based programing model and a web development model for building componentized content-centric applications (such as wikis, blogs, content-oriented websites, etc.). Summary: WebEngine is a lightweight, versatile, content-centric, open source web framework to quickly build and deliver next generation content-oriented web applications. WebEngine relies heavily on the REST paradigm: URLs are mapped to the hierarchical content repository, content is accessed using GETs, user actions are GETs and POSTs, etc. Hence it’s very easy and straightforward to write RESTful apps using WebEngine. WebEngine is fully extensible and componentized, thanks to OSGi (all components are OSGi bundles) and Nuxeo Runtime’s extension points. WebEngine can run either standalone (with startup time <4s) using the Nuxeo Runtime launcher and the embedded Jetty 6, or in a full-blown Java EE app server such as JBoss. WebEngine can also be connected to any Nuxeo EP instance (and Nuxeo Core repository) and be used to expose / publish its content to the web. (click to run the slideshow) Features highlights: Scripting (Groovy, JavaScript, Ruby, Python…) or Java code for business logic Advanced content model Leverage Nuxeo Platform’s ECM services Smart URLs management Powerful templating (based on the FreeMarker engine) Wikitext renderer (using Wikimodel) Open source under the LGPL license Join the community! Download it and give it a try Check out the presentation on SlideShare Read the reference documentation Join the discussion to get help and give feedback on the forums Learn how WebEngine fits in the overall Nuxeo roadmap and… Contribute]! :-)
July 11, 2008
by Stefane Fermigier
· 1,390 Views
article thumbnail
ASP.NET Client Side State Management
There are two types of state management - server side and client side. In this post I'm going to introduce the client side state management and explain when to choose client side or server side state management. Client Side State Management Techniques Client side state management include the following techniques: ViewState - ASP.NET track the state of the controls on the pages through the ViewState. Also, ViewState is used to store small custom values. Don't hold big custom data object because it will affect the performance of your web page. Hidden fields - hidden fields are html input control with a type of hidden - . They are used to store data in the html without presenting the data in the browser. The data of hidden field is available when the form is processed or when using javascript. The ViewState use hidden field to save its data. You can use the View Source operation (if enabled) to scan the web page's source and to look for hidden fields. Query strings - query string state is stored in the URL that is sent to the browser. Unlike ViewState and hidden fields, the user can see the values without using special operations like View Source. The parameters are passed in the URL separated by the & symbol. For example, in the following URL the query string is built out of two data parameters, a and b, which hold the values 1 and 2. Cookies - the cookies store their values in the user's browser and the browser send them back to the server every time a request is performed. Cookies are maintained during the session of a user in the web site and therefore can be used in multiple web pages of the site. Control state - when building custom controls you should use the control state to save your state if EnableViewState property is set to false. This is the only reason to use this technique. How do you know when to choose the client side or server side state management? When to Choose Client Side State Management Choose client side for better scalability and support multiple servers. The client side helps to get better scalability by storing state data in the user's browser instead of using the web server's memory. The client side support multiple servers because all the state is located in the browser. This way when you are redirected to another server you don't need to worry about your state. The thing I wrote doesn't means that you can't use server side state management for the supporting of multiple server. To enable server side state management to be able to support multiple servers you'll have to use centralized state management (state server or store state in database) or you'll have to use techniques like sticky connection for load balancing. When to Choose Server Side State Management Choose server side for better security and to reduce bandwidth and web page's size. Server side state management is more secure. The state is saved on the server and therefore isn't delivered to the client. Confidential state data shouldn't be used with client side state management. Server side reduce the traffic to and from the client because data isn't sent to the browser and it's saved on the server. You should always remember that using client side state management sends data to the user's browser and that data is sent back to the server every time. This situation increases bandwidth usage and therefore your application will be less responsive and you'll suffer from performance issues. In the next post I'll drill down into the client side techniques and explain how to use them.
June 30, 2008
by Gil Fink
· 44,719 Views
article thumbnail
Concurrency and HashMap
In theory everyone knows Hash Map is not Thread Safe and it shouldn’t be used in multi Threaded applications. But still people come out with their own theories that they can use HashMap in their context. Some say they are just reading the data and map is not written to a lot. Unfortunately none of these explanations holds good when one lands up in a synchronization issue. Normally most of the guys do not understand fundamentals around Java Memory Model and Concurrency .One cannot blame them for not knowing their fundamentals as its hard for people to visualize concurrent executions since from college days we are used to sequential executions of program. Enuf of blame game, now lets look into some code.Have a look at the code mentioned below and come out with all your theories of what can go wrong with it or theories which say its all correct. public class MapTestTask implements Runnable { private Map hashMap; private Object value = new Object(); public MapTestTask(Map map) { this.hashMap = map; } public void run() { hashMap.put(Thread.currentThread(), value); Object retrieved = hashMap.get(Thread.currentThread()); if (retrieved == null) { // Can it ever Happen } } } Now question is when we run multiple such Threads can we ever see retrieved as null. If we look from sequential point of view it can never happen.But when concurrency comes into picture this can happen. I will give you a code with which can reproduce this scenario. This is my sincere advise : do not over engineer when Concurreny is involved. Because none of the theories will stand the test of time in a concurrent environment. As a rule of thumb use Thread safe collections wherever concurrency is involved. Finally i will leave you with this interesting bug in Java which says HashMap can get into infinite loop when used in muti Threaded Environment which further reiterates my point that not all the possible scenarios can be visualized in a multi threaded environment and therefore one should rely on basics rather than trying a smart creativity which has high probability of landing into trouble at some point in future. Source Code for Reproducing package test; import java.util.Map; public class MapTestTask implements Runnable { private Map hashMap; private Object value = new Object(); public MapTestTask(Map map) { this.hashMap = map; } public void run() { hashMap.put(Thread.currentThread(), value); Object retrieved = hashMap.get(Thread.currentThread()); if (retrieved == null) { // Can it ever Happen System.out.println("Oh My God it can happen."); } } } package test; import java.util.Map; import java.util.HashMap; public class TestMap { public static void main(String[] args) throws InterruptedException { Map map = new HashMap(); int NUM_THREADS = 1000; Thread[] threads = new Thread[NUM_THREADS]; for (int i = 0; i < NUM_THREADS; i++) { threads[i] = new Thread(new MapTestTask(map)); } for (int i = 0; i < NUM_THREADS; i++) { threads[i].start(); } for (int i = 0; i < NUM_THREADS; i++) { threads[i].join(); } } } From http://pitfalls.wordpress.com/2008/06/29/concurrencyandhashmap/
June 30, 2008
by Pavitar Singh
· 37,891 Views
article thumbnail
Create New Eclipse Workspace - With All Your Old Settings
It's all a matter of taste. Do you like to have just one workspace for all your projects, or do you prefer to have multiple separate workspaces? Sure, the first way seems to be the official, supported. It should be easy to manage the workspace -- given the tools like working sets (and working set filters), mylyn and the ability to close projects. But I still don't get it. I hate when my workspace is overflowing with projects, I want to have as many workspaces as projects. So I create new workspace and live happily ever after. But wait -- all my settings are gone. All my carefully crafted custom templates, all my keybindings, my font settings, everything is gone. It's all text, fortunately Lucky us. All eclipse settings are saved as a plain text in the workspace directory. So if you want to create new workspace, but preserve your settings, I have two answers for you: The short answer All settings are stored in the .metadata/.plugins/org.eclipse.core.runtime/.settings directory. I mean -- all relevant settings. If you look into .metadata/.plugins directory there are many more directories with settings, but they are too project specific. I've walked trough these configuration files one by one, believe me, nothing useful lies hidden there. So the short answer is: If you want to create a new eclipse workspace and preserve all your settings, simply copy the .metadata/.plugins/org.eclipse.core.runtime/.settings directory into your new workspace directory. Do not copy other directories! There are project specific settings and since your old projects are left in your old workspace, the copied settings would not be valid and you would get some nasty exceptions at eclipse startup. The long answer Let the code do the talk for me. I have created a (simple) shell script that automates new workspace creation. The downside is that it requires either a *nix shell or windows with cygwin. Nevertheless, you can always manually copy the .settings directory (see the short answer). The script has been tested by me, I and myself so it should work (most of the time). To use it, save it somewhere, make it executable (chmod +x new-workspace.sh) and run it either in interactive mode ./new-workspace.sh -i where it will ask you the details, or with paths to your workspaces (the new workspace directory will be created for you, just specify the path) ./new-workspace.sh old-workspace new-workspace The script will create new workspace directory and copy the relevant settings from your old workspace. If it doesn't work for you, drop me a comment. Feel free to improve it. Update: the pastebin page expired (although I'd swear I checked the keep forever option), so I moved the script over to github.
June 30, 2008
by Tomas Kramar
· 237,689 Views
article thumbnail
Python Development in NetBeans IDE
The nbPython project aims to provide support for Python development to NetBeans IDE. Below follow the step-by-step instructions for getting started! The bits are available (milestone releases) for download now as NetBeans modules (nbms). Let us get started: Download/lnstall the M3 NBMs from here. Fire up NetBeans IDE. Go to Tools -> Plugins -> Downloaded Tab. Click on "Add Plugins" and select all the NBM files in the unzipped directory. Now, you will see all the chosen modules along with their descriptions. You should see something like this: Press the "Install" button. Using NetBeans IDE 6.1, I get a dependency error: Then, I downloaded a latest build NetBeans IDE Build 200806220002 from here and repeated the above steps. This time there was no dependency problem. You will get a "Validation Warning". Press "Continue": The installation will continue and will be over without any further user interaction required. Creating a New Python Project Go to File-> New Project-> Python-> Python Project. Choose a Project Name- say "HelloNbPython". Click on Finish. A new project gets created and is visible in the Project Explorer window. No default file is created. Right Click on the project name, and select New-> Empty Python File. Enter a name, say- HelloWorld. A Python script will be generated for you which will simply print "Hello World". The project tree is now as follows: HelloNbPython/ pyproject/ project.properties src/ HelloWorld.py Runing a Python Script Right Click on HelloWorld.py in the project explorer and click "Run Python Script". First time I did this, I got the following exception message: java.io.IOException: Cannot run program "/home/amit/netbeans-dev-200806220002/nbpython/jython-2.5/bin/jython" (in directory "/home/amit/NetBeansProjects/HelloNbPython"): java.io.IOException: error=13, Permission denied So I checked the file permission of the 'jython' executable: $ ls -l jython -rw-r--r-- 1 amit amit 5101 2008-06-22 15:05 jython As you can see there is no executable permission for 'jython' as indicated by the absence of the 'x' flag So, I made the 'jython' executable by: chmod +x jython Run the script again and you should get "Hello World" in the output window.
June 22, 2008
by Amit Saha
· 33,234 Views
article thumbnail
Glimmer - Using Ruby to Build SWT User Interfaces
Glimmer is a JRuby DSL that enables easy and efficient authoring of user-interfaces using the robust platform-independent Eclipse SWT library. Glimmer comes with built-in data-binding support to greatly facilitate synchronizing UI with domain models. The goal of the Glimmer project is to create a JRuby framework on top of Eclipse technologies to enable easy and efficient authoring of desktop applications by taking advantage of the Ruby language. With Glimmer having just become an Eclipse project, it's a good time to find out more. Philosophy Glimmer's design philosophy can be summarized as follows: Concise and DRY Asks for minimum info needed to accomplish task Convention over configuration As predictable as possible for existing SWT developers Conventions Since Glimmer relies on Ruby, it is different in its syntax and conventions from what typical Java SWT developers would expect: Method parentheses are optional Java-vs-Ruby example: show() => show Method names follow underscored syntax Java-vs-Ruby example: addListener => add_listener Classes are constructed using the new(...) method (as opposed to new keyword): Java-vs-Ruby example: new GridLayout() => GridLayout.new Download Please download Glimmer from RubyForge: https://rubyforge.org/projects/glimmer/ NOTE: Glimmer is moving to Eclipse.org. Please visit http://andymaleh.blogspot.com for up-to-date news on the move and the upcoming download location on the Eclipse website. Installation Extract the Glimmer zip file and follow the installation instructions in the README file. NOTE: While Glimmer is platform-independent, its functionality has only been verified on Windows. Feedback from Mac and Linux users would be greatly appreciated. Tutorial Let's start with a very simple Glimmer Hello World example: shell { label { text “Hello World!” } } This will render the following: [img_assist|nid=3586|title=|desc=|link=none|align=undefined|width=126|height=48] In the SWT library a shell represents an application's window. It acts as a frame around the application widgets, which are visual components that display information and/or enable interaction with the user. One widget that was used in the Hello World example is the label widget, which simply displays text on the screen. Shell is also considered a widget, except it is a special kind of widget called composite. The shell keyword, which declared the application's shell, was followed by a block of code encased in curly braces. This block contains the shell content declarations, such as the Hello World label. The label keyword was also followed by a block of code. However, this block contained a property declaration for the label, stating that the text value is “Hello World!” So, to declare a widget, simply state its name followed by a block of code. The block may specify property values or nest other widget declarations for composite widgets. Now, let's move on to a more advanced example: shell { text "User Profile" composite { layout GridLayout.new(2, false) group { text "Name" layout GridLayout.new(2, false) layout_data GridData.new(fill, fill, true, true) label {text "First"}; text {text "Bullet"} label {text "Last"}; text {text "Tooth"} } group { layout_data GridData.new(fill, fill, true, true) text "Gender" button(radio) {text "Male"; selection true} button(radio) {text "Female"} } group { layout_data GridData.new(fill, fill, true, true) text "Role" button(check) {text "Student"; selection true} button(check) {text "Employee"; selection true} } group { text "Experience" layout RowLayout.new layout_data GridData.new(fill, fill, true, true) spinner {selection 5}; label {text "years"} } button { text "save" layout_data GridData.new(right, center, true, true) } button { text "close" layout_data GridData.new(left, center, true, true) } } }.open This will render the following: [img_assist|nid=3587|title=|desc=|link=none|align=undefined|width=195|height=209] The example contains a variation of widgets from SWT: Composite: a widget that can simply contain other widgets and manage their layout Group: Similar to Composite except that it usually has a border and a title. Text Field: Enables user to type in text information Checkbox Button: Allows user to make a selection from different options Radio Button: Allows user to make a selection between options that are mutually exclusive Spinner: Enables user to type in numeric information or spinning a number selection by mouse Push Button: Enables user to initiate actions Given that Glimmer relies on the Eclipse SWT library, developers may consult the SWT API as a reference on all the widgets, including their properties and layout options: http://help.eclipse.org/stable/nftopic/org.eclipse.platform.doc.isv/reference/api/index.html Keep in mind the following rules when reading the SWT API: Any widget available in SWT, including custom widgets written by developers, can be accessed from Glimmer by downcasing/underscoring the widget's name (e.g. Composite -> composite, LabledText -> labeled_text) Properties available on SWT widgets are specified by listing them followed by their values, each on a line or separated by semicolons within the widget's block (e.g. label {text "Username:"; font some_font}) Property names are also downcased/underscored in Glimmer. SWT widgets must have a style value specified, which is a constant available on the “SWT” class. Glimmer generally hides that by relying on smart defaults. Here is a listing of the defaults configured in Glimmer: text: SWT::BORDER table: SWT::BORDER spinner: SWT::BORDER button: SWT::PUSH Nonetheless, to customize a widget, a style value may be optionally specified within parentheses after the widget name. For an example, “button(SWT::RADIO)” renders a radio button and “button(SWT::CHECK)” renders a checkbox button. Glimmer's syntax also has syntactic sugar for specifying the style. Simply state the name of the style in the standard Ruby downcased/underscored format without the “SWT::” prefix. For example, button(SWT::RADIO) becomes button(radio). SWT composite widgets, such as shell, composite, and group can have a layout manager that lays out child widgets according to a certain pattern without the need to specify the (x, y) position of each child widget explicitly. Layout managers come in many flavors, such as GridLayout, offering a grid-like layout; FillLayout, allowing child widgets to fill the whole available area; and RowLayout, rendering child widgets one after the other in a row by default. Glimmer is configured with smart defaults for layout managers too: shell: FillLayout composite: GridLayout with one column group: GridLayout with one column GridLayout is a particularly useful SWT layout, so I will go over it in a little more detail here. GridLayout allows you to lay widgets out in a grid similar to HTML tables. To instantiate a custom GridLayout, you must specify the number of columns and whether they are of equal width or not. Here is a block of code demonstrating a group box having a GridLayout with 2 columns of unequal width: group { layout GridLayout.new(2, false) } Now, suppose we add four elements to that group box: group { layout GridLayout.new(2, false) label {text "First"}; text {text "Bullet"} label {text "Last"}; text {text "Tooth"} } The specified GridLayout will lay out the child widgets in the grid from left to right and top to bottom: The label with the text “First” will go into the 1st column of the 1st row. The text box with the text “Bullet” will go into the 2nd column of the 1st row. The label with the text “Last” will go into the 1st column of the 2nd row. The text box with the text “Tooth” will go into the 2nd column of the 2nd row. The group was actually a part of the advanced example illustrated earlier. It was given a title (by specifying the text attribute,) and the widget declarations were written in a way that maps visually to how they appear on the screen. Notice how text box declarations are on the same line as the label declarations since both the label and text box go under the same row, which helps improve code readability and maintainability: group { text "Name" layout GridLayout.new(2, false) layout_data GridData.new(fill, fill, true, true) label {text "First"}; text {text "Bullet"} label {text "Last"}; text {text "Tooth"} } That renders the following: [img_assist|nid=3588|title=|desc=|link=none|align=undefined|width=89|height=73] Layout of specific widgets may be further customized by specifying layout data. For GridLayout, layout data is specified through GridData objects. For example, we may decide to have the text boxes in the previous example have a greater width: group { layout GridLayout.new(2, false) label {text "First"}; text { text "Bullet" layout_data GridData.new(100, default) } label {text "Last"}; text { text "Tooth" layout_data GridData.new(100, default) } } This renders the following: [img_assist|nid=3589|title=|desc=|link=none|align=undefined|width=125|height=73] The used GridData constructor takes two parameters: width hint and height hint. The width was set to 100 pixels for both text boxes. The height was kept at the default value (SWT::DEFAULT) For more details about GridLayout, GridData, and other layout managers, please refer to the SWT API documentation. So far we have covered how to construct user-interfaces that can display data and gather input from the user. Next, we will demonstrate how to perform work based on actions taken by the user. SWT widgets can be monitored for certain user-interface events, such as mouse clicks, focus gain and loss, and key presses. With the original SWT API, events can be monitored by adding listeners to widgets. For example, to monitor the push of a button, you would add a SelectionListener that does some work in its widgetSelected event method. With Glimmer, events can be monitored by declaring their name (following Ruby conventions) prefixed by “on” Here is an example of how to monitor button selection: import org.eclipse.swt.widgets.MessageBox @shell1 = shell { composite { button { text 'Save' on_widget_selected { message_box = MessageBox.new(@shell1.widget, SWT::NULL) message_box.text = 'Information' message_box.message = 'Saved!' message_box.open } } } } @shell1.open This renders the following: [img_assist|nid=3590|title=|desc=|link=none|align=undefined|width=170|height=145] On click of the button, a message box is opened to let the user know that the information entered is saved. MessageBox is a class from SWT that represents message dialogs. It was imported using the JRuby import method. Its constructor takes a parent and style. To obtain the parent, we assigned the shell object to a Ruby class variable @shell1. Since Glimmer wraps all SWT constructed objects with Glimmer decorators ( e.g. Shell is wrapped with RShell,) to obtain the SWT Shell class and pass it as the parent to the MessageBox constructor, the widget method was called (e.g. @shell1.widget.) In the original SWT API, MessageBox has setter methods to set its text and message attributes. However in JRuby, the developer has the option to set them following the Ruby attribute conventions (e.g. message_box.text = 'value') because JRuby automatically enhances all Java objects with methods that follow the Ruby convention. Another example that benefits from event monitoring is field validation on loss of focus. For example, let's say we are validating the ZIP code on an address form, and we would like to display an error message if its value does not have a valid ZIP code format (e.g. 12345 or 12345-1234,) here is how we would do it with Glimmer (please add the following code before the button in the previous example): import org.eclipse.swt.widgets.MessageBox @shell1 = shell { composite { label { text "ZIP Code" } text { on_focus_lost { |focus_event| zip_code = focus_event.widget.text unless zip_code =~ /^\d{5}([-]\d{4})?$/ message_box = MessageBox.new(@shell1.widget, SWT::NULL) message_box.text = 'Validation Error' message_box.message = 'Format must match ##### or #####-####' message_box.open focus_event.widget.set_focus end } } button { text 'Save' on_widget_selected { message_box = MessageBox.new(@shell1.widget, SWT::NULL) message_box.message = 'Saved!' message_box.open } } } } @shell1.open Here is what it produces: [img_assist|nid=3591|title=|desc=|link=none|align=undefined|width=346|height=151] Notice how the on_focus_lost block has a FocusEvent object as a parameter. This parameter may be specified optionally whenever some information is needed from the event object. Again, this maps to the focusLost method on the FocusListener class in the original SWT API, which also takes a FocusEvent object as a parameter. While widgets in the original SWT API have a setFocus event to grab the user interface focus, in JRuby set_focus may be used instead following the Ruby naming conventions. Now, in order to cleanly separate event-driven behavior from user-interface code, we can rely on Glimmer's data-binding support. Stay tuned for the next tutorial, which will cover data-binding and how to achieve clean code separation with the Model-View-Presenter pattern. References: Glimmer Eclipse Technology Project Proposal: http://www.eclipse.org/proposals/glimmer/ Glimmer Newsgroup: http://www.eclipse.org/newsportal/thread.php?group=eclipse.technology.glimmer Glimmer at RubyForge: http://rubyforge.org/projects/glimmer/ Author Blog: http://andymaleh.blogspot.com Andy Maleh (andy at obtiva.com), Senior Consultant, Obtiva Corp.
June 19, 2008
by Andy Maleh
· 60,591 Views
article thumbnail
Running the Table With JMesa
Shhhh. I’ll tell you a secret. I don’t like tables. I know. Shocking, isn't it? Don't get me wrong: I don't dislike tables per se. They're great for displaying tabular material. (For page organization, not so much.) But I so dislike the code needed to build a table within a JSP. It usually comes down to something like this: User IDNameEmail${row.userID}${row.name}${row.email} All that iterative logic simply looks incomprehensible to me. It's still better than scriptlets or custom tag libraries (both of which were, to be sure, phenomenal in their time), but it's an undigestible mass, and even if I do step through it line by line and understand what it does, I'm still left with just a table. Users accustomed to active, Javascript-assisted widgets don't respond to tables that just lie there. Many more lines of code will be needed to enable them to do useful things like paginating through long lists of items, sorting by column values, and the like. It'll be an unholy mix of HTML, JSP directives, JSP tags, EL, Javascript, Java, XML, properties files, and so forth. The whole thing seems so error-prone (note to self: more code + more languages = more "opportunities" for bugs). But recently I discovered an open-source Java library called JMesa that provides another way. I'm going to share with you some of the things I've found in JMesa, building up an HTML page containing a table from nothing to, well, considerably more than nothing. There's a good deal of code here, to give you a sense of the JMesa API; hopefully. you'll come away with some ideas about how you can use JMesa in your own projects. I won't bother with package declarations, imports, or code not relevant to the point at hand; the complete code is available for download in the form of an Eclipse project. Installation instructions will be found at the end of this article. Join me in exploring JMesa! Preparation A Page to Show Before we can get to JMesa, though, we'll need a few things: a page within which to display our table, for instance. In fact, we'll learn even more if we put this page in a context. I have recently fallen in like with Spring MVC and so will use that to build a simple site with a few pages. Just to be clear, while Spring dependency injection and utilities are woven into the code below, JMesa does not depend upon Spring. The pages are not fancy, and I am going to skip most of the setup. Everything is included in the download, of course. One thing I shouldn't skip is the controller for the search results page, the page within which we will build our table. We'll start with pretty much the simplest functionality we can: public class SimpleSearchController extends AbstractController { @Override protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception { return new ModelAndView("simple-results", "results", "Here we will display search results"); } } For those not familiar with Spring MVC, the ModelAndView return value contains a string that will be resolved to a view (in this project, it is resolved to "/WEB-INF/jsp/simple-results.jsp"), and a key-value pair (the second and third constructor arguments) that can be accessed using EL on the JSP page: ${results} Finally, we use the Spring jmesa-servlet.xml configuration file to create and associate a URL with our controller: welcomeController simpleSearchController Clicking on the "Search" link in the menu now produces: [img_assist|nid=3678|title=Figure 1.|desc=A simple page for our table|link=none|align=left|width=757|height=240] All right, not much. But it's the page we need. Something to Display Another thing we need before we can build a table is something to show in it. This "domain" object should be pretty easy to display: public class HelloWorld implements Comparable { private int pk; private String hello = "Hello"; private String world = "world"; private String from = "from"; private String firstName; private String lastName; private String format = "{0}, {1}! {2} {3} {4}"; // ... accessors and mutators public String toString() { return MessageFormat.format(getFormat(), hello, world, from, getFirstName(), getLastName()); } // ... implementations of equals, hashCode, and compareTo } Persistence Service Of course, we need instances of this domain object. Normally, we'd get them from a persistence service; for now, we'll just create them in memory: public class HelloWorldService { private int nextId; private Set helloWorlds = new TreeSet(); public HelloWorldService() { nextId = 1; helloWorlds.add(newInstance("Albert", "Einstein")); helloWorlds.add(newInstance("Grazia", "Deledda")); helloWorlds.add(newInstance("Francis", "Crick")); helloWorlds.add(newInstance("Linus", "Pauling")); helloWorlds.add(newInstance("Theodore", "Roosevelt")); helloWorlds.add(newInstance("Hideki", "Yukawa")); helloWorlds.add(newInstance("Harold", "Urey")); helloWorlds.add(newInstance("Barbara", "McClintock")); helloWorlds.add(newInstance("Hermann", "Hesse")); helloWorlds.add(newInstance("Mikhail", "Gorbachev")); helloWorlds.add(newInstance("Amartya", "Sen")); helloWorlds.add(newInstance("Albert", "Gore")); helloWorlds.add(newInstance("Amnesty", "International")); helloWorlds.add(newInstance("Daniel", "Bovet")); helloWorlds.add(newInstance("William", "Faulkner")); helloWorlds.add(newInstance("Otto", "Diels")); helloWorlds.add(newInstance("Marie", "Curie")); } public Set findAll() { return helloWorlds; } private HelloWorld newInstance(String firstName, String lastName) { HelloWorld hw = new HelloWorld(); hw.setPk(nextId++); hw.setFirstName(firstName); hw.setLastName(lastName); return hw; } } That's that. Now we're ready to focus on JMesa. JMesa Let's start with something extremely simple. On the very first page of the JMesa web site we find four lines of code that we can appropriate and refashion for a Spring controller: public class BasicJMesaSearchController extends AbstractController { private HelloWorldService helloWorldService; public void setHelloWorldService(HelloWorldService helloWorldService) { this.helloWorldService = helloWorldService; } @Override protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception { Set results = helloWorldService.findAll(); TableFacade tableFacade = new TableFacadeImpl("results",request); tableFacade.setItems(results); tableFacade.setColumnProperties("pk", "firstName", "lastName", "format"); return new ModelAndView("results", "results", tableFacade.render()); } } We let Spring inject the HelloWorldService, which we use to retrieve a set of items to display. Then we create and configure the JMesa TableFacade class. This class takes an HTTP request in its constructor: TableFacade is going to send itself messages passed as parameters in the request (more on this in a moment). We supply it with the set of items and with which JavaBean property of those items we want displayed in each column. We'll also need a bit of new code in the search results page (in the project, this is actually a different search results page, as you, oh sharp-eyed reader, have already noticed): ${results} And we'll need to create and point to the new controller in jmesa-servlet.xml: ... basicSearchController Redeploy, and the results look like magic. How did we get them? [img_assist|nid=3679|title=Figure 2.|desc=Using JMesa "out of the box"|link=none|align=left|width=757|height=405] The key is in the variable results, which now holds the entire text of the table generated by the JMesa TableFacade when we called its render method. We also put a self-submitting HTML form around the JMesa table that it will use to send itself messages about how to alter itself. This makes possible many amazing features. The table automagically paginates itself. It allows the user to change the number of rows displayed. It allows sorting on any column or combination of columns. It provides color striping of table rows and onMouseOver row highlighting. And every bit of this came for free: we did nothing to enable it but what you have already seen. (OK, we played around with some of JMesa's images and CSS style sheets to make it fit in with our color scheme, but that really shouldn't count.) To demonstrate, we'll use the select at the top of the form to change the number of rows displayed to 16, sort by first name ascending and last name descending (by clicking on the first column header once and the second twice), and mouse over the third row to see the highlighting: [img_assist|nid=3681|title=Figure 3.|desc=JMesa search results sorted and highlighted|link=none|align=left|width=757|height=565] Now Al Gore and Einstein appear in the order we asked for. You will have noticed the images in the table toolbar. Those on the left are standard first, previous, next, and last navigation icons. The select we've already mentioned. But there are two other images as well: these turn filtering, another amazing feature of JMesa that is active by default, on and off. Filtering allows the user to apply expressions to a column in order to display only rows having matching values in that column. While filtering can take setup beyond the scope of this article, even by default it's astonishing. Try typing "Einstein" in the text field that appears above the last-name column header and clicking on the filter icon (the magnifying glass). The results show only the row containing Einstein's name in the last name column. And we didn't have to do a thing! [img_assist|nid=3680|title=Figure 4.|desc=JMesa search results filtered|link=none|align=left|width=757|height=280] See the JMesa web site for details about filtering, editable tables that keep track of your changes for you, and much, much more: it's impressive stuff. Customizing And now, to business. The JMesa default is astounding, but no default is ever exactly like you want it. The ability to customize is critical. Also, defaults rarely exercise every feature, and this one is no exception. Let's start with some requirements: We will display the value of each HelloWorld item's toString method in an additional column We will display more user-friendly values in the format column We will ensure that columns that cannot be reasonably sorted are made unsortable We will add columns containing links to edit and delete pages for the HelloWorld items We will display images in the edit and delete columns We will not display the Pk property of each item, but will pass its value to edit and delete pages as needed We will enable the user to retrieve a comma-separated-values (CSV) copy of the table contents We will enable the user to retrieve an Excel spreadsheet copy of the table contents We will disable filtering and highlighting We will reorganize the toolbar items in a different order Believe it or not, implementing each of these features will be quite easy! and you'll begin to get a sense for the possibilities of JMesa. ToString Column Each HelloWorld item produces a formatted string within its toString method. This is not a JavaBean property method, so we cannot directly point the TableFacade at it. We want this value to be rendered (to use JMesa terminology) as the contents of a (a cell) in each HTML row. Cell contents are produced by implementations of the CellEditor interface. Its getValue method is passed the item to be displayed, the property to be called, and the current row count. Since only the item itself is actually needed for our purpose, the implementation is simple: public class ToStringCellEditor implements CellEditor { @Override public Object getValue(Object item, String property, int rowcount) { if (item == null) { return ""; } return item.toString(); } } Of course, we'll need a column into which to put the results. All we need do is add an arbitrary value to the column properties list: tableFacade.setColumnProperties("firstName", "lastName", "format", "toString"); This value is used to retrieve the column: Row row = tableFacade.getTable().getRow(); Column column = row.getColumn("toString"); column.getCellRenderer().setCellEditor(new ToStringCellEditor()); Of course, this means that the getValue method of the ToStringCellEditor will always be passed a bogus property value, but since the editor doesn't use it, that's no problem. (Note that we've also left off the pk column as per requirements.) User-Friendly Format Column We continue by introducing a more user-friendly value into the format column. The format string "{0}, {1}! {2} {3} {4}" looks ugly and most likely won't be understood by an end user. The only real information it conveys is that it is the default value. We'll use a Spring MessageSource to supply something a little easier on the eyes at runtime. First, we'll add a property to the messages.properties file loaded by Spring at application startup: format.{0},\ {1}!\ {2}\ {3}\ {4}=Default format (The backslashes are needed to escape the white space in the key.) As we have already seen, a CellEditor is needed to change a cell's displayed value. Using MessageSource to produce the display value at runtime requires a few more lines than the ToStringCellEditor: public class SpringMessageCellEditor implements CellEditor { MessageSource source; String prefix; Locale locale; public SpringMessageCellEditor(MessageSource source, String prefix, Locale locale) { this.source = source; this.prefix = prefix; this.locale = locale; } public Object getValue(Object item, String property, int rowcount) { if (item != null) { try { return source.getMessage(prefix + "." + PropertyUtils.getProperty(item, property), null, locale); } catch (IllegalAccessException ignore) { } catch (InvocationTargetException ignore) { } catch (NoSuchMethodException ignore) { } } return null; } } We still have to add this editor to the column displaying the format property: Column column = row.getColumn("format"); column.getCellRenderer().setCellEditor(new SpringMessageCellEditor(messageSource, "format", locale); Unsortable Columns Next, we want the table to know that some columns are unsortable. Columns are typically sorted by property value, but we just added a column that corresponds to no property, that displays the output of the toString method. If the user clicked on the header of that column, he or she would wind up with a very ugly NullPointerException message. Making a column (actually, we need to have an HtmlColumn, but most columns qualify) unsortable is very simple: htmlColumn.setSortable(false); With this, no onClick method will be generated for the column header, preventing users from accidentally causing a mess. Edit and Delete Columns Now we'll add columns containing links to edit and delete pages for HelloWorld items. I prefer using icons to buttons saying "Edit" and "Delete", as it reduces the amount of textual information the user must process. Tables typically present a lot of information in a compact space, making user overload a problem worthy of attention. To do this, we'll need a CellEditor (by now, you knew that was coming!). Since this is functionality I use a lot, let's design it for reuse, refactoring out reusable code into one class, and code tailored to this project into another. ImageCellEditor encapsulates the general process of setting up an image with a link, and includes a method that will let subclasses override the default processing of the link: public class ImageCellEditor extends AbstractContextSupport implements CellEditor { private String image; private String alt; private String link; public ImageCellEditor(String image, String alt, String link) { this.image = image; this.alt = alt; this.link = link; } public Object getValue(Object item, String property, int rowcount) { CoreContext context = getCoreContext(); String imagePath = context.getPreference("html.imagesPath"); StringBuilder img = new StringBuilder(); if (link != null && link.trim().length() != 0) { img.append(""); } img.append(""); if (link != null && link.trim().length() != 0) { img.append(""); } return img.toString(); } /** * This method can be overridden by subclasses to handle specific * HTML link needs. */ public String processLink(Object item, String property, int rowcount, String link) { return link; } } This is our opportunity to introduce CoreContext and WebContext, two important classes that plug our code into the JMesa infrastructure. Extending AbstractContextSupport gets us JavaBean property methods for these objects (just a convenience; I could have implemented the interface ContextSupport, but then I would have had to write the property methods myself). The CoreContext has many uses; our immediate purpose for it is to retrieve a value configured in the jmesa.properties file. This was pointed to in web.xml: jmesaPreferencesLocation WEB-INF/jmesa.properties It contains a preference called "html.imagesPath" that replaces the default path from which JMesa retrieves images: html.imagesPath=/images/ This means we won't have to hard-code a part of the image URL. (There are a lot more configurable preferences: for details, see the JMesa web site.) The WebContext provides us with the servlet context path, again letting us avoid hard-coding the image URL: getWebContext().getContextPath() Getting back to the two image columns, we have a requirement to pass the Pk property of the appropriate HelloWorld to the edit or delete pages when the images are clicked. Adding this property to the link is easy, using the MessageFormat class to process the link argument of the application-specific subclass: public class HelloWorldImageCellEditor extends ImageCellEditor { public String processLink(Object item, String property, int rowcount, String link) { return MessageFormat.format(link, ((HelloWorld) item).getPk()); } } After creating the editor, we can retrieve the context objects for it from the TableFacade: ImageCellEditor editor = new HelloWorldImageCellEditor("edit.gif", messageSource.getMessage("image.edit.alt", null, locale), "edit.html?pk={0,number,integer}"); editor.setWebContext(tableFacade.getWebContext()); editor.setCoreContext(tableFacade.getCoreContext()); Now we have the images and the links. But it would be awfully nice if the images could be centered within the column, something notoriously difficult to achieve with CSS style sheets. What would work would be to use the align and valign attributes of the cell. How can we do that? The cell itself, as opposed to its contents, is rendered by the interface CellRenderer. Unfortunately, the HtmlCellRenderer sub-interface that comes with JMesa has no method for adding attributes. The Decorator and Template patterns, however, come to the rescue. Again, we implement the functionality for reuse as two classes, the first a generic decorator with an additional template method: public abstract class AttributedHtmlCellRendererDecorator implements HtmlCellRenderer { // all other methods will be delegated to this renderer protected HtmlCellRenderer renderer; public AttributedHtmlCellRendererDecorator(HtmlCellRenderer renderer) { this.renderer = renderer; } public Object render(Object item, int rowcount) { HtmlBuilder html = new HtmlBuilder(); html.td(2); html.width(getColumn().getWidth()); addAttributes(html); html.style(getStyle()); html.styleClass(getStyleClass()); html.close(); String property = getColumn().getProperty(); Object value = getCellEditor().getValue(item, property, rowcount); if (value != null) { html.append(value.toString()); } html.tdEnd(); return html.toString(); } /** * Subclasses will add attributes. */ public abstract void addAttributes(HtmlBuilder html); } The second will be a subclass that adds the specific attributes we need: public class AlignedHtmlCellRendererDecorator extends AttributedHtmlCellRendererDecorator { private String align; private String valign; public AlignedHtmlCellRendererDecorator(HtmlCellRenderer renderer, String align, String valign) { super(renderer); this.align = align; this.valign = valign; } @Override public void addAttributes(HtmlBuilder html) { html.align(align); html.valign(valign); } } Whew, that was a mouthful! However, our images will come out nicely centered in the column, and we've learned a good deal more about how the JMesa API works. There will be edit and delete pages to link to, of course, but these are not of interest here and are completely trivial in the Eclipse project. CSV and Excel Output In JMesa terminology, output other than HTML is called exporting the table. As complex as it might seem, it's actually the easiest part of the process. Again, a single line of code will do all we need: tableFacade.setExportTypes(response, org.jmesa.limit.ExportType.CSV, org.jmesa.limit.ExportType.EXCEL); That's really all there is to it! (OK, you have to include some JAR files in the library, but what did you expect, magic?) Filtering and Highlighting Making a row (we need an HtmlRow) unfilterable and unhighlighted is just as simple as making a column unsortable: htmlRow.setFilterable(false); htmlRow.setHighlighter(false); With this, no filtering row or icons will be generated above the column header and the highlighting feature will be turned off. Toolbar The code to reorganize the toolbar is quite straightforward; while we're at it, we need to include icons for the various output formats: public class ReorderedToolbar extends AbstractToolbar { @Override public String render() { if (ViewUtils.isExportable(getExportTypes())) { addExportToolbarItems(getExportTypes()); addToolbarItem(ToolbarItemType.SEPARATOR); } MaxRowsItem maxRowsItem = (MaxRowsItem) addToolbarItem(ToolbarItemType.MAX_ROWS_ITEM); if (getMaxRowsIncrements() != null) { maxRowsItem.setIncrements(getMaxRowsIncrements()); } addToolbarItem(ToolbarItemType.SEPARATOR); addToolbarItem(ToolbarItemType.FIRST_PAGE_ITEM); addToolbarItem(ToolbarItemType.PREV_PAGE_ITEM); addToolbarItem(ToolbarItemType.NEXT_PAGE_ITEM); addToolbarItem(ToolbarItemType.LAST_PAGE_ITEM); return super.render(); } } I arranged the icons by simply specifying the order in which they are added to the toolbar. They look more natural to me this way; your mileage may vary. Note that we delegate the messy work of actually rendering the toolbar to the JMesa superclass. Putting It All Together We'll refactor out reusable code once more in writing a Factory to encapsulate building our customized table, starting with an abstract class: public abstract class AbstractTableFactory { protected abstract String getTableName(); protected abstract void configureColumns(TableFacade tableFacade, Locale locale); protected abstract void configureUnexportedTable(TableFacade tableFacade, Locale locale); protected abstract ImageCellEditor getEditImageCellEditor(Locale locale); protected abstract ImageCellEditor getDeleteImageCellEditor( Locale locale); public TableFacade createTable(HttpServletRequest request, HttpServletResponse response, Collection items) { TableFacade tableFacade = new TableFacadeImpl(getTableName(), request); tableFacade.setItems(items); tableFacade.setStateAttr("return"); configureTableFacade(response, tableFacade); Locale locale = request.getLocale(); configureColumns(tableFacade, locale); if (! tableFacade.getLimit().isExported()) { configureUnexportedTable(tableFacade, locale); } return tableFacade; } public void configureTableFacade(HttpServletResponse response, TableFacade tableFacade) { tableFacade.setExportTypes(response, getExportTypes()); tableFacade.setToolbar(new ReorderedToolbar()); Row row = tableFacade.getTable().getRow(); if (row instanceof HtmlRow) { HtmlRow htmlRow = (HtmlRow) row; htmlRow.setFilterable(false); htmlRow.setHighlighter(false); } } protected ExportType[] getExportTypes() { return null; } protected void configureColumn(Column column, String title, CellEditor editor) { configureColumn(column, title, editor, false, true); } protected void configureColumn(Column column, String title, CellEditor editor, boolean filterable, boolean sortable) { column.setTitle(title); if (editor != null) { column.getCellRenderer().setCellEditor(editor); } if (column instanceof HtmlColumn) { HtmlColumn htmlColumn = (HtmlColumn) column; htmlColumn.setFilterable(filterable); htmlColumn.setSortable(sortable); } } protected void configureEditAndDelete(Row row, WebContext webContext, CoreContext coreContext, Locale locale) { HtmlComponentFactory factory = new HtmlComponentFactory(webContext, coreContext); HtmlColumn col = factory.createColumn((String) null); col.setFilterable(false); col.setSortable(false); CellRenderer renderer = col.getCellRenderer(); ImageCellEditor editor = getEditImageCellEditor(locale); editor.setWebContext(webContext); editor.setCoreContext(coreContext); renderer.setCellEditor(editor); col.setCellRenderer(new AlignedHtmlCellRendererDecorator((HtmlCellRenderer) renderer, "center", "middle")); row.addColumn(col); col = factory.createColumn((String) null); col.setFilterable(false); col.setSortable(false); renderer = col.getCellRenderer(); editor = getDeleteImageCellEditor(locale); editor.setWebContext(webContext); editor.setCoreContext(coreContext); renderer.setCellEditor(editor); col.setCellRenderer(new AlignedHtmlCellRendererDecorator((HtmlCellRenderer) renderer, "center", "middle")); row.addColumn(col); } } This has a lot of code (note the abstract methods ), in part because I know I usually want edit and delete columns. One line that might pass by unnoticed in all this, however, is really quite something: tableFacade.setStateAttr("return"); When this attribute is set, JMesa uses the Memento design pattern to save the state of its tables. When you return to a table page and include the attribute you specify here in the URL, you return to the exact place you left: the page number to which you had moved before leaving the table, the number of values displayed per page, and so forth. The application-specific concrete class, after all this, can be pretty simple: public class HelloWorldTableFactory extends AbstractTableFactory { protected MessageSource messageSource; public void setMessageSource(MessageSource messageSource) { this.messageSource = messageSource; } @Override protected String getTableName() { return "results"; } @Override protected ExportType[] getExportTypes() { return new ExportType[] { CSV, EXCEL }; } @Override protected void configureColumns(TableFacade tableFacade, Locale locale) { tableFacade.setColumnProperties("firstName", "lastName", "format", "toString"); Row row = tableFacade.getTable().getRow(); configureColumn(row.getColumn("firstName"), messageSource.getMessage("column.firstName", null, locale), null); configureColumn(row.getColumn("lastName"), messageSource.getMessage("column.lastName", null, locale), null); configureColumn(row.getColumn("format"), messageSource.getMessage("column.format", null, locale), new SpringMessageCellEditor(messageSource, "format", locale), false, false); configureColumn(row.getColumn("toString"), messageSource.getMessage("column.toString", null, locale), new ToStringCellEditor(), false, false); } @Override protected void configureUnexportedTable(TableFacade tableFacade, Locale locale) { HtmlTable table = (HtmlTable) tableFacade.getTable(); table.setCaption(messageSource.getMessage("table.caption", null, locale)); configureEditAndDelete(table.getRow(), tableFacade.getWebContext(), tableFacade.getCoreContext(), locale); } @Override protected ImageCellEditor getEditImageCellEditor(Locale locale) { return new HelloWorldImageCellEditor("edit.gif", messageSource.getMessage("image.edit.alt", null, locale), "edit.html?pk={0,number,integer}"); } @Override protected ImageCellEditor getDeleteImageCellEditor(Locale locale) { return new HelloWorldImageCellEditor("delete.gif", messageSource.getMessage("image.delete.alt", null, locale), "delete.html?pk={0,number,integer}"); } } Controller We end as we began, with a Spring MVC Controller to launch all this infrastructure. Since the details of table creation are encapulated in a factory, this is uncluttered: the only decision to be made is whether or not the table is to be exported. If it is exported, the results will be written directly to the output stream of the response; if not, they'll be rendered as a string containing our HTML table: public class CustomJMesaSearchController extends AbstractController { private HelloWorldService helloWorldService; private HelloWorldTableFactory tableFactory; public void setHelloWorldService(HelloWorldService helloWorldService) { this.helloWorldService = helloWorldService; } public void setTableFactory(HelloWorldTableFactory tableFactory) { this.tableFactory = tableFactory; } @Override protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception { Set results = helloWorldService.findAll(); TableFacade tableFacade = tableFactory.createTable(request, response, results); if (tableFacade.getLimit().isExported()) { tableFacade.render(); return null; } return new ModelAndView("results", "results", tableFacade.render()); } } We are actually reusing the same JSP page as in the basic JMesa setup: the only difference is in the Java code that generates the table. One more change in jmesa-servlet.xml to create everything and tie it all together: ... customSearchController And how different the display looks!: [img_assist|nid=3682|title=Figure 5.|desc=A customized search result|link=none|align=left|width=757|height=465] Ajax Finally, the table looks like we want it to, but it's irritating having to resubmit the form each time we want to make a change. Isn't that the sort of thing Ajax is supposed to help us avoid? The answer is, of course, yes! So how do we leverage Ajax to help us? Fortunately, the JMesa folks have already worked that out. There are two parts to the solution: changes to the controller and changes to the JSP page. ${results} In our previous solution, the onInvokeAction Javascript method called createHiddenInputFieldsForLimitAndSubmit, which submitted the form. In the Ajax solution, it assembles parameters for the TableFacade class and sends a request for the HTML for table display, adding a parameter to indicate that it's an Ajax request. Then a callback Javascript function substitutes the returned HTML for the contents of the that now holds the table. The simplicity and unusual syntax of the latter code come courtesy of the jQuery Ajax library, which is thoughtfully used by JMesa: The controller, of course, needs to interpret this new request correctly. This is just one more branch on the decision tree we saw in the previous controller: public class AjaxJMesaSearchController extends AbstractController { @Override protected ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) throws Exception { Set results = helloWorldService.findAll(); TableFacade tableFacade = tableFactory.createTable(request, response, results); if (tableFacade.getLimit().isExported()) { tableFacade.render(); return null; } else if ("true".equals(request.getParameter("ajax"))) { String encoding = response.getCharacterEncoding(); byte[] contents = tableFacade.render() .getBytes(encoding); response.getOutputStream().write(contents); return null; } return new ModelAndView("ajax-results", "results", tableFacade.render()); } } Of course, we have to make Spring aware of the controller change in jmesa-servlet.xml: ... ajaxSearchController That's all there is to it! The table looks and acts just as it did, except now it refreshes without resubmitting the form each time. Conclusion Now I don't have to like tables: I can program them in Java and not worry about them on a display JSP. This makes the page cleaner, gives me more functionality out-of-box, and enables me to nix at least some of the languages I'd otherwise have to fuss with. What's not to like? I hope you'll take a good look at JMesa and see if it can make your life easier, and that this article helps you decide. Good luck! Installation of the Eclipse Project Installing the Eclipse project is not difficult; the included Ant build file and these instructions assume Tomcat as the deployment target (I'm using version 6.0.14 with JDK 6.0_03). If you want to use another servlet container, though, feel free to modify the instructions and the Ant file as needed: download the ZIP archive unzip the archive to any directory; it will create its own top-level subdirectory open the project as a Java project in Eclipse the project must use the Java 6 compiler (available from "http://java.sun.com/javase/6/") the Tomcat installation must be version 6 (available from "http://tomcat.apache.org/download-60.cgi") open the build file and modify the path to the Tomcat root add an external JAR file to the Eclipse project build path from the Tomcat installation: lib/servlet-api.jar run the Ant "deploy" target, which will build automatically open a browser and point it to "http://localhost:8080/running-jmesa-examples/" or to an equivalent URL for your setup (N.B. Some code in the project has been refactored from the way it appears in the article.)
June 18, 2008
by David Sills
· 56,523 Views
article thumbnail
ASP.NET - Preventing SQL Injection Attacks
Consider a simple web application that requires user input in some fields, lets say some search box. Suppose a user types the following string in that textbox: '; DROP DATABASE pubs -- On submit our application executes the following dynamic SQL statement SqlDataAdapter myCommand = new SqlDataAdapter("SELECT OrderId, OrderNumber FROM Orders WHERE OrderNumber = '" + OrderNumberTextBox.Text + "'", myConnection); Or stored procedure: SqlDataAdapter myCommand = new SqlDataAdapter("uspGetOrderList '" + OrderNumberTextBox.Text + "'", myConnection); The intention being that the user input would be run as: SELECT OrderId, OrderNumber FROM Orders WHERE OrderNumber = 'PO123' However, the code inserts the user's malicious input and generates the following query: SELECT OrderId, OrderNumber FROM Orders WHERE OrderNumber = ''; DROP DATABASE pubs --' In this case, the ' (single quotation mark) character that starts the rogue input terminates the current string literal in the SQL statement. As a result, the opening single quotation mark character of the rogue input results in the following statement. SELECT OrderId, OrderNumber FROM Orders WHERE OrderNumber = '' The; (semicolon) character tells SQL that this is the end of the current statement, which is then followed by the following malicious SQL code. ; DROP DATABASE pubs Finally, the -- (double dash) sequence of characters is a SQL comment that tells SQL to ignore the rest of the text. In this case, SQL ignores the closing ' (single quotation mark) character, which would otherwise cause a SQL parser error. --' Using stored procedures doesn’t solve the problem either because the generated query would be: uspGetOrderList ''; DROP DATABASE pubs--' Or perhaps this was your login page and your query being: SELECT UserId FROM Users WHERE LoginId = AND Password = AND IsActive = 1 Someone could easily login by typing in the following in your login textbox: ' OR 1 = 1; -- Which makes our query: SELECT UserId FROM Users WHERE LoginId = '' OR 1 = 1; --' AND Password = '' AND IsActive = 1 Viola, the attacker has now successfully logged in to your site using SQL injection attack. SQL injection can occur, as demonstrated above, when an application uses input to construct dynamic SQL statements or when it uses stored procedures to connect to the database. Conventional security measures, such as the use of SSL and IPSec, do not protect your application from SQL injection attacks. Successful SQL injection attacks enable malicious users to execute commands in an application's database. Common vulnerabilities that make your data access code susceptible to SQL injection attacks include: Weak input validation. Dynamic construction of SQL statements without the use of type-safe parameters. Use of over-privileged database logins. So what can we do to help protect our application from such attacks? To counter SQL injection attacks, we need to: Constrain and sanitize input data Check for known good data by validating for type, length, format, and range and using a list of acceptable characters to constrain input. Create a list of acceptable characters and use regular expressions to reject any characters that are not on the list. Using the list of unacceptable characters is impractical because it is very difficult to anticipate all possible variations of bad input. Start by constraining input in the server-side code for your ASP.NET Web pages. Do not rely on client-side validation because it can be easily bypassed. Use client-side validation only to reduce round trips and to improve the user experience. Check my other blog on Validation Application Block for server-side validation. If in the previous code example, the Order Number value is captured by an ASP.NET TextBox control, you can constrain its input by using a RegularExpressionValidator control as shown in the following. If the Order Number input is from another source, such as an HTML control, a query string parameter, or a cookie, you can constrain it by using the Regex class from the System.Text.RegularExpressions namespace. The following example assumes that the input is obtained from a cookie. using System.Text.RegularExpressions; if (Regex.IsMatch(Request.Cookies["OrderNumber"], "^PO\d{3}-\d{2}$")) { // access the database } else { // handle the bad input } Performing input validation is essential because almost all application-level attacks contain malicious input. You should validate all input, including form fields, query string parameters, and cookies to protect your application against malicious command injection. Assume all input to your Web application is malicious, and make sure that you use server validation for all sources of input. Use client-side validation to reduce round trips to the server and to improve the user experience, but do not rely on it because it is easily bypassed. Apply ASP.NET request validation during development to identify injection attacks ASP.NET request validation detects any HTML elements and reserved characters in data posted to the server. This helps prevent users from inserting script into your application. Request validation checks all input data against a hard-coded list of potentially dangerous values. If a match occurs, it throws an exception of type HttpRequestValidationException. Request validation is enabled by ASP.NET by default. You can see the following default setting in the Machine.config.comments file. Confirm that you have not disabled request validation by overriding the default settings in your server's Machine.config file or your application's Web.config file. You can disable request validation in your Web.config application configuration file by adding a element with validateRequest="false" or on an individual page by setting ValidateRequest="false" on the @ Pages element. NOTE: You should disable Request Validation only on the page with a free-format text field that accepts HTML-formatted input. You can test the effects of request validation. To do this, create an ASP.NET page that disables request validation by setting ValidateRequest="false", as follows: When you run the page, "Hello" is displayed in a message box because the script in txtString is passed through and rendered as client-side script in your browser. If you set ValidateRequest="true" or remove the ValidateRequest page attribute, ASP.NET request validation rejects the script input and produces an error similar to the following. A potentially dangerous Request. Form value was detected from the client (txtString=" Use type-safe SQL parameters for data access Parameter collections such as SqlParameterCollection provide type checking and length validation. If you use a parameters collection, input is treated as a literal value, and SQL Server does not treat it as executable code. An additional benefit of using a parameters collection is that you can enforce type and length checks. Values outside of the range trigger an exception. You can use these parameters with stored procedures or dynamically constructed SQL command strings. Using stored procedures does not necessarily prevent SQL injection. The important thing to do is use parameters with stored procedures. If you do not use parameters, your stored procedures can be susceptible to SQL injection if they use unfiltered input. The following code shows how to use SqlParameterCollection when calling a stored procedure: using System.Data; using System.Data.SqlClient; using (SqlConnection connection = new SqlConnection(connectionString)) { DataSet userDataset = new DataSet(); SqlDataAdapter myCommand = new SqlDataAdapter("uspGetOrderList", connection); myCommand.SelectCommand.CommandType = CommandType.StoredProcedure; myCommand.SelectCommand.Parameters.Add("@OrderNumber", SqlDbType.VarChar, 11); myCommand.SelectCommand.Parameters["@OrderNumber"].Value = OrderNumberTextBox.Text; myCommand.Fill(userDataset); } The @OrderNumber parameter is treated as a literal value and not as executable code. Also, the parameter is checked for type and length. In the preceding code example, the input value cannot be longer than 11 characters. If the data does not conform to the type or length defined by the parameter, the SqlParameter class throws an exception. You should review your application's use of stored procedures because simply using stored procedures with parameters does not necessarily prevent SQL injection. For example, the following parameterized stored procedure has several security vulnerabilities. CREATE PROCEDURE dbo.uspRunQuery @var ntext AS exec sp_executesql @var GO The stored procedure executes whatever statement is passed to it. Consider the @var variable being set to: DROP TABLE ORDERS; If you cannot use stored procedures, you should still use parameters when constructing dynamic SQL statements. The following code shows how to use SqlParametersCollection with dynamic SQL. using System.Data; using System.Data.SqlClient; using (SqlConnection connection = new SqlConnection(connectionString)) { DataSet userDataset = new DataSet(); SqlDataAdapter myDataAdapter = new SqlDataAdapter("SELECT OrderId, OrderNumber FROM Orders WHERE OrderNumber = @OrderNumber", connection); myCommand.SelectCommand.Parameters.Add("@OrderNumber", SqlDbType.VarChar, 11); myCommand.SelectCommand.Parameters["@OrderNumber"].Value = OrderNumberTextBox.Text; myDataAdapter.Fill(userDataset); } If you concatenate several SQL statements to send a batch of statements to the server in a single round trip, you can still use parameters if you make sure that parameter names are not repeated i.e. use unique parameter names during SQL text concatenation. SELECT OrderId, OrderNumber FROM Orders WHERE OrderNumber = 'PO123' using System.Data; using System.Data.SqlClient; using (SqlConnection oConn = new SqlConnection(connectionString)) { SqlDataAdapter oAdapter = new SqlDataAdapter( "SELECT CustomerID INTO #Temp1 FROM Customers " + "WHERE CustomerID > @custIDParm; " + "SELECT CompanyName FROM Customers " + "WHERE Country = @countryParm and CustomerID IN " + "(SELECT CustomerID FROM #Temp1);", oConn); SqlParameter custIDParm = oAdapter.SelectCommand.Parameters.Add("@custIDParm", SqlDbType.NChar, 5); custIDParm.Value = customerID.Text; SqlParameter countryParm = oAdapter.SelectCommand.Parameters.Add("@countryParm", SqlDbType.NVarChar, 15); countryParm.Value = country.Text; oConn.Open(); DataSet dataSet = new DataSet(); oAdapter.Fill(dataSet); } Use a least privileged account that has restricted permissions in the database Ideally, you should only grant execute permissions to selected stored procedures in the database and provide no direct table access. The problem is more severe if your application uses an over-privileged account to connect to the database. For example, if your application's login has privileges to eliminate a database, then without adequate safeguards, an attacker might be able to perform this operation. If you use Windows authentication to connect, the Windows account should be least-privileged from an operating system perspective and should have limited privileges and limited ability to access Windows resources. Additionally, whether or not you use Windows authentication or SQL authentication, the corresponding SQL Server login should be restricted by permissions in the database. Consider the example of an ASP.NET application running on Microsoft Windows Server 2003 that accesses a database on a different server in the same domain. By default, the ASP.NET application runs in an application pool that runs under the Network Service account. This account is a least privileged account. Create a SQL Server login for the Web server's Network Service account. The Network Service account has network credentials that are presented at the database server as the identity DOMAIN\WEBSERVERNAME$. For example, if your domain is called XYZ and the Web server is called 123, you create a database login for XYZ\123$. Grant the new login access to the required database by creating a database user and adding the user to a database role. Establish permissions to let this database role call the required stored procedures or access the required tables in the database. Only grant access to stored procedures the application needs to use, and only grant sufficient access to tables based on the application's minimum requirements. If the ASP.NET application only performs database lookups and does not update any data, you only need to grant read access to the tables. This limits the damage that an attacker can cause if the attacker succeeds in a SQL injection attack. Use Character Escaping Techniques In situations where parameterized SQL cannot be used, consider using character escaping techniques. If you are forced to use dynamic SQL and parameterized SQL cannot be used, you need to safeguard against input characters that have special meaning to SQL Server (such as the single quote character). If not handled, special characters such as the single quote character in the input can be utilized to cause SQL injection. Escape routines add an escape character to characters that have special meaning to SQL Server, thereby making them harmless. private static string GetStringForSQL(string inputSQL) { return inputSQL.Replace("'", "''"); } Special input characters pose a threat only with dynamic SQL and not when using parameterized SQL. Your first line of defense should always be to use parameterized SQL. Avoid disclosing database error information In the event of database errors, make sure you do not disclose detailed error messages to the user. Use structured exception handling to catch errors and prevent them from propagating back to the client. Log detailed error information locally, but return limited error details to the client. If errors occur while the user is connecting to the database, be sure that you provide only limited information about the nature of the error to the user. If you disclose information related to data access and database errors, you could provide a malicious user with useful information that he or she can use to compromise your database security. Attackers use the information in detailed error messages to help deconstruct a SQL query that they are trying to inject with malicious code. A detailed error message may reveal valuable information such as the connection string, SQL server name, or table and database naming conventions. See my other post on Exception Handling - Do's and Dont's. You can use the element to configure custom, generic error messages that should be returned to the client in the event of an application exception condition. Make sure that the mode attribute is set to "remoteOnly" in the web.config file as shown in the following example. After installing an ASP.NET application, you can configure the setting to point to your custom error page as shown in the following example. Conclusion The above list is just some points found on MSDN on how you can make your site more secure by effectively preventing SQL injection attacks. You should always be reviewing your code to find these or other security vulnerabilities; remember all type of attacks start with some input, and your first line of defense should be input validation using both client-side and server-side validation. Original Author Original article written by Misbah Arefin
June 18, 2008
by Schalk Neethling
· 90,815 Views
article thumbnail
Understanding the GWT Compiler
[img_assist|nid=3421|title=|desc=|link=url|url=http://www.manning.com/affiliate/idevaffiliate.php?id|align=left|width=208|height=388]The GWT compiler is the fulcrum of GWT. The entire approach GWT takes, encapsulating browser differences and compiling JavaScript from Java, is made possible by the design and architecture of the compiler. The GWT compiler compiles Java into JavaScript, but it’s important to understand that the compiler doesn’t compile Java the same way javac does. The GWT compiler is really a Java source to JavaScript source translator. The GWT compiler needs hints about the work that it must perform partly because it operates from source. These hints come in the form of the module descriptor, the marker interfaces that denote serializable types, the JavaDoc style annotations used in serializable types for collections, and more. Although these hints may sometimes seem like overkill, they’re needed because the GWT compiler will optimize your application at compile time. This doesn’t just mean compressing the JavaScript naming to the shortest possible form; it also includes pruning unused classes, and even methods and attributes, from your code. The core engineering goal of the GWT compiler is summarized succinctly: you pay for what you use. This optimization offers big advantages over other Ajax/JavaScript libraries, where a large initial download of a library may be needed even if just a few elements are used. In Java, serialization marked by the java.io.Serializable interface is handled at the bytecode level. GWT examines your code and only provides serialization for the classes where you explicitly need it. Like GWTShell, GWTCompiler supports a set of useful command-line options. They’re described in table 1: GWTCompiler [-logLevel level] [-gen dir] [-out dir] [-treeLogger] [-style style] module Table 1 GWTCompiler parameters Option Description -logLevel The logging level: ERROR, WARN, INFO, TRACE, DEBUG, SPAM, or ALL -gen The directory into which generated files will be written for review -out The directory to which output files will be written (defaults to the current directory) -treeLogger Logs output in a graphical tree view -style The script output style: OBF[uscated], PRETTY, or DETAILED(defaults to OBF) module The name of the module to compile The -gen and -out command-line options specify where generated files and the final output directory are to be, respectively. And -logLevel, as in the case of GWTShell, is used to indicate the level of logging performed during the compilation. You can even use the -treeLogger option to bring up a window to view the hierarchical logging information you would see in the shell’s console display. The GWT compiler supports several styles of output, each of use in looking at how your code is executing in the browser. JavaScript output style When working with the GWT compiler, you can use several values with the -style command-line option to control what the generated JavaScript looks like. These options are as follows: OBF - Obfuscated mode. This is a non-human-readable, compressed version suitable for production use. PRETTY - Pretty-printed JavaScript with meaningful names. DETAILED - Pretty-printed JavaScript with fully qualified names. To give you an idea of what these options mean, let’s look at examples of java.lang.StringBuffer compiled in the three different modes. First, in listing 1, is the obfuscated mode. Listing 1 StringBuffer in obfuscated compilation function A0(){this.B0();return this.js[0];}function C0(){if(this.js.length > 1){this.js = [this.js.join('')];this.length = this.js[0].length;}function D0(E0){this.js = [E0];this.length = E0.length;}function Ez(F0,a1){return F0.yx(yZ(a1));}function yB(b1){c1(b1);return b1;}function c1(d1){d1.e1('');}function zB(){}_ = zB.prototype = new f();_.yx = w0;_.vB = A0;_.B0 = C0;_.e1 = D0;_.i = 'java.lang.StringBuffer';_.j = 75;function f1(){f1 = a;g1 = new iX();h1 = new iX();return window;} Obfuscated mode is just that. This is intended to be the final compiled version of your application, which has names compressed and whitespace cleaned. Next is the pretty mode, shown in listing 2. Listing 2 StringBuffer in pretty compilation function _append2(_toAppend){ var _last = this.js.length - 1; var _lastLength = this.js[_last].length; if (this.length > _lastLength * _lastLength) { this.js[_last] = this.js[_last] + _toAppend; } else { this.js.push(_toAppend); } this.length += _toAppend.length; return this;}function _toString0(){ this._normalize(); return this.js[0];}// Some stuff omitted.function _$StringBuffer(_this$static){ _$assign(_this$static); return _this$static;}function _$assign(_this$static){ _this$static._assign0('');}function _StringBuffer(){}_ = _StringBuffer.prototype = new _Object();_._append = _append2;_._toString = _toString0;_._normalize = _normalize0;_._assign0 = _assign;_._typeName = 'java.lang.StringBuffer'; _._typeId = 75; append() becomes _append2() to avoid collision> toString() becomes _toString0()> _typeName holds name of original Java class> Pretty mode is useful for debugging as method names are somewhat preserved. However, collisions are resolved with suffixes, as the _toString0() method name shows . Last, we have the detailed mode, as displayed in listing 3. Listing 3 StringBuffer in detailed compilation function java_lang_StringBuffer_append__Ljava_lang _String_2(toAppend){ var last = this.js.length - 1; var lastLength = this.js[last].length; if (this.length > lastLength * lastLength) { this.js[last] = this.js[last] + toAppend; } else { this.js.push(toAppend); } this.length += toAppend.length; return this;}function java_lang_StringBuffer_toString__(){ this.normalize__(); return this.js[0];}function java_lang_StringBuffer_normalize__(){ if (this.js.length > 1) { this.js = [this.js.join('')]; this.length = this.js[0].length; }// . . . some stuff omittedfunction java_lang_StringBuffer(){}_ = java_lang_StringBuffer.prototype = new java_lang_Object();_.append__Ljava_lang_String_2 = java_lang_StringBuffer_append__Ljava_lang_String_2;_.toString__ = java_lang_StringBuffer_toString__;_.normalize__ = java_lang_StringBuffer_normalize__;_.assign__Ljava_lang_String_2 = java_lang_StringBuffer_assign__Ljava_lang_String_2;_.java_lang_Object_typeName = 'java.lang.StringBuffer';_.java_lang_Object_typeId = 75; Detailed mode preserves the full class name, as well as the method name #2. For overloaded methods, the signature of the method is encoded into the name, as in the case of the append() method #1. There are some important concepts to grasp about this compilation structure, especially given the way GWT interacts with native JavaScript, through the JavaScript Native Interface (JSNI), which will be discussed in the section on the compiler lifecycle. The names of your classes and methods in their JavaScript form aren’t guaranteed, even for different compilations of the same application. Use of the special syntax provided with JSNI will let you invoke known JavaScript objects from your Java code and invoke your compiled Java classes from within JavaScript; but you can’t freely invoke your JavaScript when using obfuscated style, predictably. This imposes certain limitations on your development: If you intend to expose your JavaScript API for external use, you need to create the references for calls into GWT code using JSNI registrations. You can’t rely on JavaScript naming in an object hash to give you java.lang.reflect.* type functionality, since the naming of methods isn’t reliable. Although they’re rare, you should consider potential conflicts with other JavaScript libraries you’re including in your page, especially if you’re publishing using the PRETTY setting. In addition to being aware of the available compiler output options and how they affect your application, you should also be familiar with a few other compiler nuances. Additional compiler nuances Currently, the compiler is limited to J2SE 1.4 syntactical structures. This means that exposing generics or annotations in your GWT projects can cause problems. Other options are available for many of the purposes for which you might wish to use annotations. For example, you can often use JavaDoc-style annotations, to which GWT provides its own extensions. Along with the J2SE 1.4 limitations, you also need to keep in mind the limited subset of Java classes that are supported in the GWT Java Runtime Environment (JRE) emulation library. This library is growing, and there are third-party extensions, but you need to be aware of the constructs you can use in client-side GWT code—the complete JRE you’re accustomed to isn’t available. Of course, one of the great advantages of GWT’s approach to compiling JavaScript from plain Java is that you get to leverage your existing toolbox while building your Ajax application. When you execute the hosted mode browser, you’re running regularly compiled Java classes. This, again, means you can use all the standard Java tooling—static analysis tools, debuggers, IDEs, and the like. These tools are useful for writing any code, but they become even more important in the GWT world because cleaning up your code means less transfer time to high-latency clients. Also, because JavaScript is a fairly slow execution environment, such cleanup can have a large impact on ultimate performance. The GWT compiler helps by optimizing the JavaScript it emits to include only classes and methods that are on the execution stack of your module and by using native browser functions where possible, but you should always keep the nature of JavaScript in mind. To that end, we’ll now take a closer look at the lifecycle of a GWT compilation and at how this JavaScript is generated. The compiler lifecycle When the GWT compiler runs, it goes through several stages for building the final compiled project. In these stages, the need for the GWT module definition file becomes clear. First, the compiler identifies which combinations of files need to be built. Then, it generates any client-side code using the generator metaprogramming model. Last, it produces the final output. We’ll look at each of these steps in the compiler lifecycle in more detail. dentifying build combinations One of the great strengths of GWT is that it builds specific versions of the application, each exactly targeted to what the client needs (user agent, locale, so on). This keeps the final download, and the operational overhead at the client level, very lean. A particular build combination is defined in the GWT module definition using a tag. This establishes a base set of values that are used for the build. The first and very obvious property is the user agent that the JavaScript will be built for. Listing 4 shows the core GWT UserAgent module definition. Listing 4 The GWT UserAgent definition = 6000) { return "ie6"; } } } else if (ua.indexOf("gecko") != -1) { var result = /rv:([0-9]+)\.([0-9]+)/.exec(ua); if (result && result.length == 3) { if (makeVersion(result) >= 1008) return "gecko1_8"; } return "gecko"; } return "unknown";]]> Here the tag establishes a number of different builds that the final compiler will output #1: in this case, ie6, gecko, gecko1_8, safari, and opera. This means each of these will be processed as a build of the final JavaScript that GWT emits. GWT can then switch implementations of classes based on properties using the tag in the module definition. As the GWT application starts up, the JavaScript snippet contained within the tag determines which of these implementations is used #2. This snippet is built into the startup script that determines which compiled artifact is loaded by the client. At the core of the GWT UI classes is the DOM class. This gets replaced based on the user.agent property. Listing 5 shows this definition. Listing 5 Changing the DOM implementation by UserAgent Now you can see the usefulness of this system. The basic DOM class is implemented with the same interface for each of the browsers, providing a core set of operations on which cross-platform code can easily be written. Classes replaced in this method can’t be instantiated with simple constructors but must be created using the GWT.create() method. In practice, the DOM object is a singleton exposing static methods that are called by applications, so this GWT.create() invocation is still invisible. This is an important point to remember if you want to provide alternative implementations based on compile-time settings in your application. You can also define your own properties and property providers for switching implementations. We have found that doing this for different runtime settings can be useful. For example, we have defined debug, test, and production settings, and replacing some functionality in the application based on this property can help smooth development in certain cases. This technique of identifying build combinations and then spinning off into specific implementations during the compile process is known in GWT terms as deferred binding. The GWT documentation sums this approach up as “the Google Web Toolkit answer to Java reflection.” Dynamic loading of classes (dynamic binding) isn’t truly available in a JavaScript environment, so GWT provides another way. For example, obj.getClass().getName() isn’t available, but GWT.getTypeName(obj) is. The same is true for Class.forName("MyClass"), which has GWT.create(MyClass) as a counterpart. By using deferred binding, the GWT compiler can figure out every possible variation, or axis, for every type and feature needed at compile time. Then, at runtime, the correct permutation for the context in use can be downloaded and run. Remember, though, that each axis you add becomes a combinatory compile. If you use 4 languages and 4 browser versions, you must compile 16 final versions of the application; and if you use several runtime settings, you end up with many more combinations in the mix. This concept is depicted in figure 1. Figure 1 Multiple versions of a GWT application are created by the compiler for each axis or variant application property, such as user agent and locale. Compiling a new monolithic version of your application for each axis doesn’t affect your end users negatively. Rather, this technique allows each user to download only the exact application version they need, without taking any unused portion along for the ride. This is beneficial for users, but it slows compile time considerably, and it can be a painful point for developers. Reducing the compile variants to speed up compile time Even though GWT compile time can be long, keep in mind that the end result for users is well optimized. Also, the GWT module system allows you to tweak the compile time variants for the situation. During day-to-day development, you may want to use the tag in your module definition to confine the compile to a single language, or single browser version, to speed up the compile step. Another important use for module properties is in code generation, which is the next step of the compilation process. Generating code GWT’s compiler includes a code generation or metaprogramming facility that allows you to generate code based on module properties at compile time. Perhaps the best example is the internationalization support. The i18n module defines several no-method interfaces that you extend to define Constants, Messages (which include in-text replacement), or Dictionary classes (which extract values from the HTML host page). The implementations of each of these classes are built at compile time using the code generation facility, producing a lean, custom version of your application in each language. The i18n module does this through the tag, which lets you add additional iterative values to a property in a module. Listing 6 demonstrates the use of this concept to add French and Italian support to a GWT application. Listing 6 Defining French and Italian using extend-property When an application inherits the i18n module, the GWT compiler searches for interfaces that extend one of the i18n classes and generates an implementation for the class based on a resource bundle matching the language code. This is accomplished via the tag in the i18n module definition. Listing 7 shows this along with the tag, which is used for establishing which language will be needed at runtime. Listing 7 The i18n module’s locale property declarations = 0) { var language = args.substring(startLang); var begin = language.indexOf("=") + 1; var end = language.indexOf("&"); if (end == -1) { end = language.length; } locale = language.substring(begin, end); } } if (locale == null) { // Look for the locale on the web page locale = __gwt_getMetaProperty("locale") } if (locale == null) { return "default"; } while (!__gwt_isKnownPropertyValue("locale", locale)) { var lastIndex = locale.lastIndexOf("_"); if (lastIndex == -1) { locale = "default"; break; } else { locale = locale.substring(0,lastIndex); } } return locale; } catch(e) { alert("Unexpected exception in locale "+ "detection, using default: " + e); return "default"; } ]]> The module first establishes the locale property. This is the property we extended in listing 6 to include it and fr. Next, the property provider is defined . The value is checked first as a parameter on the request URL in the format locale=xx and then as a tag on the host page in the format ; finally, it defaults to default. The last step is to define a generator class. Here it tells the compiler to generate implementations of all classes that extend or implement Localizable with the LocalizableGenerator . This class writes out Java files that implement the appropriate interfaces for each of the user’s defined Constants, Messages, or Dictionary classes. Notice that, to this point, nowhere have we dealt specifically with JavaScript outside of small snippets in the modules. GWT will produce the JavaScript in the final step. Producing output You can do a great deal from Java with the GWT module system, but you may need to get down to JavaScript-level implementations at some point if you wish to integrate existing JavaScript or extend or add lower-level components. For this, GWT includes JSNI. This is a special syntax that allows you to define method implementations on a Java class using JavaScript. Listing 8 shows a simple JSNI method. Listing 8 A simple JSNI method public class Alert { public Alert() { super(); } public native void alert(String message) /*-{ alert(message); }-*/; } In this listing, we first define the alert() method using the native keyword. Much like the Java Native Interface (JNI) in regular Java, this indicates that we are going to use a native implementation of this method. Unlike JNI, the implementation lives right in the Java file, surrounded by a special /*- -*/ comment syntax. To reference Java code from JavaScript implementations, syntax similar to JNI is provided. Figure 2 shows the structure of calls back into Java from JavaScript. Figure 2 The structure of JSNI call syntax GWT reuses the JNI typing system to reference Java types. The final compiled output will have synthetic names for methods and classes, so the use of this syntax is important to ensure that GWT knows how to direct a call from JavaScript. In the final step of the compilation process, GWT takes all the Java files, whether provided or generated, and the JSNI method implementations, and examines the call tree, pruning unused methods and attributes. Then it transforms all of this into a number of unique JavaScript files, targeted very specifically at each needed axis. This minimizes both code download time and execution time in the client. Although this complex compilation process can be time consuming for the developer, it ensures that the end user’s experience is the best it can possibly be for the application that is being built. This article is excerpted from Chapter 1 of GWT in Practice, by Robert Cooper and Charlie Collins, and published in May 2008 by Manning Publications.
June 17, 2008
by Schalk Neethling
· 126,294 Views
article thumbnail
Protect Your Java Code From Reverse Engineering
If you are developing a Java application, it is important to understand that the Java class files can be easily reverse-engineered using Java decompilers. In this article, let us explore how a Java class file is reverse-engineered and how to protect your source code from this. Java source code is compiled to a class file that contains byte code. The Java Virtual Machine needs only the class file for execution. The problem is that the class file can easily be decompiled into the original source code using Java decompiler tools. The best solution to prevent reverse-engineering is to obfuscate the class file so that is will be very hard to reverse-engineer. According to the dictionary Obfuscate means “to make obscure or unclear”. That is exactly what lot of Java obfuscator tools do as explained below. Decompile Java class file. Before understanding how to obfuscate the java code, let us first try to understand how someone can reverse engineer your java application. Following 3 steps explains how a class file is reverse engineered to the original java source code. 1. Create HelloWorld.java as shown below. public class HelloWorld { public static void main (String args[]) { String userMessage = “Hello World!”; int userCount = 100; userCount = userCount + 1; System.out.println(userMessage); System.out.println(userCount); } } 2. Compile HelloWorld.java program and execute it to make sure it works properly. $ javac HelloWorld.java $ java HelloWorld Hello World! 101 Java class file contains only byte code. If you try to view a class file, it will be non-readable as shown below. $ vi HelloWorld.class Ãþº¾^@^@^@2^@ ^@^G^@^P^H^@^Q ^@^R^@^S ^@^T^@^V^G^@^W^G^@^X^A^@^F^A^@^C()V^A^@^DCode^A^@^OLineNumberTable ^A^@^Dmain^A^@^V([Ljava/lang/String;)V^A^@ SourceFile^A^@^OHelloWorld.java^L^@^H^@ ^A^@^LHello World!^G^@^Y^L^@^Z^@^[^G^@^\^L^@^]^@^^^L^@^]^@^_^A^@ HelloWorld^A^@^Pjava/lang/Object^A^@^Pjava/lang/System^A^@^Cout^A^@^ULjava/io/PrintStream;^A ^@^Sjava/io/PrintStream^A^@^Gprintln^A^@^U(Ljava/lang/String;)V^A^@^D(I)V^@!^@^F^@^G^@^@^@^@^@^B^@^A^@^H^@ ^@^A^@ 3. Decompile HelloWorld.class file and view the original source. For this demonstration let us use Jad decompiler which is free for non-commercial use. Download the appropriate jad for your platform. Use jad to reverse-engineer the HelloWorld.class file to get the original source as shown below. $ unzip jadls158.zip $ ./jad HelloWorld.class Parsing HelloWorld.class... Generating HelloWorld.jad $ vi HelloWorld.jad Obfuscate your java application Let us review how to obfuscate and protect your source code from reverse engineering using ProGuard a free GPL licensed software. 1. Download and Install ProGuard $ cd /home/jsmith $ unzip proguard4.2.zip 2. Create a proguard config file Create myconfig.pro that contains all the information about your java application. -injar : Specify the location of your jar file. i.e the compiled java application that contains the class files -outjar: This is the jar file proguard will create after obfuscation. This will contain all the mangled, obscure naming convention of the methods and variables in the class file if someone tries to reverse engineer. -printmapping: ProGurad outputs all the mapping information in this file for your reference. -keep: Indicate the class files or the methods that you don’t want ProGuard to obfuscate. For e.g. mypkg.MainAppFrame contains the entry point for the application with the main class, which will not get obfuscated in this example. $ cat myconfig.pro -injars /home/jsmith/myapp.jar -outjars /home/jsmith/myapp-obfuscated.jar This is the obfuscated jar file -libraryjars /usr/java/jdk1.5.0_14/jre/lib/rt.jar -printmapping proguard.map -verbose -keep public class mypkg.MainAppFrame 3. Execute ProGuard. $ cd /home/jsmith/proguard4.2/lib $ java -jar proguard.jar @myconfig.pro This creates the following two files: myapp-obfuscated.jar: Contains the obfuscated class files of your application. You can distribute this without having to worry about someone reverse engineering your application easily. proguard.map: This file contains the mapping information for your reference. 4. Sample proguard.map file This is a sample proguard.map file that indicates the original name of the java source objects (classfile, methods, variable etc.) and the new obfuscated name. myapp.AppToolBar -> myapp.ae: javax.swing.JButton btnNew -> d javax.swing.JButton btnOpen -> e 5. Sample java source code (myapp.AppToolBar) before obfuscation. btnNew = changeButtonLabel(btnNew, language.getText("new")); btnOpen = changeButtonLabel(btnOpen, language.getText("open")); 6. Sample java source code that was decompiled from the class file (myapp.ae) after obfuscation. d = a(d, n.a("new")); e = a(e, n.a("open")); You can see that the line “btnNew = changeButtonLabel(btnNew, language.getText(”new”));” got translated to “d = a(d, n.a(”new”));”, by the ProGuard, which will not make any sense to someone who is using java decompiler tools to reverse engineer the class file. From http://www.thegeekstuff.com
June 17, 2008
by Ramesh Natarajan
· 109,444 Views · 4 Likes
article thumbnail
Exploring Erlang with Map/Reduce
Under the category of "Concurrent Oriented Programming", Erlang has got some good attention recently due to some declared success from Facebook engineers of using Erlang in large scale applications. Tempted to figure out the underlying ingredients of Erlang, I decided to spent some time to learn the language. Multi-threading Problem Multiple threads of execution is a common programming model in modern languages because it enable a more efficient use of computing resources provided by multi-core and multi-machine architecture. One of question to be answered though, is how these parallel threads of execution interact and work co-operative to solve the application problem. There are basically two models for communication between concurrent executions. One is based on a "Shared Memory" model which one thread of execution write the information into a shared place where other threads will read from. Java's thread model is based on such a "shared memory" semantics. The typical problem of this model is that concurrent update requires very sophisticated protection scheme, otherwise uncoordinated access can result in inconsistent data. Unfortunately, this protection scheme is very hard to analyze once there are multiple threads start to interact in combinatorial explosion number of different ways. Hard to debug deadlock problem are frequently pop up. To reduce the complexity, using a coarse grain locking model is usually recommended but this may reduce the concurrency. Erlang has picked the other model based on "message passing". In this model, any information that needs to be shared will be "copied" into a message and send to other executions. In this model, each thread of execution has its state "completely local" (not viewable by other thread of executions). Their local state is updated when they learn what is going on in other threads by receiving their messages. This model mirrors how people in real life interact with each other. Erlang Sequential Processing Coming from an object oriented imperative programming background, there are a couple of things I need to unlearn/learn in Erlang. Erlang is a functional programming language and have no OO concepts. Erlang code is structured as "function" at a basic unit, grouped under a "module". Each "function" takes a number of inputs parameters and produce an output value. Like many functional programming language, Erlang encourage the use of "pure function" which is "side-effect-free" and "deterministic". "Side-effect-free" means there is no state changes within the execution of the function. "Deterministic" means the same output will always be produced from the same input. Erlang has a very different concept in variable assignment in that all variables in Erlang is immutable. In other words, every variable can only be assigned once and from then onwards can never be changed. So I cannot do X = X + 1, and I have to use a new variable and assigned it with the changed value, e.g. Y = X + 1. This "immutability" characteristic simplify debugging a lot because I don't need to worry about how the value of X is changed at different point of execution (it simply won't change). Another uncommon thing about Erlang is that there is no "while loop" construct in the language. To achieve the looping effect, you need to code the function in a recursive way, basically putting a terminal clause to check for the exit condition, as well as carefully structure the logic in a tail recursion fashion. Otherwise, you may run out of memory in case the stack grow too much. Tail recursion function means the function either returns a value (but not an expression) or a recursive function call. Erlang is smart enough to do tail recursion across multiple functions, such as if funcA calls funcB, which calls funcC, which call funcA. Tail recursion is especially important in writing server daemon which typically make a self recursive call after process a request. Erlang Parallel Processing The execution thread in Erlang is called a "Process". Don't be confused with OS-level processes, Erlang process is extremely light-weight, much lighter than Java threads. A process is created by a spawn(Node, Module, Function, Arguments) function call and it terminates when that function is return. Erlang processes communicate with each other by passing messages. Process ids are used by the sender to specify the recipient addresses. The send call happens asynchronously and returns immediately. The receiving process will make a synchronous receive call and specify a number of matching patterns. Arriving messages that match the pattern will be delivered to the receiving process, otherwise it will stay in the queue forever. Therefore, it is good practices to have a match all pattern to clean up garbage message. The receive call also accepts a timeout parameter so that it will return if no matched messages happen within the timeout period. Error handling in Erlang is also quite different from other programming languages. Although Erlang provides a try/catch model, it is not the preferred approach. Instead of catching the error and handle it within the local process, the process should simply die and let another process to take care of what should be done after its crash. Erlang have the concept of having processes "linked" to each other and monitor the life status among themselves. In a default setting, a dying process will propagate an exit signal to all the processes it links to (links are bi-directional). So there is a chaining effect that when one process die, the whole chain of processes will die. However, a process can redefine its behavior after receiving the exit signal. Instead of "dying", a process can choose to handle the error (perhaps by restarting the dead process). Other Erlang Features Pattern matching is a common programming construct in many places of Erlang, namely "Function calls", "Variable assignment", "Case statements" and "Receive messages". It takes some time to get used to this style. After that I feel this construct to be very powerful. Another cool feature that Erlang provides is the code hot swap. By specifying the module name when making the function call, a running Erlang process can execute the latest code without restarting itself. This is a powerful features for code evolution because you don't need to shutdown the VM when deploying new code. Since the function itself can be passed as a message to a remote process, execute code remotely is extremely easy in Erlang. The problem of installation, deployment is pretty much non-existent in Erlang Map/Reduce using Erlang After learning the basic concepts, my next step is to search for a problem and get some hands on with the language. Based on a work-partition, aggregation, parallel processing model, Map/Reduce seems to have the characteristic model that aligns very nicely into Erlang's parallel processing model. So I pick my project to implement a simple Map/Reduce framework in Erlang. Here is the Erlang implementation ... First of all, I need some Helper functions -module(mapreduce). -export([reduce_task/2, map_task/2, test_map_reduce/0]). %%% Execute the function N times, %%% and put the result into a list repeat_exec(N,Func) -> lists:map(Func, lists:seq(0, N-1)). %%% Identify the reducer process by %%% using the hashcode of the key find_reducer(Processes, Key) -> Index = erlang:phash(Key, length(Processes)), lists:nth(Index, Processes). %%% Identify the mapper process by random find_mapper(Processes) -> case random:uniform(length(Processes)) of 0 -> find_mapper(Processes); N -> lists:nth(N, Processes) end. %%% Collect result synchronously from %%% a reducer process collect(Reduce_proc) -> Reduce_proc ! {collect, self()}, receive {result, Result} -> Result end. Main function The MapReduce() function is the entry point of the system. It first starts all the R number of Reducer processes It starts all the M number of Mapper processes, passing them the R reducer processes ids For each line of input data, it randomly pick one of the M mapper processes and send the line to it Wait until the completion has finished Collect result from the R reducer processes Return the collected result The corresponding Erlang code is as follows ... %%% The entry point of the map/reduce framework map_reduce(M, R, Map_func, Reduce_func, Acc0, List) -> %% Start all the reducer processes Reduce_processes = repeat_exec(R, fun(_) -> spawn(mapreduce, reduce_task, [Acc0, Reduce_func]) end), io:format("Reduce processes ~w are started~n", [Reduce_processes]), %% Start all mapper processes Map_processes = repeat_exec(M, fun(_) -> spawn(mapreduce, map_task, [Reduce_processes, Map_func]) end), io:format("Map processes ~w are started~n", [Map_processes]), %% Send the data to the mapper processes Extract_func = fun(N) -> Extracted_line = lists:nth(N+1, List), Map_proc = find_mapper(Map_processes), io:format("Send ~w to map process ~w~n", [Extracted_line, Map_proc]), Map_proc ! {map, Extracted_line} end, repeat_exec(length(List), Extract_func), timer:sleep(2000), %% Collect the result from all reducer processes io:format("Collect all data from reduce processes~n"), All_results = repeat_exec(length(Reduce_processes), fun(N) -> collect(lists:nth(N+1, Reduce_processes)) end), lists:flatten(All_results). Map Process The Map processes, once started, will perform the following ... Receive the input line Execute the User provided Map function to turn into a list of key, value pairs For each key and value, select a reducer process and send the key, value to it The corresponding Erlang code will be as follows .. %%% The mapper process map_task(Reduce_processes, MapFun) -> receive {map, Data} -> IntermediateResults = MapFun(Data), io:format("Map function produce: ~w~n", [IntermediateResults ]), lists:foreach( fun({K, V}) -> Reducer_proc = find_reducer(Reduce_processes, K), Reducer_proc ! {reduce, {K, V} end, IntermediateResults), map_task(Reduce_processes, MapFun) end. Reduce Process On the other hand, the reducer processes will execute as follows ... Receive the key, value from the Mapper process Get the current accumulated value by the key. If no accumulated value is found, use the initial accumulated value Invoke the user provided reduce function to calculate the new accumulated value Store the new accumulated value under the key The corresponding Erlang code will be as follows .. %%% The reducer process reduce_task(Acc0, ReduceFun) -> receive {reduce, {K, V} -> Acc = case get(K) of undefined -> Acc0; Current_acc -> Current_acc end, put(K, ReduceFun(V, Acc)), reduce_task(Acc0, ReduceFun); {collect, PPid} -> PPid ! {result, get()}, reduce_task(Acc0, ReduceFun) end. Word Count Example To test the Map/Reduce framework using a word count example .. %%% Testing of Map reduce using word count test_map_reduce() -> M_func = fun(Line) -> lists:map( fun(Word) -> {Word, 1} end, Line) end, R_func = fun(V1, Acc) -> Acc + V1 end, map_reduce(3, 5, M_func, R_func, 0, [[this, is, a, boy], [this, is, a, girl], [this, is, lovely, boy]]). This is the result when execute the test program Erlang (BEAM) emulator version 5.6.1 [smp:2] [async-threads:0] Eshell V5.6.1 (abort with ^G) 1> c (mapreduce). {ok,mapreduce} 2> 2> mapreduce:test_map_reduce(). Reduce processes [<0.37.0>,<0.38.0>,<0.39.0>,<0.40.0>,<0.41.0>] are started Map processes [<0.42.0>,<0.43.0>,<0.44.0>] are started Send [this,is,a,boy] to map process <0.42.0> Send [this,is,a,girl] to map process <0.43.0> Map function produce: [{this,1},{is,1},{a,1},{boy,1}] Send [this,is,lovely,boy] to map process <0.44.0> Map function produce: [{this,1},{is,1},{a,1},{girl,1}] Map function produce: [{this,1},{is,1},{lovely,1},{boy,1}] Collect all data from reduce processes [{is,3},{this,3},{boy,2},{girl,1},{a,2},{lovely,1}] 3> Summary From this exercise of implementing a simple Map/Reduce model using Erlang, I found that Erlang is very powerful in developing distributed systems.
June 15, 2008
by Ricky Ho
· 18,930 Views
article thumbnail
Hibernate - Dynamic Table Routing
I have been searching for a method to dynamically route objects to databases at runtime using Hibernate and recently I found a solution which fit the bill.
June 13, 2008
by alvin sd
· 60,149 Views · 3 Likes
article thumbnail
Java Reporting With Jasper Reports - Part 2
Welcome back to Java Reporting series. For those who didn't read the introductory article; have a look before we get started. Today we're going to have a quick tour in JasperReport architecture, development lifecycle, report definition files, and finally we're going to set up our environment and start work in a sample application. Architecture As shown in the above figure JasperReports architecture is based on declarative XML fileswhich by convention have an extension of jrxml that contains the report layout. A lot of third-party design tools were produced to generate your jrxml file in a smooth way (like iReport or JasperAssistant)Design file is supposed to be filled by report's result which is fetched from database, XML files, Java collection, Comma-separated values or Models. Jasper can communicate with those data-sources and more, it can merge any number of data-sources together and manipulates the results of any combinations. This communication goes through JDBC, JNDI, XQuery, EJBQL, Hibernate or existing Oracle PL/SQL. You also can define your own data-source class and pass it to jasper engine directly. After defining your report design layout in jrxml format and determining your data source(s) jasper engine does the rest of work. It compiles your design file and fills it with results fetched from data-source and generates your report to the chosen exporting format (PDF, Excel, HTML, XML, RTF, TXT …, etc.) Report Definition file structure (jrxml): Jasper design file –jrxml- contains the following elements: : the root element.
June 13, 2008
by Hossam Sadik
· 293,298 Views
article thumbnail
.NET Memory control : Use GCHandle to pin down the objects
In the .NET framework memory control is mostly autonomous and is controlled by the CLR. Although in managed languages we do not care much about controlling memory but when we have to interact with other languages we have to be aware of the memory implications. When we create an object in memory there is no guarantee that the object will remain in the same location it was created in. This is because the GC moves memory around by itself when needed to. Variable movement in memory I am sure that everyone reading this have already read about how GC collects memory and generations. As you know that GC allocates memory spaces in bulk. When needed GC collects unused memory and moves it. For example if the object you are using is collected, in order make better allocation GC can move it a different block of memory and free the block you were in. Let think we have a variable called "x" and it allocated in the memory like in the position like shown below. [img_assist|nid=3441|title=|desc=Figure 1: Before GC collection|link=none|align=none|width=240|height=88] In the figure above 1 is the space that are allocated and 0 are the ones that need to garbage collected because there is no reference to it. When GC collects it might decide to move the occupied items to the top block keep the second block empty by compacting. If so our memory could look like this diagram below: [img_assist|nid=3442|title=|desc=Figure 2: After compacting the variable moved to different virtual address|link=none|align=none|width=240|height=88] As you see that GC has moved all items to the first block of memory and the second block is empty which will result in faster memory allocation for new variables and it makes sense to do to so. Our variable x has moved to a different location in memory space which means the address of the memory where the variable is kept has changed. (Please keep in mind that the address space used does not have anything is physical ram as the OS may decide to change the physical address anytime and the address we are referring to is virtual address. Usually GC collect after 256KB of memory has become occupied, depending on the version of GC and OS and the mode it runs. As GC is independent of the thread you are running, it can collect any time. [img_assist|nid=3443|title=|desc=|link=none|align=none|width=418|height=316] If you are doing unmanaged memory operation and you are using the memory address and GC can come in move the memory away as you use the address to do unmanaged operations. Pin down: API Calls to PInvoke So can can this be a problem when we are calling windows APIs? For example, what happens if the window handle (hwnd) that I specified in the pinvoke is moved to a different memory address. The variable is pinned down to memory before the PInvoke call and pinned status is released just after the call. This is done automatically by the CLR. So what would have happened if GC had come in while the unmanaged PInvoke was running? See the figure below: [img_assist|nid=3444|title=|desc=|link=none|align=none|width=391|height=578] As you can see in the figure above that the variable x did not move to another virtual address in memory. This is useful when I need some unmanaged code to access a memory address that is constant. For example lets think of a scenario where I have an integer array which I pass to some unmanaged function and then I change the values of the variable at times and unmanaged function read changed values in the integer array and does some work. In such a scenario I will need the array to remain in one constant space. So I will need GCHandle class to pin it down in memory GCHandle class We find the class in System.Runtime.InteropServices namespace. In order use this class we will need SecurityPermission with Unmanaged code flag most of the time. Each application domain has a for GC handles, with this GCHandle class we can control what items are in that table. The Garbage Collector actually reads this table and abides by it. Each entry in the table points to an object in the managed heap and how GC should treat it. There are four behaviors types as described in the GCHandleType enumeration. They are as follows, 1. Weak 2. WeakTrackResurrection 3. Normal 4. Pinned Weak and WeakTrackResurrection for tracking weak referenced objects (see earlier post WeakReference: GC knows the Best for more on weak references). The Normal type is used to keep an object alive even if there are no reference to it. We are interested in the last type called Pinned. This tells the Garbage Collector to keep this object in memory even if there are no reference to it and never to move this object around in memory. See an example below on how to use the GCHandle class: string name = "My Name"; byte[] nameinbyte = ASCIIEncoding.ASCII.GetBytes(name); // Pin down the byte array GCHandle handle = GCHandle.Alloc(nameinbyte, GCHandleType.Pinned); IntPtr address = handle.AddrOfPinnedObject (); // Do stuff ... with the pinned object address // .... handle.Free(); Things to Remember Please note that too many pinned object will make the GC slowdown. Also only blittable types and arrays of blittable types can be used. If you have written a custom object you can make it blittable by implenting a custom marshaler with the ICustomMarshaler interface.
June 10, 2008
by Shafqat Ahmed
· 41,592 Views
article thumbnail
Hibernate - Tuning Queries Using Paging, Batch Size, and Fetch Joins
This article covers queries - in particular a tuning test case and the relations between simple queries, join fetch queries, paging query results, and batch size. Paging the Query Results I will start with a short introduction about paging in EJB3: To support paging the EJB3 Query interface defines the following two methods: setMaxResults - sets the number of maximum rows to retrieve from the database setFirstResult - sets the first row to retrieve For example if our GUI displays a list of customers and we have 500,000 customers (database rows) in out database we wouldn't like to display all 500,000 records is one view (even if we put performance considerations aside - nobody can do anything with a list of 500,000 rows). The GUI design would usually include paging - we break the list of records to display into logical pages (for example 100 records per page) and the user can navigate between pages (same as Google's results navigator down the search page). When using the paging support it is important to remember that the query has to be sorted otherwise we can't be sure that when fetching the "next page" it will really be the next page (since in the absence of the 'order by' clause form a SQL query the order in which rows are fetch is unpredictable). Here is a sample use, for fetching the first tow pages of 100 rows each: Query q = entityManager.createQuery("select c from Customer c order by c.id"); q.setFirstResult(0).setMaxResults(100); .... next page ... Query q = entityManager.createQuery("select c from Customer c order by c.id"); q.setFirstResult(100).setMaxResults(100); This is a simple API and it's important (for performance) to remember using it when we need to fetch only parts of the results. Test Case Description This test cased is based on a real tuning I did for an application, I just changed the class names to Customer and Order. Let's assume that I have a Customer entity with a set of orders (lazily fetched - but it happens in eager fetch as well) and we need to: Fetch customers and their orders Do it in a "paging mode" - 100 customers per page Tuning Requirement #1 - Fetch Customers and Their Orders There are two possibilities to perform this kind of fetch: Simple select: select c from customer c order by c.id Join fetch: select distinct c from Customer c left outer join fetch c.orders order by c.id The simple select is as simple as it can be, we load a list of customers with a proxy collection in their orders field. The orders collection will be filled with data once I access it (for example c.getOrders().getSize() ). The 'join fetch' means that we want to fetch an association as an integral part of the query execution. The joined fetched entities (in the example above: c.orders) must be part of an association that is referenced by an entity returned from the query (in the example above: c). The 'join fetch' is one of the tools used for improving queries performance (see more in here). The Hibernate core documentations explains that "a 'fetch' join allows associations or collections of values to be initialized along with their parent objects, using a single select" (see here). I have in my database 18,998 customer records, each with few orders. Let's compare execution time for the two queries. My code looks the same for both queries (except of the query itself), I execute the query, then I iterate the results checking the size of of each customer orders collection and print the execution time and number of records fetch (as a sanity for the query syntax): Query q = entityManager.createQuery(queryStr); long a = System.currentTimeMillis(); List l = q.getResultList(); for (Customer c : l) { c.getOrders().size(); } long b = System.currentTimeMillis(); System.out.println("Execution time: " + (b - a)+ "; Number of records fetch: " + l.size() ); And to the numbers (avg. 3 executions): Simple select: 24,984 millis Join fetch: 1,219 millis The join fetch query execution time was 20 times faster(!) than the simple query. The reason is obvious, using the join fetch select I had only one round trip to the database. While using a simple select I had to fetch the customers (1 round trip to the database) and each time I accessed a collection I had another round trip (that's 18,998 additional round trips!). The winner is 'join fetch'. But does it? wait for the next one - the paging... Tuning Requirement #2 - Use Paging The second requirement was to do it in paging - each page will have 100 customers (so we will have 18,900/100+1 pages - the last page has 98 customers). So let's change the code above a little bit: Query q = entityManager.createQuery(queryStr); q.setFirstResult(pageNum*100).setMaxResults(100); long a = System.currentTimeMillis(); List l = q.getResultList(); for (Customer c : l) { c.getOrders().size(); } long b = System.currentTimeMillis(); System.out.println("Execution time: " + (b - a)+ "; Number of records fetch: " + l.size() ); I added the second line which limits the query result to a specific page with up to 100 records per page. And the numbers are (avg. 3 executions): Simple select: 328 millis Join fetch: 1,660 millis The wheel has turned over. Why? First a quote from the EJB3 Persistence specification: "The effect of applying setMaxResults or setFirstResult to a query involving fetch joins over collections is undefined" (section 3.6.1 - Query Interface) We could have stopped here but it is interesting to understand the issue and to see what Hibernate does. To implement the paging features Hibernate delegates the work to the database using its syntax to limit the number of records fetched by the query. Each database has its own proprietary syntax for limiting the number of fetched records, some examples: Postgres uses LIMIT and OFFSET Oracle has rownum MySQL uses its version of LIMIT and OFFSET MSSQL has the TOP keyword in the select and so on The important thing to remember here is meaning of such limit: the database returns a subset of the query result. So if we asked for the first 100 customers which their names contain 'Eyal' the outcome is logically the same as building a table in memory out of all customers that match the criteria and take from there the first 100 rows. And here is the catch: if the query with the limit includes a join clause for a collection than the first 100 row in the "logical table" will not necessarily be the first 100 customers. the outcome of the join might duplicate customers in the "logical tables" but the database doesn't aware or care about that - it performs operations on tables not on objects!. For example think of the extreme case, the customer 'Eyal' has 100 orders. The query will return 100 rows, hibernate will identify that all belong to the same customer and return only one Customer as the query result - this is not what we were asking for. This also works, of course, the other way around. If a customer had more than 100 orders and the result set size was limited to 100 rots the orders collection would not contain all of the customer's orders. To deal with that limitation Hibernate actually doesn't issue an SQL statement with a LIMIT clause. Instead it fetches all of the records and performs the paging in memory. This explains why using the 'join fetch' statement with paging took more than the one without paging - the delta is the in-memory paging done by Hibernate. If you look at Hibernate logs you will find the next warning issued by Hibernate: WARNING: firstResult/maxResults specified with collection fetch; applying in memory! Final Tuning - BatchSize Does it mean that in the case of paging we shouldn't use a join fetch? usually it does (unless your page size is very close to the actual number of records). But even if you use a simple select this is a classic case for using the @BatchSize annotation. If my session/entity manager has 100 customers attached to it than, be default, for each first access to one of the customers' order collection Hibernate will issue a SQL statement to fill that collection. At the end I will execute 100 statements to fetch 100 collections. You can see it in the log: Hibernate: /* select c from Customer c order by c.id */ select customer0_.id as id0_, customer0_.ccNumber as ccNumber0_, customer0_.name as name0_, customer0_.fixedDiscount as fixedDis5_0_, customer0_.DTYPE as DTYPE0_ from CUSTOMERS customer0_ order by customer0_.id limit ? offset ? Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id=? Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id=? Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id=? ............ Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id=? Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id=? Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id=? The @BatchSize annotation can be used to define how many identical associations to populate in a single database query. If the session has 100 customers attached to it and the mapping of the 'orders' collection is annotated with @BatchSize of size n. It means that whenever Hibernate needs to populate a lazy orders collection it checks the session and if it has more customers which their orders collections need to be populated it fetches up to n collections. Example: if we had 100 customers and the batch size was set to 16 when iterating over the customers to get their number of orders hibernate will go to the database only 7 times (6 times to fetch 16 collections and one more time to fetch the 4 remaining collections - see the sample below). If our batch size was set to 50 it would go only twice. @OneToMany(mappedBy="customer",cascade=CascadeType.ALL, fetch=FetchType.LAZY) @BatchSize(size=16) private Set orders = new HashSet(); And in the log: Hibernate: /* select c from Customer c order by c.id */ select customer0_.id as id0_, customer0_.ccNumber as ccNumber0_, customer0_.name as name0_, customer0_.fixedDiscount as fixedDis5_0_, customer0_.DTYPE as DTYPE0_ from CUSTOMERS customer0_ order by customer0_.id limit ? offset ? Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) Hibernate: /* load one-to-many par2.Customer.orders */ select orders0_.customer_id as customer4_1_, orders0_.id as id1_, orders0_.id as id1_0_, orders0_.customer_id as customer4_1_0_, orders0_.description as descript2_1_0_, orders0_.orderId as orderId1_0_ from ORDERS orders0_ where orders0_.customer_id in (?, ?, ?, ?) Back to our test case. In my example setting the batch size to 100 looks like a nice tuning opportunity. And indeed when setting it to 100 the total execution time dropped to 188 millis (that's an 132 (!!!) times faster than worse result we had). The batch size can also be set globally by setting the hibernate.default_batch_fetch_size property for the session factory. From http://www.jroller.com/eyallupu/
June 9, 2008
by Eyal Lupu
· 256,367 Views · 7 Likes
article thumbnail
Java Concurrency: Read / Write Locks
Jakob has done a great series on Java Concurrency - check out the first 14 articles at his blog. Going forward, we're delighted to announce that you'll also be able to follow the series here on JavaLobby. A read / write lock is more sophisticated lock than the Lock implementations shown in the text Locks in Java. Imagine you have an application that reads and writes some resource, but writing it is not done as much as reading it is. Two threads reading the same resource does not cause problems for each other, so multiple threads that want to read the resource are granted access at the same time, overlapping. But, if a single thread wants to write to the resource, no other reads nor writes must be in progress at the same time. To solve this problem of allowing multiple readers but only one writer, you will need a read / write lock. Java 5 comes with read / write lock implementations in the java.util.concurrent package. Even so, it may still be useful to know the theory behind their implementation. Here is a list of the topics covered in this text: Read / Write Lock Java Implementation First let's summarize the conditions for getting read and write access to the resource: Read Access If no threads are writing, and no threads have requested write access. Write Access If no threads are reading or writing. If a thread wants to read the resource, it is okay as long as no threads are writing to it, and no threads have requested write access to the resource. By up-prioritizing write-access requests we assume that write requests are more important than read-requests. Besides, if reads are what happens most often, and we did not up-prioritize writes, starvation could occur. Threads requesting write access would be blocked until all readers had unlocked the ReadWriteLock. If new threads were constantly granted read access the thread waiting for write access would remain blocked indefinately, resulting in starvation. Therefore a thread can only be granted read access if no thread has currently locked the ReadWriteLock for writing, or requested it locked for writing. A thread that wants write access to the resource can be granted so when no threads are reading nor writing to the resource. It doesn't matter how many threads have requested write access or in what sequence, unless you want to guarantee fairness between threads requesting write access. With these simple rules in mind we can implement a ReadWriteLock as shown below: public class ReadWriteLock{ private int readers = 0; private int writers = 0; private int writeRequests = 0; public synchronized void lockRead() throws InterruptedException{ while(writers > 0 || writeRequests > 0){ wait(); } readers++; } public synchronized void unlockRead(){ readers--; notifyAll(); } public synchronized void lockWrite() throws InterruptedException{ writeRequests++; while(readers > 0 || writers > 0){ wait(); } writeRequests--; writers++; } public synchronized void unlockWrite() throws InterruptedException{ writers--; notifyAll(); } } The ReadWriteLock has two lock methods and two unlock methods. One lock and unlock method for read access and one lock and unlock for write access. The rules for read access are implemented in the lockRead() method. All threads get read access unless there is a thread with write access, or one or more threads have requested write access. The rules for write access are implemented in the lockWrite() method. A thread that wants write access starts out by requesting write access (writeRequests++). Then it will check if it can actually get write access. A thread can get write access if there are no threads with read access to the resource, and no threads with write access to the resource. How many threads have requested write access doesn't matter. It is worth noting that both unlockRead() and unlockWrite() calls notifyAll() rather than notify(). To explain why that is, imagine the following situation: Inside the ReadWriteLock there are threads waiting for read access, and threads waiting for write access. If a thread awakened by notify() was a read access thread, it would be put back to waiting because there are threads waiting for write access. However, none of the threads awaiting write access are awakened, so nothing more happens. No threads gain neither read nor write access. By calling noftifyAll() all waiting threads are awakened and check if they can get the desired access. Calling notifyAll() also has another advantage. If multiple threads are waiting for read access and none for write access, and unlockWrite() is called, all threads waiting for read access are granted read access at once - not one by one. Read / Write Lock Reentrance The ReadWriteLock class shown earlier is not reentrant. If a thread that has write access requests it again, it will block because there is already one writer - itself. Furthermore, consider this case: Thread 1 gets read access. Thread 2 requests write access but is blocked because there is one reader. Thread 1 re-requests read access (re-enters the lock), but is blocked because there is a write request In this situation the previous ReadWriteLock would lock up - a situation similar to deadlock. No threads requesting neither read nor write access would be granted so. To make the ReadWriteLock reentrant it is necessary to make a few changes. Reentrance for readers and writers will be dealt with separately. Read Reentrance To make the ReadWriteLock reentrant for readers we will first establish the rules for read reentrance: A thread is granted read reentrance if it can get read access (no writers or write requests), or if it already has read access (regardless of write requests). To determine if a thread has read access already a reference to each thread granted read access is kept in a Map along with how many times it has acquired read lock. When determing if read access can be granted this Map will be checked for a reference to the calling thread. Here is how the lockRead() and unlockRead() methods looks after that change: public class ReadWriteLock{ private Map readingThreads = new HashMap(); private int writers = 0; private int writeRequests = 0; public synchronized void lockRead() throws InterruptedException{ Thread callingThread = Thread.currentThread(); while(! canGrantReadAccess(callingThread)){ wait(); } readingThreads.put(callingThread, (getAccessCount(callingThread) + 1)); } public synchronized void unlockRead(){ Thread callingThread = Thread.currentThread(); int accessCount = getAccessCount(callingThread); if(accessCount == 1){ readingThreads.remove(callingThread); } else { readingThreads.put(callingThread, (accessCount -1)); } notifyAll(); } private boolean canGrantReadAccess(Thread callingThread){ if(writers > 0) return false; if(isReader(callingThread) return true; if(writeRequests > 0) return false; return true; } private int getReadAccessCount(Thread callingThread){ Integer accessCount = readingThreads.get(callingThread); if(accessCount == null) return 0; return accessCount.intValue(); } private boolean isReader(Thread callingThread){ return readers.get(callingThread) != null; } } As you can see read reentrance is only granted if no threads are currently writing to the resource. As you can see, if the calling thread already has read access this takes precedence over any writeRequests. Write Reentrance Write reentrance is granted only if the thread has already write access. Here is how the lockWrite() and unlockWrite() methods look after that little change: public class ReadWriteLock{ private Map<Thread, Integer> readingThreads = new HashMap<Thread, Integer>(); private int writeAccesses = 0; private int writeRequests = 0; private Thread writingThread = null; public synchronized void lockWrite() throws InterruptedException{ writeRequests++; Thread callingThread = Thread.currentThread(); while(! canGrantWriteAccess(callingThread)){ wait(); } writeRequests--; writeAccesses++; writingThread = callingThread; } public synchronized void unlockWrite() throws InterruptedException{ writeAccesses--; if(writeAccesses == 0){ writingThread = null; } notifyAll(); } private boolean canGrantWriteAccess(Thread callingThread){ if(hasReaders()) return false; if(writingThread == null) return true; if(writingThread != callingThread) return false; return true; } private boolean hasReaders(){ return readingThreads.size() > 0; } } Notice how the thread currently holding the write lock is now taken into account when determining if the calling thread can get write access. Read to Write Reentrance Sometimes it is necessary for a thread that have read access to also obtain write access. For this to be allowed the thread must be the only reader. To achieve this the writeLock() method should be changed a bit. Here is what it would look like: public class ReadWriteLock{ private Map readingThreads = new HashMap(); private int writeAccesses = 0; private int writeRequests = 0; private Thread writingThread = null; public synchronized void lockWrite() throws InterruptedException{ writeRequests++; Thread callingThread = Thread.currentThread(); while(! canGrantWriteAccess(callingThread)){ wait(); } writeRequests--; writeAccesses++; writingThread = callingThread; } public synchronized void unlockWrite() throws InterruptedException{ writeAccesses--; if(writeAccesses == 0){ writingThread = null; } notifyAll(); } private boolean canGrantWriteAccess(Thread callingThread){ if(isOnlyReader(callingThread)) return true; if(hasReaders()) return false; if(writingThread == null) return true; if(writingThread != callingThread) return false; return true; } private boolean hasReaders(){ return readingThreads.size() > 0; } private boolean isOnlyReader(Thread thread){ return readers == 1 && readingThreads.get(callingThread) != null; } } Now the ReadWriteLock class is read-to-write access reentrant. Write to Read Reentrance Sometimes a thread that has write access needs read access too. A writer should always be granted read access if requested. If a thread has read access no other threads can have read nor write access, so it is not dangerous. Here is how the canGrantReadAccess() method will look with that change: public class ReadWriteLock{ private boolean canGrantReadAccess(Thread callingThread){ if(isWriter(callingThread)) return true; if(writingThread != null) return false; if(isReader(callingThread) return true; if(writeRequests > 0) return false; return true; } } Fully Reentrant ReadWriteLock Below is the fully reentran ReadWriteLock implementation. I have made a few refactorings to the access conditions to make them easier to read, and thereby easier to convince yourself that they are correct. public class ReadWriteLock{ private Map readingThreads = new HashMap(); private int writeAccesses = 0; private int writeRequests = 0; private Thread writingThread = null; public synchronized void lockRead() throws InterruptedException{ Thread callingThread = Thread.currentThread(); while(! canGrantReadAccess(callingThread)){ wait(); } readingThreads.put(callingThread, (getReadAccessCount(callingThread) + 1)); } private boolean canGrantReadAccess(Thread callingThread){ if( isWriter(callingThread) ) return true; if( hasWriter() ) return false; if( isReader(callingThread) ) return true; if( hasWriteRequests() ) return false; return true; } public synchronized void unlockRead(){ Thread callingThread = Thread.currentThread(); if(!isReader(callingThread)){ throw new IllegalMonitorStateException("Calling Thread does not" + " hold a read lock on this ReadWriteLock"); } int accessCount = getReadAccessCount(callingThread); if(accessCount == 1){ readingThreads.remove(callingThread); } else { readingThreads.put(callingThread, (accessCount -1)); } notifyAll(); } public synchronized void lockWrite() throws InterruptedException{ writeRequests++; Thread callingThread = Thread.currentThread(); while(! canGrantWriteAccess(callingThread)){ wait(); } writeRequests--; writeAccesses++; writingThread = callingThread; } public synchronized void unlockWrite() throws InterruptedException{ if(!isWriter(Thread.currentThread()){ throw new IllegalMonitorStateException("Calling Thread does not" + " hold the write lock on this ReadWriteLock"); } writeAccesses--; if(writeAccesses == 0){ writingThread = null; } notifyAll(); } private boolean canGrantWriteAccess(Thread callingThread){ if(isOnlyReader(callingThread)) return true; if(hasReaders()) return false; if(writingThread == null) return true; if(!isWriter(callingThread)) return false; return true; } private int getReadAccessCount(Thread callingThread){ Integer accessCount = readingThreads.get(callingThread); if(accessCount == null) return 0; return accessCount.intValue(); } private boolean hasReaders(){ return readingThreads.size() > 0; } private boolean isReader(Thread callingThread){ return readingThreads.get(callingThread) != null; } private boolean isOnlyReader(Thread callingThread){ return readingThreads.size() == 1 && readingThreads.get(callingThread) != null; } private boolean hasWriter(){ return writingThread != null; } private boolean isWriter(Thread callingThread){ return writingThread == callingThread; } private boolean hasWriteRequests(){ return this.writeRequests > 0; } } Calling unlock() From a finally-clause When guarding a critical section with a ReadWriteLock, and the critical section may throw exceptions, it is important to call the readUnlock() and writeUnlock() methods from inside a finally-clause. Doing so makes sure that the ReadWriteLock is unlocked so other threads can lock it. Here is an example: lock.lockWrite(); try{ //do critical section code, which may throw exception } finally { lock.unlockWrite(); } This little construct makes sure that the ReadWriteLock is unlocked in case an exception is thrown from the code in the critical section. If unlockWrite() was not called from inside a finally-clause, and an exception was thrown from the critical section, the ReadWriteLock would remain write locked forever, causing all threads calling lockRead() or lockWrite() on that ReadWriteLock instance to halt indefinately. The only thing that could unlock the ReadWriteLockagain would be if the ReadWriteLock is reentrant, and the thread that had it locked when the exception was thrown, later succeeds in locking it, executing the critical section and calling unlockWrite() again afterwards. That would unlock the ReadWriteLock again. But why wait for that to happen, if it happens? Calling unlockWrite() from a finally-clause is a much more robust solution.
June 9, 2008
by Jakob Jenkov
· 79,312 Views
article thumbnail
Easy Unwrapping with IntelliJ IDEA
In complicated code constructs that contain numerous nested statements, you sometimes need to accurately get rid of the enclosing parts. When you try to manually delete such statements, it is too easy to break the syntax of the whole construct. With the new IntelliJ IDEA’s unwrapping feature, this task becomes just a snap. Consider the following example, where a string variable is being analyzed: [img_assist|nid=3370|title=|desc=|link=none|align=left|width=640|height=299] Let’s remove one of the branches. To do that, place the cursor at the branch containing the statement, and press Ctrl+ Shift+Delete. IntelliJ IDEA shows you which results you can obtain: the part that will remain is displayed on the blue background, while the grey background denotes the pieces of code to be deleted: [img_assist|nid=3371|title=|desc=|link=none|align=left|width=640|height=258] After clicking the Remove ‘else…’ option, we get the promised results: [img_assist|nid=3372|title=|desc=|link=none|align=left|width=640|height=226] As you see, all parentheses are properly balanced, and the syntax is correct. However, Java developers are not the only ones who can enjoy this powerful feature. The same “stripping” can be done in XML or HTML code. Consider a situation when you have to clean up some HTML code and get rid of inline styles, for example, remove bold fonts. What will you do in this case – delete an opening tag, then delete the closing tag, with the real possibility to lose something in process. With IntelliJ IDEA, all you have to do is to place the caret somewhere inside the tags, and press Ctrl+Shift+Delete. The unnecessary tags are removed silently. This facility is even more helpful for XML constructs, which can have deep nesting. You can sequentially remove enclosing tags level by level, until getting to the innermost one you want to preserve. Do I need to mention that such unwrapping can be undone? Enjoy!
June 7, 2008
by Irina Megorskaya
· 10,408 Views · 1 Like
article thumbnail
dotNET WPF Dialogs and DialogResult
In WPF Dialogs are quite different from Windows Forms. The behavior is still the same i.e when you have a Dialog opened (by calling the ShowDialog() method) the user must close the dialog in order to use the Window that opened the Dialog Window. The difference is the way you handle the Dialog Result and also how to set Dialog Results. Let’s say that you created a Window in Windows Forms and you have a Button. You can set the DialogResult property of the button so that when the user click on that button the Dialog ends with the Dialog Result that you have set. (also if the user clicks the exit button of the Dialog the DialogResult would be a Cancel). The code to handle the DialogResult in Windows Forms would look like this: DialogResult result = new Form1().ShowDialog(); if (result == DialogResult.OK) MessageBox.Show(“User clicked OK”); else if (result == DialogResult.Cancel) MessageBox.Show(“User clicked Cancel”); In the WPF this is different. There is no DialogResult property on controls and also the ShowDialog does not return a DialogResult instead it returns a Nullable. So to handle a DialogResult after calling the ShowDialog() in WPF your code would look like this: MyDialog dialog = new MyDialog(); dialog.ShowDialog(); if (dialog.DialogResult.HasValue && dialog.DialogResult.Value) MessageBox.Show(“User clicked OK”); else MessageBox.Show(“User clicked Cancel”); ShowDialog can return True which is equivalent to DialogResult.Ok or False which is equivalent to DialogResult.Cancel. So let’s have a look at how you can create a Dialog in WPF and set the DialogResult of the Dialog. We can do this by setting a property on the Window called DialogResult. Once you set this property on the Window the Window will automatically close and the ShowDialog methods returns the result that you have set in the DialogResult property. If you want to have a “Cancel” Button for your Dialog you can use the IsCancel property and set this to true. When the user clicks the button the DialogResult will be set to false and the Dialog will close. This will also allow the user to click ESC to cancel the dialog ( which is something that every user would expect from a dialog). Please note that if you have more than one button with the IsCancel = “True” the ESC does not work as expected instead the focus will be given to the first button that has the IsCancel = “True”. The XAML for this button would look like this: If you want to have an “Ok” button you can set the IsDefault property. This property will NOT set the DialogResult to true for you, instead it will allow the user to click “Enter” and the Click event handler of the button is automatically called (so as such it doesn’t really have to do with Dialogs, it is just more useful when you are using Dialogs because users would expect such a behavior). You must code the event handler (and obviously register the handler to the click event) yourself and set the DialogResult to true. Something like this: private void ButtonOkClick(object sender, RoutedEventArgs e) { DialogResult = true; } That’s basically it. For more information you might want to have a look at the MSDN documentation on this here. Download Sample Dialog application
June 6, 2008
by Marlon Grech
· 21,402 Views
article thumbnail
Common REST Design Pattern
Based on the same architectural pattern of the web, "REST" has a growing dominance of the SOA (Service Oriented Architecture) implementation these days. In this article, we will discuss some basic design principles of REST. SOAP : The Remote Procedure Call Model Before the REST become a dominance, most of SOA architecture are built around WS* stack, which is fundamentally a RPC (Remote Procedure Call) model. Under this model, "Service" is structured as some "Procedure" exposed by the system. For example, WSDL is used to define the procedure call syntax (such as the procedure name, the parameter and their structure). SOAP is used to define how to encode the procedure call into an XML string. And there are other WS* standards define higher level protocols such as how to pass security credentials around, how to do transactional procedure call, how to discover the service location ... etc. Unfortunately, the WS* stack are getting so complicated that it takes a steep learning curve before it can be used. On the other hand, it is not achieving its original goal of inter-operability (probably deal to different interpretation of what the spec says). In the last 2 years, WS* technology development has been slowed down and the momentum has been shifted to another model; REST. REST: The Resource Oriented Model REST (REpresentation State Transfer) is introduced by Roy Fielding when he captured the basic architectural pattern that make the web so successful. Observing how the web pages are organized and how they are linked to each other, REST is modeled around a large number of "Resources" which "link" among each other. As a significant difference with WS*, REST raises the importance of "Resources" as well as its "Linkage", on the other hand, it push down the importance of "Procedures". Under the WS* model, "Service" in the SOA is organized as large number of "Resources". Each resource will have a URI that make it globally identifiable. A resource is represented by some format of "Representation" which is typically extracted by an idempotent HTTP GET. The representation may embed other URI which refers to other resources. This emulates an HTML link between web pages and provide a powerful way for the client to discover other services by traversing its links. It also make building SOA search engine possible. On the other hand, REST down play the "Procedure" aspect and define a small number of "action" based on existing HTTP Methods. As we discussed above, HTTP GET is used to get a representation of the resource. To modify a resource, REST use HTTP PUT with the new representation embedded inside the HTTP Body. To delete a resource, REST use HTTP DELETE. To get metadata of a resource, REST use HTTP HEAD. Notice that in all these cases, the HTTP Body doesn't carry any information about the "Procedure". This is quite different from WS* SOAP where the request is always made using HTTP POST. At the first glance, it seems REST is quite limiting in terms of the number of procedures that it can supported. It turns out this is not the case, REST allows any "Procedure" (which has a side effect) to use HTTP POST. Effectively, REST categorize the operations by its nature and associate well-defined semantics with these categories (ie: GET for read-only, PUT for update, DELETE for remove, all above are idempotent) while provide an extension mechanism for application-specific operations (ie: POST for application procedures which may be non-idempotent). URI Naming Convention Since resource is usually mapped to some state in the system, analyzing its lifecycle is an important step when designing how a resource is created and how an URI should be structured. Typically there are some eternal, singleton "Factory Resource" which create other resources. Factory resource typically represents the "type" of resources. Factory resource usually have a static, well-known URI, which is suffixed by a plural form of the resource type. Some examples are ... http://xyz.com/books http://xyz.com/users "Resource Instance", which are created by the "Factory Resource" usually represents an instance of that resource type. "Resource instances" typically have a limited life span. Their URI typically contains some unique identifier so that the corresponding instance of the resource can be located. Some examples are ... http://xyz.com/books/4545 http://xyz.com/users/123 "Dependent Resource" are typically created and owned by an existing resource during part of its life cycle. Therefore "dependent resource" has an implicit life-cycle dependency on its owning parent. When a parent resource is deleted, all the dependent resource it owns will be deleted automatically. Dependent resource use an URI which has prefix of its parent resource URI. Some examples are ... http://xyz.com/books/4545/tableofcontent http://xyz.com/users/123/shopping_cart Creating Resource To create a resource instance of a particular resource type, make an HTTP POST to the Factory Resource URI. If the creation is successful, the response will contain a URI of the resource that has been created. To create a book ... POST /books HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 Content-Length: nnn Ricky Ho HTTP/1.1 201 Created Content-Type: application/xml; charset=utf-8 Location: /books/4545 http://xyz.com/books/4545 To create a dependent resource, make an HTTP POST to its owning resource's URI To upload the content of a book (using HTTP POST) ... POST /books/4545 HTTP/1.1 Host: example.org Content-Type: application/pdf Content-Length: nnnn {pdf data} HTTP/1.1 201 Created Content-Type: application/pdf Location: /books/4545/content http://xyz.com/books/4545/tableofcontent HTTP POST is typically used to create a resource when its URI is unknown to the client before its creation. However, if the URI is known to the client, then an idempotent HTTP PUT should be used with the URI of the resource to be created. To upload the content of a book (using HTTP PUT) ... HTTP/1.1 201 Created Content-Type: application/pdf Location: /books/4545/content http://xyz.com/books/4545/tableofcontent HTTP/1.1 200 OK Finding Resources Make an HTTP GET to the factory resource URI, criteria pass in as parameters. (Note that it is up to the factory resource to interpret the query parameter). To search for books with a certain author ... GET /books?author=Ricky HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 HTTP/1.1 200 OK Content-Type: application/xml; charset=utf-8 Content-Length: nnn http://xyz.com/books/4545 Ricky http://xyz.com/books/4546 Ricky Another school of thoughts is to embed the criteria in the URI path, such as ... http://xyz.com/books/author/Ricky I personally prefers the query parameters mechanism because it doesn't imply any order of search criteria. Lookup a particular resource Make an HTTP GET to the resource object URI Lookup a particular book... GET /books/4545 HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 HTTP/1.1 200 OK Content-Type: application/xml; charset=utf-8 Content-Length: nnn Ricky Ho In case the resource have multiple representation format. The client should specify within the HTTP header "Accept" of its request what format she is expecting. Lookup a dependent resource Make an HTTP GET to the dependent resource object URI Download the table of content of a particular book... GET /books/4545/tableofcontent HTTP/1.1 Host: xyz.com Content-Type: application/pdf HTTP/1.1 200 OK Content-Type: application/pdf Content-Length: nnn {pdf data} Modify a resource Make an HTTP PUT to the resource object URI, pass in the new object representation in the HTTP body Change the book title ... PUT /books/4545 HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 Content-Length: nnn Ricky Ho HTTP/1.1 200 OK Delete a resource Make an HTTP DELETE to the resource object URI Delete a book ... DELETE /books/4545 HTTP/1.1 Host: xyz.com HTTP/1.1 200 OK Resource Reference In some cases, we do not want to create a new resource, but we want to add a "reference" to an existing resource. e.g. consider a book is added into a shopping cart, which is another resource. Add a book into the shopping cart ... POST /users/123/shopping_cart HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 Content-Length: nnn http://xyz.com/books/4545 HTTP/1.1 200 OK Show all items of the shopping cart ... GET /users/123/shopping_cart HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 HTTP/1.1 200 OK Content-Type: application/xml; charset=utf-8 Content-Length: nnn http://xyz.com/books/4545 ... Note that the shopping cart resource contains "resource reference" which acts as links to other resources (which is the books). Such linkages create a resource web so that client can discovery and navigate across different resources. Remove a book from the shopping cart ... POST /users/123/shopping_cart HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 Content-Length: nnn http://xyz.com/books/4545 HTTP/1.1 200 OK Note that we are using HTTP POST rather than HTTP DELETE to remove a resource reference. This is because we are remove a link but not the actual resource itself. In this case, the book still exist after it is taken out from the shopping cart. Checkout the shopping cart ... POST /orders HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 Content-Length: nnn http://xyz.com/users/123/shopping_cart HTTP/1.1 201 Created Content-Type: application/xml; charset=utf-8 Location: /orders/2008/04/10/1001 http://xyz.com/orders/2008/04/10/1001 Note that here the checkout is implemented by creating another resource "Order" which is used to keep track of the fulfillment of the purchase. Transaction Resource One of the common criticism of REST is because it is so tied in to HTTP (which doesn't support a client callback mechanism), doing asynchronous service or notification on REST is hard. So how do we implement long running transactions (which typically require asynchronicity and callback support) in REST ? The basic idea is to immediately create a "Transaction Resource" to return back to the client. While the actual processing happens asynchronously in the background, the client at any time, can poll the "Transaction Resource" for the latest processing status. Lets look at an example to request for printing a book, which may take a long time to complete Print a book POST /books/123 HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 Content-Length: nnn ?xml version="1.0" ?> http://xyz.com/printers/abc HTTP/1.1 200 OK Content-Type: application/xml; charset=utf-8 Location: /transactions/1234 http://xyz.com/transactions/1234 Note that a response is created immediately which contains the URI of a transaction resource, even before the print job is started. Client can poll the transaction resource to obtain the latest status of the print job. Check the status of the print Job ... GET /transactions/1234 HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 HTTP/1.1 200 OK Content-Type: application/xml; charset=utf-8 Content-Length: nnn PrintJob In Progress It is also possible to cancel the transaction if it is not already completed. Cancel the print job POST /transactions/1234 HTTP/1.1 Host: xyz.com Content-Type: application/xml; charset=utf-8 Content-Length: nnn HTTP/1.1 200 OK Conclusion The Resource Oriented Model that REST advocates provides a more natural fit for our service web. Therefore, I suggest that SOA implementation should take the REST model as a default approach.
June 4, 2008
by Ricky Ho
· 133,782 Views
  • Previous
  • ...
  • 1617
  • 1618
  • 1619
  • 1620
  • 1621
  • 1622
  • 1623
  • 1624
  • 1625
  • 1626
  • ...
  • Next
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook
×