Now that YouTrack has been announced and adopted by most JetBrains products, it's time to show how it actually works. That's because you report bugs to JetBrains now and then, don't you? Let's start with the way you search for issues. You do that by typing a natural-language-like search query in the search box. For example, the following query finds you all critical issues in project "YouTrack" that should be (or should have been) fixed for version 1.0. Even when you're somewhere outside of the search box, no need to grab your mouse to get there: just press Esc. The primary building block of YouTrack search queries is an attribute-value pair. Issue attributes include issue ID, project, state, type, priority, assignee, reporter, date of creation/update, associated tags etc. For example, to find all critical issues that won't fix in project "IDEA Development", you type the following query: state: {Won't fix} priority: critical project: IDEADEV You can make your query even shorter by skipping unambiguous keywords. For example, in this case, keywords “state”, "priority" and "project" can be safely omitted: #{Won't fix} #critical #IDEADEV On the other hand, "today" would be an ambiguous value because it is valid for two attributes: "updated" and "created". That's why using it without explicitly specifying one of those attributes could render inaccurate search results. Several things make YouTrack query language special, and one of those things is the use of shortcut keywords. When looking for all issues assigned to or reported by yourself, no need to enter a cumbersome query like assigned to: me or reported by: me Instead, type #my Of course, search queries may also contain full-text search items: for example, typing #unresolved light bulbs returns all unresolved issues that contain words "light" and "bulb" in summary, description, comments etc. When you type a query into the search box, YouTrack displays suggestions using query completion: We suggest that you use query completion extensively, for two reasons: Plain and simple: it lets you enter queries faster. More importantly, it automatically inserts special syntax items for you: Colons (:) after attributes. Hash marks (#) before values specified without attributes. Curly braces ({}) around values that contain spaces. In most cases, query completion displays automatically but whenever you need to call it at will, just press Ctrl+Space or Alt+Down. With any character you type, the completion list shrinks to show only the matching options: To help you get the feel of YouTrack search queries, here are some practical examples from our own YouTrack instance: Issues from three IntelliJ IDEA projects assigned to Max Shafirov that were updated this week. Unresolved dotTrace features scheduled for version 4.0. RubyMine issues supporting YAML implemented in version 1.5. Unresolved YouTrack issues sorted by number of votes in descending order. Let's sum it all up. In YouTrack, instead of setting up complicated filters in forms that bolster your inferiority complex, you just type what you are looking for in a manner very similar to how you would say it in plain English, aided by query completion. There have been other issue trackers that used search queries as a filtering option but there are none to my knowledge that rely as heavily on them and make them as usable as YouTrack. For additional information on YouTrack search queries, see the following docs: YouTrack Quick Start Guide: Searching for Issues. YouTrack search demo. YouTrack Search Keyword Reference. YouTrack search features at the JetBrains website. You can download YouTrack 1.0 Beta or use our Early Access Program to receive latest builds. Track with pleasure!

When you have collections of associated objects in domain objects, you generally want to specify some kind of default sort order. For example, suppose I have domain objects Timeline and Event: @Entity class Timeline { @Required String description @OneToMany(mappedBy = "timeline") @javax.persistence.OrderBy("startYear, endYear") Set events } @Entity class Event { @Required Integer startYear Integer endYear @Required String description @ManyToOne Timeline timeline } In the above example I've used the standard JPA (Java Persistence API) @OrderBy annotation which allows you to specify the order of a collection of objects via object properties, in this example a @OneToMany association . I'm ordering first by startYear in ascending order and then by endYear, also in ascending order. This is all well and good, but note that I've specified that only the start year is required. (The @Required annotation is a custom Hibernate Validator annotation which does exactly what you would expect.) How are the events ordered when you have several events that start in the same year but some of them have no end year? The answer is that it depends on how your database sorts null values by default. Under Oracle 10g nulls will come last. For example if two events both start in 2001 and one of them has no end year, here is how they are ordered: 2001 2002 Some event 2001 2003 Other event 2001 Event with no end year What if you want to control how null values are ordered so they come first rather than last? In Hibernate there are several ways you could do this. First, you could use the Hibernate-specific @Sort annotation to perform in-memory (i.e. not in the database) sorting, using natural sorting or sorting using a Comparator you supply. For example, assume I have an EventComparator helper class that implements Comparator. I could change Timeline's collection of events to look like this: @OneToMany(mappedBy = "timeline") @org.hibernate.annotations.Sort(type = SortType.COMPARATOR, comparator = EventCompator) Set events Using @Sort will perform sorting in-memory once the collection has been retrieved from the database. While you can certainly do this and implement arbitrarily complex sorting logic, it's probably better to sort in the database when you can. So we now need to turn to Hibernate's @OrderBy annotation, which lets you specify a SQL fragment describing how to perform the sort. For example, you can change the events mapping to : @OneToMany(mappedBy = "timeline") @org.hibernate.annotations.OrderBy("start_year, end_year") Set events This sort order is the same as using the JPA @OrderBy with "startYear, endYear" sort order. But since you write actual SQL in Hibernate's @OrderBy you can take advantage of whatever features your database has, at the possible expense of portability across databases. As an example, Oracle 10g supports using a syntax like "order by start_year, end_year nulls first" to order null end years before non-null end years. You could also say "order by start_year, end year nulls last" which sorts null end years last as you would expect. This syntax is probably not portable, so another trick you can use is the NVL function, which is supported in a bunch of databases. You can rewrite Timeline's collection of events like so: @OneToMany(mappedBy = "timeline") @org.hibernate.annotations.OrderBy("start_year, nvl(end_year , start_year)") Set events The expression "nvl(end_year , start_year)" simply says to use end_year as the sort value if it is not null, and start_year if it is null. So for sorting purposes you end up treating end_year as the same as the start_year if end_year is null. In the contrived example earlier, applying the nvl-based sort using Hibernate's @OrderBy to specify SQL sorting criteria, you now end with the events sorted like this: 2001 Event with no end year 2001 2002 Some event 2001 2003 Other event Which is what you wanted in the first place. So if you need more complex sorting logic than what you can get out of the standard JPA @javax.persistence.OrderBy, try one of the Hibernate sorting options, either @org.hibernate.annotations.Sort or @org.hibernate.annotations.OrderBy. Adding a SQL fragment into your domain class isn't necessarily the most elegant thing in the world, but it might be the most pragmatic thing.

Faz o cálculo da idade de uma pessoa utilizando sql oracle Select Trunc ( (SYSDATE - to_date('14/07/1980','dd/mm/yyyy')) /365, 0 ) as "age" from Dual

i will continue the jpa implementation patterns series by discussing the relative merits of field access vs. property access. the jpa specification allows two ways for the persistence provider to access the persistent state of an entity. the persistence provider can either invoke javabeans style property accessors (getters and setters) or access the instance fields of the entity directly. which method is used depends on whether you have annotated the properties or the fields of an entity. the jpa 1.0 specification does not allow you to mix access types within an entity or even an entity hierarchy. if you have annotated both fields and properties, the behaviour is undefined. the jpa 2.0 specification has the @access annotation that makes it possible mix access types within an entity or entity hierarchy. but the interesting question remains; which access type to use? a question that has been discussed before , but one i couldn't resist commenting on too. encapsulation - property access is said to provide better encapsulation, because directly accessing fields is bad, right? well actually, using property access obliges you to write getters and setters for all your persistent properties. these methods not only allow the jpa provider to set the fields, they also allow any other user of your entity class to do this! using field access allows you to write only the getters and setters you want to ( they're evil , remember?) and also write them as you want them, for example by doing validation in your setters or some calculation in your getters. in contrast, making these methods smarter when using property access is just asking for trouble . performance - some people prefer field access because it supposedly offers better performance than property access. but that is a very bad reason to choose field access. modern optimizing jvms will make property access perform just as fast as field access and in any case database calls are orders of magnitude slower than either field access or method invocations. lazy loading in hibernate - hibernate's lazy loading implementation always initializes a lazy proxy when any method on that proxy is invoked. the only exception to this is the method annotated with the @id annotation when you use property access. but when you use field access there is no such method and hibernate initializes the proxy even when invoking the method that returns the identity of the entity. while some propose to use property access until this bug is fixed, i am not in favour of basing design decisions on framework bugs. if this bug really hurts your performance you might want to try and get the id of entity with the following code: serializable id = ((hibernateproxy) entity).gethibernatelazyinitializer().getidentifier() it's nasty, but at least this code will be localized to where you really need it. field access in hibernate - it is good to know that while field access is ok for hibernate to populate your entities, your code should still access those values through methods. otherwise you will fall into the first of the hibernate proxy pitfalls mentioned by my colleague maarten winkels. to summarize i think field access is the way to go because it offers better encapsulation (without it properly managing bidirectional associations is impossible) and the performance impact is negligible (#1 on the performance problems top 10 is still the interplay between the database and your java app). the only downside are some snafu's in hibernate's lazy loading implementation that require you to take extra care when using field access. what access type do you prefer? do you see any difference in the way field access and property access are implemented in jpa providers other than hibernate? please let me know by leaving a comment below. see you at the next jpa implementation patterns blog in which i will talk about mapping inheritance hierarchies in jpa. from http://blog.xebia.com

logging is a fundamental part of applications. every application has a varying flavor of logging mechanism. a well designed logging system is a huge utility for system administrators and developers, especially the support team. logs save many valuable hours for both the support team or developers. as users execute programs at the front end, the system invisibly builds a vault of event information (log entries) for system administrators and the support team. after stating its value, let’s try to figure out the logging requirements for an application. java has a standard logging api in its new versions (java.util.logging). log4j is also a well-known library for logging. we implemented a transparent “logging service” in our application framework. you may prefer some kind of logging implementations but there are some other important questions you have to answer in your application design which are what to log, when to log, how much to log, how to control logging and how correlate it with your exception system. what to log some application exceptions should be logged: why not log all exceptions? some exceptions are managed exceptions which are thrown by application as a warning or as a validation error to the user. if all validation errors or application exceptions are included in logs, the logs will lose their usefulness. it would contain many entries, it makes it hard to reach valuable entries. here, we should discriminate exceptions if they should be logged. we used a loggable mark interface to determine if it is included in logs. another best practice is to have a single global exception handler. in this way, application developers don’t worry about logging of any exception they generate. a single exception handler means a single unified and neat exception logging mechanism. some application events should be logged: for major components of an application we may log lifecycle events like start, stop and restart. some security-related events may be logged such as unauthorized url access attempts, user logins etc. some resource thresholds may be exceeded and should also be logged. some application states should be logged: in some codes, we should ask that “what could go wrong here in this code”. if this state occurs, we may throw an exception or log it (if we don’t want to interrupt current process) with some levels like error, warning or information. for example if a connection is normally released while it is uncommitted, that info may be logged as uncommitted connection release that points some application code problems. some debug information may be logged: in some applications, you may have some errors and can’t find why this is happening. you may add some debug logs into your code and redeploy it to diagnose the problem. some chronic bugs deserve debug traces which can’t be detected in development environment. executed sqls may be logged: in some conditions, we may want to get the sql statement executed by an application. we should easily switch logging on without start/stop of system. let’s say user complain about wrong report result when executing a report. if we don’t have a clue about that we may log sql and trace it. user http requests may be logged: we may need what is coming from the user with full parameter details. to achieve this kind of logging we have to plug a log service in servlets and jsp pages. executing threads may be logged: i mentioned a blackbox implementation in our applications in one of my previous posts. you may log executing thread information to that black box to figure out what may go wrong in a system crash. javascript errors may be logged: this may be considered same with first item but its implementations totally differs. you need to have a global javascript exception handler. in the handler, you submit javascript error with ajax to log on the server (assuming it is not an ajax error). some best practices as i said above, the exception handling system is an important plug point for logging. in all our servlets and jsps, the main code blocks have the following structure to enable global exception handling. this standard eliminates the work of logging exceptions by developers (transparent unified logging): try { } catch(exception e) { globalhandler.handleexception(e); } we named “log medium” to mean where to store log entries but in java logging api it is called handlers. for some type of logs, database logs are handy since you can run powerful sql queries against log table which is merely possible in files. to assign log messages to the appropriate log levels is also important. otherwise some wrong alerts would mislead system administrators. we used system.err stream for application-level logging which is same place with e.printstacktrace() which may be printed with our global handler log entry at the same time. we wouldn’t need all exception’s stack trace but some may be very useful to see where the root of the problem (i.e. stackoverflowexception). when logging exceptions we used following format. some articles recommends that we’d better to include problem failover suggestion but i think this information should be given to the user not included in the logs. logs are rarely read but exceptions are in front of users. log entry should answer following questions: who (username) , when (timestamp) , where (context, servletorpage,database) , what (command) , result (exception) [errorhandlername] username: userabc databasename: abc timestamp: 15.07.2009 13:02:08 context:servlet page: /prod/sales/salesorders.jsp window: windwname command: savesalesorder exception: shortdescriptionofexception exceptionstacktracehere in a clustered environment, logging should be considered. we separated same type of log files with a cluster node name suffix to the file name. otherwise, concurrent log writes may lead to some problems. system_node01.log system_node02.log as a last item, logging may be a very interesting performance killer. if applications begin to log a lot, application performance may severely fall down. i have a real life story for that. once we had forgotten to include an image file in deployment package. it was being used in every page. when we started system after installment, page response was so slow. log files seem small enough that may not incur problem. after, opening and reading log files, we realized that log frequency (stating image is missing) was high and that was causing the problem. in conclusion, your log file size and log writing frequency should be small.

Good Database schema design is important for performance. One of the most basic optimizations is to design your tables to take as little space on the disk as possible , this makes disk reads faster and uses less memory for query processing. Data Types You should use the smallest data types possible, especially for indexed fields. The smaller your data types, the more indexes (and data) can fit into a block of memory, the faster your queries will be. Normalization Database Normalization eliminates redundant data, which usually makes updates faster since there is less data to change. However a Normalized schema causes joins for queries, which makes queries slower, denormalization speeds retrieval. More normalized schemas are better for applications involving many transactions, less normalized are better for reporting types of applications. You should normalize your schema first, then de-normalize later. Applications often need to mix the approaches, for example use a partially normalized schema, and duplicate, or cache, selected columns from one table in another table. With JPA O/R mapping you can use the @Embedded annotation for denormalized columns to specify a persistent field whose @Embeddable type can be stored as an intrinsic part of the owning entity and share the identity of the entity. Database Normalization and Mapping Inheritance Hiearchies The Class Inheritance hierarchy shown below will be used as an example of JPA O/R mapping. In the Single table per class mapping shown below, all classes in the hierarchy are mapped to a single table in the database. This table has a discriminator column (mapped by @DiscriminatorColumn), which identifies the subclass. Advantages: This is fast for querying, no joins are required. Disadvantages: wastage of space since all inherited fields are in every row, a deep inheritance hierarchy will result in wide tables with many, some empty columns. In the Joined Subclass mapping shown below, the root of the class hierarchy is represented by a single table, and each subclass has a separate table that only contains those fields specific to that subclass. This is normalized (eliminates redundant data) which is better for storage and updates. However queries cause joins which makes queries slower especially for deep hierachies, polymorphic queries and relationships. In the Table per Class mapping (in JPA 2.0, optional in JPA 1.0), every concrete class is mapped to a table in the database and all the inherited state is repeated in that table. This is not normlalized, inherited data is repeated which wastes space. Queries for Entities of the same type are fast, however polymorphic queries cause unions which are slower. Know what SQL is executed You need to understand the SQL queries your application makes and evaluate their performance. Its a good idea to enable SQL logging, then go through a use case scenario to check the executed SQL. Logging is not part of the JPA specification, With EclipseLink you can enable logging of SQL by setting the following property in the persistence.xml file: With Hibernate you set the following property in the persistence.xml file: Basically you want to make your queries access less data, is your application retrieving more data than it needs, are queries accessing too many rows or columns? Is the database query analyzing more rows than it needs? Watch out for the following: queries which execute too often to retrieve needed data retrieving more data than needed queries which are too slow you can use EXPLAIN to see where you should add indexes With MySQL you can use the slow query log to see which queries are executing slowly, or you can use the MySQL query analyzer to see slow queries, query execution counts, and results of EXPLAIN statements. Understanding EXPLAIN For slow queries, you can precede a SELECT statement with the keyword EXPLAIN to get information about the query execution plan, which explains how it would process the SELECT, including information about how tables are joined and in which order. This helps find missing indexes early in the development process. You should index columns that are frequently used in Query WHERE, GROUP BY clauses, and columns frequently used in joins, but be aware that indexes can slow down inserts and updates. Lazy Loading and JPA With JPA many-to-one and many-to-many relationships lazy load by default, meaning they will be loaded when the entity in the relationship is accessed. Lazy loading is usually good, but if you need to access all of the "many" objects in a relationship, it will cause n+1 selects where n is the number of "many" objects. You can change the relationship to be loaded eagerly as follows : However you should be careful with eager loading which could cause SELECT statements that fetch too much data. It can cause a Cartesian product if you eagerly load entities with several related collections. If you want to override the LAZY fetch type for specific use cases, you can use Fetch Join. For example this query would eagerly load the employee addresses: In General you should lazily load relationships, test your use case scenarios, check the SQL log, and use @NameQueries with JOIN FETCH to eagerly load when needed. Partitioning The main goal of partitioning is to reduce the amount of data read for particular SQL operations so that the overall response time is reduced Vertical Partitioning splits tables with many columns into multiple tables with fewer columns, so that only certain columns are included in a particular dataset, with each partition including all rows. Horizontal Partitioning segments table rows so that distinct groups of physical row-based datasets are formed. All columns defined to a table are found in each set of partitions. An example of horizontal partitioning might be a table that contains historical data being partitioned by date. Vertical Partitioning In the example of vertical partitioning below a table that contains a number of very wide text or BLOB columns that aren't referenced often is split into two tables with the most referenced columns in one table and the seldom-referenced text or BLOB columns in another. By removing the large data columns from the table, you get a faster query response time for the more frequently accessed Customer data. Wide tables can slow down queries, so you should always ensure that all columns defined to a table are actually needed. The example below shows the JPA mapping for the tables above. The Customer data table with the more frequently accessed and smaller data types is mapped to the Customer Entity, the CustomerInfo table with the less frequently accessed and larger data types is mapped to the CustomerInfo Entity with a lazily loaded one to one relationship to the Customer. Horizontal Partitioning The major forms of horizontal partitioning are by Range, Hash, Hash Key, List, and Composite. Horizontal partitioning can make queries faster because the query optimizer knows what partitions contain the data that will satisfy a particular query and will access only those necessary partitions during query execution. Horizontal Partitioning works best for large database Applications that contain a lot of query activity that targets specific ranges of database tables. Hibernate Shards Partitioning data horizontally into "Shards" is used by google, linkedin, and others to give extreme scalability for very large amounts of data. eBay "shards" data horizontally along its primary access path. Hibernate Shards is a framework that is designed to encapsulate support for horizontal partitioning into the Hibernate Core. Caching JPA Level 2 caching avoids database access for already loaded entities, this makes reading frequently accessed unmodified entities faster, however it can give bad scalability for frequent or concurrently updated entities. You should configure L2 caching for entities that are: read often modified infrequently Not critical if stale You should also configure L2 (vendor specific) caching for maxElements, time to expire, refresh... References and More Information: JPA Best Practices presentation MySQL for Developers Article MySQL for developers presentation MySQL for developers screencast Keeping a Relational Perspective for Optimizing Java Persistence Java Persistence with Hibernate Pro EJB 3: Java Persistence API Java Persistence API 2.0: What's New ? High Performance MySQL book Pro MySQL, Chapter 6: Benchmarking and Profiling EJB 3 in Action sharding the hibernate way JPA Caching Best Practices for Large-Scale Web Sites: Lessons from eBay

JPA has 2 levels of caching. The first level of caching is the persistence context. The JPA Entity Manager maintains a set of Managed Entities in the Persistence Context. The Entity Manager guarantees that within a single Persistence Context, for any particular database row, there will be only one object instance. However the same entity could be managed in another User's transaction, so you should use either optimistic or pessimistic locking as explained in JPA 2.0 Concurrency and locking The code below shows that a find on a managed entity with the same id and class as another in the same persistence context , will return the same instance. @Stateless public ShoppingCartBean implements ShoppingCart { @PersistenceContext EntityManager entityManager; public OrderLine createOrderLine(Product product,Order order) { OrderLine orderLine = new OrderLine(order, product); entityManager.persist(orderLine); //Managed OrderLine orderLine2 =entityManager.find(OrderLine, orderLine.getId())); (orderLine == orderLine2) // TRUE return (orderLine); } } The diagram below shows the life cycle of an Entity in relation to the Persistent Context. The code below illustrates the life cycle of an Entity. A reference to a container managed EntityManager is injected using the persistence context annotation. A new order entity is created and the entity has the state of new. Persist is called, making this a managed entity. because it is a stateless session bean it is by default using container managed transactions , when this transaction commits , the order is made persistent in the database. When the orderline entity is returned at the end of the transaction it is a detached entity. The Persistence Context can be either Transaction Scoped-- the Persistence Context 'lives' for the length of the transaction, or Extended-- the Persistence Context spans multiple transactions. With a Transaction scoped Persistence Context, Entities are "Detached" at the end of a transaction. As shown below, to persist the changes on a detached entity, you call the EntityManager's merge() operation, which returns an updated managed entity, the entity updates will be persisted to the database at the end of the transaction. An Extended Persistence Context spans multiple transactions, and the set of Entities in the Persistence Context stay Managed. This can be useful in a work flow scenario where a "conversation" with a user spans multiple requests. The code below shows an example of a Stateful Session EJB with an Extended Persistence Context in a use case scenario to add line Items to an Order. After the Order is persisted in the createOrder method, it remains managed until the EJB remove method is called. In the addLineItem method , the Order Entity can be updated because it is managed, and the updates will be persisted at the end of the transaction. The example below contrasts updating the Order using a transaction scoped Persistence Context verses an extended Persistence context. With the transaction scoped persistence context, an Entity Manager find must be done to look up the Order, this returns a Managed Entity which can be updated. With the Extended Persistence Context the find is not necessary. The performance advantage of not doing a database read to look up the Entity, must be weighed against the disadvantages of memory consumption for caching, and the risk of cached entities being updated by another transaction. Depending on the application and the risk of contention among concurrent transactions this may or may not give better performance / scalability. JPA second level (L2) caching JPA second level (L2) caching shares entity state across various persistence contexts. JPA 1.0 did not specify support of a second level cache, however, most of the persistence providers provided support for second level cache(s). JPA 2.0 specifies support for basic cache operations with the new Cache API, which is accessible from the EntityManagerFactory, shown below: If L2 caching is enabled, entities not found in persistence context, will be loaded from L2 cache, if found. The advantages of L2 caching are: avoids database access for already loaded entities faster for reading frequently accessed unmodified entities The disadvantages of L2 caching are: memory consumption for large amount of objects Stale data for updated objects Concurrency for write (optimistic lock exception, or pessimistic lock) Bad scalability for frequent or concurrently updated entities You should configure L2 caching for entities that are: read often modified infrequently Not critical if stale You should protect any data that can be concurrently modified with a locking strategy: Must handle optimistic lock failures on flush/commit configure expiration, refresh policy to minimize lock failures The Query cache is useful for queries that are run frequently with the same parameters, for not modified tables. The EclipseLink JPA persistence provider caching Architecture The EclipseLink caching Architecture is shown below. Support for second level cache in EclipseLink is turned on by default, entities read are L2 cached. You can disable the L2 cache. EclipseLink caches entities in L2, Hibernate caches entity id and state in L2. You can configure caching by Entity type or Persistence Unit with the following configuration parameters: Cache isolation, type, size, expiration, coordination, invalidation,refreshing Coordination (cluster-messaging) Messaging: JMS, RMI, RMI-IIOP, … Mode: SYNC, SYNC+NEW, INVALIDATE, NONE The example below shows configuring the L2 cache for an entity using the @Cache annotation The Hibernate JPA persistence provider caching Architecture The Hibernate JPA persistence provider caching architecture is different than EclipseLink: it is not configured by default, it does not cache enities just id and state, and you can plug in different L2 caches. The diagram below shows the different L2 cache types that you can plug into Hibernate. The configuration of the cache depends on the type of caching plugged in. The example below shows configuring the hibernate L2 cache for an entity using the @Cache annotation For More Information: Introducing EclipseLink EclipseLink JPA User Guide Hibernate Second Level Cache Speed Up Your Hibernate Applications with Second-Level Caching Hibernate caching Java Persistence API 2.0: What's New ? Beginning Java™ EE 6 Platform with GlassFish™ 3 Pro EJB 3: Java Persistence API (JPA 1.0)

Model your complete database with all its relations with this JPA pattern for lazy loading.

DELIMITER ; DROP FUNCTION IF EXISTS multiurldecode; DELIMITER | CREATE FUNCTION multiurldecode (s VARCHAR(4096)) RETURNS VARCHAR(4096) DETERMINISTIC CONTAINS SQL BEGIN DECLARE pr VARCHAR(4096) DEFAULT ''; IF ISNULL(s) THEN RETURN NULL; END IF; REPEAT SET pr = s; SELECT urldecode(s) INTO s; UNTIL pr = s END REPEAT; RETURN s; END; | DELIMITER ;

SELECT attributes_ean FROM products_attributes WHERE LENGTH(attributes_ean) = 13 AND SUBSTRING((10 - (((( SUBSTRING(attributes_ean FROM 2 FOR 1) + SUBSTRING(attributes_ean FROM 4 FOR 1) + SUBSTRING(attributes_ean FROM 6 FOR 1) + SUBSTRING(attributes_ean FROM 8 FOR 1) + SUBSTRING(attributes_ean FROM 10 FOR 1) + SUBSTRING(attributes_ean FROM 12 FOR 1) )*3) + ( SUBSTRING(attributes_ean FROM 1 FOR 1) + SUBSTRING(attributes_ean FROM 3 FOR 1) + SUBSTRING(attributes_ean FROM 5 FOR 1) + SUBSTRING(attributes_ean FROM 7 FOR 1) + SUBSTRING(attributes_ean FROM 9 FOR 1) + SUBSTRING(attributes_ean FROM 11 FOR 1) )) MOD 10)) FROM -1 FOR 1) != SUBSTRING(attributes_ean FROM 13 FOR 1)

During Jazoon 2009 I got a few minutes of private attention from Mike Keith to my last article about domain models. That small time was worthy the whole conference for me since Mike pointed the gaps in my text as well as some valuable hints on how to better translate domain models in JPA annotations. From that short conversation, a special sentence remains alive in my memory: don't break the optimistic locking. After a review on my original code I agreed with Mike that I was ignoring the optimistic locking in my service layer - a common mistake noticed over the Internet and also in conversation with other friends. The problem is not new and the solution is neither new, but I decided to blog it shortly to my personal reference and eventually for your help. The problem: breaking the optimistic locking. When exposing domain models through web-services you should serialize your entities between the client and the service, and every time you do that you have a detached JPA entity. In order to persist the detached objects in the database you need to re-attach them in to a new persistence context - and that's where the problem begins. Concurrent threads can access the same write method, reading a same entity, modifying it and then writing back the detached entity in the database. In my original code I was reading the latest version of the entity and then copying the field values from the external entity to the latest one. In this way I guaranteed the unbreakable writing code but I felt in the most basic mistake of JPA: I broken the consistency of the entities. From the Mike book: it is just an accident waiting to happen. Below you find the trap example from my original code: @Override public FpUser update(FpUser entity) throws Exception { FpUser attached = manager.find(FpUser.class, entity.getId()); // Here I am modifying the latest entity and not the detached one. attached.setEmail(entity.getEmail()); attached.setName(entity.getName()); return manager.merge(attached); } From the code above, we can enumerate the steps required to bypass the optimistic locking: Client A reads entity.v1 Client B reads entity.v1 Client A modifies the entity.version1 and starts an update.transaction#1 Client B modifies the entity.version1 and starts an update.transaction#2 update.transaction#1 updates the fields received from Client A, merge the entity - that receives the version v2 - but get suspended before to finish. update.transaction#2 updates the fields and received from Client B, updates the version to v3 and finishes returning the entity.v3 to the Client B. update.transaction#1 finishes returning the entity.v2 to the Client A. At the end of the above execution, we have the following scenario: Client A has an instance of the entity version 2 Client B has an instance of the entity version 3 (it actually jumped directly from version 1 to 3, without even noticing the changes of the version 2) The database has the data from version 3 The worse side effect of this trap is that Client A believes the current data persisted in the database is the ones from version 2, but actually it is wrong since the version 3 is currently stored in the database. The inconsistency could be easily detected by the optimistic locking of JPA, but since I am reading the latest version on every update operation the code won't throw the proper exception and the clients will become inconsistent with the server side. Solution: keep it simple The default mechanism specified in JPA to avoid inconsistencies is a Version field applicable to the entities through the @Version annotation. Once you included the version field in your entities, you can just invoke the merge operation to re-attach detached objects and the container will handle the versioning for you - simple and easy (and safe). The above code can be rewritten in a sound manner: @Override public FpUser update(FpUser entity) throws Exception { return manager.merge(entity); } And that's it, fewer lines of code with a more sound and more robust code. I will fix the code in the footprint repository, so the article readers will find a better code in the repository - and perhaps the java.net staff help me to include an addendum to my article warning the readers about that. At least we both know about that from now on :) Other interesting blogs about similar problems: Some J2EE Performance Tips - from Carol McDonald's Retrying transactions in Java - from Panagiotis Astithas TopLink JPA - from Oracle EclipseLink JPA - it replaces the TopLink in JavaEE 6 Before to release your eyes to a next blogger, let me ask you the intrigant question: What if I care only partially about my Entity locking? It is subject for another blog entry, but during my talk with Mike he confessed the next JPA 2.0 includes this feature: the ability to lock partially an entity. In this way, I don't need to throw an exception in a transaction that will affect minor priority fields (the idea behind the common trap I demonstrated above). People that implement a code to avoid exceptions during updates are actually preventing the client to receive exceptions, resolving manually the locking problems. This suicidal trick seems to make sense where some fields support data overwriting - usually an optional or very low priority data. As soon I got a good example I return to this point, now you are free to give me your feedback or to find something else to have fun on the web. My vacations are over :) time to update my working environment and nothing better than a short blog to warm up my brain to the third quarter of Java in 2009. Next step: to conclude the second part of the article, reviewing the gaps and offering a good quality material of Java EE 5 - the last step before to start my complete migration to Java EE 6. From http://weblogs.java.net/blog/felipegaucho

Optimistic locking lets concurrent transactions process simultaneously, but detects and prevent collisions, this works best for applications where most concurrent transactions do not conflict. JPA Optimistic locking allows anyone to read and update an entity, however a version check is made upon commit and an exception is thrown if the version was updated in the database since the entity was read. In JPA for Optimistic locking you annotate an attribute with @Version as shown below: public class Employee { @ID int id; @Version int version; The Version attribute will be incremented with a successful commit. The Version attribute can be an int, short, long, or timestamp. This results in SQL like the following: “UPDATE Employee SET ..., version = version + 1 WHERE id = ? AND version = readVersion” The advantages of optimistic locking are that no database locks are held which can give better scalability. The disadvantages are that the user or application must refresh and retry failed updates. Optimistic Locking Example In the optimistic locking example below, 2 concurrent transactions are updating employee e1. The transaction on the left commits first causing the e1 version attribute to be incremented with the update. The transaction on the right throws an OptimisticLockException because the e1 version attribute is higher than when e1 was read, causing the transaction to roll back. Additional Locking with JPA Entity Locking APIs With JPA it is possible to lock an entity, this allows you to control when, where and which kind of locking to use. JPA 1.0 only supported Optimistic read or Optimistic write locking. JPA 2.0 supports Optimistic and Pessimistic locking, this is layered on top of @Version checking described above. JPA 2.0 LockMode values : OPTIMISTIC (JPA 1.0 READ): perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch. OPTIMISTIC_FORCE_INCREMENT (JPA 1.0 WRITE) perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch, force an increment to the version at the end of the transaction, even if the entity is not modified. PESSIMISTIC: lock the database row when reading PESSIMISTIC_FORCE_INCREMENT lock the database row when reading, force an increment to the version at the end of the transaction, even if the entity is not modified. There are multiple APIs to specify locking an Entity: EntityManager methods: lock, find, refresh Query methods: setLockMode NamedQuery annotation: lockMode element OPTIMISTIC (READ) LockMode Example In the optimistic locking example below, transaction1 on the left updates the department name for dep , which causes dep's version attribute to be incremented. Transaction2 on the right gives an employee a raise if he's in the "Eng" department. Version checking on the employee attribute would not throw an exception in this example since it was the dep Version attribute that was updated in transaction1. In this example the employee change should not commit if the department was changed after reading, so an OPTIMISTIC lock is used : em.lock(dep, OPTIMISTIC). This will cause a version check on the dep Entity before committing transaction2 which will throw an OptimisticLockException because the dep version attribute is higher than when dep was read, causing the transaction to roll back. OPTIMISTIC_FORCE_INCREMENT (write) LockMode Example In the OPTIMISTIC_FORCE_INCREMENT locking example below, transaction2 on the right wants to be sure that the dep name does not change during the transaction, so transaction2 locks the dep Entity em.lock(dep, OPTIMISTIC_FORCE_INCREMENT) and then calls em.flush() which causes dep's version attribute to be incremented in the database. This will cause any parallel updates to dep to throw an OptimisticLockException and roll back. In transaction1 on the left at commit time when the dep version attribute is checked and found to be stale, an OptimisticLockException is thrown Pessimistic Concurrency Pessimistic concurrency locks the database row when data is read, this is the equivalent of a (SELECT . . . FOR UPDATE [NOWAIT]) . Pessimistic locking ensures that transactions do not update the same entity at the same time, which can simplify application code, but it limits concurrent access to the data which can cause bad scalability and may cause deadlocks. Pessimistic locking is better for applications with a higher risk of contention among concurrent transactions. The examples below show: reading an entity and then locking it later reading an entity with a lock reading an entity, then later refreshing it with a lock The Trade-offs are the longer you hold the lock the greater the risks of bad scalability and deadlocks. The later you lock the greater the risk of stale data, which can then cause an optimistic lock exception, if the entity was updated after reading but before locking. The right locking approach depends on your application: what is the risk of risk of contention among concurrent transactions? What are the requirements for scalability? What are the requirements for user re-trying on failure? For More Information: Preventing Non-Repeatable Reads in JPA Using EclipseLink Java Persistence API 2.0: What's New ? What's New and Exciting in JPA 2.0 Beginning Java™ EE 6 Platform with GlassFish™ 3 Pro EJB 3: Java Persistence API (JPA 1.0)

Hibernate is a powerful, high performance object/relational persistence and query service. Hibernate lets you develop persistent classes following object-oriented idiom - including association, inheritance, polymorphism, composition, and collections. Hibernate allows you to express queries in its own portable SQL extension (HQL), as well as in native SQL, or with an object-oriented Criteria and Example API. Quintessential to using any ORM framework like hibernate is to know how to leverage the various performance tuning methods supported by the framework. In this volume Wings Jiang discusses three performance tuning strategies for hibernate: SQL Optimization Session Management Data Caching SQL Optimization When using Hibernate in your application, you already have been coding HQL (Hibernate Query Language) somewhere. For example, “from User user where user.name = ‘John’”. If issuing your SQL statement like this, Hibernate cannot use the SQL cache implemented by database because name of the user, in most scenarios, is extremely distinct. On the contrary, while using placeholder to achieve this, like “from User user where user.name =?” will be cached by the Database to fulfill the performance improvement. You can also set some Hibernate properties to improve performance, such as setting the number of records retrieved while fetching records via configuring property hibernate.jdbc.fetch_size, setting the batch size when committing the batch processing via configuring property hibernate.jdbc.batch_size and switching off the SQL output via setting property hibernate.show_sql to false in product environments. In addition, the performance tuning of your target Database is also significant, like SQL clauses tuning, reasonable indexes, delicate table structures, data partitions etc. Session Management Undoubtedly, Session is the pith of Hibernate. It manages the Database related attributes, such as JDBC connections, data entities’ states. Managing the Session efficiently is the key to getting high performance in enterprise applications. One of the many commonly used and equally elegant approaches to session management in hibernate is to use ThreadLocal. Threadlocal will create a local copy of session for every thread. Thus synchronization problems are averted, when objects are put in the Threadlocal, . To understand how ThreadLocal variables are used in Java, refer to Sun Java Documentation at http://java.sun.com/j2se/1.5.0/docs/api/java/lang/ThreadLocal.html Data Caching Before accomplishing any data caching, it is essential to set the property hibernate.cache.user_query_cache = true. There are three kinds of commonly used Caching Strategies in Hibernate: Using cache based on Session level (aka Transaction layer level cache). This is also called first-level cache. Using cache based on SessionFactory level (Application layer level cache). This is also called second-level cache. Using cluster cache which is employed in distributed application (in different JVMs). In fact, some techniques, like loading data by id, lazy initialization which betokens loading appropriate data in proper time rather than obtaining a titanic number of useless records, which are fairly useless in the subsequent operations are consummated via data caching. First Level Cache (aka Transaction layer level cache) Fetching an object from database always has a cost associated with it. This can be offset by storing the entities in hibernate session. Next time the entities are required, they are fetched from the session, rather than fetching from the database. To clear an object from the session use: session.evict(object). To clear all the objects from the session use session.clear(). Second Level Cache (aka Application layer level cache) In this approach, if an object is not found in session, it is searched for in the session factory before querying the database for the object. If an object is indeed fetched from database, the selected data should be put in session cache. This would improve the performance when the object is required next time. To remove an entity from session factory use the various overloaded implementations of evict() method of SessionFactory. In fact, Hibernate lets you tailor your own caching implementation by specifying the name of a class that implements org.hibernate.cache.CacheProvider using the property hibernate.cache.provider_class. But it is recommended to employ a few built-in integrations with open source cache providers (listed below). Cache Type Cluster Safe Query Cache Supported Hashtable Memory NO YES EHCache Memory, Disk NO YES OSCache Memory, Disk NO YES SwarmCache Clustered YES (clustered invalidation) NO JBoss TreeCache Clustered YES (replication) YES Terracota Clustered YES YES In order to use second level caching, developers have to append some configurations in hibernate.cfg.xml (for example, using EHCache here). net.sf.ehcache.hibernate.Provider In addition, developers also need to create a cache specific configuration file (Example: ehcache.xml for EHCache). (1) diskStore : Sets the path to the directory where cache .data files are created. The following properties are translated: a.user.home - User's home directory b.user.dir - User's current working directory c.java.io.tmpdir (Default temp file path) maxElementsInMemory : Sets the maximum number of objects that will be created in memory. eternal : Sets whether elements are eternal. If eternal, timeouts are ignored and the element is never expired. timeToIdleSeconds : Sets the time to idle for an element before it expires. Is only used if the element is not eternal. Idle time is now - last accessed time. timeToLiveSeconds : Sets the time to live for an element before it expires. Is only used if the element is not eternal. TTL is now - creation time overflowToDisk : Sets whether elements can overflow to disk when the in-memory cache has reached the maxInMemory limit. Finally the cache concurrency strategy has to be specified in mapping files. For example, the following code fragment shows how to configure your cache strategy. … … Cache Concurrency Strategies There are four kinds of built-in cache concurrency strategies provided by Hibernate. Chosing a right concurrency strategy for your hibernate implementation is the key to cache performance optimization. Besides to ensure data consistency and transaction integrity it is indispensable to master these strategies. read-only If your application needs to read but never modify instances of a persistent class, a read-only cache may be used. This is the simplest and best performing strategy. It's even perfectly safe for use in a cluster. nonstrict-read-write If the application only occasionally needs to update data (For example, if it is extremely unlikely that two transactions would try to update the same item simultaneously) and strict transaction isolation is not required, a nonstrict-read-write cache might be appropriate. read-write If the application needs to update data, a read-write cache might be appropriate. This cache strategy should never be used if serializable transaction isolation level is required. transactional If the application seldom needs to update data and at the same time, application also needs to avoid “dirty read” and “repeatable read”, this kind of concurrency strategy can be employed. The transactional cache strategy provides support for fully transactional cache providers such as JBoss TreeCache. The following table lists cache concurrency strategy supported by various cache providers. Cache Read-only Nonstrict-read-write Read-write Transactional Hashtable YES YES YES N/A EHCache YES YES YES N/A OSCache YES YES YES N/A SwarmCache YES YES N/A N/A JBoss TreeCache YES N/A N/A YES Cluster Cache (in different JVMs) Hibernate also supports cluster caching in disparate JVMs. At present, both SwarmCache and JBoss TreeCache support cluster caching across multiple JVMs. In some situations, especially at the level of enterprise, certain application has to support the concurrency accessing of thousands of users, at that time, cluster cache can help you because the cluster can provide failover and load balancing which improve the performance of application. Points to Note When employing one of the four cache strategies above, pay close attention to the following situation: Data cached almost immutable If data you want to cache is almost constant, you can use data caching which can improve the performance of the application. On the contrary, if the caching data are quiet volatile, Hibernate have to maintain and update the caching over time which extremely leads to performance hit. Data sizes in reasonable range If the size of data you is caching is massive, Hibernate will occupy the most memories of system, which causes the long waiting time of the whole application. Low frequency of data updating If data you are caching needs to be modified frequently, Hibernate have to take an array of time to update and modify the data in caching, which impacts the performance of the application as well. High frequency of data querying If data you are caching is steady, which means that most of the operations are querying, searching, no updating and modifying, making the most use of caching will be affording huge performance improvement. None crucial data Because of existing some incongruities when keeping the data in caching, so if the data you are caching is fairly crucial, do not use caching. By contrast, if the data in caching is insignificant, just use it without any vacillation. Summary Actually, after employing SQL Optimization, Session Management, Data Caching, we will obtain great battalions of performance gains, which make applications achieve acceptable waiting time for the final customers. External Links for Further Study http://www.hibernate.org/hib_docs/reference/en/html/performance.html http://blogs.jboss.com/blog/acoliver/2006/01/23/Hibernate_EJB3_Tuning.txt About Author I am Wings Jiang from BCM China. I have mainly focused on J2EE technologies in recent years and worked in several projects involving Struts/Tapestry, Spring, Hibernate, WebLogic, Websphere, Oracle, DB2 etc. I have experience in design and code of several Java applications. Hibernate performance is one of the areas I pay close heed to in my current working.

Not that long ago, there was just ASP.NET. But ASP.NET has expanded to include different approaches to development. Recently, I was chatting with Microsoft’s Scott Hunter and Steve Walther over drinks at DevConnections Orlando and Steve mentioned “the four pillars of ASP.NET.” The four pillars are the four ways you can build web sites with ASP.NET, both today and in the future. The four pillars are: 1. ASP.NET Web Forms. Until recently, this was the only pillar of ASP.NET. Everything was done using the familiar rich server-side controls that many have grown to love (and some have learned to despise which is the seed for the other three pillars). So what’s wrong with Web Forms? Well, many developers believe that ASP.NET Web Forms are too much of a black box with a healthy-dose of black magic. There’s a price you pay for the Web Form event model and lifecycle and that price is control over the exchange of data between server and browser. Some say the Web Form model with its incessant ViewState is a pig. The Web Form model also makes it difficult to create testable applications and practice test-driven development (TDD). On the other hand, ASP.NET AJAX and ASP.NET MVC (pillars 2 and 3, respectively) come with a steep learning curve, and for many developers will take longer to develop the equivalent application. Pillar 4 (Dynamic Data), on the other hand, has limited applicability, IMO. My opinion is that with the possible exception of Dynamic Data for the creation of simpler intranet applications, nothing in my mind beats ASP.NET Web Forms (especially when paired with the ASP.NET AJAX UpdatePanel control) for creating good, solid, and responsive applications that, while not the leanest and meanest of applications nor very testable, are easy to grok and master for the novice through advanced developer. Web forms are also the best match for the developer looking to make the move to ASP.NET from Microsoft desktop (Win Forms or WPF) development. And with each new version, Microsoft is making it easier to tame the pig (ViewState) and reduce its size: ASP.NET 2.0 introduced ControlState and ASP.NET 4.0 changes the inheritance model of ViewState so it will be easier to turn off ViewState by default and only turn it on when needed. 2. ASP.NET AJAX. This pillar came to life with the ASP.NET AJAX Extensions for ASP.NET 2.0 / Visual Studio 2005. ASP.NET AJAX is now integrated into ASP.NET and Visual Studio 2008 and consists of both a set of server-side controls as well as the AJAX client-side libraries. In regards to its existence as a “pillar,” I would argue that Microsoft is really only talking about the client-side of ASP.NET AJAX. The reason I say this is that the Upgrade Panel and other server-side AJAX controls merely extend the Web Form pillar, giving Web Form controls the ability to do asynchronous postbacks. There are many in the hard-core ASP.NET AJAX community that believe that a true AJAX application needs to be built from the ground up without the use of Web Forms and the server-side controls. Using this approach, ASP.NET merely becomes a way to emit HTML with embedded JavaScript (and references to the Microsoft ASP.NET AJAX and JQuery libraries) that calls back to ASP.NET (or WCF) web services. When the HTML and JavaScript hit the browser, that’s when the action begins. The promise of this approach is a much snappier user interface and a much more scalable web site. On the downside, this requires programming in a loosely-typed language with a weird inheritance model, spotty support for IntelliSense, and, while improved, lousy debugger support. Another downside, the lack of smart client-side controls is likely to be remedied in the ASP.NET 4.0 timeframe. Microsoft is busy improving the client-side story, complete with client-side controls, for ASP.NET 4.0 (if you are curious, check out http://asp.net/ajax and click on the “ASP.NET AJAX Preview on CodePlex” link for a peek at what’s coming). Regardless, I believe this pillar will always be for a subset of ASP.NET developers who don’t shun the client-side. 3. ASP.NET MVC. This pillar is the newest to emerge from Microsoft. In fact, as of this writing, it’s only a couple of weeks old, having been released at Mix09. Some ASP.NET curmudgeons would call this a throwback to the days of ASP “classic” spaghetti code, but for many others--especially the alt.net crowd and transplants from Ruby and Java--this represents the cat’s pajamas on the Microsoft web stack. (Of course, it’s amazing how quickly developers find problems in the latest programmer’s paradise--usually before its release--and I’m sure the MVC aficionados are already looking to the next release.) The basic idea behind ASP.NET MVC is to separate out the three concerns of the MVC pattern: the model, view, and controller. The model represents the data model, the view is the user interface that presents the data and interacts with the user, and the controller is the command center that takes inputs from the view, pushes and pulls data into/from the model, and decides what to do next. By separating out these concerns (as purely as possible), you improve the ability to create unit tests for your applications and, at least on some level, improve application maintainability. If you are into test driven development, then this is the pillar to hook your horse to. 4. Dynamic Data. IMO, Dynamic Data is a misnomer. From its name, one would tend to think this is yet another data access technology from Microsoft. It is not. I would have preferred Microsoft to use the name Dynamic Application or Dynamic Application Builder. MSDN Help says, “ASP.NET Dynamic Data is a framework that lets you create data-driven ASP.NET Web applications easily.” You start with Dynamic Data by creating a database and then using either LINQ to SQL or the Entity Framework to create a model over your data. Next, create a new project or web site in Visual Studio 2008 (with SP1) using one of the Dynamic Data Web Application templates, make a fairly simple change to the web site’s Global.asax, and Visual Studio builds a dynamic ASP.NET application over your data model. The resulting site support the creation, retrieval, updating, and deletion (commonly referred to as the CRUD operations) of rows in the underlying database. Dynamic Data uses what is termed dynamic scaffolding to construct your application. This means if you change your data model, the application will be instantly updated: there’s no code that needs to be re-generated. Dynamic Data is customizable. Thus, if you don’t like how Dynamic Data presents a datatype or a particular field, or want to change how it performs validation of a field, you can change the templates behind these. You can also change the master page used as well as a number of other details. Kudos to Microsoft for Dynamic Data--even though I hate the name. Just realize that this pillar, unlike the other three, is only applicable to a subset of ASP.NET applications that fit in well with the Dynamic Data approach to applications: applications, which are primarily data-centric, intranet-based applications. That said I could see many places where I might use Dynamic Data, though I am still trying to work through the security implications of opening up the database for pretty much unrestricted reading and writing of database tables. (For those who would like to see an MVC version of Dynamic Data: I’d expect to see such a beast come down the pike at some point from Microsoft.) Conclusion So what does this mean to the developer? I have both good and bad news. The good news is that you now have a choice in how you develop your ASP.NET applications. If you love the responsiveness and coolness of AJAX or you need to scale your applications big-time, then you’ll love ASP.NET AJAX. If, OTOH, you are into unit tests or TDD, you will love ASP.NET MVC. Finally, if you were looking for an easier way to build you basic CRUD web application, you’ll want to take a look at Dynamic Data. Don’t need any of the above? There’s no need to despair--ASP.NET Web Forms are here for the long haul! The bad news is that you now have a choice in how you develop your ASP.NET applications. (Wait a second, wasn’t that also the good news?)This means you have more things to learn. It also means that in trying to support all four pillars, Microsoft may be taking some of its focus off Web Forms. After all, there’s only so many PMs and devs at Microsoft on the ASP.NET team. Furthermore, this means that if you are a manager like me worried about maintainability of applications and hiring developers, your job just got more difficult because one person’s idea of ASP.NET development is not necessarily someone else’s. Still, I think the good news here, outweighs the bad. Microsoft used to present ASP.NET Web Forms vs. MVC as a choice between a car and a motorcycle. Both will get you to your job , but some (the majority of the population, I might add) prefer driving a car, while a sizable minority love their motorcycles which give you better gas mileage and independence, but don’t protect you in the rain. To stretch this analogy to its breaking point, let me suggest that ASP.NET AJAX is like riding a bicycle to work (lean and mean, best gas mileage, but it requires you to exercise for your commute and exposes you to the elements like the motorcycle) while Dynamic Data is like taking the bus to work (let metro do the driving for you.) What about Silverlight? Silverlight is really just the next generation of a one-click desktop application, that is hosted in the browser and runs on a multi-platform .NET virtual machine. Not to take anything away from Silverlight, but it’s not ASP.NET. Does this mean that ASP.NET Web Forms is going away? Although some people “in the know” have stated in the past couple of years that either ASP.NET AJAX or ASP.NET MVC was the future of ASP.NET, this is not the official position of Microsoft. Think about it, since 98%--yes, this is a total educated guess--of ASP.NET development is still done using ASP.NET Web Forms, and this percentage is unlikely to change significantly over the next several years, Microsoft would be stupid to kill off Web Forms. It doesn’t make any economic sense, and since Microsoft is a for-profit entity, I think Web Forms will be a major thrust of ASP.NET for many years to come. In my opinion, Microsoft has added the three new pillars for the following reasons: · Microsoft is always trying to follow innovation and buzz in the industry. Three years ago, that buzz was in AJAX. Today, MVC, separation of concerns, and TDD is all the rage so Microsoft has countered with ASP.NET MVC. · Microsoft is always trying to broaden the appeal of ASP.NET. Microsoft has been trying to woo the open source community for years (after initially discounting it). And in order to do this, it needs to embrace many of the ideals of this community, which tends to live on the bleeding edge and is into AJAX, testability, and TDD, amongst other things. · Microsoft truly wants to improve the productivity of its customers. After all, if you improve the ability of your customers, in our case corporate and independent developers, to get their jobs done, you’ve gone a long way to attracting and retaining customers. In Microsoft’s eyes (and many of its customers), this is a win, win situation. I, for one, would like to thank Microsoft for the four pillars of ASP.NET. In producing the four pillars, Microsoft has given ASP.NET developers the choices they both want and need.

JSF has a simple Inversion-of-Control (IoC) container called the JSF Managed Bean Facility (MBF). Although it has a verbose XML syntax, and is not as robust as the Spring BeanFactory, PicoContainer, or the JBoss Microcontainer, the MBF does have the basics of an IoC container, and offers features like dependency injection. When a POJO is managed by the JSF MBF, it is typically referred to as a managed-bean. But if you're going to create a maintainable JSF webapp/portlet, it is necessary to distinguish between different kinds of managed-beans. This practice will also preserve the clean separation of concerns that JSF provides by implementing the Model-View-Controller (MVC) design pattern: Managed-Bean Type Nickname Typical Scope Model Managed-Bean model-bean session Description: This type of managed-bean participates in the "Model" concern of the MVC design pattern. When you see the word "model" -- think DATA. A JSF model-bean should be a POJO that follows the JavaBean design pattern with getters/setters encapsulating properties. The most common use case for a model bean is to be a database entity, or to simply represent a set of rows from the result set of a database query. Backing Managed-Bean backing-bean request Description: This type of managed-bean participates in the "View" concern of the MVC design pattern. The purpose of a backing-bean is to support UI logic, and has a 1::1 relationship with a JSF view, or a JSF form in a Facelet composition. Although it typically has JavaBean-style properties with associated getters/setters, these are properties of the View -- not of the underlying application data model. JSF backing-beans may also have JSF actionListener and valueChangeListener methods. Controller Managed-Bean controller-bean request Description: This type of managed-bean participates in the "Controller" concern of the MVC design pattern. The purpose of a controller bean is to execute some kind of business logic and return a navigation outcome to the JSF navigation-handler. JSF controller-beans typically have JSF action methods (and not actionListener methods). Support Managed-Bean support-bean session / application Description: This type of bean "supports" one or more views in the "View" concern of the MVC design pattern. The typical use case is supplying an ArrayList to JSF h:selectOneMenu drop-down lists that appear in more than one JSF view. If the data in the dropdown lists is particular to the user, then the bean would be kept in session scope. However, if the data applies to all users (such as a dropdown lists of provinces), then the bean would be kept in application scope, so that it can be cached for all users. Utility Managed-Bean utility-bean application Description: This type of bean provides some type of "utility" function to one or more JSF views. A good example of this might be a FileUpload bean that can be reused in multiple web applications. Now... One of the main benefits in making fine distinctions like this is loose coupling. What's that you ask? Well let's first take a look at an example of tight coupling, where MVC concerns can be smashed/confused into a single managed-bean: public class ModelAndBackingAndControllerBean { private String fullName; // model-bean property private boolean privacyRendered; // backing-bean property // model-bean getter public String getFullName() { return fullName; } // model-bean setter public void setFullName(String fullName) { this.fullName = fullName; } // backing-bean getter public boolean isPrivacyRendered() { return privacyRendered; } // backing-bean setter public void setPrivacyRendered(boolean privacyRendered) { this.privacyRendered = privacyRendered; } // backing-bean actionListener for UI support logic public void togglePrivacySection(ActionEvent actionEvent) { privacyRendered = !privacyRendered; } // controller-bean business logic public String submit() { System.out.println("fullName=" + fullName); return "success"; } } The problem here is that the bean would have to be kept in session scope because of the model-bean property. Additionally, what if we wanted to do some unit testing with mock model data? Can't do it. So in order to fix these problems, and to promote loose coupling, we would have three separate Java classes: public class ModelBean { private String fullName; public void setFullName(String fullName) { this.fullName = fullName; } public String getFullName() { return fullName; } } public class BackingBean { private boolean privacyRendered; public void setPrivacyRendered(boolean privacyRendered) { this.privacyRendered = privacyRendered; } public boolean isPrivacyRendered() { return privacyRendered; } public void togglePrivacySection(ActionEvent actionEvent) { privacyRendered = !privacyRendered; } } public class ControllerBean { private ModelBean modelBean; public ModelBean getModelBean() { return modelBean; } public void setModelBean(ModelBean modelBean) { // Dependency injected from the JSF managed-bean facility this.modelBean = modelBean; } public String submit() { System.out.println("fullName=" + getModelBean().getFullName()); return "success"; } } Now that the beans are found in different classes, they can all be kept in their appropriate scopes. The model-bean can be kept in session scope, and the backing-bean and controller-bean can be kept in request scope, thus saving memory resources on the server. Finally, we can use the dependency injection features of the JSF MBF in order to inject the model-bean into the controller-bean. This can be seen in the following WEB-INF/faces-config.xml example, where the #{modelBean} Expression Language (EL) binding is used: modelBean myproject.ModelBean session backingBean myproject.BackingBean request controllerBean myproject.ControllerBean request modelBean #{modelBean} From http://blog.icefaces.org/

Rules are statements that define business procedures and policies. Normally, every business application contains many embedded business rules that determine business process flow and execution. When we program these rules with static languages like Java, there is no problem at all. However, the problem is that some business rules change over time very frequently. It is very expensive to fix these changing parts since it requires new versions and software development cycles; develop, deploy on a test server, test, package, deploy on a production server etc. Here “Dynamic Programming” comes to our rescue. Our program doesn’t have to necessarily be written with a dynamic language. We can use this advantage in Java with Rule Engine mechanism. Rule Engines don’t serve only the dynamic programming. They evaluate rules and extract some inferences by using some algorithms like RETE. This inference mechanism is hardly possible with many “if-else” statements in normal program code. Rule Engines have generally 4 types of rules; Constraint Rule: We may need to define restriction or limits for some entities. “A customer sales order amount must be lower than 100 if the plant is over capacity ” Validation Rule: Some validation rules may change over time. “Production order can’t be accepted if current day is Sunday” Action Rule: Some actions may be triggered. “Send a notification e-mail if a purchase order is over $1000 to boss”. Computation Rule: Some formula may be executed. “Discount is 20% if customer is Mr. Anderson” We can use Rule Engines on the market but even we can write our own Rule Engine to benefit the dynamic programming in Java. By using Java compilers, classloaders, it is not so much hard. My favorite rule definition format is Java source code instead of some natural languages or XML. This can also reduce the overall development cost of your custom rule engine. By writing Java rule definitions, we can also use the IDE debugger features without learning a new syntax. If we develop our own Rule Engine, we may skip the inference engine part since we only want dynamic programming feature for now. Best place for plugging rule execution is database access layer. Rules should work invisibly behind the database objects. In some cases, some rules might be invoked from static program codes. The benefits that we have when using Rule Engine are: Dynamic programming with hotswapping feature that provides fix&run easiness. Minimal cost of business rule changes that occurs many times. Accumulation of business rules in one place that may result re-usability and code repetition prevention. Dynamic validations that may change over time. Constraints can be defines for default values, assertion checks etc. In any time rule may be turned off/on. Some of the features of Rule Engine may remind us database constraints and triggers. Using database may hinder portability and require DBA involvement. I think Rule Engine is better for business rule definitions. These database features may be used for other purposes. The critical question we should ask ourselves is which part of business rules should be taken into Rule Engine from the programs. The answer is “frequently changing” rules. Some changing parts can be defined within parameter tables (processing options) but we can’t define computation or many if-else statements in these tables. But we should prefer parameter table if it is enough for the problem. Links: http://java.sun.com/developer/technicalArticles/J2SE/JavaRule.html http://www.javaworld.com/javaworld/jw-04-2005/jw-0425-ruleengine.html http://www.javaworld.com/javaworld/jw-06-2006/jw-0612-dynamic.html http://www.amazon.com/Principles-Business-Addison-Wesley-Information-Technology/dp/0201788934/ref=sr_1_3?ie=UTF8&s=books&qid=1236927606&sr=1-3 http://www.amazon.com/Business-Rules-Applied-Building-Approach/dp/0471412937/ref=sr_1_1?ie=UTF8&s=books&qid=1236927606&sr=1-1

Each day this week will feature a new article with an in-depth look at the creation process behind setting up a Java project and implementing the frontend with Apache Wicket. Wicket is a Java web application framework which allows “Designers” (people good with Dreamweaver) and “Developers” (people good with Java and Databases) to collaborate on a project with minimal chances of stepping on each other’s toes or wearing each other’s hats. The beauty of Wicket is that it uses plain xhtml pages as it’s templating markup. This means that html pages can be loaded into Dreamweaver (or whatever tool the Designer is comfortable with) and they will look very close to the same as they would when rendered on the deployment web server. Workflow The basic workflow involved in creating and maintaining html rendered by Wicket is as follows: The Designer creates the html for the website and fleshes it out with “mock” sections. For instance in the application we intend to create during our Five Days of Wicket will be a pastebin application called “Mystic Paste”. In our application we’ll have an “Add Code to Mystic Paste” page, mock data might include some user created content in the textarea of the page. All css/images, etc… are setup such that if they were to be put on a webserver, everything would work. The Developer needs to flesh out the dynamic areas of the webpage, that is, he needs to instruct Wicket where it will need to show information from the server. The developer does this by decorating the designer’s html page with special Wicket tags and attributes. Because these tags and attributes are just considered part of another namespace separate from xhtml’s, editors like Dreamweaver and browsers will simply ignore them - It is important to note: The developer will still keep the “mocked” sections of the page intact, this is so the page renders and looks fleshed out on its own. The mocked sections will be replaced by real data when rendered by Wicket. The Developer hands the file back to the Designer. The Designer is free to make further edits, so long as he/she does not remove or manipulate the Wicket tags and attributes present in the file. If the Designer does need to remove any Wicket tags or attributes, they need to consult the Developer as such an action will “break” the webpage when Wicket renders it. Example Wicket Page Here is an example of a Wicket page. This example was taken from Manning Publishing’s book “Wicket in Action”: ... Gouda Gouda is a Dutch... $1.99 add to cart Emmental Emmental is a Swiss che... $2.99 add to cart ... This looks almost 100% like a normal webpage would look, the only difference is the addition of the “wicket:XXX” attributes and tags sprinkled through the document. The parts of the document using the special Wicket namespace modifiers will be replaced/removed in the final markup when Wicket renders the page to the user’s browser. Notice the “” element? This is where your designer can put a “mocked” version of what that area of the page should look like. You as a developer can take that mocked html and divide it out into a template that is dynamically driven from the backend. Here is how the final page looks if you were to simply load the page into a web browser (or Dreamweaver) from your hard drive: Preparing for Setup Deviating a bit from the Standard Wicket Convention One of the first things a developer notices when starting out with Wicket is the convention where Wicket likes having its html template files live at the same level and in the same packages as it’s Java source files. Sure you can jump through hoops to get Wicket to load the html template files from elsewhere but a nice compromise is to simply keep your html template files within the same package directory structure but in a source folder separate from the Java classes. Why? Well quite simply to keep your designers (Dreamweaver folks!) from having to grab Java source files along with the html files they are working on. It will just confuse them and clutter their directories. You can of course stick with the typical “Java source files along side html” convention if you wish, but I find it much cleaner to separate them during design time, and have Maven combine them only at build time into the target war (which it will gladly do automagically). Project Folder Structure . |-- pom.xml |-- src | `-- main | |-- filters | |-- java | | `-- com | | `-- mysticcoders | | `-- mysticpaste | | |-- model | | |-- persistence | | | |-- exception | | | |-- hibernate | | |-- services | | `-- web | | |-- ajax | | |-- pages | | |-- panels | | `-- servlet | |-- resources | | |-- com | | | `-- mysticcoders | | | `-- mysticpaste | | | |-- spring | | | `-- web | | | |-- ajax | | | |-- pages | | | `-- panels | | `-- log4j.properties | |-- sql | `-- webapp | |-- WEB-INF | |-- images | |-- js | `-- css src/main: maven builds source and resources from this directory to the main deployable target (i.e. our war file) filters: we keep a set of “filters” files that maven can use to interpolate variables at build time. What does this mean? It means that inside your configuration files, the files you use to setup database connections or file paths, you can insert variable place holders like ${db.host}. When maven does a build, it looks up the correct filter file to use and looks for the key=value part corresponding to “db.host” and inserts it into the configuration file for you. This ensures that you are able to configure your application per environment you deploy to (i.e. DEV, QA, PROD, etc…) by having different filter files with the same keys but different values. For more information see Maven’s documentation on filtering resource. java/*: this folder will contain all of the application’s source code. Everything from the database access code, wicket code and services code for the mysticpaste application (see below). model: all “domain” classes, that is, classes that represent the objects in the application. For mysticpaste you’ll see classes like “PasteItem” which represents an item pasted to the mysticpaste. persistence: at this level of the persistence package a list of interfaces will be kept. The interfaces comprise the basic access layer the services layer will use to save, retrieve and update items to/from the paste bin. exception: the peristence layer needs to tell the services layer when things have gone wrong. It does this via delcaring and throwing exceptions. hibernate: such is our case, our persistence interfaces will be implemented via the ORM known as Hibernate. This package will store all of the custom hibernate implementations and hibernate specific classes services: The services layer will be stored here. Both the generic interfaces and their implementation classes. The persistence layer will be injected via spring. web: this folder is where our Wicket classes will reside and it’s split into several category packages which are as follows: ajax: mysticpaste uses Ajax to render portions of its UI. The wicket classes which render the xml/html to be injected dynamically into the page are stored here. pages: standard Wicket page classes which are used throughout the application are stored here panels: reusable panel classes are stored here. Panels may be included within Wicket pages for sake of templating servlet: any run of the mill servlet code we need is stored here. A good example might be an ImageUploadServlet resources/*: the resources folder will hold our non-java based files. Noteably html files and spring confguration files spring: Holds any spring configuration files needed to wire the services and persistence layer web: this folder and all subfolders mirror the packages under src/main/java/…/web and hold the .html files that the Wicket page/panel classes use as their templates. As described above, a “standard” wicket application simply stores the .html files along side their Wicket classes under src/main/java/…/web, however we want to keep these files separate from the Java source so as to keep the directory our designers checkout from version control contianing only the files they need to work on. sql: any sql scripts we need to keep handy for building the mysticpaste database. webapp: this folder will keep the files which live at the base directory of our war file WEB-INF: where you keep your web.xml file images: any image resource, .gif/.png/.jpg files your webapp will reference js: javascript files your webapp will reference css: style sheets your webapp uses src/test/*: All files which reside under this folder are test classes and resources needed to support the tests. Maven will build everything under src/main/java and add it to the class path of the JUnit or TestNG classes you create. java: JUnit or TestNG test classes which will be run during a build resources: resource files which are needed to support the tests Getting Started Since we are using Maven as our build tool we can take advantage of the fact that the fine folks at the Wicket project have created a specialized “archetype“ which creates a skeleton web application complete with a folder structure which mimics roughly what we have outlined above and Maven pom.xml file used to build a war. The Wicket contributors have even gone one step further and have created a little web page which will, based off a few drop down options, generate the maven command you need to execute in order to create the boiler plate Wicket project. You can find this web page over on the Apache Wicket site under the “Quick Start” link. Copying the above Maven command creates a Skeleton Wicket Project To be precise, the command I used was: mvn archetype:create -DarchetypeGroupId=org.apache.wicket -DarchetypeArtifactId=wicket-archetype-quickstart -DarchetypeVersion=1.3.5 -DgroupId=com.mysticcoders -DartifactId=mysticpaste And I ended up with the following folder structure: . `-- mysticpaste |-- pom.xml `-- src |-- main | |-- java | | `-- com | | `-- mysticcoders | | |-- HomePage.html | | |-- HomePage.java | | `-- WicketApplication.java | |-- resources | | `-- log4j.properties | `-- webapp | `-- WEB-INF | `-- web.xml `-- test `-- java `-- com `-- mysticcoders |-- Start.java `-- TestHomePage.java Now obviously we’ll have to rearrange a few things, for instance I want my base package to be com.mysticcoders.mysticpaste, but that’s easy enough to do once we are in an IDE. For now, let’s test this example webapp out and see if it works. To do that switch into the mysticpaste directory (the directory that has pom.xml in it) and type the following: mvn jetty:run This will start up a Jetty webapp container running on port 8080 (if you have something running there already, use the -Djetty.port= option). Startup a webbrowser and navigate to http://localhost:8080/mysticpaste/ You should see: Your IDE Sooner or later you’re going to want to crack open your IDE and start hacking away. Maven makes this extremely easy by allowing you to create IDE specific project files based off of the Maven pom.xml file. Eclipse mvn eclipse:eclipse For eclipse you’ll also have to set the M2_REPO classpath variable for the workspace your project resides under. Do this by entering the following command: mvn -Declipse.workspace= eclipse:add-maven-repo IntelliJ IDEA mvn idea:idea -OR- in IDEA 7+ simply open the pom.xml file Netbeans Netbeans supports maven out of the box, just “Open Project” and choose the mysticpaste directory that contains the pom.xml file When generating the project files through Maven, the project is setup such that classpath entries point to your local Maven repository (i.e. ~/.m2/repository, or C:Documents and Settingsyourusername.M2repository on Windows). It also sets up src/main/java, src/main/resources as “source folders”. You may add other folders to the source folder list as per your IDE if needed, the only thing you have to remember is if you ever use mvn eclipse:clean followed by mvn eclipse:eclipse again, those other source folders will have to be readded through your IDE. Instead, you should add the source/resource folders directly to your pom.xml, this way they will be maintained. Spring The Mystic Paste application will use Spring, and really you should too. Unless you have been hiding under a rock or work in a corporate environment so lame as to which technologies newer than 2002 are forbidden you should learn to accept Spring as a defacto standard. Dependency injection for the win! We add the following to our pom.xml: org.apache.wicket wicket-spring-annot org.springframework spring org.springframework spring-test org.springframework spring-tx wicket-spring-annot: allows us to wire our Wicket application via handy dependency injection annotations (i.e. @SpringBean, see Wicket documentation for more detail) spring: is just the core spring libraries spring-test: is a set of Spring integration classes for Unit testing spring-tx: is the Spring Transaction Management api for declarative transactions web.xml additions for Spring In order for Spring to manage our Wicket application we need to setup the Wicket filter with a Spring-aware application factory. This allows us to wire our Wicket Application class in our applicationContext.xml file, which is really handy if you have a services and configuration settings you want to inject into the Wicket Application object so the rest of your application can access them. To do this, we change the original Wicket filter like so: wicket.mysticpaste org.apache.wicket.protocol.http.WicketFilter applicationFactoryClassName org.apache.wicket.spring.SpringWebApplicationFactory As well, we want our Spring context to be available to our webapp if ever there is a need for one of our pages to access the Spring managed beans directly: contextConfigLocation classpath:com/mysticcoders/mysticpaste/spring/applicationContext.xml org.springframework.web.context.ContextLoaderListener Hibernate Hibernate is our ORM of choice, it will allow us to persist and retrieve our model objects to and from the underlying database, whatever that database may be. We add the following to our pom.xml: org.hibernate hibernate-annotations c3p0 c3p0 commons-dbcp commons-dbcp javax.transaction jta 1.0.1B hibernate-annotations: used so we can annotate our model classes with mapping information, instead of having to create a separate mysticpaste.hbm.xml file. c3p0: provides a connection pooling library Hibernate can use commons-dbcp: another connection pooling library, we’ll add it as well and decide whether to use it or c3p0 later jta: this is the Java Transaction API which is needed by Hibernate (Hibernate provides an implementation of the API) web.xml additions for Hibernate To have a Hibernate Session open and ready for our webapplication during a Request Cycle we need to setup a Hibernate filter like so (otherwise, good luck getting lazy loading working!): open.hibernate.session.in.view org.springframework.orm.hibernate3.support.OpenSessionInViewFilter open.hibernate.session.in.view /* As the comment states above, make sure this filter-mapping exists *before* your wicket.mysticpaste filter or else it just plain won’t work. Database For the Mystic Paste we decided to use the freely available PostgreSQL. Adding support for PostgreSQL is very easy, unlike with some of the commercial DBMSes where you have to download and install their JDBC driver into your repository. To add support for Postgres, we simply add the following to our pom.xml: postgresql postgresql Servlets Regardless of which webapplication framework you choose there are just some times when a plain jane Servlet comes in really handy. If you have a need for Servlets and the Servlet must have access to the Wicket session add the following to your web.xml: wicket.session org.apache.wicket.protocol.http.servlet.WicketSessionFilter filterName wicket.mysticpaste And then, after your other filter-mappings add the following (assuming you mount your servlet-mappings under /servlet/): wicket.session /servlet/* Maven Filters and Profiles In order to build our Mystic Paste project for various environments (DEV/QA/PROD) we need to implement both Maven profiles and filters. Filters Filters allow you to place variables inside your configuration files and have those variables filled in durring build time. This is very handy for setting environment specific things such as database connection information. Enabling filters is quite easy, we open up the pom.xml file and find the section for and set the value for the element to true as follows: true src/main/resources . . . But for filtering to work, we need to specify a filters file. It’s not enough to specify only one filter file because we need to specify different filters per environment and we’ll do that by using Maven Profiles. Profiles To setup a profile, create a new set of elements following the section in your pom.xml file. Like so: DEV DEV DEV QA QA PROD PROD and just above your tag underneith your tag you would add the following elements: mysticpaste src/main/filters/filters-${env}.properties true src/main/resources src/main/filters will contain the following files. |-- pom.xml |-- src | `-- main | |-- filters | | `-- filters-DEV.properties | | `-- filters-QA.properties | | `-- filters-PROD.properties filters-DEV.properties jdbc.url=jdbc:postgresql://localhost:5432/mysticpaste jdbc.user=mysticpaste jdbc.password=password image.upload.path=/tmp/pasetbin/userimages image.upload.size.limit=4096K filters-PROD.properties jdbc.url=jdbc:postgresql://192.168.2.10:5432/mysticpaste jdbc.user=mysticpaste jdbc.password=CrYp71c image.upload.path=/mysticpaste/userimages image.upload.size.limit=4096K Now within any file under src/main/resources that has variables of the form ${variable.name} will have those variables replaced with the values specified in the proper filters file located under src/main/filters. For instance here is an example of a Spring applicationContext.xml file which will be interpolated with proper variables values at build time: applicationContext.xml To determine which filters file will be used depends on the profile chosen when building. For example, to build to production using the filters-PROD.properties we would execute the following: mvn clean deploy -P PROD The profile you use with the -P switch must match one of the values of the element for a profile. Conclusion Although it’s quite easy to get started with the Maven QuickStart project it is sometimes a bit frustrating putting some of the additonal pieces together. Building to several environments, setting up depenencies not included in the QuickStart project and strucuturing your project in an effort to make life easy for yourself as a developer and for your designer. I hope our Day 1 tutorial leaves you with a good sense of how a Wicket project is setup, now we can move onto coding the app! In the next four days we will be covering: Writing the Tests Designing the backend Designing the Wicket components Putting it all together Mystic Coders, LLC has been coding web magic since 2000. Mystic is a full-service Development Agency specializing in Enterprise development with Java. They are usually involved in developing enterprise-grade software for companies large and small, and have experience working in diverse industries, including b2b, b2c, and government-based projects. Mystic has done work with large companies such as LeapFrog, Nestlé, Harrah's Entertainment and the Los Angeles Conventions & Visitor's Bureau, among others. Andrew Lombardi, CTO of Mystic, is available for speaking engagements. For more about Mystic, check us out at http://www.mysticcoders.com

The industry is recognizing the fact that performance testing & engineering should be part of the project execution road map starting from the requirements gathering phase. At many times during project executions, performance engineering related activities are executed based on customer need or slow response time of application after development phase gets completed. Glassbox can be leveraged (by developers/testers/business users) during and after the development cycle to monitor the response times of requests with-out being aware of underlying application structure and code details. Analysis generated by Glassbox gives direct pointers on where is the bottleneck which causes slow response time for that particular request/page/URL. About Glassbox Glassbox is an open source web application which aid in performance monitoring and troubleshooting of multiple web applications deployed in container. Troubleshooting It contains the built-in knowledge repository of common problems which are used to pinpoint the issues and suggestions on causes as Java code executes. Performance Monitoring It monitors the requests as Java code executes and provides details about response times. Glassbox web client (AJAX GUI) provides nice summary dashboard view which contains various attributes like (server-name, application name, operation/request-URL, average time, no. of executions, status (slow / OK) and analysis details). By default, an operation that takes more than 1 sec execution time is marked as SLOW status. Such SLA can be modified using Glassbox properties file. Analysis part describes the problem precisely and very clearly in plain English words, rather than displaying large code/exception trace. This definitely increases developer productivity by reducing developer’s time spent in log files and using IDE debuggers. Internals The two main components of Glassbox are Monitor and Agent. Monitor uses Aspect-Oriented Programming (AOP) to monitor the JVM activity. Agent diagnoses and presents the monitoring results and uses knowledge repository to cross reference the problem with suggestions/solutions. Glassbox agent supports viewing of the analysis results using JMX (eg. Java 5 JConsole) Consoles. Glassbox extensively uses the AOP approach internally to monitor the Java code. This gives the benefit of not making any changes to source code or build-process and hence can work with any legacy web application/jar file as well. Technologies Glassbox should work on any application server that supports Servlet 2.3 or later. The servers where Glassbox is tested and installation process is automated are Apache Tomcat, weblogic, websphere, Resin, Oracle OC4J, websphere, Resin, Jetty & GlassFish. Overhead Having Glassbox application running on same container would generate a performance overhead. Typically this would affect the response time and memory overhead. Hence it is recommended to start the Glassbox application only when it’s required for performance monitoring. Licensing Glassbox is an open source project, it is free to download and run. Glassbox uses the GNU Lesser General Public License to distribute software and documentation. Demo Application Development & Deployment to Tomcat To test the capabilities of Glassbox, a sample application is developed which has a TestServlet class. This servlet calls DelayGenerator class’s generateDelay() method. This method calls Thread class’s sleep() method which suspends the execution of servlet. A counter is being initialized in DelayGenerator class which determines the time interval till which servlet is needed to be suspended. TestServlet.java /** * File: TestServlet.java * @author Viral Thakkar */ package com.infosys.star.glassbox; import java.io.IOException; import java.io.PrintWriter; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; public class TestServlet extends HttpServlet { protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { DelayGenerator delayObj = new DelayGenerator(); int delay = delayObj.generateDelay(); response.setContentType("text/html"); PrintWriter out = response.getWriter(); out.println(""); out.println(" Hello World from Test Servlet : "+delay+" milliseconds "); out.println(""); out.flush(); } } DelayGenerator.java /** * File: DelayGenerator.java * @author Viral Thakkar */ package com.infosys.star.glassbox; public class DelayGenerator { private static int counter = 1; public int generateDelay() { try { Thread.sleep(counter * 100); counter++; } catch (InterruptedException e) { e.printStackTrace(); } return counter*100; } } Glassbox Installation & Integration to Apache Tomcat 6.0 Glassbox installation is very straightforward for non-clustered environment for the server where it’s automated. Simply drop the glassbox.war file at the appropriate folder inside server folder or perform the server specific steps/configuration to deploy the war file. Browse to server url with context root as glassbox – http://<>:/glassbox. Follow the instructions available on this page. According to specific server, this page would suggest the configuration changes for a server. Please refer to Glassbox User Guide document for details on how to install Glassbox for clustered application server environment. For Apache Tomcat 6.0- Add following command line arguments to Tomcat’s Java options: -Dglassbox.install.dir=C:\Tomcat6.0\lib\glassbox -Djava.rmi.server.useCodebaseOnly=true -javaagent:C:\Tomcat6.0\lib\aspectjweaver.jar Monitoring & Technical Analysis Glassbox web client (URL- http://<>:<>/glassbox ) shows the summary and detailed view of all the requests/operations that container/JVM has executed. Summary Section View Different attributes (columns) which gets displayed in this table are as below - Attribute / Column Name Comments Status This indicates whether operation/request is performing OK, SLOW or FAILING Analysis For SLOW/FAILING status, this value provides the small summary of the cause of the problem. Operation This is name of the operation/request of an application Server Name of the server where monitoring is being done. In a clustered environment, this allows to distinguish operations on different servers. Executions This value indicates how many times this operation has run since the application server was started or Glassbox’s statistics were last reset. Click the request in above summary table to view its detailed analysis in below detailed section. Detailed Section View The details area provides information relating to operations selected in the summary table. Different sub-sections which gets displayed in this view are as below - Sub-section Name Comments Executive Summary High level summary view of the selected operation gets displayed in a table format. This is neat view to senior stake holders who are not interested in technical details. Technical Summary This section contains more technical details in paragraph and table representation formats to provide insight into root cause of the problem if any, like which operation, query is slow and statistics of same. Details like stack trace, thread lock name are provided to find and fix the problem. “Common solutions” sub section shows pointers to resolve the identified problem/s. “Glassbox has ruled out other potential problems” sub section saves time to know what problems have already been ruled out. Executive Summary View Technical Summary -> Technical Details Views Above two snapshots are parts of the Technical Details section and provide minute details at code level with line number so as to pinpoint where the problem is. Here cause is identified at Class com.infosys.star.glassbox.DelayGenerator inside Method generateDelay at line number 12 where Thread.sleep is invoked. Perform Load Testing Using JMeter and Monitor Using Glassbox Apache JMeter is used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. It can be used to make a graphical analysis of performance or to test server/script/object behavior under heavy concurrent load. Using JMeter, create a test plan that simulates 10 users requesting for 1 page 5 times. i.e. 10 x 1 x 5 = 50 HTTP requests. First step is to add a Thread Group element. The Thread Group tells JMeter the number of users to simulate, how often the users should send requests, and the how many requests they should send. Next step is to add HTTP Request element to added Thread Group. In parallel, have the Glassbox up and running to monitor response time statistics of the load generated by JMeter application. Below is the Executive summary view of above test in Glassbox web UI interface. Section “Monitoring & Technical Analysis” contains the details to understand the Glassbox generated analysis. Conclusion Glassbox is not the replacement for performance testing tool like load runner. Glassbox aids in the project to various stakeholders in finding, conveying and fixing the performance problems at all phases starting build (development) to post deployment. Glassbox application to be started/installed only during monitoring time so as to avoid the performance overhead for other applications due to CPU & memory footprint occupied by Glassbox application on the container. During load testing of the application, Glassbox turns out to be good option to figure out the root causes inside an application code. References Glassbox web site - http://www.glassbox.com/glassbox/Home.html Glassbox User Guide - http://nchc.dl.sourceforge.net/sourceforge/glassbox/Glassboxv2.0UserGuide.pdf Apache JMeter - http://jakarta.apache.org/jmeter/ Download & Support Glassbox Download Link - http://www.glassbox.com/glassbox/Downloads.html Glassbox forum Link - http://sourceforge.net/forum/forum.php?forum_id=575670 About Author Viral Thakkar is a Technical Architect with the Banking and Capital Markets vertical at Infosys. He has 9.5 years of technology consulting experience mainly on Java/JEE technologies and frameworks with large banks and financial institutions across the globe. He has been part of many small and large-scale initiatives related to application development, architecture creation and strategy definition. From http://viralpatel.net/blogs

Java:SWT: click on table column header to sort table package ca.freewill.swt.test; import java.util.Arrays; import org.eclipse.jface.viewers.TableViewer; import org.eclipse.jface.viewers.Viewer; import org.eclipse.jface.viewers.ViewerComparator; import org.eclipse.swt.SWT; import org.eclipse.swt.events.SelectionAdapter; import org.eclipse.swt.events.SelectionEvent; import org.eclipse.swt.widgets.Table; import org.eclipse.swt.widgets.TableColumn; public class TableSorter { private final TableViewer tableViewer; public TableSorter(TableViewer tableViewer) { this.tableViewer = tableViewer; addColumnSelectionListeners(tableViewer); tableViewer.setComparator(new ViewerComparator() { public int compare(Viewer viewer, Object e1, Object e2) { return compareElements(e1, e2); } }); } private void addColumnSelectionListeners(TableViewer tableViewer) { TableColumn[] columns = tableViewer.getTable().getColumns(); for (int i = 0; i < columns.length; i++) { addColumnSelectionListener(columns[i]); } } private void addColumnSelectionListener(TableColumn column) { column.addSelectionListener(new SelectionAdapter() { public void widgetSelected(SelectionEvent e) { tableColumnClicked((TableColumn) e.widget); } }); } private void tableColumnClicked(TableColumn column) { Table table = column.getParent(); if (column.equals(table.getSortColumn())) { table.setSortDirection(table.getSortDirection() == SWT.UP ? SWT.DOWN : SWT.UP); } else { table.setSortColumn(column); table.setSortDirection(SWT.UP); } tableViewer.refresh(); } private int compareElements(Object e1, Object e2) { IColumnContentProvider columnValueProvider = (IColumnContentProvider) tableViewer.getContentProvider(); Table table = tableViewer.getTable(); int index = Arrays.asList(table.getColumns()).indexOf(table.getSortColumn()); int result = 0; if (index != -1) { Comparable c1 = columnValueProvider.getValue(e1, index); Comparable c2 = columnValueProvider.getValue(e2, index); result = c1.compareTo(c2); } return table.getSortDirection() == SWT.UP ? result : -result; } } // Content provider must implement this interface package ca.freewill.swt.test; public interface IColumnContentProvider { Comparable getValue(Object element, int column); }

It all started with a task to do: Print all members of the group within Active Directory, including members of the nested groups. And a deadline: 15 minutes. Given the deadline, I had no chance to get it done in time. Having 15 minutes means you need to get it right from the first run. Googling for groovy ldap brought Gldapo. But after looking at it and seeing how much configuration has to be done, I searched for some alternatives. Groovy LDAP was beautifully simple and had no external dependencies. I downloaded the jar, dropped it into my GROOVY_HOME/lib directory and started to write the script: import org.apache.directory.groovyldap.LDAP ldap = LDAP.newInstance('ldap://ldap.mycompany.com:389/dc=mycompany,dc=com') After reading through the sample scripts, I already had the main part: ldap.eachEntry ('&(objectClass=person)(memberOf=cn=mygroup') { person -> println "${person.displayName} (${person.cn})" } I saved it as listGroup.groovy and ran it from the command line: groovy listGroup It worked out of the box, printing on the console all the members of the group: John Smith (smithj) Amanda McDonald (mcdonaa) Isabelle Dupre (duprei) Of course, the script was not printing members of the nested groups. In order to do that, I had to turn the snippet into the Groovy recurrent function and avoid hardcoding a group's name in favor of taking it as a command line parameter. Here is the entire script: import org.apache.directory.groovyldap.LDAP import org.apache.directory.groovyldap.SearchScope List getMembersOfAGroup(connection, groupName) { def members = [] def result = connection.searchUnique("cn=$groupName”); connection.eachEntry("memberOf=${result.dn}") { member -> if (member.objectclass.contains("group")) members.addAll(getMembersOfAGroup(connection, member.cn)) else members.add("${member.displayName} (${member.cn})") } return members } LDAP ldap = LDAP.newInstance("ldap://ldap.mycompany.com:389/dc=mycompany,dc=com") getMembersOfAGroup(ldap, args[0]).each { println it } If your directory contains circular group relations, the script has to be further adjusted. This detail has been omitted for simplicity reasons. Please note, that the examples in this article work only with Microsoft Active Directory, because they use vendor specific structure and schema elements. In other directory solutions for instance, group membership is often stored in group entries only, while in Active Directory it is stored in both group and member object. But the examples can easily be adjusted to fit another directory's solution, e.g. by modifying filter expressions. What is this LDAP thing you're talking about? LDAP 101: LDAP stands for Lightweight Directory Access Protocol. A directory is a storage organized as a tree of directory entries. The tree usually reflects political, geographical and/or organizational boundaries. Every directory entry consists of a set of attributes (name/value pairs). These attributes are defined in the LDAP schema. Each directory entry has a unique identifier named DN (Distinguished Name). For more information please read Apache Directory introductory article. Project background Groovy LDAP is a small library started by Stefan Zoerner from the Apache Directory project. Its goal was to create minimalistic LDAP API for Groovy, with metaphors understood by the LDAP community (e.g. members of the Apache Directory team). As such, the only two dependencies of Groovy LDAP are: Java SE (5 or later) Groovy 1.0 or later Under the hood, JNDI is used to perform LDAP queries, but fortunately Groovy LDAP hides it and lets you use a bunch of useful methods and objects, instead. It actually reminds me of the time when Netscape LDAP API was widely used. It defines a set of methods to perform basic LDAP operations: create, modify, delete, compare, search. Groovy LDAP is written in Java, not Groovy. The only Groovy dependency is a reference to a Closure class, which is used as a parameter in a couple of search methods. So with the exception of the method taking the closure, others can be also used in Java programs. How to get it The simplest way is to get the binaries from the Groovy LDAP download page. After downloading and expanding the zip file you need to look for groovy-ldap.jar in the dist directory. Drop it into your GROOVY_HOME/lib directory and you’re ready to write your first script. How to build it If you want to build the library on your own, you will need: Apache Ant 1.7.1 Ivy 1.4.1 or later After you download and install Ant, drop Ivy's jar (ivy-1.4.1.jar) into your ANT_HOME/lib directory. Now you can check out the source files from Apache Directory sandbox Subversion repository. Once the files are checked out, just type ant and wait until the distribution jar is built in the dist directory. Connecting to the directory The first thing you will want to do is to connect to the directory. Groovy LDAP offers here two types of connection: anonymous bind and simple bind. Anonymous bind happens when you connect to the directory without providing your credentials. Many directories allow anonymous bind if the client is only reading from the directory. In corporations anonymous bind is often disabled for security reasons. So, in order to connect you need to instantiate LDAP class using newInstance() method, with the following variants: public LDAP newInstance() public LDAP newInstance(url) A non-parameter method connects to the default address, which is localhost:389. It proves to be useful for various short proof-of-concept scripts. The second method takes the url of the directory as a second parameter. If anonymous bind is not allowed or not sufficient there is an equivalent method, taking additionally user credentials: public LDAP newInstance(url, user, password) Once the connection is established, you can perform any other actions. One tip is to always provide a baseDN as a part of the connection url e.g. ldap://ldap.mycompany.com:389/dc=mycompany,dc=com By doing so you define the default base, upon which searches will be performed, which in turn allows you to use convenient one parameter search methods, instead of specifying a search base and scope each time. Reading and searching directory entries You may want to start with checking if a specific directory entry exists: def found = ldap.exists('cn=smithj,dc=mycompany,dc=com') exists() method is searching the directory by DN (Distinguished Name) and returning a boolean result detailing whether an entry was found. As a companion there is read() method, that reads directory entry, specified by its DN: if (found) def entry = ldap.read('cn=smithj,dc=mycompany,dc=com') This method returns either a boolean value or a given entry, accordingly. But there might be cases when you do not want to search by DN, but by another attribute which is also unique. A good example of this is a userId attribute, which is usually unique within a company. def entry = ldap.searchUnique('userId=smithj') This method assumes uniqueness of an object. If more than one result is returned from the search, you will get an exception. When more results are expected, you can use search() method: and then iterate over a result set: results = ldap.search('(objectClass=user)') println 'Found: $results.size entries' results.each { entry -> println entry.dn } Searches can be also performed with more compact and more Groovy method eachEntry() taking a closure as the last parameter: ldap.eachEntry('(objectClass=user)') { entry -> println entry.dn } As you see, when you have the entry object, you can reference all its properties using native map syntax e.g. entry.dn. This is possible, because all result objects returned from Groovy LDAP search methods are Maps or Lists of Maps. But, how does Groovy LDAP know in which subtree you would like to perform your search? It doesn't, because you haven't specified anything else, but the basic query. So it assumed you want to search in baseDN (hopefully specified, when connecting to the directory). When you want to have more control over how the query is performed, there is a different version of search(), searchUnique() and eachEntry() methods that support it e.g. public List or Search class instance as parameters, but we'll leave them as for now. When you deal with LDAP directories as a part of your daily job, you may want to have a look at Apache Directory Studio, a full-fledged LDAP client tool, which allows you to connect, browse and modify any LDAP-compatible directory. It can also be used as diagnostic tool when your query in Groovy LDAP doesn't work as expected. Adding, modifying and deleting directory entries When you know how to search and read from the directory, it's time to do some modifications. Let's start from adding a new entry: def attributes = [ objectclass: ['top', 'person'], cn: 'smithc', displayName: 'John Smith' ] ldap.add('cn=smithc,dc=example,dc=com', attributes) add() method takes DN and a Map with attributes as parameters. You need to remember not to put DN in the attributes map, as it is not an attribute but rather the unique identifier of an entry. Removing a directory entry is even more straightforward: ldap.delete('cn=smithc,dc=example,dc=com') delete() method will throw an exception, if an object with the given DN does not exist. Modifying a directory entry is not very Groovyish for the time being. Adding single attributes is still relatively easy: def dn = 'cn=smithj,dc=mycompany,dc=com' def email = [ email: '[email protected]' ] ldap.modify(dn, 'ADD', email) Performing batch modifications could be more readable using Builder-like syntax.. The current way to do this is the following: def modifications = [ [ 'REPLACE', [email: '[email protected]'] ], [ 'ADD', [phone: '+48 99 999 99 99'] ] ] ldap.modify(dn, modifications) The same operation, using more expressive syntax, would potentially look like: ldap.modify ('cn=smithj,dc=mycompany,dc=com') { replace(email: '[email protected]') add(phone: '+48 99 999 99 99') } Summary As you can see, Groovy LDAP is a neat little library, delivering simple but convenient API to deal with LDAP directories, which makes it an ideal candidate to use in various administrator scripts and short programs. As a project it resides in Apache Directory sandbox, so when you have a chance, contribute and help Groovy LDAP to become an official subproject of the Apache Directory. Thanks I would like to thank Stefan Zoerner and Carolyn Harman for thorough review of the article. Resources Apache Directory Project Groovy LDAP Gldapo Apache Ant Apache Ivy Groovy