Data Mapper is one of the most advanced persistence-related patterns: an implementation of a Data Mapper stores objects (in general a whole object graph) in a database, and decouples the object model from the backend data representation, moving objects back and forth from the data store without introducing hardcode dependencies towards it. The database back end used by most of the implementations are usually relational: Object-relational mappers are some of the widely used tools today (and are even fading in some areas where different types of database are preferred.) Dependencies The interfaces for a Data Mapper can be put in the domain layer, but actual implementatons are in the category of infrastructure adapters and should be kept out of it, to promote the reuse and testing of domain layer classes without the need for a database back end or driver to be present. When this pattern is employed in an application, there are no more dependencies from the domain layer to external components, and no subclassing like in the Active Record case. Domain entities and value objects become Plain Old PHP Objects which do not extend anything (extends keyword) and do not need to reflect any database schema (if they are saved in a relational db), ensuring the maximum freedom of modelling to the developers. Different kind of implementations Early implementations of Data Mapper did not store an inner reference to a database connection or object that represent the link with the data store; in this case, result sets or some kind of raw data are passed to the Data Mapper, which reconstitutes the objects and encapsulates the process. Currently it is preferred to put all the references to the database as internals of the Data Mapper implementation (or in an abstraction layer under it). Anyway the Data Mapper hides as much as possible, like the type of the database and related knowledge, from the client code (domain layer or an upper one). The interface of the modern Data Mappers become from store() (insert and update) and remove() to one that comprehends also find() methods or a more complex system of querying; the implementation of querying is out of the scope of this pattern, but can be mixed up with it easily. A distinction in the implementations of Data Mapper is in their scope. A Data Mapper can be specific to a particular Entity/Aggregate Root (single class or class with composed objects), or a generic implementation can be customized with metadata (annotations, XML configuration) to work with different classes. Generic implementations are usually very complex, and specific ones may become much more easy to code due to simplifications. However, generic Data Mappers are prone to reuse and present less bugs than the project-specific ones, which were the only alternative in the last years. Issues The difficulties in implementing such a pattern are clear. Given a transaction, like an http request, the mapper has to keep track of the changed objects, and generate automatically the right DML queries to issue (SELECT, UPDATE, DELETE), in the right order and without leaving out any part of the modified data, avoid duplicating rows or update ones that do not exist anymore. This is a case of simple interface and complex implementation. To avoid breaking encapsulation, implementations usually employ reflection to access private fields of the object to store or that the mapper is reconstituting. Other possible choices for the data access are specific constructors for reconstitution or specific interfaces for domain mapping, but this solution still breaks encapsulation by providing to the client code methods that are not meant to be called, or fields that should not even be seen out of the objects but are actually accessed. This results in a unclear Api which may promote dependencies on persistence-related items. Providing metadata breaks encapsulation too, of course, but at least it is kept in the immediate so that it can change with the domain classes. Annotations are the preferred mean to specify metadata such as column names or relationships, and in PHP they are hidden in the docblock comments so that when the Data Mapper is not used they are just ignored. Data Mapper does not provide a total illusion (abstraction) of an in-memory collection of objects: the knowledge that there is some kind of external data store scatters into the application upper layers. Moreover, eventually some particular issue of the storage leaks into the object part of the application. As an example, consider the performance of queries, which is often the object of discussion when using object-relational mappers. Usually not all the object graph is instantiated as it may be very large; tuning how large the instantiated part will be is a trade-off which depends on the underlying database. Furthermore, generated queries may result very inefficient to the point that much of the client code must hint the joins to perform via the Api. Examples The generic Data Mapper Doctrine 2 (now in beta) is one of the few implementations in PHP of this pattern. As we've seen before in this article, specific implementations are dependent on the domain layer, so they are usually not reusable. A working copy of Doctrine 2 would be too large in size for inclusion in this post, so we are only analyzing the interface that most of the client code would see: the Entity Manager (name borrowed from Hibernate and JPA, since Java application used Data Mappers for years before this pattern has seen adoption from PHP ones.) The Entity Manager is not a domain specific interface, but other patterns like the Repository one can then compose the mapper to provid segregated interfaces for a particular class (aggregate root). As always, I have removed the less interesting methods or code to show the Api, and expanded the comments. * @author Guilherme Blanco * @author Jonathan Wage * @author Roman Borschel */ class EntityManager { /** * Flushes all changes to objects that have been queued up to now to the database. * This effectively synchronizes the in-memory state of managed objects with the * database. * No query is executed before this method is called from client code. * * @throws Doctrine\ORM\OptimisticLockException If a version check on an entity that * makes use of optimistic locking fails. */ public function flush() { $this->_errorIfClosed(); $this->_unitOfWork->commit(); } /** * Finds an Entity by its identifier. * This method is often combined with query-oriented ones. * * @param string $entityName the class name * @param mixed $identifier usually primary key * @param int $lockMode * @param int $lockVersion * @return object */ public function find($entityName, $identifier, $lockMode = LockMode::NONE, $lockVersion = null) { return $this->getRepository($entityName)->find($identifier, $lockMode, $lockVersion); } /** * Tells the EntityManager to make an instance managed and persistent. * * The entity will be entered into the database at or before transaction * commit or as a result of the flush operation. * * NOTE: The persist operation always considers entities that are not yet known to * this EntityManager as NEW. Do not pass detached entities to the persist operation. * * @param object $object The instance to make managed and persistent. */ public function persist($entity) { if ( ! is_object($entity)) { throw new \InvalidArgumentException(gettype($entity)); } $this->_errorIfClosed(); $this->_unitOfWork->persist($entity); } /** * Removes an entity instance. * * A removed entity will be removed from the database at or before transaction commit * or as a result of the flush operation. * * @param object $entity The entity instance to remove. */ public function remove($entity) { if ( ! is_object($entity)) { throw new \InvalidArgumentException(gettype($entity)); } $this->_errorIfClosed(); $this->_unitOfWork->remove($entity); } /** * Refreshes the persistent state of an entity from the database, * overriding any local changes that have not yet been persisted. * * @param object $entity The entity to refresh. */ public function refresh($entity) { if ( ! is_object($entity)) { throw new \InvalidArgumentException(gettype($entity)); } $this->_errorIfClosed(); $this->_unitOfWork->refresh($entity); } /** * Determines whether an entity instance is managed in this EntityManager. * * @param object $entity * @return boolean TRUE if this EntityManager currently manages the given entity, FALSE otherwise. */ public function contains($entity) { return $this->_unitOfWork->isScheduledForInsert($entity) || $this->_unitOfWork->isInIdentityMap($entity) && ! $this->_unitOfWork->isScheduledForDelete($entity); } /** * Factory method to create EntityManager instances. * * @param mixed $conn An array with the connection parameters or an existing * Connection instance. * @param Configuration $config The Configuration instance to use. * @param EventManager $eventManager The EventManager instance to use. * @return EntityManager The created EntityManager. */ public static function create($conn, Configuration $config, EventManager $eventManager = null); }

I just installed TPTP into my Eclipse 3.5 under Ubuntu 9.04 and tried to profile a class. The Profile Configuration opened with a red warning reading “the launch requires at least one data collector to be selected“. Clicking the configuration’s Monitor tab reveals a more detailed error (and nothing to select): IWATO435E An error occured when connecting to the host. A quick check of the error log (Window – Show View – Other… – General – Error Log) reveals the cause: RAServer generated the following output: [Error Stream]:ACServer: error while loading shared libraries: /home/jholy/development/tools/eclipse-ide/pulse2-2.4.2/Common/plugins/org.eclipse.tptp.platform.ac.linux_ia32_4.4.202.v201002100300/agent_controller/bin/../lib/libtptpUtils.so.4: file too short Checking the content of the lib/ folder revealed an interesting thing: -rw-r–r– 1 jholy jholy 17 2010-02-16 23:16 libtptpUtils.so -rw-r–r– 1 jholy jholy 21 2010-02-16 23:16 libtptpUtils.so.4 -rwxr-xr-x 1 jholy jholy 100K 2010-02-16 23:16 libtptpUtils.so.4.5.0 As also the content of the two small files suggests (they contain a name of the corresponding file with a longer name), the *.so and *.so.4 files should have been links but the installer failed to create them. Solution List all files in the lib/ folder, you will see that there are many real files like libtptpUtils.so.4.5.0 and libxerces-c.so.26.0 and many should-be-links files. The solution is, of course, to replace all those files that shoud be links with actual links. For me the solution was: $ cd .../plugins/org.eclipse.tptp.platform.ac.linux_ia32_4.4.202.v201002100300/agent_controller/lib # Move out the files that are OK lib$ mkdir tmp lib$ mv libswt-* libcbe.so tmp/ # Fix the links lib$ for FILE in `ls *.so`; do ln -sf "${FILE}.4.5.0" $FILE; ln -sf "${FILE}.4.5.0" "${FILE}.4"; done # Move the correct files back lib$ mv tmp/* . lib$ rmdir tmp # Fix links for files with *.26 instead of *.4.5.0 lib$ ln -sf libxerces-c.so.26.0 libxerces-c.so.26 lib$ ln -sf libxerces-c.so.26.0 libxerces-c.so lib$ ln -sf libxerces-depdom.so.26.0 libxerces-depdom.so.26 lib$ ln -sf libxerces-depdom.so.26.0 libxerces-depdom.so lib$ rm libxerces-depdom.so.4 libxerces-c.so.4 # Done! Try to open the profile configuration now, the IWATO435E should have disappeared and you should be able to select a data collector. If not, restart Eclipse, try again, check the error log. My environment Ubuntu 9.04 Eclipse 3.5 TPTP – see above From http://theholyjava.wordpress.com/2010/05/13/eclipse-profile-configuration-the-launch-requires-at-least-one-data-collector/

The Active Record pattern effectively prescribes to wrap a row of a database table in a domain object with a 1:1 relationship, managing its state and adding business logic in the wrapping class code. An Active Record implementation is in fact a classical C structure aka Record aka associative array of data, with the addition of utility methods that encapsulate behavior that acts on these data. The most useful method is usually the save() one, which updates the database reflecting in the row the current state of the record. Thus, the Active Record transparently works with SQL queries and provides an higher-level Api. Although Active Record is similar in implementation to the Row Data Gateway pattern, it is distinguished from it in the fact that it defines methods with domain-specific logic. The consequence of the presence of domain-specific logic is that generic implementations of Active Record provided by libraries must be customized to met the need of the object model. Typically this customization is done with a thin subclassing, which at least renames the library class with a domain name (like User or Post) and may specify metadata on the database table where the Active Records state is kept, if they are not inferred. The issue of subclassing Subclassing allows the developer to create new methods and properties to represent business logic, and to build a richer and more specific interface than the one constituted by simple Row Data Gateway objects. Despite these advantage, this interface is not much segregated, as subclassing exposes all the public methods of the base library class, on which the developers of the domain model have no control. Subclassing also ties the domain layer to the infrastructure one, being it a library or a framework or every kind of data persistence layer (examples of PHP ORMs that use Active Record are the Zend_Db component, Doctrine 1 and Propel). Domain objects cannot be created or even their class source code loaded without having the library code available. This is an issue when reusing the model in a different environment, and even in test suites. If the library is powerful enough, it may provide adapters for different databases so that a lightweight database instance can be created in the testing environment. Another caveat of Active Record is the fundamental assumption that a domain entity is always a row of a table of a relational database; this constraint is forced even when it is not appropriate, and the database and object model must match. In fact, part of the database (like foreign keys) often scatter into the domain model, as an Active Record with an external one-to-one relationship will usually store not only the related object but also its foreign key. Another example of mirroring of the relational model into the object graph is for the management of M-to-N associating entities, often forced to become real entities even when they do not make sense (the famous UserGroup classes that tie together User and Group rows). Diffusion Thus, the Active Record pattern puts at risk the freedom of implementing a powerful Domain Model, where the object graph is a mix of state-carrying and behavior-carrying objects, like Strategies and Specifications. It is however, a radical simplification in implementation of domain models where CRUD functions are all the rage, and there is no gain in implementing objects that do not simply map to a relational database. Note that in the case of PHP, most of the custom web applications developed in this language are deeply influenced by the back end, assumed as a relational database or even as MySQL. But while the technologies for user-to-application and application-to.application interaction on the web continue to grow, the situation will continue to evolve and if PHP wants to keep up with the pace of other dynamic languages, Java and .NET, it needs to finally decouple from the relational database as the unique model of data. By the way, current implementation of persistence frameworks are transitioning towards a Data Mapper approach (not only in PHP but also in Ruby, while Java has done that years ago with Hibernate and JPA), which is less invasive on the Domain Model source code, and does not introduce an hard dependency from the domain layer to infrastructure components. Examples This sample implementation of the Active Record pattern is taken from the Doctrine 1.2 ORM. The base class in this framework id Doctrine_Record (together with Doctrine_Record_Abstract), while a base class with the schema metadata is regenerated from a model, and it is subclassed for orthogonality of customization and synchronization by the developer. The base Active Record class looks like this: /** * Implements also __get() and __set(), not shown along with many other dozen methods. */ abstract class Doctrine_Record extends Doctrine_Record_Abstract implements Countable, IteratorAggregate, Serializable { /** * Empty template method to provide concrete Record classes with the possibility * to hook into the saving procedure. */ public function preSave($event) { } /** * Empty template method to provide concrete Record classes with the possibility * to hook into the saving procedure. */ public function postSave($event) { } /** * applies the changes made to this object into database * this method is smart enough to know if any changes are made * and whether to use INSERT or UPDATE statement * * this method also saves the related components * * @param Doctrine_Connection $conn optional connection parameter * @return void */ public function save(Doctrine_Connection $conn = null) { if ($conn === null) { $conn = $this->_table->getConnection(); } $conn->unitOfWork->saveGraph($this); } /** * returns a string representation of this object */ public function __toString() { return (string) $this->_oid; } } If we have an Article entity, it will be represented via subclassing of the generic Active Record. In Doctrine 1, a subclass can be generated by writing a compact Yaml model, or even reverse engineered from an existing database: /** * BaseOtk_Content_Article * * This class has been auto-generated by the Doctrine ORM Framework * * @property integer $id * @property integer $section_id * @property integer $author_id * @property integer $image_id * @property string $title * @property string $description * @property string $text * @property integer $visits * @property boolean $draft * @property boolean $closed * @property Otk_Content_Section $section * @property Otk_User $author * @property Otk_File $image * @property Doctrine_Collection $sections * @property Doctrine_Collection $Otk_Content_Tag * @property Doctrine_Collection $comments * */ abstract class BaseOtk_Content_Article extends Otk_Model_Record { public function setTableDefinition() { $this->setTableName('oss_content_articles'); $this->hasColumn('id', 'integer', 3, array('type' => 'integer', 'primary' => true, 'autoincrement' => true, 'length' => '3')); $this->hasColumn('section_id', 'integer', 2, array('type' => 'integer', 'notnull' => true, 'length' => '2')); $this->hasColumn('author_id', 'integer', 3, array('type' => 'integer', 'length' => '3')); $this->hasColumn('image_id', 'integer', 3, array('type' => 'integer', 'length' => '3')); $this->hasColumn('title', 'string', 255, array('type' => 'string', 'length' => '255')); $this->hasColumn('description', 'string', 1000, array('type' => 'string', 'length' => '1000')); $this->hasColumn('text', 'string', null, array('type' => 'string')); $this->hasColumn('visits', 'integer', 3, array('type' => 'integer', 'length' => '3')); $this->hasColumn('draft', 'boolean', null, array('type' => 'boolean', 'notnull' => true, 'default' => false)); $this->hasColumn('closed', 'boolean', null, array('type' => 'boolean')); } public function setUp() { $this->hasOne('Otk_Content_Section as section', array('local' => 'section_id', 'foreign' => 'id')); $this->hasOne('Otk_User as author', array('local' => 'author_id', 'foreign' => 'id')); $this->hasOne('Otk_File as image', array('local' => 'image_id', 'foreign' => 'id')); $this->hasMany('Otk_Content_Section as sections', array('refClass' => 'Otk_Content_Tag', 'local' => 'article_id', 'foreign' => 'section_id')); $this->hasMany('Otk_Content_Tag', array('local' => 'id', 'foreign' => 'article_id')); $this->hasMany('Otk_Content_Comment as comments', array('local' => 'id', 'foreign' => 'article_id')); $timestampable0 = new Doctrine_Template_Timestampable(); $sluggable0 = new Doctrine_Template_Sluggable(array('fields' => array(0 => 'title'), 'canUpdate' => true, 'unique' => true)); $this->actAs($timestampable0); $this->actAs($sluggable0); } } To support further regenerations of the subclass as the schema evolves, another subclassing step is necessary. This class will never be touched by the regeneration process and it is the one referred in client code. /** * This class defines the domain logic via addition of methods. */ class Otk_Content_Article extends BaseOtk_Content_Article { public function getTags() { $tags = array(); foreach ($this->sections as $section) { $tags[$section->slug] = $section->name; } return $tags; } }

This article shows 2 ways to convert Java Map to String. Approach 1: simple, lightweight – produces query string like output, but restrictive. Approach 2: uses Java XML bean serialization, more robust but produces overly verbose output. Approach 1: Map to query string format Approach 1 converts a map to a query-string like output. Here’s what an output looks like: name1=value1&name2=value2 Full Code: import java.io.UnsupportedEncodingException; import java.net.URLDecoder; import java.net.URLEncoder; import java.util.HashMap; import java.util.Map; public class MapUtil { public static String mapToString(Map map) { StringBuilder stringBuilder = new StringBuilder(); for (String key : map.keySet()) { if (stringBuilder.length() > 0) { stringBuilder.append("&"); } String value = map.get(key); try { stringBuilder.append((key != null ? URLEncoder.encode(key, "UTF-8") : "")); stringBuilder.append("="); stringBuilder.append(value != null ? URLEncoder.encode(value, "UTF-8") : ""); } catch (UnsupportedEncodingException e) { throw new RuntimeException("This method requires UTF-8 encoding support", e); } } return stringBuilder.toString(); } public static Map stringToMap(String input) { Map map = new HashMap(); String[] nameValuePairs = input.split("&"); for (String nameValuePair : nameValuePairs) { String[] nameValue = nameValuePair.split("="); try { map.put(URLDecoder.decode(nameValue[0], "UTF-8"), nameValue.length > 1 ? URLDecoder.decode( nameValue[1], "UTF-8") : ""); } catch (UnsupportedEncodingException e) { throw new RuntimeException("This method requires UTF-8 encoding support", e); } } return map; } } Example usage code Map map = new HashMap(); map.put("color", "red"); map.put("symbols", "{,=&*?}"); map.put("empty", ""); String output = MapUtil.mapToString(map); Map parsedMap = MapUtil.stringToMap(output); for (String key : map.keySet()) { Assert.assertEquals(parsedMap.get(key), map.get(key)); } Output with Approach 1: symbols=%7B%2C%3D%26*%3F%7D&color=red∅= Caveat Only supports String keys and values. Due to the nature of serialization, null keys and values are not supported. Null will be converted to an empty String. This is because there is no way to distinguish between a null and an empty String in the serialized form. If you need support for null keys and values, use java.beans.XMLEncoder as shown below. Approach 2: Java Bean XMLEncoder: Map to String Java provides XMLEncoder and XMLDecoder classes as part of the java.beans package as a standard way to serialize and deserialize objects. This Map map = new HashMap(); map.put("color", "red"); map.put("symbols", "{,=&*?}"); map.put("empty", ""); ByteArrayOutputStream bos = new ByteArrayOutputStream(); XMLEncoder xmlEncoder = new XMLEncoder(bos); xmlEncoder.writeObject(map); xmlEncoder.flush(); String serializedMap = bos.toString() System.output.println(serializedMap); Output with Approach 2 The serialized value is shown below. As you can see this is more verbose, but can accommodate different data types and null keys and values. symbols {,=&*?} color red empty symbols {,=&*?} color red empty Java Bean XMLDecoder: String to Map XMLDecoder xmlDecoder = new XMLDecoder(new ByteArrayInputStream(serializedMap.getBytes())); Map parsedMap = (Map) xmlDecoder.readObject(); for (String key : map.keySet()) { Assert.assertEquals(parsedMap.get(key), map.get(key)); } Summary While Java provides a standard (and overly verbose) way to serialize and deserialize objects, this articles discusses an alternative lightweight way to convert a Java Map to String and back. If you are serializing a map with non-null String keys and values, then you should be able to use this alternative way, otherwise use the Java bean serialization. From http://www.vineetmanohar.com/2010/05/07/2-ways-to-convert-java-map-to-string

The Table Data Gateway pattern is the object-oriented equivalent of a relational table. In fact, this pattern's intent is to encapsulate the full interaction with a database table, holding all the logic specific to this particular implementation of the back end. In the majority of cases, a Table Data Gateway deals with a relational model, having a 1:1 relationship with the main tables of the database. Minor tables may not need a specific class, or can be managed via Table Data Gateways of tables that link them with foreign keys (for example entities introduced to store M:N relationships are usually not first-class citizens.) In the relational implementation, the Table Data Gateway handles all SQL queries, presenting a domain-specific interface when a class is coded for a specific table, or a generic one when a generic implementation is reused throughout different applications. The difference between the two APIs may be something like findBy($field, $value) (generic) versus findByPrice($price) (domain-specific). Note that in PHP magic methods are often used to implement domain-specific interfaces without code generation: a __call() implementation can catch the various findBy*() method and throw exceptions if the methods is not applicable. Related patterns Although, the concept of table is already correlated with a relational model (and it does not hold when the back end is an object-oriented database or one of the key-value stores so trendy today), this pattern is named Gateway because it is a specialization of the Gateway category of pattern, which decouple an object graph (or any in-memory structures) from external infrastructure like databases, web services, filesystems and so on. In fact, there is an alternate name for this pattern: Data Access Object (or DAO for friends). Although if I was pedantic I would highlight the differences between the implementations of DAOs and Table Data Gateway, their intent is really the same and there are differences in an individual pattern implementations that are greater than the ones between the different patterns. There's no clear demarcation line between the two. Another related pattern is the Table Module one. Table Data Gateway does not work against, but with a Table Module, providing a separation of concerns: the first object takes the rows out of the database, while the second performs in-memory operation on them (generally by composing the Table Data Gateway or its results). The in-memory operations of a Table Module are easier to test, but the SQL-based operations of the Table Data Gateway are pushed on the database side: there is a trade-off between the logic should be kept in each class. When used in isolation, the Table Data Gateway is also a Factory for also for Row Data Gateways or Active Records, both again implemented with generic or domain-specific interfaces. Many frameworks and first-generation PHP ORMs based on Active Record are also based on Table Data Gateway to provide a collection-level access to the objects stored as rows. In the context of Active Record, the only alternative to a Table Data Gateway to handle operations like find() is to place static methods on the Active Record class, with all the testability and dishonest API issues that ensue. Both Zend Framework and Doctrine 1.x represent tables as first-class objects. Examples Zend Framework's component Zend_Db, which is explored in the sample code, provides always generic implementations of Zend_Db_Table, and the possibility of optional subclassing (to add domain-specific methods). It is not recommend to expose the API of Table Data Gateway in front-end code, but it's a simple solution when the business logic does not warrant a full-featured Domain Model. Even when working with a Domain Model, and before the introduction of generic Data Mappers for PHP, the Table Data Gateway can be used in a composition solution (wrapped) to craft a simple API for a domain-specific Data Mapper, resulting in decoupling from the database. As I've written earlier, the sample code is taken from the Zend_Db_Table class of Zend Framework (actually from its parent abstract class, Zend_Db_Table_Abstract). I've enriched the docblock comments and left out all the methods not part of the main API (most of getters and setters for configuration and protecte|private members). $value) { switch ($key) { case self::ADAPTER: $this->_setAdapter($value); break; case self::DEFINITION: $this->setDefinition($value); break; case self::DEFINITION_CONFIG_NAME: $this->setDefinitionConfigName($value); break; case self::SCHEMA: $this->_schema = (string) $value; break; case self::NAME: $this->_name = (string) $value; break; case self::PRIMARY: $this->_primary = (array) $value; break; case self::ROW_CLASS: $this->setRowClass($value); break; case self::ROWSET_CLASS: $this->setRowsetClass($value); break; case self::REFERENCE_MAP: $this->setReferences($value); break; case self::DEPENDENT_TABLES: $this->setDependentTables($value); break; case self::METADATA_CACHE: $this->_setMetadataCache($value); break; case self::METADATA_CACHE_IN_CLASS: $this->setMetadataCacheInClass($value); break; case self::SEQUENCE: $this->_setSequence($value); break; default: // ignore unrecognized configuration directive break; } } return $this; } /** * Inserts a new row. * The data structure is as generic as possible. The list of columns is * known by configuration. * $this->_db is a light abstraction over PDO, which already encapsulates * most of the SQL. Database abstraction is not a banal task and segregating * the functionalities in different classes is very helpful. * * @param array $data Column-value pairs. * @return mixed The primary key of the row inserted. */ public function insert(array $data) { $this->_setupPrimaryKey(); /** * Zend_Db_Table assumes that if you have a compound primary key * and one of the columns in the key uses a sequence, * it's the _first_ column in the compound key. */ $primary = (array) $this->_primary; $pkIdentity = $primary[(int)$this->_identity]; /** * If this table uses a database sequence object and the data does not * specify a value, then get the next ID from the sequence and add it * to the row. We assume that only the first column in a compound * primary key takes a value from a sequence. */ if (is_string($this->_sequence) && !isset($data[$pkIdentity])) { $data[$pkIdentity] = $this->_db->nextSequenceId($this->_sequence); } /** * If the primary key can be generated automatically, and no value was * specified in the user-supplied data, then omit it from the tuple. */ if (array_key_exists($pkIdentity, $data) && $data[$pkIdentity] === null) { unset($data[$pkIdentity]); } /** * INSERT the new row. */ $tableSpec = ($this->_schema ? $this->_schema . '.' : '') . $this->_name; $this->_db->insert($tableSpec, $data); /** * Fetch the most recent ID generated by an auto-increment * or IDENTITY column, unless the user has specified a value, * overriding the auto-increment mechanism. */ if ($this->_sequence === true && !isset($data[$pkIdentity])) { $data[$pkIdentity] = $this->_db->lastInsertId(); } /** * Return the primary key value if the PK is a single column, * else return an associative array of the PK column/value pairs. */ $pkData = array_intersect_key($data, array_flip($primary)); if (count($primary) == 1) { reset($pkData); return current($pkData); } return $pkData; } /** * Updates existing rows. * Again we see generic data structures, not tied to PDO * or to particular adapters. * * @param array $data Column-value pairs. * @param array|string $where An SQL WHERE clause, or an array of SQL WHERE clauses. * @return int The number of rows updated. */ public function update(array $data, $where) { $tableSpec = ($this->_schema ? $this->_schema . '.' : '') . $this->_name; return $this->_db->update($tableSpec, $data, $where); } /** * Deletes existing rows. * * @param array|string $where SQL WHERE clause(s). * @return int The number of rows deleted. */ public function delete($where) { $tableSpec = ($this->_schema ? $this->_schema . '.' : '') . $this->_name; return $this->_db->delete($tableSpec, $where); } /** * Fetches rows by primary key. The argument specifies one or more primary * key value(s). To find multiple rows by primary key, the argument must * be an array. * * This method accepts a variable number of arguments. If the table has a * multi-column primary key, the number of arguments must be the same as * the number of columns in the primary key. To find multiple rows in a * table with a multi-column primary key, each argument must be an array * with the same number of elements. * * The find() method always returns a Rowset object, even if only one row * was found. * * @param mixed $key The value(s) of the primary keys. * @return Zend_Db_Table_Rowset_Abstract Row(s) matching the criteria. * @throws Zend_Db_Table_Exception */ public function find() { $this->_setupPrimaryKey(); $args = func_get_args(); $keyNames = array_values((array) $this->_primary); if (count($args) < count($keyNames)) { require_once 'Zend/Db/Table/Exception.php'; throw new Zend_Db_Table_Exception("Too few columns for the primary key"); } if (count($args) > count($keyNames)) { require_once 'Zend/Db/Table/Exception.php'; throw new Zend_Db_Table_Exception("Too many columns for the primary key"); } $whereList = array(); $numberTerms = 0; foreach ($args as $keyPosition => $keyValues) { $keyValuesCount = count($keyValues); // Coerce the values to an array. // Don't simply typecast to array, because the values // might be Zend_Db_Expr objects. if (!is_array($keyValues)) { $keyValues = array($keyValues); } if ($numberTerms == 0) { $numberTerms = $keyValuesCount; } else if ($keyValuesCount != $numberTerms) { require_once 'Zend/Db/Table/Exception.php'; throw new Zend_Db_Table_Exception("Missing value(s) for the primary key"); } $keyValues = array_values($keyValues); for ($i = 0; $i < $keyValuesCount; ++$i) { if (!isset($whereList[$i])) { $whereList[$i] = array(); } $whereList[$i][$keyPosition] = $keyValues[$i]; } } $whereClause = null; if (count($whereList)) { $whereOrTerms = array(); $tableName = $this->_db->quoteTableAs($this->_name, null, true); foreach ($whereList as $keyValueSets) { $whereAndTerms = array(); foreach ($keyValueSets as $keyPosition => $keyValue) { $type = $this->_metadata[$keyNames[$keyPosition]]['DATA_TYPE']; $columnName = $this->_db->quoteIdentifier($keyNames[$keyPosition], true); $whereAndTerms[] = $this->_db->quoteInto( $tableName . '.' . $columnName . ' = ?', $keyValue, $type); } $whereOrTerms[] = '(' . implode(' AND ', $whereAndTerms) . ')'; } $whereClause = '(' . implode(' OR ', $whereOrTerms) . ')'; } // issue ZF-5775 (empty where clause should return empty rowset) if ($whereClause == null) { $rowsetClass = $this->getRowsetClass(); if (!class_exists($rowsetClass)) { require_once 'Zend/Loader.php'; Zend_Loader::loadClass($rowsetClass); } return new $rowsetClass(array('table' => $this, 'rowClass' => $this->getRowClass(), 'stored' => true)); } return $this->fetchAll($whereClause); } /** * Fetches a new blank row (not from the database). * Thanks to the metadata, a new Row Data Gateway can be created. This * if a Factory Method. The dynamic nature of PHP makes configuring the * subclass for the Row Data Gateway as simple as defining a string. * * @param array $data OPTIONAL data to populate in the new row. * @param string $defaultSource OPTIONAL flag to force default values into new row * @return Zend_Db_Table_Row_Abstract */ public function createRow(array $data = array(), $defaultSource = null) { $cols = $this->_getCols(); $defaults = array_combine($cols, array_fill(0, count($cols), null)); // nothing provided at call-time, take the class value if ($defaultSource == null) { $defaultSource = $this->_defaultSource; } if (!in_array($defaultSource, array(self::DEFAULT_CLASS, self::DEFAULT_DB, self::DEFAULT_NONE))) { $defaultSource = self::DEFAULT_NONE; } if ($defaultSource == self::DEFAULT_DB) { foreach ($this->_metadata as $metadataName => $metadata) { if (($metadata['DEFAULT'] != null) && ($metadata['NULLABLE'] !== true || ($metadata['NULLABLE'] === true && isset($this->_defaultValues[$metadataName]) && $this->_defaultValues[$metadataName] === true)) && (!(isset($this->_defaultValues[$metadataName]) && $this->_defaultValues[$metadataName] === false))) { $defaults[$metadataName] = $metadata['DEFAULT']; } } } elseif ($defaultSource == self::DEFAULT_CLASS && $this->_defaultValues) { foreach ($this->_defaultValues as $defaultName => $defaultValue) { if (array_key_exists($defaultName, $defaults)) { $defaults[$defaultName] = $defaultValue; } } } $config = array( 'table' => $this, 'data' => $defaults, 'readOnly' => false, 'stored' => false ); $rowClass = $this->getRowClass(); if (!class_exists($rowClass)) { require_once 'Zend/Loader.php'; Zend_Loader::loadClass($rowClass); } $row = new $rowClass($config); $row->setFromArray($data); return $row; } }

The architectural pattern I'd like to talk about in this article is the overly famous Domain Model. An application's Domain Model is simply defined as an object graph created from domain-specific classes; when present, a Domain Model is the core of the application, where all the business logic resides. This object graph is employed by upper layers of an application which present it to the user. The metaphor for this methodology In software development, the term domain (or business domain) is an umbrella for the area the application is built in, and that it will serve. The new domains we encounter as we move to new projects are one of the most interesting points of software development, where we are constantly embracing new fields and gaining knowledge. Given a domain such as a particular industry (chemical, electronics) or business (air travelling, e-commerce), the point of connection of an application with these activities is its model. A model is an abstract representation of the reality of the domain, which captures its interesting and relevant aspects. The practice of modelling is not a specific trait of software development (in particular model-driven development), but it is a more general scientifical process. For example, everyone who works in the field of information technology knows the voltage/current relationships for simple components such as resistors and capacitors (Ohm's law and current derivative of the voltage). The specific domain here is electronics, and this model is named lumped component model, essentially because it lets a designer connect isolated one-port (two terminals) components to build his desired circuit. This model is a simplification of much more complex models of reality: the Maxwell equations and the propagation of electromagnetic fields; the lumped component model is valid whenever the frequency of the voltage/current signals in the circuit is low, so that the wavelengths of these signals are far greater than the dimensions of the circuit (if that goes over your head, don't worry, it's the field of electrical engineers.) When designers consider larger circuits, such as a transmission line, this model ceases to give correct results and more general ones must be employed. The domain is almost the same, but the model serves a different purpose and has to be necessarily different from the one used in small scale circuits. This complex example is here only to show that given a domain, there is no single model for it, but there are many possible ones which may adapt more or less reliably to the goals of an application. Starting from a modelling phase and deep understanding of the domain are key points of Domain-Driven Design, one of the ascending methodologies for developing complex enterprise software. Software models While there are standard mathematical models for many domains in the scientific world, software developers usually build a tailored one in every different application, performing an analysis of the domain (or at least they should.) The result of the modelling can comprehend document or diagrams, but the most powerful artifact is an executable model. Object-oriented programming is a almost perfect paradigm when it comes to modelling the real world, and lets the developers construct a Domain Model in the form of a set of classes. In a correct implementation of a Domain Model, these classes should be behaviorally complete: they must encapsulate their data as much as possible and expose a set of methods, while avoiding their usage as dumb data containers. The bread and butter of a Domain Model are the classical example of User, Post, Forum, Group, PrivateMessage classes, which are usually in a one to one relationship with database tables. But the Domain Model is not limited to these Entity classes: it also "comprehends" ValueObjects (modelization of domain-specific data types) and various kinds of Services. Every class that encapsulates business logic is welcome, so that this logic is not duplicated in upper layers, which are the primary clients of the Domain Model. Dependencies and purity Another key trait of the classes included in the Domain Model is the absence of external dependencies, like a library to store in the data contained in the objects in a database. The code artifact in a Domain Model are either interfaces, or Plain Old Php Objects (classes which do not extend any external abstract superclass.) Active Record approaches should be avoided because not only a relational database is an infrastructure detail not included in the Domain Model itself, but the very concept of persistence is abstracted away. As far as the clients of the Domain Model are concerned, the state and behavior of the application are represented by an in-memory object graph, whose methods expose functionalities and which client code can play with. There are no dependencies from a Domain Model towards infrastructure classes, because these dependencies must be inverted. The resulting system is an instance of the hexagonal architecture, where the Domain Model defines ports (interfaces) and infrastructure can be chosen to provide adapters for these ports (implementations in the form of classes extraneous to the model). The implementaton of non-invasive persistence is the subject of the Data Mapper pattern, which will be treated later in this series, but every kind of service implementation which communicate with the outside of the core object graph (databases, network, filesystem) is only defined as a contract in the Domain Model. Persistence is almost always dealt with a library in other object-oriented languages, now also in PHP with a non-invasive ORM such as Doctrine 2. Nothing obstructs the developers from implementing a specific Data Mapper by hand, but it's a very repetitive and prone to errors task. While in origin simpler, invasive patterns such as Active Record could be used in a Domain Model, nowadays with Data Mapper availables it is considered an hack. Sample Returning to the subject of the Domain Model as the core of an application, the diffused opinion is that the more complex the business logic and the data involved, the more the application benefits from a rich Domain Model. Thus, this pattern should not be used in small-sized applications where there is no much more logic than CRUD screens for data containers, which unfortunately were a target for PHP in the last ten years. I hope PHP keeps evolving to finally break in the enterprise segment, where this pattern is most valuable. Due to the size and scope of this article, I am forced to keep the sample code short. Forgive me if you think that you can achieve the same functionality with fewer lines of code, but this pattern is about architecture and should highlight the separation of concerns between classes more than the KISS principle. Another problem with code samples in modelling is that you have to actually know the domain well to follow the discussion. For this reason I chose a webmail system for this example. _sender; } /** * Do we need setters and getters? Every field should be * analyzed. If we can keep it private and inaccessible, * it's usually better. */ public function setSender($sender) { $this->_sender = $sender; } /** * @return string */ public function getRecipient() { return $this->_recipient; } public function setRecipient($recipient) { $this->_recipient = $recipient; } /** * @return string */ public function getSubject() { return $this->_subject; } public function setSubject($subject) { $this->_subject = $subject; } /** * @return string */ public function getText() { return $this->_text; } public function setText($text) { $this->_text = $text; } public function __toString() { return $this->_subject . ' > ' . substr($this->_text, 0, 20) . '...'; } public function reply() { $reply = new Email(); $reply->setRecipient($this->_sender); $reply->setSender($this->_recipient); $reply->setSubject('Re: ' . $this->_subject); $reply->setText($this->_sender . " wrote:\n" . $this->_text); return $reply; } } /** * Interface for a service. This is part of the Domain Model, * implementations will be plugged in depending on the environment. */ interface EmailRepository { /** * @return array * @TypeOf(Email) */ public function getEmailsFor($recipient); } // client code $mail = new Email(); $mail->setSender("[email protected]"); $mail->setRecipient("[email protected]"); $mail->setSubject('Hello'); $mail->setText('This is a test of an Email object, which is part of our Domain Model.'); echo $mail, "\n"; $reply = $mail->reply(); echo $reply, "\n";

while recently fiddling with some more runtime generic type extraction for deployit , i was caught out by some unexpected behaviour by the reflection api. a check of the javadocs quickly revealed that i had once again been too hasty in relying on "common sense". still, the case seems sufficiently unintuitive to merit discussion. in this case, the issue centres on the interplay between class.gettypeparameters and parameterizedtype . the gist of the code looks something like: interface spying {} // small class hierarchy class person {} class professional extends person {} class agent extends professional {} class assassin extends professional {} class bystander extends person {} ... person jbond = new agent(); system.out.println("generic superclass type argument: " + trygetsuperclassgenerictypeparam(jbond)); person joepublic = new bystander(); system.out.println("generic superclass type argument: " + trygetsuperclassgenerictypeparam(joepublic)); person oddjob = new assassin(); system.out.println("generic superclass type argument: " + trygetsuperclassgenerictypeparam(oddjob)); ... type trygetsuperclassgenerictypeparam(object obj) { class clazz = obj.getclass(); class superclass = clazz.getsuperclass(); // elvis would be preferred, but for the sake of clarity... if (superclass.gettypeparameters().length > 0) { return ((parameterizedtype) clazz.getgenericsuperclass()).getactualtypearguments()[0]; } else { return null; } } so...what happens? trygetsuperclassgenerictypeparam is where the action happens. it seems fairly straightforward: see if the object's superclass is generic (i.e. takes type parameters) and, if so, cast its type representation to parameterizedtype to extract the actual value for the type parameter. if the superclass is not generic, simply return null. when this code is run, the first two invocations of trygetsuperclassgenerictypeparam result in the expected: generic superclass type argument: interface spying generic superclass type argument: null what about the third one? well, given the fact that we've omitted to specify a generic type parameter for professional we might assume 1 that we'd also get null. the actual output, however, is: exception in thread "main" java.lang.classcastexception: java.lang.class cannot be cast to java.lang.reflect.parameterizedtype at trygetsuperclassgenerictypeparam(...) huh? in order to figure out what's going on here, let's have a look at the javadoc for class.gettypeparameters: returns an array of typevariable objects that represent the type variables declared by the generic declaration represented by this genericdeclaration object, in declaration order. returns an array of length 0 if the underlying generic declaration declares no type variables. in other words, this is returning class-level information about the declaration of, in our case, the professional class, which of course does have a type parameter. however, if we look at class.getgenericsuperclass 2 , which we invoke next, we find that it: returns the type representing the direct superclass of the entity [...] represented by this class. if the superclass is a parameterized type, the type object returned must accurately reflect the actual type parameters used in the source code. here, the information returned is specific to the actual declaration of the class, which may (or may not, as in our case) specify type paramaters for its superclass. and therein lies the problem: professional.class.gettypearguments looks at the declaration of the professional class, discovering a type argument, whereas assassin.class.getgenericsuperclass looks at the occurrence of professional in the declaration of assassin and discovers no type parameters. hence, it returns a class rather than a parameterizedtype and blows up our code. ergo to cut a long story short: if an object's superclass has type arguments as determined by class.gettypearguments that does not mean that object.getclass().getgenericsuperclass() will be a parameterizedtype. footnotes read "i assumed" it's a pity that class.getgenericsignature , which determines the "generic or not" behaviour of class.getgenericsuperclass, is private, native and undocumented. from http://blog.xebia.com/2010/04/22/when-a-class-with-type-parameters-is-not-a-parameterized-class-a-java-generics-puzzler/

For readability’s sake, it’s almost always a good idea to replace magic numbers and string literals with constants. That’s all good, but it can take a bit of time to refactor these to constants, especially strings or parts of strings. For example, in the code below we want to refactor “shovel and spade” to a private static final String called TOOLS. To do that manually would take some time. It goes even slower if we only want to extract “spade” to a constant because we first have to convert the string to a concatenation. String tools = "shovel and spade"; ... String otherTools = "shovel and spade"; Luckily, Eclipse has a couple of ways to instantly convert literals to constants. Coupled with tools to speed up string selection and to pick out part of a string, you have the ability to create a constant in about 2 seconds flat. I’ll discuss all these features below. Extract a constant from a string/number There are 2 ways to extract a constant, the one uses a quick fix and the other a refactoring. I’ll show the quick fix method first and then the refactoring and discuss the (small) differences between the two. The example uses a string, but everything is true for numbers as well. Follow these steps to use the quick fix: First select the string. The fastest way is to place the cursor on the string and press Alt+Shift+Up (Select Enclosing Element; a nifty shortcut that I discuss in Select strings and methods with a single keystroke). After selecting the string, press Ctr+1 (Quick Fix) and then select Extract to constant. Eclipse will do the following: (a) Create a private final static variable of type String with a default name, (b) replace all occurrences of that string with the constant and (c) place the cursor on the constant’s declaration to give you a chance to change the name, type and visibility of the variable using placeholders that you can Tab through. Once you’re happy with the constant details, press Enter to go back to the line on which you initiated the quick fix. Here’s a short video with an example of using quick fix. We’ll extract a constant (called TOOLS) from a string literal (“shovel and spade”) that’s used in two places. Note: You can use Tab to move from one placeholder to another and pressing Enter will get you back to your original line. The other way to extract a constant is by using the Extract Constant refactoring. Again, select the string, then select Refactor > Extract Constant… (Alt+T, A) from the application menu. A dialog appears prompting you for the constant’s name, its visibility and whether to replace all occurrences of the string with the constant. After you’ve entered the details, press Enter and you’ll have your constant defined. Here’s a short video with an example using refactoring. We’ll use the same example as above. The differences between the two? Not much, the biggest difference being when you enter the details of the constant (ie. before the change is made or after). The refactoring dialog also provides an option to add the qualifying type name before the constant’s usage, but most of time this is redundant. I’d recommend using the quick fix, unless you’re more comfortable with dialogs. BTW, you can assign custom keyboard shortcuts to either command by mapping either Quick Assist – Extract Constant or the command Extract Constant. Pick out part of a string Sometimes you’ll want to break up a string into multiple parts and convert one of those parts into a constant. Eclipse can do this automatically. Select the part of the string you want to pick out (don’t worry about quotes), press Ctrl+1 and choose Pick out selected part of String. Eclipse will convert that part into a string with quotes, concatenate it to the rest of the string and select it. You can then use any of the Extract Constant tools above. Here’s an example of how to use this feature. Notice how the string’s already selected so we can use the Extract Constant quick fix immediately. Related Tips Select entire strings and methods in Eclipse with a single keystroke Convert string concatenations into StringBuilder or MessageFormat calls with Eclipse’s Quick Fix How to manage keyboard shortcuts in Eclipse and why you should Join/split if statements and rearrange expressions using Eclipse Quick Fix More tips on using quick fixes and making editing faster.

The purpose of this article is to give you the details of our 100 node cluster demo. This demo is recorded and you can watch the 5 minute screencast Hazelcast is an open source clustering and highly scalable data distribution platform for Java. JVMs that are running Hazelcast will dynamically cluster and allow you to easily share and partition your application data across the cluster. Hazelcast is a peer-to-peer solution (there is no master node, every node is a peer) so there is no single point of failure. Communication among cluster members is always TCP/IP with Java NIO beauty. The default configuration comes with 1 backup so if a node fails, no data will be lost (you can specify the backup count). It is as simple as using java.util.{Map, Queue, Set, List}. Just add the hazelcast.jar into your classpath and start coding. When you download the Hazelcast, you will find a test.sh under bin directory. The test.sh runs an application which randomly makes 40% get, 40% put and 20% remove on a distributed map. In this demo the same test application will be used to see how it performs on 100 node cluster. Amazon EC2 and S3 An easy to use and scalable cloud environment was needed for demo so we decided to use Amazon EC2 for server instances (nodes) and S3 service to store demo application zip and configuration files. With its newly announced Java SDK, it is very simple to start/stop server instances and upload files to S3 programatically. Hazelcast AMI & Launcher The challenge here is that we are running an application on 100 nodes and dealing with each and every server in the cluster is a huge task. We don't want to ssh into every server and manually start the application. This part is automated by creating a special server image (AMI). The AMI contains Java Runtime and a launcher application we developed, which will download the demo application from Amazon S3, unzip it, and run the hazelcast/bin/test.sh in it. The Launcher is actually so generic that it can run any application; it doesn't care/know what test.sh contains. Deployer Deployment of the demo application is also automated so that we don't need to login into AWS Management Console and manually start instances. Deployer instantiates any number of Amazon EC2 servers with any AMI and also uploads the demo application zip file to S3. So the idea here is that, the Deployer will store the application into S3 and launch 100 EC2 instances with our image. The Launcher on each instance will download the application from S3 and run it. Demo Details. The smallest EC2 instances (m1.small) are used to run the demo. These are the virtual instances with CPU about 1.0 GHz. Also keep in mind that EC2 platform suffers from considerable amount of network latency. That's why we increased the thread count to 250 in our application. The following steps performed during the demo Download hazelcast-1.8.3.zip from www.hazelcast.com. Unzip the file and move the monitoring war file into tomcat6/webapps directory. Edit the test.sh under the bin directory: Add -Xmx1G -Xms1G Add -Dhazelcast.initial.wait.seconds=100 to make the cluster evenly partition on start so that migration can be avoided for better performance. Add t250 as an argument to the application to set thread count to 250. Remember the latency issue. Run the Deployer from IDE. Check from EC2 Management Console if 100 servers started. Start tomcat. Copy the public DNS name of one of the servers to connect to from monitoring tool. Go to http://localhost:8080/hazelcast-monitor-1.8.3/ (Hazelcast Monitoring Tool). Paste the address and connect to the cluster. Enjoy! Results You should always look for programatic ways of launching applications on the cloud. With these tools we were able to deploy and run the demo application on 100 servers in minutes. The entire Hazelcast cluster was making over 400,000 operations per second on the smallest EC2 instances. In our next demo we will experiment Hazelcast on large data set and even bigger cluster. Watch the screencast

In this article, I will explain how to debug Hibernate’s generated SQL so that unexpected query results be traced faster either to a faulty dataset or a bug in the query. There’s no need to present Hibernate anymore. Yet, for those who lived in a cave for the past years, let’s say that Hibernate is one of the two main ORM frameworks (the second one being TopLink) that dramatically ease database access in Java. One of Hibernate’s main goal is to lessen the amount of SQL you write, to the point that in many cases, you won’t even write one line. However, chances are that one day, Hibernate’s fetching mechanism won’t get you the result you expected and the problems will begin in earnest. From that point and before further investigation, you should determine which is true: either the initial dataset is wrong or the generated query is or both if you’re really unlucky Being able to quickly diagnose the real cause will gain you much time. In order to do this, the greatest step will be viewing the generated SQL: if you can execute it in the right query tool, you could then compare pure SQL results to Hibernate’s results and assert the true cause. There are two solutions for viewing the SQL. Show SQL The first solution is the simplest one. It is part of Hibernate’s configuration and is heavily documented. Just add the following line to your hibernate.cfg.xml file: ... true The previous snippet will likely show something like this in the log: select this_.PER_N_ID as PER1_0_0_, this_.PER_D_BIRTH_DATE as PER2_0_0_, this_.PER_T_FIRST_NAME as PER3_0_0_, this_.PER_T_LAST_NAME as PER4_0_0_ from T_PERSON this_ Not very readable but enough to copy/paste in your favourite query tool. The main drawback of this is that if the query has parameters, they will display as ? and won’t show their values, like in the following output: select this_.PER_N_ID as PER1_0_0_, this_.PER_D_BIRTH_DATE as PER2_0_0_, this_.PER_T_FIRST_NAME as PER3_0_0_, this_.PER_T_LAST_NAME as PER4_0_0_ from T_PERSON this_ where (this_.PER_D_BIRTH_DATE=? and this_.PER_T_FIRST_NAME=? and this_.PER_T_LAST_NAME=?) If they’re are too many parameters, you’re in for a world of pain and replacing each parameter with its value will take too much time. Yet, IMHO, this simple configuration should be enabled in all environments (save production), since it can easily be turned off. Proxy driver The second solution is more intrusive and involves a third party product but is way more powerful. It consists of putting a proxy driver between JDBC and the real driver so that all generated SQL will be logged. It is compatible with all ORM solutions that rely on the JDBC/driver architecture. P6Spy is a driver that does just that. Despite its age (the last release dates from 2003), it is not obsolete and server our purpose just fine. It consists of the proxy driver itself and a properties configuration file (spy.properties), that both should be present on the classpath. In order to leverage P6Spy feature, the only thing you have to do is to tell Hibernate to use a specific driver: com.p6spy.engine.spy.P6SpyDriver ... This is a minimal spy.properties: module.log=com.p6spy.engine.logging.P6LogFactory realdriver=org.hsqldb.jdbcDriver autoflush=true excludecategories=debug,info,batch,result appender=com.p6spy.engine.logging.appender.StdoutLogger Notice the realdriver parameter so that P6Spy knows where to redirect the calls. With just these, the above output becomes: 1270906515233|3|0|statement|select this_.PER_N_ID as PER1_0_0_, this_.PER_D_BIRTH_DATE as PER2_0_0_, this_.PER_T_FIRST_NAME as PER3_0_0_, this_.PER_T_LAST_NAME as PER4_0_0_ from T_PERSON this_ where (this_.PER_D_BIRTH_DATE=? and this_.PER_T_FIRST_NAME=? and this_.PER_T_LAST_NAME=?)|select this_.PER_N_ID as PER1_0_0_, this_.PER_D_BIRTH_DATE as PER2_0_0_, this_.PER_T_FIRST_NAME as PER3_0_0_, this_.PER_T_LAST_NAME as PER4_0_0_ from T_PERSON this_ where (this_.PER_D_BIRTH_DATE=’2010-04-10′ and this_.PER_T_FIRST_NAME=’Johnny’ and this_.PER_T_LAST_NAME=’Be Good’) Of course, the configuration can go further. For example, P6Spy knows how to redirect the logs to a file, or to Log4J (it currently misses a SLF4J adapter but anyone could code one easily). If you need to use P6Spy in an application server, the configuration should be done on the application server itself, at the datasource level. In that case, every single use of this datasource will be traced, be it from Hibernate, TopLink, iBatis or plain old JDBC. In Tomcat, for example, put spy.properties in common/classes and update the datasource configuration to use P6Spy driver. The source code for this article can be found here. To go further: P6Spy official site Log4jdbc, a Google Code contender that aims to offer the same features From http://blog.frankel.ch/debugging-hibernate-generated-sql

First of all, let’s see what is WMI and what it offers. WMI is an acronym for Windows Management Instrumentation, which is basically an interface to the Windows OS system settings, drivers and parameters. It also allows managing Windows personal computers and servers through it. A .NET developer can use WMI to obtain information about drivers installed on the client machine, verify whether the system is licensed or not, check for hardware configuration and a lot more. Quoting Linus Torvalds, “Talk is cheap. Show me the code”, let’s get to the basics of WMI usage. To get data through WMI, a SQL-like query is used. The specific query type is called WQL (WMI Query Language). Don’t let the name confuse you. It is still very similar to SQL. Before diving into code, you should know that Windows comes with a tool called WMI Test Tool, which lets you test WQL queries, to check their correctness and returned results. It is a bit harder to track wrong query results in code, so this tool can save some time for the developer. To run it, just start the Run dialog (or the Command Prompt) and type wbemtest. Once it is started, you will see a window like this: Click on Connect and you will see a dialog like this: It lets you connect to a namespace on your local Windows computer. You can use your credentials (although for the most queries this is not a requirement) and select the impersonation and authentication levels (once again, for the most queries the default settings are acceptable). Once you click connect, you will be able to execute WMI queries, as well as perform other tasks (for example, enumerate classes in a superclass to review its possibilities). Before creating a query, you need to understand what information you want to obtain. The query is executed against a WMI class – you can read the complete list here. Let’s take the Win32_Processor class as an example here. Querying against this class will give us the information about the CPU installed on a machine. If the machine runs with multiple CPUs, a query result will be returned for each one of them. The Win32_Processor class exposes the following properties: AddressWidth Architecture Availability Caption ConfigManagerErrorCode ConfigManagerUserConfig CpuStatus CreationClassName CurrentClockSpeed CurrentVoltage DataWidth Description DeviceID ErrorCleared ErrorDescription ExtClock Family InstallDate L2CacheSize L2CacheSpeed L3CacheSize L3CacheSpeed LastErrorCode Level LoadPercentage Manufacturer MaxClockSpeed Name NumberOfCores NumberOfLogicalProcessors OtherFamilyDescription PNPDeviceID PowerManagementCapabilities[] PowerManagementSupported ProcessorId ProcessorType Revision Role SocketDesignation Status StatusInfo Stepping SystemCreationClassName SystemName UniqueId UpgradeMethod Version VoltageCaps Most of these are have self-descriptive names, but if you are ever confused about one of them, you can always refer to the MSDN documentation for the class, that explains each one of them. Now, let’s try to get the values of the above mentioned properties in your .NET application. In my examples I am using C#, but if you are using another .NET language, you shouldn’t have a problem adapting the code. First of all, you need to add a reference to the System.Management and System. Management.Instrumentation namespaces. This is done by right-clicking on References in the Solution Explorer and selecting Add Reference. Then, you can select the above mentioned libraries from the .NET list: Once selected, you need to reference the proper namespaces in your code: using System.Management; Now, to the actual code. I am going to create a function that can be called from anywhere in the code to simplify this task. void GetCPUInfo() { ManagementObjectSearcher searcher = new ManagementObjectSearcher("SELECT * FROM Win32_Processor"); foreach (ManagementObject obj in searcher.Get()) { if (!(obj == null)) Debug.Print(obj.Properties["CpuStatus"].Value.ToString()); } } The ManagementObjectSearcher is the key element here – it gets the returned properties based on the query. The parameter I am passing to it when instantiating is the actual query. As you see, it is very similar to SQL. My current query will retrieve all properties available in Win32_Processor. I iterate through them (note that each result is a ManagementObject – the property holder, in this case will be a separate instance for each CPU that is found) and print in the Output window the value of the CpuStatus property: The 1 here is exactly what is returned. It is a good practice to consult the documentation before reading specific properties, to understand the possible returned values. 1 for CpuStatus means that the CPU is installed and is active. Important note: Some of the readers might be curious, why there is a null value verification. Some of the classes require user authentication to get the correct data and some properties are simply not available, being the cause of multiple exceptions, depending on the authentication methods and property types. Therefore, to avoid exceptions, this code security measure is used here. If only one property is needed to be retrieved, then the query can be organized like this: SELECT CpuStatus FROM Win32_Processor The important thing to remember here is that when you only retrieve one property, the rest of them are unavailable for that specific query result. Therefore, trying to get their value will cause an exception.

Do you use Jetty and need to change the default setting for browser cache control? Have a look at the init-param element named cacheControl in webdefault.xml. Here’s the default configuration for the version of Jetty I use. Note the element is commented. To enable and configure browser cache control, uncomment and edit the param-value as appropriate. The following example instructs the browser to disable all caching. cacheControl no-store,no-cache,must-revalidate For information on Cache-Control, see RFC 2616, Section 14.9. From http://codeaweso.me/2009/09/jetty-browser-cache-control/

When a hard drive crashes, you can lose all your data. Corrupt hard drives happen out of the blue and for seemingly no good reason. If your hard drive fails, what can you do? One option is to call a hard drive recovery company. If your data is worth a lot of money to you, you can pay a forensic computer company to get the data off your hard drive. Before you write a check though, try a little Do-It-Yourself first. What is going on inside the hard drive is a bunch of little platters spinning at high speed. When data is accessed or written to the disk, a little head (sort of like on a record player) moves to the right spot and does it's magic. The space between the head and the platter is very very tiny. Freezing the hard drive will shrink the head and the platter ever so slightly, often allowing you to read data. Here is how I got the data off of a failed hard drive. Remove the hard drive from the computer. Place the hard drive inside of a zip top freezer bag. (don't buy a cheap bag.) Place the wrapped hard drive inside of ANOTHER zip top freezer bag. (yes, you need to do this) (see figure 1 below) Place the double wrapped hard drive in the coldest part of your freezer. Leave the hard drive in the freezer for 12 hours at least. You want it good and cold! (see figure 2 below) Once very chilled, install the hard drive in your computer and start pulling off data. Begin with the most valuable data. At some point, the hard drive will fail again. When it does, mark the last successfully copied data, pull out the hard drive, double wrap it again and stick it in the Chill Chest for another 12 hours. You may need to do this a number of times to get all the data you want, or until the hard drive stops working completely. Double Wrapped Hard Drive Hard Drive in the Freezer

Some of the Spock Framework 0.4 features are starting to see the light of day, with the Data Tables being explained last week in a nice blog post from Peter Niederwieser. One of the new features that I had not seen before is the new advanced @Unroll usage. Mixed with Data Tables, it produces some very cool results, and it can still be used with 0.3 style specs as well. Here's the juice: JUnit Integration and @Unroll Spock is built on JUnit, and has always had good IDE support without any effort from you as a user. For the most part, the IDEs just think Spock is another unit test. Here's the a Spock spec for the new Data Tables feature and how it shows up in an IDE. import spock.lang.* class TableTest extends Specification { def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a | b | c 3 | 7 | 7 5 | 4 | 5 9 | 9 | 9 } } The assertion will be run 3 times: once for each row in the data table. And JUnit faithfully reports the method name correctly, even when the method names has a space in it: The problem with data driven tests and xUnit is poor error location. When a test fails you will receive an error stating which method is the culprit... but what if the method runs an assertion across 50 or 60 pieces of data? The cause of a failure is almost never clear with data driven tests. At it's worst you have to step through several iterations of code waiting for an exception. Good tests have a clear point of failure, but good tests also do not repeat themselves with boilerplate. This is exactly why Spock has the @Unroll annotation. As a test author you get to write one concise unit test, and JUnit does the work of reporting results that help you isolate failures. Consider the same test method with the @Unroll annotation and the accompanying IDE output. @Unroll def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a | b | c 3 | 7 | 7 5 | 4 | 5 9 | 9 | 9 } When executed, JUnit sees three test methods instead of one: one for each row in the data table: The end result for you as a test writer is accurate failure resolution. You can pinpoint exactly which row failed. This feature is available in Spock 0.3 and you can use it today. What is new in 0.4 is the ability to change the test name dynamically. Here is a full @Unroll annotation that changes the method name: @Unroll("maximum of #a and #b is #c") def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a | b | c 3 | 7 | 7 5 | 4 | 5 9 | 9 | 9 } Notice the #variable syntax in the annotation parameter. The # produces a sort of GString-like variable substitution that lets you bind columns from your data table into your test name. The annotation parameter references #a, #b, and #c, which aligns with the data table definition of a | b | c. Check out the IDE output: Previously, the test name was just the iteration number within the test. The new @Unroll parameter allows you to make the test name much more meaningful. Your tests will improve because failures become more descriptive. Unrolled failure messages before simply had the iteration name embedded in them, while now they can have meaningful data that you prescribe. My favorite part of playing with the new @Unroll was to see the default value of the parameter within the Spock source code: java.lang.String value() default "#featureName[#iterationCount]"; Talk about eating your own dog food... the default value is a test name template, just like you could have written in your own test. Makes you wonder what other variables are in scope, huh? Spock snapshot builds for 0.4 are available at: http://m2repo.spockframework.org. Get it before the link breaks. From http://hamletdarcy.blogspot.com

Perhaps the most striking thing about about the Play! framework is that its biggest advantage over other Java web application development frameworks does not fit into a neat feature list, and is only apparent after you have used it to build something. That advantage is usability. Note that usability is separate from functionality. In what follows, I am not suggesting that you cannot do this in some other framework: I merely claim that it is easier and more pleasant in Play! I need to emphasise this because geeks often have a total blind spot for usability because they enjoying figuring out difficult things, and under-appreciate the value of things that Just Work. Written by web developers for web developers The first hint that something different is going on here is when you first hear that the Play! framework is 'written by web developers for web developers', an unconventional positioning that puts the web's principles and conventions first and Java's second. Specifically, this means that the Play! framework is more in line with the W3C's Architecture of the World Wide Web than it is with Java Enterprise Edition (Java EE) conventions. URLs for perfectionists For example, the Play! framework, like other modern web frameworks, provides first-class support for arbitrary 'clean' URLs, which has always been lacking from the Servlet API. It is no coincidence that at the time of writing, Struts URLs for perfectionists, a set of work-arounds for the Servlet API-based Struts 1.x web framework, remains the third-most popular out of 160 articles on www.lunatech-research.com despite being a 2005 article about a previous-generation Java web technology. In Servlet-based frameworks, the Servlet API does not provide useful URL-routing support; Servlet-based frameworks configure web.xml to forward all requests to a single controller Servlet, and then implement URL routing in the framework, with additional configuration. At this point, it does not matter whether the Servlet API was ever intended to solve the URL-routing problem and failed by not being powerful enough, or whether it was intended to be a lower-level API that you do not build web applications in directly. Either way, the result is the same: web frameworks add an additional layer on top of the Servlet API, itself a layer on top of HTTP. Play! combines the web framework, HTTP API and the HTTP server, which allows it to implement the same thing more directly with fewer layers and a single URL routing configuration. This configuration, like Groovy's and Cake PHP's, reflects the structure of an HTTP request - HTTP method, URL path, and then the mapping: # Play! 'routes' configuration file… # Method URL path Controller GET / Application.index GET /about Application.about POST /item Item.addItem GET /item/{id} Item.getItem GET /item/{id}.pdf Item.getItemPdf In this example, there is more than one controller. We also see the use of an id URL parameter in the last two URLs. HttpServletRequest Another example is Play!'s Http.Request class, which is a far simpler than the Servlet API's HttpServletRequest interface. In addition, Play! uses a class where Java EE 6 uses the Java EE convention of using an interface. This interface is also split between HttpServletRequest and the more generic ServletRequest interface. This separation may be useful if you want to use Servlets for things other than web applications, or if you want to allow for the unlikely possibility of the web changing protocol, but for most of us it is merely irrelevant complexity. In other words, the Servlet API is always used with a framework on top these days because it is sub-optimised for building web applications, which is what all of us actually use it for. Play! fixes that. Better usability is not just for normal people Another way of looking at the idea that Play! is by and for web developers is to consider how a web developer might approach software design differently to a Java EE developer. When you write software, what is the primary interface? If you are a web developer, the primary interface is a web-based user-interface constructed with HTML, CSS and (increasingly) JavaScript. A Java EE developer, on the other hand, may consider their primary interface to be a Java API, or perhaps a web services API, for use by other layers in the system. This difference is a big deal, because a Java interface is intended for use by other programmers, while a web user-interface interface is intended for use by non-programmers. In both cases, good design includes usability, but usability for normal people is not the same as usability for programmers. In a way, usability for everyone is a higher standard than usability for programmers, when it comes to software, because programmers can cope better with poor usability. This is a bit like the Good Grips kitchen utensils: although they were originally designed to have better usability for elderly people with arthritis, it turns out that making tools easier to hold is better for all users. The Play! framework is different because the usability that you want to achieve in your web application is present in the framework itself. For example, the web interface to things like the framework documentation and error messages shown in the browser is just more usable. Along similar lines, the server's console output avoids the pages full of irrelevant logging and pages of stack traces when there is an error, leaving more focused and more usable information for the web developer. $ play run phase ~ _ _ ~ _ __ | | __ _ _ _| | ~ | '_ \| |/ _' | || |_| ~ | __/|_|\____|\__ (_) ~ |_| |__/ ~ ~ play! 1.0, http://www.playframework.org ~ ~ Ctrl+C to stop ~ Listening for transport dt_socket at address: 8000 10:15:58,629 INFO ~ Starting /Users/peter/Documents/work/workspace/phase 10:16:00,007 WARN ~ You're running Play! in DEV mode 10:16:00,424 INFO ~ Listening for HTTP on port 9000 (Waiting a first request to start) ... 10:16:11,847 INFO ~ Connected to jdbc:hsqldb:mem:playembed 10:16:13,448 INFO ~ Application 'phase' is now started ! 10:16:14,825 INFO ~ starting DispatcherThread 10:16:48,168 ERROR ~ @61lagcl6i Internal Server Error (500) for request GET /application/startprocess?account=x Java exception (In /app/controllers/Application.java around line 41) IllegalArgumentException occured : Person not found for account x play.exceptions.JavaExecutionException: Person not found for account x at play.mvc.ActionInvoker.invoke(ActionInvoker.java:200) at Invocation.HTTP Request(Play!) Caused by: java.lang.IllegalArgumentException: Person not found for account x at controllers.Application.startProcess(Application.java:41) at play.utils.Java.invokeStatic(Java.java:129) at play.mvc.ActionInvoker.invoke(ActionInvoker.java:127) ... 1 more Try to imagine a JSF web application producing a stack trace this short. In fact, Play! goes further: instead of showing the stack trace, the web application shows the last line of code within the application that appears in the stack trace. After all, what you really want to know is where things first went wrong in your own code. This kind of usability does not happen by itself; the Play! framework goes to considerable effort to filter out duplicate and irrelevant information, and focus on what is essential. Quality is in the details In the Play! framework, much of the quality turns out to be in the details: they may be small things individually, rather than big important features, but they add up to result in a more comfortable and more productive development experience. The warm feeling you get when building something with Play! is the absence of the frustration that usually results from fighting the framework. We recommend that you go to http://www.playframework.org/, download the latest binary release, and spend half an hour on the tutorial. Peter Hilton is a senior software developer at Lunatech Research.

This article shows you step by step how to cache your entire tomcat web application with Squid reverse Proxy without writing any Java code. What is Squid Squid is a free proxy server for HTTP, HTTPS and FTP which saves bandwidth and increases response time by caching frequently requested web pages. While squid can be used as a proxy server when users try to download pages from the internet, it can be also used as a reverse-proxy by putting squid between the user and your webapp. All user requests first hit Squid. If the requested page already exists in Squid’s cache it is served directly from the cache without hitting your Webapp. If the page does not exist in Squid’s cache, it is fetched from your web application and stored in the cache for future requests. Squid reduces hits to your server by caching response pages. You don’t have to worry about building page level caching in every application that your write, Squid takes care of that part. When should I use Squid Ideally you should use Squid for pages which have a high ratio of reads to writes. In other words, a page that changes less frequently but is accessed very often. Here are some scenarios: A dynamical web page which displays news and is updated once an hour, and receives hundreds of hits during the hour A static web page accessed freqently. Squid can give performance boost by caching frequently accessed static web pages in memory When should I not use Squid In most cases, if the request URL is the only factor which determines the response then you can safely use Squid. See more specific examples below: If the entire apps is very dynamic in nature, and the validity of pages changes immediately. Squid is not suitable for apps which require login. This unfortunately is a large number of applications. Such applications need to resort to back end caching, for example use other caching frameworks like Ehcache to cache re-usable page fragments and/or cache database queries and/or other performance bottlenecks. Apps which heavily use browser cookies. Squid relies on URLs to cache pages. If the page served is computed from URLs + cookies, then you should not cache those pages in Squid. How does the overall setup work Apache Squid Tomcat architecture Apache receives requests on port 80. Apache calls Squid with the request. Squid checks its cache to see if it has the response cached from before. If yes and if the response is not expired, it returns the cached response.In this case: Squid will write the following header to the response X-Cache: HIT from www.vineetmanohar.com X-Cache: HIT from www.vineetmanohar.com If the response is not found in Squid’s cache, squid will make a call to Tomcat on port 8082. Tomcat’s proxy connector is listening on this port. It processes the request and sends the response back to Squid. Squid saves the response in its cache, unless caching is disabled for that URL. Squid returns the final response to Apache which sends the response back to the user. What if I don’t want to use Apache Using Apache is not required to use Squid. You can run Squid on port 80, and point your users directly to Squid. If that is the case, skip section one and directly jump to section 2 below. Step 1/3: Apache Httpd Config If you are using Apache as a front end, you need to instruct Apache to forward requests to Squid at port 3128. See the following code snippet. Change the server name and paths to reflect your real values. Apache config file: /etc/httpd/conf/httpd.conf ServerName www.vineetmanohar.com DocumentRoot /home/webadmin/www.vineetmanohar.com/html # forward requests to squid running on port 3128 ProxyPass / http://localhost:3128/ ProxyPassReverse / http://localhost:3128/ /etc/httpd/conf/httpd.conf ServerName www.vineetmanohar.com DocumentRoot /home/webadmin/www.vineetmanohar.com/html # forward requests to squid running on port 3128 ProxyPass / http://localhost:3128/ ProxyPassReverse / http://localhost:3128/ In addition to the above, you also need mod_proxy installed. If you see the following in your httpd.conf, you probably already have mod_proxy installed. If you first need to install mod_proxy LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so Step 2/3: Squid Config First make sure that Squid is installed on your server. You can download Squid from here. The squid config file on Linux/Unix is located at this location /etc/squid/squid.conf /etc/squid/squid.conf The config file is pretty long. Follow these instructions and set the values appropriately. 1. # leave the port to 3128 2. http_port 3128 3. 4. # how much memory cache do you want? depends on how much memory you have on the machine 5. cache_mem 200 MB 6. 7. # what's the biggest page that you want stored in memory. If you home page is 100 KB and 8. # you want it stored in memory, you may set it to a number bigger than that. 9. maximum_object_size_in_memory 100 KB 10. 11. # how much disk cache do you want. It is 6400 MB in the following example, change it as per 12. # your needs. Make sure you have that much disk space free. 13. cache_dir ufs /var/spool/squid 6400 16 256 14. 15. # this is probably the most important config section. Here you can configure the cache life for 16. # each URL pattern. 17. 18. # Time is in minutes 19. # 1 day = 1440, 2 days = 2880, 7 days = 10080, 28 days = 40320 20. 21. # do not cache url1 22. refresh_pattern ^http://127.0.0.1:8082/url1/ 0 20% 0 23. 24. # cache url2 for 1 day 25. refresh_pattern ^http://127.0.0.1:8082/url2/ 1440 20% 1440 override-expire override-lastmod reload-into-ims ignore-reload 26. 27. # cache css for 7 days 28. refresh_pattern ^http://127.0.0.1:8082/css 10080 20% 10080 override-expire override-lastmod reload-into-ims ignore-reload 29. 30. # by default cache the whole website for 1 minute 31. refresh_pattern ^http://127.0.0.1:8082/ 0 20% 0 override-expire override-lastmod reload-into-ims ignore-reload 32. 33. # how long should the errors should be cached for. For example 404s, HTTP 500 errors 34. negative_ttl 0 seconds 35. 36. # On which host does tomcat run. Set 127.0.0.1 for localhost 37. httpd_accel_host 127.0.0.1 38. 39. # this is the proxy port as defined in Tomcat server.xml. By default it is "8082" 40. httpd_accel_port 8082 41. 42. # set this to "on". Read more documentation if you want to change this. 43. httpd_accel_single_host on 44. 45. # To access Squid stats via the manager interface, you need to enter a password here 46. cachemgr_passwd your_clear_text_password all 47. 48. # Say "off" if you want the query string to appear in the squid logs. 49. strip_query_terms off # leave the port to 3128 http_port 3128 # how much memory cache do you want? depends on how much memory you have on the machine cache_mem 200 MB # what's the biggest page that you want stored in memory. If you home page is 100 KB and # you want it stored in memory, you may set it to a number bigger than that. maximum_object_size_in_memory 100 KB # how much disk cache do you want. It is 6400 MB in the following example, change it as per # your needs. Make sure you have that much disk space free. cache_dir ufs /var/spool/squid 6400 16 256 # this is probably the most important config section. Here you can configure the cache life for # each URL pattern. # Time is in minutes # 1 day = 1440, 2 days = 2880, 7 days = 10080, 28 days = 40320 # do not cache url1 refresh_pattern ^http://127.0.0.1:8082/url1/ 0 20% 0 # cache url2 for 1 day refresh_pattern ^http://127.0.0.1:8082/url2/ 1440 20% 1440 override-expire override-lastmod reload-into-ims ignore-reload # cache css for 7 days refresh_pattern ^http://127.0.0.1:8082/css 10080 20% 10080 override-expire override-lastmod reload-into-ims ignore-reload # by default cache the whole website for 1 minute refresh_pattern ^http://127.0.0.1:8082/ 0 20% 0 override-expire override-lastmod reload-into-ims ignore-reload # how long should the errors should be cached for. For example 404s, HTTP 500 errors negative_ttl 0 seconds # On which host does tomcat run. Set 127.0.0.1 for localhost httpd_accel_host 127.0.0.1 # this is the proxy port as defined in Tomcat server.xml. By default it is "8082" httpd_accel_port 8082 # set this to "on". Read more documentation if you want to change this. httpd_accel_single_host on # To access Squid stats via the manager interface, you need to enter a password here cachemgr_passwd your_clear_text_password all # Say "off" if you want the query string to appear in the squid logs. strip_query_terms off Step 3/3: Tomcat Config Make sure that the HTTP Proxy Connector is defined in TOMCAT_HOME/conf/server.xml. If needed, see additional documentation on Tomcat proxy connector. Squid Manager Interface You can access the Squid config and stats via the Squid Manger HTTP interface. Make sure that the “cachemgr.cgi” file which ships with squid installation is in your cgi-bin directory. More documentation on setting that up here. Once you’ve set it up, you can access the cache manager via this URL: http:///cgi-bin/cachemgr.cgi http:///cgi-bin/cachemgr.cgi To continue enter the following values: Cache host: localhost Cache port: 3128 Manager name: manager Password: Cache host: localhost Cache port: 3128 Manager name: manager Password: Store Directory Stats shows you how much disk space is used by the disk cache. Cache Client List show you the cache HIT/MISS ratio as %. You should monitor this frequently and tune your cache to get a higher hit %. Reload Squid Config without restarting Edit the squid config using “vi” or your favorite editor vi /etc/squid/squid.conf vi /etc/squid/squid.conf Once you are done editing, reload the new config without restarting Squid /usr/sbin/squid -k reconfigure /usr/sbin/squid -k reconfigure Clearing Squid Cache To clear Squid cache: 1) Set the memory cache to 4 MB (or a lower number) cache_mem 8 MB cache_mem 8 MB 2) Set the disk cache to 8 MB (or a lower number). The disk cache must be higher that the memory cache. cache_dir ufs /var/spool/squid 20 16 256 cache_dir ufs /var/spool/squid 20 16 256 3) Reload squid config without restart as described in the previous section 4) You may need to wait a few hours for the cache to get cleared. Once the cache is clear, you may restore the previous cache sizes and reload the new config again. You can monitor the cache size through the Squid Manager HTTP interface. Bypassing Squid If for some reason you need to bypass Squid, reconfigure Apache to directly send requests to Tomcat. Edit the Apache config file /etc/httpd/conf/httpd.conf # forward requests directly to Tomcat's proxy connector running on port 8082 ProxyPass / http://localhost:8082/ ProxyPassReverse / http://localhost:8082/ # forward requests directly to Tomcat's proxy connector running on port 8082 ProxyPass / http://localhost:8082/ ProxyPassReverse / http://localhost:8082/ You will need to restart Apache after making this change. /etc/init.d/httpd restart Conclusion Squid is a very powerful tool for caching. It is not for all applications. Please examine the need of your application and use squid appropriately. I’ve used squid for several years for caching the output from a Java data mashup application and am very satisfied with the ease of use and benefits. Hope you found this tutorial useful. Feel free to post a comment or share your experience with squid. References Squid official website From http://www.vineetmanohar.com

For almost a year now, the idea of "NoSQL" has been spreading due to the demand for relational database alternatives. Maybe the biggest motivation behind NoSQL is scalability. Relational databases don't lend themselves well to the kind of horizontal scalability that's required for large-scale social networking or cloud applications, and ORMs can abstract away impedance mismatch only so much. In other cases, companies just don't need as many of the complex features and rigid schemas provided by relational databases. Most people are not suggesting that we all ditch the RDBMS, in fact, many companies don't really need to switch. Relational databases will probably be necessary for many applications years and years from now. In essence, NoSQL is a movement that aims to reexamine the way we structure data and draw attention to innovation in hopes of finding the solution to the next generation's data persistence problems. Here are some of the better known open source data stores/models labeled as "NoSQL": CouchDB- Document Store Maps keys to data It provides a RESTful JSON API and is written in Erlang You can upload functions to index data and then you can call those functions Has a very simple REST interface Provides an innovative replication strategy - nodes can reconnect, sync, and reconcile differences after being disconnected for long periods of time Enables new distributed types of applications and data MongoDB - Document Store Free-form key-value-like data store with good performance Powerful, expansive query model Usability rivals that of Redis Good for complex data storage needs. Production-quality sharding capabilities Neo4j - GraphDB Disk-based Has a restricted, single-threaded model for graph traversal Has optional layers to expose Neo4j as an RDF store Can handle graphs of several billion nodes, relationships, or properties on a single machine Released under a dual license - free for non-commercial use Apache Hbase - Wide Column Store/Column Families Built on top of Hadoop, which has functionality similar to Google's GFS and MapReduce systems Hadoop's HDFS provides a mechanism that reliably stores and organizes large amounts of data Random access performance is on par with MySQL Has a high performance Thrift gateway Cascading source and sink modules Redis - Key Value/Tuple Store Provides a rich API and does more operations in memory, using disk only periodically. It's extremely fast Lets you append a value to the end of a list of items that's already been stored on a key. Has atomic operations, making it a best-of-breed tally server. Memcached - Key Value/Tuple Store High-performance, distributed memory object caching Free and open source Generic and agnostic to the objects/strings it caches It's all in-memory data Simple yet elegant design enables easy development and deployment Language neutral caching scheme. Most of the large properties on the web are using it now, except for Microsoft Project Voldemort - Eventually Consistent Key Value Store Used by LinkedIn Handles server failure transparently Pluggable serialization supports rich keys and values including lists and tuples with named fields Supports common serialization frameworks including Protocol Buffers, Thrift, and Java Serialization Data items are versioned Supports pluggable data placement strategies Memory caching and the storage system are combined Tokyo Cabinet and Tokyo Tyrant - Key Value/Tuple Store Supports hashtable mode, b-tree mode, and table mode It's fast and straightforward Good for small to medium-sized amounts of data that require rapid updating and can be easily modeled in terms of keys and values Cassandra - Wide Column Store/Column Families First developed by Facebook SuperColumns can turn a simple key-value architecture into an architecture that handles sorted lists, based on an index specified by the user. Can scale from one node to several thousand nodes clustered in different data centers. Can be tuned for more consistency or availability Smooth node replacement if one goes down ____ Some other well known NoSQL-style data stores that are closed source include Google BigTable and Amazon SimpleDB. GigaSpaces is a popular space-based Grid solution that has NoSQL qualities. Check out this informative post on NoSQL patterns.

Learn the Abstract Factory Design Pattern with easy Java source code examples as James Sugrue continues his design patterns tutorial series, Design Patterns Uncovered

This week, I searched for free online SVN repositories for closed-source projects.

It seems that the Groovy has a project for just about anything. That's one of the reasons why the language is so popular. The GPars library is an especially useful project in this new era of mult-core processors and concurrent programming. Formerly known as GParallelizer, GPars offers a framework for handling tasks concurrently and asynchronously while safe-guarding mutable values. DZone recently got an update on the project's latest news from project lead Václav Pech, and we've provided some examples of GPars concepts. GPars uses some of the best concepts from emerging languages and implements them for Groovy. Its actor support was inspired by the Actors library in Scala and the SafeVariable class in GPars was inspired by Agents in Clojure. The Groovy-based APIs in GPars are used to declare which parts of the code should be run concurrently. Objects can be enhanced with asynchronous methods to perform collections-based operations in parallel based on the fork/join model. GPars also has a Dataflow concurrency model that offers an alternative model that is inherently safe and robust due to algorithms that prevent having to deal with live-locks and race-conditions. The SafeVariable class is another technology in GPars that alleviates problems with concurrency by providing a non-blocking mt-safe reference to mutable state when Java libraries are integrated. Finally the Parallelizer, Asynchronizer, and Actors are some of the most interesting concepts in GPars. Actors Actors can be generated quickly to consume and send messages between each other even across distributed machines. You can build a messaging-based concurrency model with actors that are not limited by the number of threads. What was once only available to Scala developers, GPars now brings to Java and Groovy developers. Actors perform three different operations - send messages, receive messages and create new actors. New actors are created with the actor() method passing in the actor's body as a closure parameter. Inside the actor's body loop() is used to iterate, react() to receive messages, and reply() to send a message to the actor, which has sent the currently processed message. Here is how to create an actor that prints out all messages that it receives: import static groovyx.gpars.actor.Actors.* def console = actor { loop { react { println it } } } The loop() method ensures that the actor doesn't stop after processing the first message. Messages are sent using the send() method or the << operator. Here is an example of the sendAndWait () method in a message: actor << 'Message' actor.send 'Message' def reply1 = actor.sendAndWait('Message') def reply2 = actor.sendAndWait(10, TimeUnit.SECONDS, 'Message') def reply3 = actor.sendAndWait(10.seconds, 'Message') The sendAndWait() family blocks the caller until a reply from the actor becomes available. The reply is returned from sendAndWait() as a return value. For non-blocking message retrieval, calling the react() method, with or without a timeout parameter, from within the actor's code will consume the next message from the actor's inbox: println 'Waiting for a gift' react {gift -> if (myWife.likes gift) reply 'Thank you!' } Here is a more 'real world' example of an event-driven actor that receives two numeric messages, generates a sum, and sends the result to the console actor: import static groovyx.gpars.actor.Actors.* //not necessary, just showing that a single-threaded pool can still handle multiple actors defaultPooledActorGroup.resize 1 final def console = actor { loop { react { println 'Result: ' + it } } } final def calculator = actor { react {a -> react {b -> console.send(a + b) } } } calculator.send 2 calculator.send 3 calculator.join() Since Actors can share a relatively small thread pool, they bypass the threading limitations of the JVM and don't require excessive system resources even if an application consists of thousands of actors. There are some more sophisticated actor examples on the old GParallelizer wiki and there's also a nice article on the key concepts behind actors in erlang and scala. The documentation on GPars Actors can be found here. Asynchronizer A major feature of GPars is the Asynchronizer class, which runs tasks asynchronously in the background. It enables a Java Executor Service-based DSL on collections and closures. Inside the Asynchronizer.doParallel() blocks, asynchronous methods can be added to the closures. async() creates a variant of the supplied closure returning a future for the potential return value when invoked. callAsync() calls a closure in a separate thread supplying the given arguments and also returns a future for the potential value. Here is one example of Asynchronizer use: Asynchronizer.doParallel() { Closure longLastingCalculation = {calculate()} Closure fastCalculation = longLastingCalculation.async() //create a new closure, which starts the original closure on a thread pool Future result=fastCalculation() //returns almost immediately //do stuff while calculation performs … println result.get() } Parallelizer Finally, there's the Parallelizer, which is a concurrent collection processor. The common pattern to process collections takes elements sequentially, one at a time. This algorithm however, won't work well on multi-core hardware. The min() function on a dual-core chip can only leverage 50% of the computing power - 25% for a quad-core. Instead, GPars uses a tree-like structure for parallel processing. The Parallelizer class enables a ParallelArray(from JSR-166y)-based DSL on collections. Here is a use exapmle: doParallel { def selfPortraits = images.findAllParallel{it.contains me}.collectParallel {it.resize()} //a map-reduce functional style def smallestSelfPortrait = images.parallel.filter{it.contains me}.map{it.resize()}.min{it.sizeInMB} } Václav Pech told DZone that GPars currently has two new people joining the project - Jon Kerridge and Kevin Chalmers. The two developers are bringing their JCSP Groovy library with them into GPars. Pech said, "Apart from experimenting with the CSP concept, we will also enhance actor remoting and polish a couple of rough edges on the APIs. The documentation and especially the samples also deserve more attention." GPars' documentation is already quite robust. Pech says there quite a few issues queued up in their JIRA, but the previously listed issues remain the top priorities. Depending on the amount of time required to make progress with CSP, the GPars 1.0 release might happen in the summer says Pech. GPars is also developed by Alex Tkachman, a leading developer on the Groovy++ project. In an with Andres Almiray, Tkachman said that some of the work that comes out of Groovy++ might be assimilated into GPars, but no plans are in place yet since Groovy++ developers are still experimenting.