As promised I am going to provide an article on how we can use Sphinx with Java to perform a full text search. I will begin the article with an introduction to Sphinx. Introduction to Sphinx Databases are continually growing and sometimes tend to hold about 100M records and need an external solution for full text search to be performed. I have picked Sphinx, an open source full-text search engine, distributed under GPL version 2 to perform a full text search on such a huge amount of data. Generally, it's a standalone search engine meant to provide fast, size-efficient and relevant full-text search functions to other applications very much compatible with an SQL Database. So my example will be based on the MySQL database, as we cannot produce millions of data to evaluate the real power of Sphinx, we will have a small amount of data and I think that should not be a problem. Here are few Sphinx Unique Features: high indexing speed (up to 10 MB/sec on modern CPUs) high search speed (avg query is under 0.1 sec on 2-4 GB text collections) high scalability (up to 100 GB of text, upto 100 M documents on a single CPU) provides distributed searching capabilities provides searching from within MySQL through pluggable storage engine supports boolean, phrase, and word proximity queries supports multiple full-text fields per document (upto 32 by default) supports multiple additional attributes per document (ie. groups, timestamps, etc) supports MySQL natively (MyISAM and InnoDB tables are both supported) The important features which have been adopted to perform a full text search are the provision of the Java API to integrate easily with the web application and considerably high indexing and searching speed with an average of 4-10 MB/sec & 20-30 ms/q @5GB,3.5M docs(wikipedia) Sphinx Terms & How It Works The fist principle part of sphinx is indexer. It is solely responsible for gathering the data that will be searchable. From the Sphinx point of view, the data it indexes is a set of structured documents, each of which has the same set of fields. This is biased towards SQL, where each row corresponds to a document, and each column to a field. Sphinx builds a special data structure optimized for our queries from the data provided. This structure is called index; and the process of building index from data is called indexing and the element of sphinx which carries out these tasks is called indexer. Indexer can be executed either from a regular script or command-line interface. Sphinx documents are equal to records in DB. Document is set of text fields and number attributes + unique ID – similar to row in DB Set of fields and attributes is constant for index – similar to table in DB Fields are searchable for FullText queries Attributes may be used for filtering, sorting, grouping searchd is the second principle tools as part of Sphinx. It is the part of the system which actually handles searches; it functions as a server and is responsible for receiving queries, processing them and returning a dataset back to the different APIs for client applications. Unlike indexer, searchd is not designed to be run either from a regular script or command-line calling, but instead either as a daemon to be called from init.d (on Unix/Linux type systems) or to be called as a service (on Windows-type systems). I am going to focus on Windows environment so later I will show you how we can install sphinx on windows as a service. Finally search is one of the helper tools within the Sphinx package. Whereas searchd is responsible for searches in a server-type environment, search is aimed at testing the index quickly without building a framework to make the connection to the server and process its response. This will only be used for testing sphinx from command – line and with respect to application’s requirement; searchd service will be used to query the MySql Server with a pre created index. Installation on Windows So now we come to the part of installing Sphinx on Windows: Download Sphinx from the official Sphinx download site i.e http://sphinxsearch.com (I downloaded Win32 release binaries with MySQL support: sphinx-0.9.9-win32.zip) Unzip the file to some folder, I unzipped to C:\devel\sphinx-0.9.9-win32 and added the bin directory to the windows path variable Well Sphinx is installed. Nice, simple, easy. Later I will tell how to set up indexes and search. Sample Application Till now I guess the whole motto of this article is clear to you, let's move ahead to define our sample application. We all use the Address Book to search for people by using their name or e-mail address when we want to immediately address an e-mail message to a specific person, people, or distribution list. We also search for people by using other basic information, such as e-mail alias, office location, and telephone number etc. I think most of the people on this planet are quire familiar with this kind of search, so let's make outlook address book as our sample database schema. Most of the fields are mapped from microsoft outlook, the only additional column is date of joining so that we can filter our queries based on joining dates of the employees. The example that I am going to put forth will use Sphinx to search for a particluar address entry using free text search, meaning the user is free to type in anything, here is our search screen, the DOJ (date of joining) search parameter is optional. The screen is self explanatory, let's move ahead and define our database. As Sphinx works well with MySQL and MySQL being free also, lets create our db scripts around mysql database (Those who wish to install MySQL can dowload it from http://www.mysql.com) Let's create our sample database 'addressbook' mysql> create database addressbook; Query OK, 1 row affected (0.03 sec) mysql> use addressbook; Database changed Note: The fields defined in the following tables are for the purpose of learning only and may not contain a complete set of fields that microsoft address book or any similar software may provide. mysql> CREATE TABLE addressbook ( Id int(11) NOT NULL, FirstName varchar(30) NOT NULL, LastName varchar(30) NOT NULL, OfficeId int(11) DEFAULT NULL, Title varchar(20) DEFAULT NULL, Alias varchar(20) NOT NULL, Email varchar(50) NOT NULL, DOJ date NOT NULL, PhoneNo varchar(20) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; mysql> CREATE TABLE CompanyLocations ( Id int(11) NOT NULL, Location varchar(60) NOT NULL, Country varchar(20) NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; It's time to put some dummy data into the table, so let's fill our tables. Our virtual company 'gogs.it' has six offices across India and Singapore as defined in the following insert script. mysql> insert into CompanyLocations (Id, Location, Country) VALUES (1, 'Tower One, Harbour Front, Singapore', 'SG'); insert into CompanyLocations (Id, Location, Country) VALUES (2, 'DLF Phase 3, Gurgaon, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (3, 'Hiranandani Gardens, Powai, Mumbai, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (4, 'Hinjwadi, Pune, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (5, 'Toll Post, Nagrota, Jammu, India', 'IN'); insert into CompanyLocations (Id, Location, Country) VALUES (6, 'Bani (Kathua), India', 'IN'); Now comes the real stuff... The data sphinx is going to index, let's populate that as well...wooooo mysql> INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (1,'Aabheer','Kumar',1,'Mr','u534','[email protected]','2008-9-3', '+911234599990'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (2,'Aadarsh','Gupta',6,'Mr','u668','[email protected]','2007-2-23','+911234599991'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (3,'Aachman','Singh',5,'Mr','u2766','[email protected]','2006-12-18','+911234599992'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (4,'Aadesh','Shrivastav',5,'Mr','u3198','[email protected]','2007-11-23','+911234599993'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (5,'Aadi','manav',1,'Mr','u2686','[email protected]','2010-7-20','+911234599994'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (6,'Aadidev','singh',4,'Mr','u572','[email protected]','2010-8-18','+911234599995'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (7,'Aafreen','sheikh',4,'Smt','u1092','[email protected]','2007-7-11','+911234599996'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (8,'Aakar','Sherpa',5,'Mr','u1420','[email protected]','2009-10-3','+911234599997'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (9,'Aakash','Singh',4,'Mrs','u2884','[email protected]','2008-6-11','+911234599998'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (10,'Aalap','Singhania',4,'Mrs','u609','[email protected]','2010-10-8','+911234599999'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (11,'Aandaleeb','mahajan',1,'Smt','u131','[email protected]','2010-10-21','+911234580001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (12,'Mamata','kumari',5,'Sh','u2519','[email protected]','2009-6-12','+911234580002'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (13,'Mamta','sharma',6,'Smt','u4123','[email protected]','2009-2-8','+911234580003'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (14,'Manali','singh',6,'Mr','u1078','[email protected]','2008-6-14','+911234580004'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (15,'Manda','saxena',1,'Mrs','u196','[email protected]','2010-9-4','+911234580005'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (16,'Salila','shetty',3,'Miss','u157','[email protected]','2009-11-15','+911234580006'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (17,'Salima','happy',3,'Mrs','u3445','[email protected]','2006-7-14','+911234580007'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (18,'Salma','haik',5,'Sh','u4621','[email protected]','2008-6-23','+911234580008'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (19,'Samita','patil',3,'Smt','u3156','[email protected]','2006-6-7','+911234580009'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (20,'Sameena','sheikh',5,'Mrs','u952','[email protected]','2008-8-13','+911234580010'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (21,'Ranita','gupta',5,'Mrs','u2664','[email protected]','2008-10-20','+911234580011'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (22,'Ranjana','sharma',1,'Sh','u3085','[email protected]','2010-6-21','+911234580012'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (23,'Ranjini','singh',6,'Mrs','u4200','[email protected]','2007-4-13','+911234580013'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (24,'Ranjita','vyapari',2,'Smt','u1109','[email protected]','2008-1-22','+911234580014'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (25,'Rashi','gupta',6,'Mrs','u3492','[email protected]','2006-2-2','+911234580015'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (26,'Rashmi','sehgal',3,'Mr','u3248','[email protected]','2008-9-9','+911234580016'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (27,'Rashmika','sexy',1,'Mrs','u4599','[email protected]','2009-3-12','+911234580017'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (28,'Rasika','dulari',3,'Smt','u2089','[email protected]','2009-1-24','+911234580018'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (29,'Dilber','lover',6,'Mr','u4241','[email protected]','2007-10-11','+911234580019'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (30,'Dilshad','happy',1,'Mr','u1564','[email protected]','2007-4-8','+911234580020'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (31,'Dipali','lights',5,'Sh','u1127','[email protected]','2006-11-1','+911234580021'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (32,'Dipika','lamp',1,'Sh','u2271','[email protected]','2010-12-17','+911234580022'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (33,'Dipti','brightness',5,'Smt','u422','[email protected]','2010-9-25','+911234580023'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (34,'Disha','singh',3,'Sh','u4604','[email protected]','2006-5-2','+911234580024'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (35,'Maadhav','Krishna',1,'Miss','u2561','[email protected]','2007-11-6','+911234580025'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (36,'Maagh','month',5,'Miss','u874','[email protected]','2008-5-8','+911234580026'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (37,'Maahir','Skilled',4,'Mr','u3372','[email protected]','2007-8-4','+911234580027'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (38,'Maalolan','Ahobilam',5,'Mrs','u3498','[email protected]','2007-7-9','+911234580028'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (39,'Maandhata','King',1,'Smt','u2089','[email protected]','2009-9-3','+911234580029'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (40,'Maaran','Brave',2,'Miss','u4020','[email protected]','2008-4-5','+9112345606001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (41,'Maari','Rain',2,'Sh','u3593','[email protected]','2007-12-5','+9112345606002'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (42,'Madan','Cupid',4,'Mrs','u795','[email protected]','2007-11-11','+9112345606003'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (43,'Madangopal','Krishna',3,'Sh','u438','[email protected]','2007-2-19','+9112345606004'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (44,'sahil','gogna',1,'Sh','u2273','[email protected]','2007-10-7','+9112345606005'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (45,'nikhil','gogna',2,'Mr','u1240','[email protected]','2009-9-14','+9112345606006'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (46,'amit','gogna',5,'Sh','u3879','[email protected]','2006-2-8','+9112345606007'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (47,'krishan','gogna',4,'Miss','u3632','[email protected]','2010-9-20','+9112345606008'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (48,'anil','kashyap',4,'Smt','u3939','[email protected]','2010-3-15','+9112345606009'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (49,'sunil','kashyap',5,'Mrs','u3493','[email protected]','2008-3-16','+9112345606010'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (50,'sandy','singh',6,'Mrs','u4691','[email protected]','2009-6-2','+9112345606011'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (51,'vishal','kapoor',3,'Mr','u1087','[email protected]','2010-5-13','+9112345606012'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (52,'bala','ji',5,'Mrs','u4762','[email protected]','2007-8-9','+9112345606013'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (53,'karan','sarin',4,'Miss','u3030','[email protected]','2008-4-8','+9112345606014'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (54,'abhishek','kumar',4,'Miss','u1093','[email protected]','2008-12-21','+9112345605001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (55,'babu','the',1,'Miss','u1055','[email protected]','2008-7-2','+9112345506001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (56,'sandeep','gainda',3,'Miss','u1320','[email protected]','2010-5-14','+9112345606301'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (57,'dheeraj','kumar',3,'Miss','u3685','[email protected]','2007-10-14','+9112345606091'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (58,'dharmendra','chauhan',1,'Smt','u3235','[email protected]','2008-8-1','+9112345806001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (59,'max','alan',3,'Smt','u3465','[email protected]','2009-5-5','+9112345608011'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (60,'hidayat','khan',3,'Smt','u958','[email protected]','2007-11-18','+911234599101'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (61,'himnashu','singh',4,'Miss','u2027','[email protected]','2008-3-2','+911234599102'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (62,'dinesh','kumar',6,'Sh','u3233','[email protected]','2008-5-9','+911234599103'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (63,'toshi','prakash',1,'Mr','u3766','[email protected]','2010-9-17','+911234599104'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (64,'niti','puri',3,'Mr','u3575','[email protected]','2009-11-15','+911234599105'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (65,'pawan','tikki',3,'Sh','u3919','[email protected]','2006-3-19','+911234599106'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (66,'gaurav','sharma',2,'Sh','u413','[email protected]','2010-4-2','+911234599107'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (67,'himanshu','verma',2,'Mrs','u4732','[email protected]','2009-3-20','+911234599108'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (68,'priyanshu','verma',3,'Sh','u183','[email protected]','2010-8-12','+911234599109'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (69,'nitika','luthra',2,'Mrs','u4259','[email protected]','2010-7-12','+911234599110'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (70,'neeru','gogna',2,'Sh','u1633','[email protected]','2010-6-23','+91532110000'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (71,'bindu','gupta',1,'Sh','u1859','[email protected]','2006-11-10','+91532110001'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (72,'gurleen','bakshi',5,'Miss','u1423','[email protected]','2007-7-1','+91532110003'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (73,'rahul','gupta',3,'Sh','u1223','[email protected]','2009-8-11','+91532110004'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (74,'jagdish','salgotra',3,'Mr','u12','[email protected]','2008-5-19','+91532110005'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (75,'vikas','sharma',3,'Smt','u465','[email protected]','2006-6-2','+91532110006'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (76,'poonam','mahendra',2,'Sh','u1744','[email protected]','2009-12-2','+91532110007'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (77,'pooja','kulkarni',3,'Mrs','u1903','[email protected]','2008-10-6','+91532110008'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (78,'priya','mahajan',6,'Sh','u4205','[email protected]','2010-8-5','+91532110009'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (79,'manoj','zerger',1,'Mrs','u3369','[email protected]','2009-12-4','+91532110010'); INSERT INTO AddressBook(Id, FirstName, LastName, OfficeId, Title, Alias, Email, DOJ, PhoneNo) VALUES (80,'mohan','master',5,'Mr','u2841','[email protected]','2010-10-7','+91532110011'); Please note that above employee data is just a data *only data* I created using a small java programme using random number generators and reading some names file, so you may find titles getting messed up :( We next create a procedure that we will use from java to fetch records that we just inserted. DROP PROCEDURE IF EXISTS search_address_book; CREATE PROCEDURE search_address_book(IN address_ids VARCHAR(1000) ) BEGIN DECLARE search_address_query VARCHAR(2000) DEFAULT ''; SET address_ids = CONCAT('\'', REPLACE(address_ids, ',', '\',\''), '\''); SET search_address_query = CONCAT(search_address_query, ' select ab.Id as Id , ab.FirstName as FName, ab.LastName as LName, cl.Location as Location, ab.Title as Title, ab.Alias as Alias, ab.Email as Email, ab.DOJ as DOJ, ab.PhoneNo as PhoneNo ' ); SET search_address_query = CONCAT(search_address_query, ' from AddressBook ab left join CompanyLocations cl on ab.OfficeId=cl.Id '); SET search_address_query = CONCAT(search_address_query, ' where ab.id IN (', address_ids ,') '); SET @statement = search_address_query; PREPARE dynquery FROM @statement; EXECUTE dynquery; DEALLOCATE PREPARE dynquery; END; # To get records for ids 1, 6 and 7, we run following commands: call search_address_book('1,6,7'); Configuring Sphinx It turns out that it was not terribly difficult to setup sphinx, but I had a hard time finding instructions on the web, so I'll post my steps here. By default Sphinx looks for 'sphinx.co.in' configuration file to come with indexes and other stuff, lets create and define source and index for our sample application addressbook.conf (read between the lines) ############################################################################# ## data source definition ############################################################################# source addressBookSource { ## SQL settings for 'mysql' ## type = mysql # some straightforward parameters for SQL source types sql_host = localhost sql_user = root sql_pass = root sql_db = addressbook sql_port = 3306 # optional, default is 3306 # pre-query, executed before the main fetch query sql_query_pre = SET NAMES utf8 # main document fetch query, integer document ID field MUST be the first selected column sql_query = \ select ab.Id as Id , ab.FirstName as FName, ab.LastName as LName, cl.Location as Location, \ ab.Title as Title, ab.Alias as Alias, ab.Email as Email, UNIX_TIMESTAMP(ab.DOJ) as DOJ, ab.PhoneNo as PhoneNo \ from AddressBook ab left join CompanyLocations cl on ab.OfficeId=cl.Id sql_attr_timestamp = DOJ # document info query, ONLY for CLI search (ie. testing and debugging) , optional, default is empty must contain $id macro and must fetch the document by that id sql_query_info = SELECT * FROM AddressBook WHERE id=$id } ############################################################################# ## index definition ############################################################################# # local index example, this is an index which is stored locally in the filesystem index addressBookIndex { # document source(s) to index source = addressBookSource # index files path and file name, without extension, make sure you have this folder path = C:\devel\sphinx-0.9.9-win32\data\addressBookIndex # document attribute values (docinfo) storage mode docinfo = extern # memory locking for cached data (.spa and .spi), to prevent swapping mlock = 0 morphology = none # make sure this file exists exceptions =C:\devel\sphinx-0.9.9-win32\data\exceptions.txt enable_star = 1 } ############################################################################# ## indexer settings ############################################################################# indexer { # memory limit, in bytes, kiloytes (16384K) or megabytes (256M) # optional, default is 32M, max is 2047M, recommended is 256M to 1024M mem_limit = 32M # maximum IO calls per second (for I/O throttling) # optional, default is 0 (unlimited) # # max_iops = 40 # maximum IO call size, bytes (for I/O throttling) # optional, default is 0 (unlimited) # # max_iosize = 1048576 # maximum xmlpipe2 field length, bytes # optional, default is 2M # # max_xmlpipe2_field = 4M # write buffer size, bytes # several (currently up to 4) buffers will be allocated # write buffers are allocated in addition to mem_limit # optional, default is 1M # # write_buffer = 1M } ############################################################################# ## searchd settings ############################################################################# searchd { # hostname, port, or hostname:port, or /unix/socket/path to listen on listen = 9312 # log file, searchd run info is logged here # optional, default is 'searchd.log' log = C:\devel\sphinx-0.9.9-win32\data\log\searchd.log # query log file, all search queries are logged here # optional, default is empty (do not log queries) query_log = C:\devel\sphinx-0.9.9-win32\data\log\query.log # client read timeout, seconds # optional, default is 5 read_timeout = 5 # request timeout, seconds # optional, default is 5 minutes client_timeout = 300 # maximum amount of children to fork (concurrent searches to run) # optional, default is 0 (unlimited) max_children = 30 # PID file, searchd process ID file name # mandatory pid_file = C:\devel\sphinx-0.9.9-win32\data\log\searchd.pid # max amount of matches the daemon ever keeps in RAM, per-index # WARNING, THERE'S ALSO PER-QUERY LIMIT, SEE SetLimits() API CALL # default is 1000 (just like Google) max_matches = 1000 # seamless rotate, prevents rotate stalls if precaching huge datasets # optional, default is 1 seamless_rotate = 1 # whether to forcibly preopen all indexes on startup # optional, default is 0 (do not preopen) preopen_indexes = 0 } # --eof-- Once the configuration is done, its time to index our sql data, the command to use is 'indexer' as shown below. C:\devel\sphinx-0.9.9-win32\bin>indexer.exe --all --config C:\devel\sphinx-0.9.9-win32\addressbook.conf CONSOLE: Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff using config file 'C:\devel\sphinx-0.9.9-win32\addressbook.conf'... indexing index 'addressBookIndex'... collected 80 docs, 0.0 MB sorted 0.0 Mhits, 100.0% done total 80 docs, 5514 bytes total 0.057 sec, 96386 bytes/sec, 1398.43 docs/sec total 2 reads, 0.000 sec, 3.5 kb/call avg, 0.0 msec/call avg total 7 writes, 0.000 sec, 2.5 kb/call avg, 0.0 msec/call avg Note: As I told earlier that Sphinx creates 1 document for each row, as we had 80 rows in the database so a total of 80 docs are created. Time taken is also very very small, believe me I tried with half million rows and it took around 3-4 seconds :) cool isn't it? Once the index is up let's try to search few records, the utility command to perform search is 'search'. Ok Sphinx maharaj* please search for employee whose alias is u4732 C:\devel\sphinx-0.9.9-win32\bin>search.exe --config C:\devel\sphinx-0.9.9-win32\addressbook.conf u4732 CONSOLE: Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff using config file 'C:\devel\sphinx-0.9.9-win32\addressbook.conf'... index 'addressBookIndex': query 'u4732 ': returned 1 matches of 1 total in 0.001 sec displaying matches: 1. document=67, weight=1, doj=Fri Mar 20 00:00:00 2009 Id=67 FirstName=himanshu LastName=verma OfficeId=2 Title=Mrs Alias=u4732 [email protected] DOJ=2009-03-20 PhoneNo=+911234599108 words: 1. 'u4732': 1 documents, 1 hits words: 1. 'u4732': 1 documents, 1 hits As you can see above this is a unique record for Himanshu. Note: You see a lot of information for the result, this is because of following line in our configuration file sql_query_info = SELECT * FROM AddressBook WHERE id=$id If you want to see less columns you need to change the sql_query_info in configuration file. Let's try another search, sphinx maharaj* please tell me which all rows have gurleen or toshi in them. C:\devel\sphinx-0.9.9-win32\bin>search.exe --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --any toshi gurleen CONSOLE: displaying matches: 1. document=63, weight=2, doj=Fri Sep 17 00:00:00 2010 Id=63 FirstName=toshi LastName=prakash OfficeId=1 Title=Mr Alias=u3766 [email protected] DOJ=2010-09-17 PhoneNo=+911234599104 2. document=72, weight=2, doj=Sun Jul 01 00:00:00 2007 Id=72 FirstName=gurleen LastName=bakshi OfficeId=5 Title=Miss Alias=u1423 [email protected] DOJ=2007-07-01 PhoneNo=+91532110003 Exactly two records were returned and this is what we were expecting. The following special operators and modifiers can be used when using the extended matching mode: operator OR: nikhil | sahil operator NOT: hello -sandy hello !sandy field search operator: @Email [email protected] For a complete set of search features , I advise you to go through http://sphinxsearch.com/docs/manual-0.9.9.html#searching link. Sphinx as Windows Service Now our main aim is to use sphinx with JAVA API, so let's move towards that now, before java can utilize the true power of Sphinx, we need to start 'searchd' as a windows service so that our java programme can connect to sphinx search engine. Let's install Sphinx as a windows service so that our java program can use this daemon service to query the index that we just created, the command is : C:\devel\sphinx-0.9.9-win32\bin>searchd.exe --install --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --servicename --port 9312 SphinxSearch CONSOLE: Sphinx 0.9.9-release (r2117) Copyright (c) 2001-2009, Andrew Aksyonoff Installing service... Service 'SphinxSearch' installed succesfully. Well now the sphinx is ready to serve us on port 9312 Note: If you try to install Sphinx without admin rights, you may get following error messages. C:\devel\sphinx-0.9.9-win32\bin>searchd.exe --install --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --servicename --port 9312 SphinxSearch CONSOLE: Installing service... FATAL: OpenSCManager() failed: code=5, error=Access is denied. Once done you can start the service as: c:\>sc start SphinxSearch (or alternatively from the services screen, start 'services.msc' in windows Run) If some how you want to delete the service , use c:\>sc delete SphinxSearch Let's create an adapter to fetch data from the database. package it.gogs.sphinx.util; import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import java.sql.CallableStatement; import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.SQLException; import java.util.ArrayList; import java.util.List; import org.apache.log4j.Logger; /** * Adapter to fetch data from the database. * * @author Munish Gogna * */ public class AddressBookAdapter { private static Logger logger = Logger.getLogger(AddressBookAdapter.class); private AddressBookAdapter() { // use in static way.. } private static Connection getConnection() throws AddressBookTechnicalException { String userName = "root"; String password = "root"; String url = "jdbc:mysql://localhost/addressbook"; try { Class.forName("com.mysql.jdbc.Driver").newInstance(); return DriverManager.getConnection(url, userName, password); } catch (Exception e) { throw new AddressBookTechnicalException("could not get connection"); } } public static List getAddressBookList(List addressIds) throws AddressBookTechnicalException, AddressBookBizException { List addressBoookList = new ArrayList(); if (addressIds == null || addressIds.size() == 0){ logger.error("AddressIds was null or empty, returning empty list"); return addressBoookList; } Connection connection = null; CallableStatement callableStatement = null; try { connection = getConnection(); callableStatement = connection.prepareCall("{ call search_address_book(?)}"); callableStatement.setString(1, Utils.toCommaString(addressIds)); callableStatement.execute(); ResultSet resultSet = callableStatement.getResultSet(); prepareResults(resultSet, addressBoookList); connection.close(); } catch (SQLException e) { logger.error("Problem connecting MYSQL - " + e.getMessage()); throw new AddressBookTechnicalException(e.getMessage()); } catch (AddressBookTechnicalException e) { logger.error("Problem connecting MYSQL - " + e.getMessage()); throw e; } finally{ if(connection != null){ try { connection.close(); } catch (SQLException e) { logger.error("Problem closing conection - " + e.getMessage()); e.printStackTrace(); } } } return addressBoookList; } private static void prepareResults(ResultSet resultSet, List addressBoookList) throws SQLException { AddressBoook addressBoook; while (resultSet.next()) { addressBoook = new AddressBoook(); addressBoook.setAlias(resultSet.getString("Alias")); addressBoook.setEmail(resultSet.getString("Email")); addressBoook.setfName(resultSet.getString("FName")); addressBoook.setlName(resultSet.getString("LName")); addressBoook.setOfficeLocation(resultSet.getString("Location")); addressBoook.setPhoneNo(resultSet.getString("PhoneNo")); addressBoook.setTitle(resultSet.getString("Title")); addressBoook.setDateOfJoining(resultSet.getDate("DOJ")); addressBoook.setId(resultSet.getLong("Id")); addressBoookList.add(addressBoook); } } } Next we create the SphinxInstance that will parse the keywords and date range and provide us a list of Ids that matches the search. package it.gogs.sphinx.util; import it.gogs.sphinx.DateRange; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.api.SphinxClient; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.api.SphinxMatch; import it.gogs.sphinx.api.SphinxResult; import it.gogs.sphinx.exception.AddressBookBizException; import java.util.ArrayList; import java.util.Date; import java.util.List; import org.apache.log4j.Logger; /** * Instance that will parse our free text and provide the results. * * Note: Make sure that 'searchd' is up and running before you use this class * @author Munish Gogna * */ public class SphinxInstance { private static String SPHINX_HOST = "localhost"; private static String SPHINX_INDEX = "addressBookIndex"; private static int SPHINX_PORT = 9312; private static SphinxClient sphinxClient; private static Logger logger = Logger.getLogger(SphinxInstance.class); static { sphinxClient = new SphinxClient(SPHINX_HOST, SPHINX_PORT); } public static List getAddressBookIds(SearchCriteria criteria) throws AddressBookBizException, SphinxException { List addressIdsList = new ArrayList(); try { if (Utils.isNull(criteria)) { logger.error("criteria is null"); throw new AddressBookBizException("criteria is null"); } if (Utils.isNull(criteria.getKeywords())) { logger.error("keyword is a required field"); throw new AddressBookBizException("keyword is a required field"); } DateRange dateRange = criteria.getDateRage(); if (!Utils.isNull(dateRange)) { if (Utils.isDateRangeValid(dateRange)) { // this is to filter results based on joining dates if they are provided sphinxClient.SetFilterRange("DOJ", getTimeInSeconds(dateRange.getFromDate()), getTimeInSeconds(dateRange.getToDate()), false); } else { logger.error(" fromDate/toDate should not be empty and 'fromDate' should be less than equal to 'toDate'"); throw new AddressBookBizException("fromDate/toDate should not be empty and 'fromDate' should be less than equal to 'toDate'"); } } sphinxClient.SetMatchMode(SphinxClient.SPH_MATCH_EXTENDED2); sphinxClient.SetSortMode(SphinxClient.SPH_SORT_RELEVANCE, ""); SphinxResult result = sphinxClient.Query(buildSearchQuery(criteria), SPHINX_INDEX, "buidling query for address book search"); SphinxMatch[] matches = result.matches; for (SphinxMatch match : matches) { addressIdsList.add(String.valueOf(match.docId)); } } catch (SphinxException e) { throw e; } catch (AddressBookBizException e) { throw e; } logger.info("Total record(s):" + addressIdsList.size()); return addressIdsList; } private static long getTimeInSeconds(Date time) { return time.getTime()/1000; } private static String buildSearchQuery(SearchCriteria criteria) throws AddressBookBizException { String keywords[] = criteria.getKeywords().split(" "); StringBuilder searchFor = new StringBuilder(); for (String key : keywords) { if (!Utils.isEmpty(key)) { searchFor.append(key); if (searchFor.length() > 1) { searchFor.append("*|*"); } } } searchFor.delete(searchFor.lastIndexOf("|*"), searchFor.length()); StringBuilder queryBuilder = new StringBuilder(); String query = searchFor.toString(); queryBuilder.append("@FName *" + query + " | "); queryBuilder.append("@LName *" + query + " | "); queryBuilder.append("@Title *" + query + " | "); queryBuilder.append("@Location *"+ query + " | "); queryBuilder.append("@Alias *" + query + " | "); queryBuilder.append("@Email *" + query + " | "); queryBuilder.append("@PhoneNo *" + query); logger.info("Sphinx Query: " + queryBuilder.toString()); return queryBuilder.toString(); } } Here is the interface that I will expose to the outside world (in my future article I will expose this interface as Web Service) import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import java.util.List; /** * * @author Munish Gogna * */ public interface AddressBook { /** * Returns the list of AddressBook objects based on search criteria. * * @param criteria * @throws AddressBookTechnicalException * @throws AddressBookBizException * @throws SphinxException */ public List getAddressBookList(SearchCriteria criteria) throws AddressBookTechnicalException, AddressBookBizException, SphinxException; } and here is the implementation class for the same. package it.gogs.sphinx.addressbook.impl; import java.util.List; import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.addressbook.AddressBook; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import it.gogs.sphinx.util.AddressBookAdapter; import it.gogs.sphinx.util.SphinxInstance; /** * Implementation for our Address Book example * * @author Munish Gogna * */ public class AddressBookImpl implements AddressBook{ public List getAddressBookList(SearchCriteria criteria) throws AddressBookTechnicalException, AddressBookBizException, SphinxException { List addressIds= SphinxInstance.getAddressBookIds(criteria); return AddressBookAdapter.getAddressBookList(addressIds); } } ok so far so good, let's run some tests now ............ package it.gogs.sphinx.test; import java.util.Calendar; import java.util.GregorianCalendar; import java.util.List; import it.gogs.sphinx.AddressBoook; import it.gogs.sphinx.DateRange; import it.gogs.sphinx.SearchCriteria; import it.gogs.sphinx.addressbook.AddressBook; import it.gogs.sphinx.addressbook.impl.AddressBookImpl; import it.gogs.sphinx.api.SphinxException; import it.gogs.sphinx.exception.AddressBookBizException; import it.gogs.sphinx.exception.AddressBookTechnicalException; import junit.framework.TestCase; /** * * @author Munish Gogna * */ public class AddressBookTest extends TestCase { private AddressBook addressBook; @Override protected void setUp() throws Exception { super.setUp(); addressBook = new AddressBookImpl(); } @Override protected void tearDown() throws Exception { super.tearDown(); } /** this should be a unique record for Himanshu */ public void test_search_for_himanshu() throws Exception { SearchCriteria criteria = new SearchCriteria(); // remember the first 'search' example?? criteria.setKeywords("u4732"); List addressList = addressBook.getAddressBookList(criteria); assertTrue(addressList.size() == 1); assertTrue("expecting himanshu here", "himanshu".equals(addressList.get(0).getfName())); } /** only two employees have name gurleen or toshi */ public void test_search_for_gurleen_or_toshi() throws Exception { SearchCriteria criteria = new SearchCriteria(); // remember the second 'search' example?? criteria.setKeywords("gurleen toshi"); List addressList = addressBook.getAddressBookList(criteria); assertTrue(addressList.size() == 2); assertTrue("expecting toshi here", "toshi".equals(addressList.get(0).getfName())); assertTrue("expecting gurleen here", "gurleen".equals(addressList.get(1).getfName())); } /** there are 16 people from jammu location */ public void test_search_for_people_from_jammu_location() throws Exception { SearchCriteria criteria = new SearchCriteria(); criteria.setKeywords("jammu"); List addressList = addressBook.getAddressBookList(criteria); assertTrue(addressList.size() == 16); } /** only Aalap, Manda and nitika are having title as Mrs and joined in 2010 */ public void test_joined_in_2010_with_title_Mrs() throws Exception { DateRange dateRange = new DateRange(); GregorianCalendar calendar1 = new GregorianCalendar(); calendar1.set(Calendar.YEAR, 2010); calendar1.set(Calendar.MONTH, Calendar.JANUARY); calendar1.set(Calendar.DAY_OF_MONTH, 1); dateRange.setFromDate(calendar1.getTime()); GregorianCalendar calendar2 = new GregorianCalendar(); calendar2.set(Calendar.YEAR, 2010); calendar2.set(Calendar.MONTH, Calendar.DECEMBER); calendar2.set(Calendar.DAY_OF_MONTH, 31); dateRange.setToDate(calendar2.getTime()); SearchCriteria criteria = new SearchCriteria(); criteria.setKeywords("Mrs"); criteria.setDateRage(dateRange); List addressList = addressBook.getAddressBookList(criteria); assertTrue("expecting 3 records here", addressList.size() == 3); } /** should get a business exception here */ public void test_without_specifying_keywords(){ SearchCriteria criteria = new SearchCriteria(); //criteria.setKeywords("Mrs"); try { addressBook.getAddressBookList(criteria); } catch (Exception e) { assertTrue(e instanceof AddressBookBizException); assertTrue(e.getMessage().indexOf("keyword is a required field") >-1); } } } How we update the Index once database changes? For these kinds of requirements, we can set up two sources and two indexes, with one "main" index for the data which only changes rarely (if ever), and one "delta" for the new documents. First Time data will go in the "main" index and the newly inserted address book entries will go into "delta". Delta index could then be reindexed very frequently, and the documents can be made available to search in a matter of minutes. Also one thing to take from this article is once 'searchd' daemon is running we can't index the data in normal way,we have to use --rotate option in such cases. For some applications where there is a timely batch update for the data, we can configure some cron job to reindex our documents in Sphinx as shown below. C:\devel\sphinx-0.9.9-win32\bin>indexer.exe --all --config C:\devel\sphinx-0.9.9-win32\addressbook.conf --rotate Capsule We asked Sphinx to provide us the Document Ids corresponding to our search parameters and then we used those Ids to fire database query. In case the data we want to return is included in Index (DOJ attribute for example in our case) we can skip the database portion, so choose wisely how much information (attributes) you want to include while you index your sql data. Well that's all ... it's time to say good bye. Take good care of your health and don't forget to vote, its a must :) - Munish Gogna

JUnit 4.8 introduced Categories: a mechanism to label and group tests, giving developers the option to include or exclude groups (or categories.) This post presents a brief overview of JUnit categories and some unexpected behavior I have found while using them. 1. Quick Introduction The following example shows how to use categories: (adapted from JUnit’s release notes) public interface FastTests { /* category marker */ } public interface SlowTests { /* category marker */ } public class A { @Category(SlowTests.class) @Test public void a() {} } @Category(FastTests.class}) public class B { @Test public void b() {} } @RunWith(Categories.class) @IncludeCategory(SlowTests.class) @ExcludeCategory(FastTests.class) @SuiteClasses({ A.class, B.class }) public class SlowTestSuite {} Lines 1, 2: we define two categories, FastTests and SlowTests. JUnit categories can be defined as classes or interfaces. Since a category acts like a label or marker, my intuition tells me to use interfaces. Line 5: we use the annotation @org.junit.experimental.categories.Category to label test classes and test methods with one or more categories. Lines 6, 9: test methods and test classes can be marked as belonging to one or more categories of tests. Labeling a test class with a category automatically includes all its test methods in such category. Lines 14 to 18: currently programmatic test suites (line 17) are the only way to specify which test categories (line 14) should be included (line 15) or excluded (line 16) when the suite is executed. I find this approach (especially the way test classes need to included in the suite) too verbose and not so flexible. Hopefully Ant, Maven and IDEs will provide support for categories (with a simpler configuration) in the very near future. Note: I recently discovered ClasspathSuite, a project that simplifies the creation of programmatic JUnit test suites. For example, we can specify we want to include in a test suite all tests whose names end with “UnitTest.” 2. Category Subtyping Categories also support subtyping. Let’s say we have the category IntegrationTests that extends SlowTests: public interface IntegrationTests extends SlowTests {} Any test class or test method labeled with the category IntegrationTests is also part of the category SlowTests. To be honest, I don’t know how handy category subtyping could be. I’ll need to experiment with it more to have an opinion. 3. Categories and Test Inheritance 3a. Method-level Categories JUnit behaves as expected when test inheritance is combined with method-level categories. For example: public class D { @Category(GuiTest.class) @Test public void d() {} } public class E extends D { @Category(GuiTest.class) @Test public void e() {} } @RunWith(Categories.class) @IncludeCategory(GuiTest.class) @SuiteClasses(E.class) public class TestSuite {} As I expected, when running TestSuite, test methods d and e are executed (both methods belong to the GuiTest category and E inherits method d from superclass D.) Nice! 3b. Class-level Categories On the other hand, unless I’m missing something, I think I found some strange behavior in JUnit in this scenario. Consider the following classes: @Category(GuiTest.class) public class A { @Test public void a() {} } public class B extends A { @Test public void b() {} } @RunWith(Categories.class) @IncludeCategory(GuiTest.class) @SuiteClasses(B.class) public class TestSuite {} As we can see, TestSuite should execute the tests in B that belong to the category GuiTest. I was expecting TestSuite to execute test method a, even though B is not marked as a GuiTest. Here is my reasoning: test method a belongs to the category GuiTest because test class A is labeled with such category test class B is an A and it inherits test method a Therefore, TestSuite should execute test method a. But it doesn’t! Here is a screenshot of the results I get (click to see full size.) There are two ways to fix this issue, depending on what test methods we want to actually run: Label class B with GuiTest. In this case, both methods, a and b, will be executed. Label method a with GuiTest. In this case, only method a will be executed. (I’ll be posting a question regarding this issue in the JUnit mailing list shortly.) 4. Categories vs. TestNG Groups (You saw this one coming, didn’t you?) Categories (or groups) have been part of TestNG for long time. Unlike JUnit’s, TestNG’s groups are defined as simple strings, not as classes or interfaces. As a static typing lover, I was pretty happy with JUnit categories. By using an IDE, we could safely rename a category or look for usages of a category within a project. Even though my observation was correct, I was missing one important point: all this works great as long as your test suite is written in Java. In the real world, I’d like to define a test suite in either Ant or Maven (or Gradle, or Rake.) In this scenario, having categories as Java types does not bring any benefit. In fact, I suspect it would be very verbose and error-prone to specify the fully-qualified name of a category in a build script. Renaming a category now would be limited to a text-based “search and replace.” Ant and Maven really need to provide a way to specify JUnit categories, clever enough to be fool-proof. As you may expect, I prefer the simplicity and pragmatism of TestNG’s groups. Update: my good friend (and creator of the TestNG framework,) Cédric, reminded me that we can use regular expressions to include or exclude groups in a test suite (details here.) This is really powerful! 5. My Usage of Categories I’m not using JUnit categories in my test suites yet. I started to look into JUnit categories because I wasn’t completely happy with the way we recognized GUI tests in FEST. We recognize test methods or test classes as “GUI tests” if they have been annotated with the @GUITest (provided by FEST.) When a “GUI test” fails, FEST automatically takes a screenshot of the desktop and includes it in the JUnit or TestNG HTML report. The problem is, our @GUITest annotation is duplicating the functionality of JUnit categories. To solve this issue, I created a JUnit extension that recognizes test methods or test classes as “GUI tests” if they belong to the GuiTest category. At this moment GuiTest is an interface provided by FEST, but I’m thinking about letting users specify their own GuiTest category as well. I also refactored this functionality out of the Swing testing module, expecting to reuse it once I implement a JavaFX testing module :) You can find the FEST code that deals with JUnit categories at github. 6. Conclusion Having the ability to label and group tests via categories is really a great feature. I still have some reservations about the practicality of defining categories as Java types, the lack of support for this feature from Ant and Maven (not JUnit’s fault,) and the unexpected behavior I noticed when combining class-level categories and test inheritance. On the brighter side, categories are still an experimental, non-final feature. I’m sure will see many improvements in future JUnit releases :) Feedback is always welcome. From http://alexruiz.developerblogs.com/?p=1711

Maven profiles, like chainsaws, are a valuable tool, with whose power you can easily get carried away, wielding them upon problems to which they are unsuited. Whilst you're unlikely to sever a leg misusing Maven profiles, I thought it worthwhile to share some suggestions about when and when not to use them. These three best practices are all born from real-world mishaps: The build must pass when no profile has been activated Never use Use profiles to manage build-time variables, not run-time variables and not (with rare exceptions) alternative versions of your artifact I'll expand upon these recommendations in a moment. First, though, let's have a brief round-up of what Maven profiles are and do. Maven Profiles 101 A Maven profile is a sub-set of POM declarations that you can activate or disactivate according to some condition. When activated, they override the definitions in the corresponding standard tags of the POM. One way to activate a profile is to simply launch Maven with a -P flag followed by the desired profile name(s), but they can also be activated automatically according to a range of contextual conditions: JDK version, OS name and version, presence or absence of a specific file or property. The standard example is when you want certain declarations to take effect automatically under Windows and others under Linux. Almost all the tags that can be placed directly in a POM can also be enclosed within a tag. The easiest place to read up further about the basics is the Build Profiles chapter of Sonatype's Maven book. It's freely available, readable, and explains the motivation behind profiles: making the build portable across different environments. The build must pass when no profile has been activated (Thanks to for this observation.) Why? Good practice is to minimise the effort required to make a successful build. This isn't hard to achieve with Maven, and there's no excuse for a simple mvn clean package not to work. A maintainer coming to the project will not immediately know that profile wibblewibble has to be activated for the build to succeed. Don't make her waste time finding it out. How to achieve it It can be achieved simply by providing sensible defaults in the main POM sections, which will be overridden if a profile is activated. Never use Why not? This flag activates the profile if no other profile is activated. Consequently, it will fail to activate the profile if any other profile is activated. This seems like a simple rule which would be hard to misunderstand, but in fact it's surprisingly easy to be fooled by its behaviour. When you run a multimodule build, the activeByDefault flag will fail to operate when any profile is activated, even if the profile is not defined in the module where the activeByDefault flag occurs. (So if you've got a default profile in your persistence module, and a skinny war profile in your web module... when you build the whole project, activating the skinny war profile because you don't want JARs duplicated between WAR and EAR, you'll find your persistence layer is missing something.) activeByDefault automates profile activation, which is a good thing; activates implicitly, which is less good; and has unexpected behaviour, which is thoroughly bad. By all means activate your profiles automatically, but do it explicitly and automatically, with a clearly defined rule. How to avoid it There's another, less documented way to achieve what aims to achieve. You can activate a profile in the absence of some property: !foo.bar This will activate the profile "nofoobar" whenever the property foo.bar is not defined. Define that same property in some other profile: nofoobar will automatically become active whenever the other is not. This is admittedly more verbose than , but it's more powerful and, most importantly, surprise-free. Use profiles to adapt to build-time context, not run-time context, and not (with rare exceptions) to produce alternative versions of your artifact Profiles, in a nutshell, allow you to have multiple builds with a single POM. You can use this ability in two ways: Adapt the build to variable circumstances (developer's machine or CI server; with or without integration tests) whilst still producing the same final artifact, or Produce variant artifacts. We can further divide the second option into: structural variants, where the executable code in the variants is different, and variants which vary only in the value taken by some variable (such as a database connection parameter). If you need to vary the value of some variable at run-time, profiles are typically not the best way to achieve this. Producing structural variants is a rarer requirement -- it can happen if you need to target multiple platforms, such as JDK 1.4 and JDK 1.5 -- but it, too, is not recommended by the Maven people, and profiles are not the best way of achieving it. The most common case where profiles seem like a good solution is when you need different database connection parameters for development, test and production environments. It is tempting to meet this requirement by combining profiles with Maven's resource filtering capability to set variables in the deliverable artifact's configuration files (e.g. Spring context). This is a bad idea. Why? It's indirect: the point at which a variable's value is determined is far upstream from the point at which it takes effect. It makes work for the software's maintainers, who will need to retrace the chain of events in reverse It's error prone: when there are multiple variants of the same artifact floating around, it's easy to generate or use the wrong one by accident. You can only generate one of the variants per build, since the profiles are mutually exclusive. Therefore you will not be able to use the Maven release plugin if you need release versions of each variant (which you typically will). It's against Maven convention, which is to produce a single artifact per project (plus secondary artifacts such as documentation). It slows down feedback: changing the variable's value requires a rebuild. If you configured at run-time you would only need to restart the application (and perhaps not even that). One should always aim for rapid feedback. Profiles are there to help you ensure your project will build in a variety of environments: a Windows developer's machine and a CI server, for instance. They weren't intended to help you build variant artifacts from the same project, nor to inject run-time configuration into your project. How to achieve it If you need to get variable runtime configuration into your project, there are alternatives: Use JNDI for your database connections. Your project only contains the resource name of the datasource, which never changes. You configure the appropriate database parameters in the JNDI resource on the server. Use system properties: Spring, for example, will pick these up when attempting to resolve variables in its configuration. Define a standard mechanism for reading values from a configuration file that resides outside the project. For example, you could specify the path to a properties file in a system property. Structural variants are harder to achieve, and I confess I have no first-hand experience with them. I recommend you read this explanation of how to do them and why they're a bad idea, and if you still want to do them, take the option of multiple JAR plugin or assembly plugin executions, rather than profiles. At least that way, you'll be able to use the release plugin to generate all your artifacts in one build, rather than a single one at a time. Further reading Profiles chapter from the Sonatype Maven book. Deploying to multiple environments (prod, test, dev): Stackoverflow.com discussion; see the first and top-rated answer. Short of creating a specific project for the run-time configuration, you could simply use run-time parameters such as system properties. Creating multiple artifacts from one project: How to Create Two JARs from One Project (…and why you shouldn’t) by Tim O'Brien of Sonatype (the Maven people) Blog post explaining the same technique Maven best practices (not specifically about profiles): http://mindthegab.com/2010/10/21/boost-your-maven-build-with-best-practices/ http://blog.tallan.com/2010/09/16/maven-best-practices/ This article is a completely reworked version of a post from my blog.

Read about what a Thread Local is, and learn how to use it in this awesome tutorial.

Devoxx, and all similar conferences, is a place where you make new discoveries, continually. One of these, in my case, at last week's Devoxx, started from a discussion with Jaroslav Bachorik from the VisualVM team. He had presented VisualVM's extensibility in a session at Devoxx. I had heard that, when creating extensions for VisualVM, one can also create new charts using VisualVM's own charting API. Jaroslav confirmed this and we created a small demo together to prove it, i.e., there's a charting API in VisualVM. Since VisualVM is based on the NetBeans Platform, I went further and included the VisualVM charts in a generic NetBeans Platform application. Then I wondered what the differences are between JFreeChart and VisualVM charts, so asked the VisualVM chart architect, Jiri Sedlacek. He sent me a very interesting answer: JFreeCharts are great for creating any kind of static graphs (typically for reports). They provide support for all types of existing chart types. The benefit of using JFreeChart is fully customizable appearance and export to various formats. The only problem of this library is that it's not primarily designed for displaying live data. You can hack it to display data in real time, but the performance is poor. That's why I've created the VisualVM charts. The primary (and so far only) goal is to provide charts optimized for displaying live data with minimal performance and memory overhead. You can easily display a fullscreen graph and it will still scroll smoothly while running and adding new values (when running on physical hardware, virtualized environment may give slightly worse results). There's a real rendering engine behind the charts which ensures that only the changed areas of the chart are repainted (no full-repaints because of a 1px change). Scrolling the chart means moving the already rendered image and only painting the newly displayed area. Last but not least, the charts are optimized for displaying over a remote X session - rendering is automatically switched to low-quality ensuring good response times and interactivity. The Tracer engine introduced in VisualVM 1.3 further improves performance of the charts. I've intensively profiled and optimized the charts to minimize the cpu cycles/memory allocations for each repaint. As of now, I believe that the VisualVM charts are the fastest real time Java charts with the lowest cpu/memory footprint. Best of all is that everything described above is in the JDK. That's because VisualVM is in the JDK. Here's a small NetBeans Platform application (though you could also use the VisualVM chart API without using the NetBeans Platform, just include these JARs on your classpath: org-netbeans-lib-profiler-charts.jar, com-sun-tools-visualvm-charts.jar, com-sun-tools-visualvm-uisupport.jar and org-netbeans-lib-profiler-ui.jar) that makes use of the VisualVM chart API outlined above: The chart that you see above is updated in real time and you can change to full screen and you can scroll through it and, at the same time, there is no lag and it is very performant. Below is all the code (from the unit test package in the VisualVM sources) that you see in the JPanel above: public class Demo extends JPanel { private static final long SLEEP_TIME = 500; private static final int VALUES_LIMIT = 150; private static final int ITEMS_COUNT = 8; private SimpleXYChartSupport support; public Demo() { createModels(); setLayout(new BorderLayout()); add(support.getChart(), BorderLayout.CENTER); } private void createModels() { SimpleXYChartDescriptor descriptor = SimpleXYChartDescriptor.decimal(0, 1000, 1000, 1d, true, VALUES_LIMIT); for (int i = 0; i < ITEMS_COUNT; i++) { descriptor.addLineFillItems("Item " + i); } descriptor.setDetailsItems(new String[]{"Detail 1", "Detail 2", "Detail 3"}); descriptor.setChartTitle("Demo Chart"); descriptor.setXAxisDescription("X Axis [time]"); descriptor.setYAxisDescription("Y Axis [units]"); support = ChartFactory.createSimpleXYChart(descriptor); new Generator(support).start(); } private static class Generator extends Thread { private SimpleXYChartSupport support; public void run() { while (true) { try { long[] values = new long[ITEMS_COUNT]; for (int i = 0; i < values.length; i++) { values[i] = (long) (1000 * Math.random()); } support.addValues(System.currentTimeMillis(), values); support.updateDetails(new String[]{1000 * Math.random() + "", 1000 * Math.random() + "", 1000 * Math.random() + ""}); Thread.sleep(SLEEP_TIME); } catch (Exception e) { e.printStackTrace(System.err); } } } private Generator(SimpleXYChartSupport support) { this.support = support; } } } Here is the related Javadoc. To get started using the VisualVM charts in your own application, read this blog, and then look in the "lib" folder of the JDK to find the JARs you will need. And then have fun with real-time data in your Java desktop applications.

When you migrate an application using Apache Axis 1.2 from Java 4 or 5 to Java 6 (JRE 1.6) you will most likely encounter a handful of strange SOAP/SAAJ/XML errors and ClassCastExceptions. This is due to the fact that Sun’s implementation of SAAJ 1.3 has been integrated directly into the 1.6 JRE. Due to this integration it’s loaded by the bootstrap class loader and thus cannot see various classes that you might be referencing in your old code. As mentioned on Spring pages: Java 1.6 ships with SAAJ 1.3, JAXB 2.0, and JAXP 1.4 (a custom version of Xerces and Xalan). Overriding these libraries by putting different version on the classpath will result in various classloading issues, or exceptions in org.apache.xml.serializer.ToXMLSAXHandler. The only option for using more recent versions is to put the newer version in the endorsed directory (see above). Fortunately, there is a simple solution, at least for Axis 1.2. Some of the exceptions that we’ve encountered Sample Axis code import javax.xml.messaging.URLEndpoint;import javax.xml.soap.MessageFactory;import javax.xml.soap.SOAPConnection;import javax.xml.soap.SOAPConnectionFactory;import javax.xml.soap.SOAPMessage;...public static callAxisWebservice() {SOAPConnectionFactory soapconnectionfactory = SOAPConnectionFactory.newInstance();SOAPConnection soapconnection = soapconnectionfactory.createConnection();MessageFactory messagefactory = MessageFactory.newInstance();SOAPMessage soapmessage = messagefactory.createMessage();...URLEndpoint urlendpoint = new URLEndpoint(string);SOAPMessage soapmessage_18_ = soapconnection.call(soapmessage, urlendpoint);...} SOAPExceptionImpl: Bad endPoint type com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Bad endPoint type http://example.com/ExampleAxisService at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(HttpSOAPConnection.java:161) This extremely confusing error is caused by the following, seemingly innocent code above, namely by the ‘… new URLEndpoint(string)’ and the call itself. The problem here is that Sun’s HttpSOAPConnection can’t see the javax.xml.messaging.URLEndpoint because it is not part of the JRE and is contained in another JAR, not visible to the classes loaded by the bootstrap loader. If you check the HttpSOAPConnection’s code (this is not exactly the version I have but close enough) you will see that it calls “Class.forName(“javax.xml.messaging.URLEndpoint”);” on line 101. For the reason mentioned it fails with a ClassNotFoundException (as indicated by the log “URLEndpoint is available only when JAXM is there” when you enable the JDK logging for the finest level) and thus the method isn’t able to recognize the type of the argument and fails with the confusing Bad endPoint message. A soluti0n in this case would be to pass a java.net.URL or a String instead of a URLEndpoint (though it might lead to other errors, like the one below). Related: Oracle saaj:soap1.2 bug SOAPExceptionImpl: Bad endPoint type. DOMException: NAMESPACE_ERR org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces. at org.apache.xerces.dom.AttrNSImpl.setName(Unknown Source) at org.apache.xerces.dom.AttrNSImpl.(Unknown Source) at org.apache.xerces.dom.CoreDocumentImpl.createAttributeNS(Unknown Source) I don’t rembember exactly what we have changed on the classpath to get this confusing exception and I’ve no idea why it is thrown. Bonus: Conflict between Axis and IBM WebSphere JAX-RPC “thin client” Additionally, if you happen to have com.ibm.ws.webservices.thinclient_7.0.0.jar somewhere on the classpath, you may get this funny exception: java.lang.ClassCastException: org.apache.axis.Message incompatible with com.ibm.ws.webservices.engine.Message at com.ibm.ws.webservices.engine.soap.SOAPConnectionImpl.call(SOAPConnectionImpl.java:198) You may wonder why Java tries to use Axis Message with WebSphere SOAP connection. Well, it’s because the SAAJ lookup mechanism prefers the websphere implementation, for it declares itself via META-INF/services/javax.xml.soap.SOAPFactory pointing to com.ibm.ws.webservices.engine.soap.SOAPConnectionFactoryImpl, but instantiates the org.apache.axis.soap.MessageFactoryImpl for message creation for the websphere thin client doesn’t provide an implementation of this factory. The solution here is the same as for all the other exception, to use exclusively Axis. But if you are interested, check the description how to correctly create a Message with the websphere runtime on page 119 of the IBM WebSphere Application Server V7.0 Web Services Guide (md = javax.xml.ws.Service.create(serviceName).createDispatch(portName, SOAPMessage.class, Service.Mode.MESSAGE); ((SOAPBinding) ((BindingProvider) md).getBinding()).getMessageFactory();). Solution The solution that my collegue Jan Nad has found is to force JRE to use the SOAP/SAAJ implementation provided by Axis, something like: java -Djavax.xml.soap.SOAPFactory=org.apache.axis.soap.SOAPFactoryImpl -Djavax.xml.soap.MessageFactory=org.apache.axis.soap.MessageFactoryImpl -Djavax.xml.soap.SOAPConnectionFactory=org.apache.axis.soap.SOAPConnectionFactoryImpl example.MainClass It’s also described in issue AXIS-2777. Check details of the lookup process in the SOAPFactory.newInstance() JavaDoc. From http://theholyjava.wordpress.com/2010/11/19/soapsaajxml-issues-when-migrating-to-java-6-with-axis-1-2/

this tutorial shows you how to create a java web start (jnlp) file for user download. when the user clicks on the downloaded jnlp file, it launches a simple awt program. here's the summary steps : create a simple awt program and jar it as testjnlp.jar add keystore into testjnlp.jar create a jnlp file put it all into the tomcat folder access testjnlp.jar from web through http://localhost:8080/test.jnlp before starting this tutorial, lets read this brief java web start explanation from oracle. java web start is a mechanism for program delivery through a standard web server. typically initiated through the browser, these programs are deployed to the client and executed outside the scope of the browser. once deployed, the programs do not need to be downloaded again, and they can automatically download updates on startup without requiring the user to go through the whole installation process again. ok, let's go ~ 1. install jdk and tomcat install java jdk/jre version above 1.5 and tomcat. 2. directory structure directory structure of this example. 3. awt + jnlp see the content of testjnlp.java, it's just a simple awt program with jnlp supported. package com.mkyong;import java.awt.*;import javax.swing.*;import java.net.*;import javax.jnlp.*;import java.awt.event.actionlistener;import java.awt.event.actionevent;public class testjnlp { static basicservice basicservice = null; public static void main(string args[]) { jframe frame = new jframe("mkyong jnlp unofficial guide"); frame.setdefaultcloseoperation(jframe.exit_on_close); jlabel label = new jlabel(); container content = frame.getcontentpane(); content.add(label, borderlayout.center); string message = "jnln hello word"; label.settext(message); try { basicservice = (basicservice) servicemanager.lookup("javax.jnlp.basicservice"); } catch (unavailableserviceexception e) { system.err.println("lookup failed: " + e); } jbutton button = new jbutton("http://www.mkyong.com"); actionlistener listener = new actionlistener() { public void actionperformed(actionevent actionevent) { try { url url = new url(actionevent.getactioncommand()); basicservice.showdocument(url); } catch (malformedurlexception ignored) { } } }; button.addactionlistener(listener); content.add(button, borderlayout.south); frame.pack(); frame.show(); } p.s if "import javax.jnlp.*;" is not found, please include the jnlp library which islocated at jre/lib/javaws.jar. 4. jar it located your java classes folder. jar it with following command in command prompt jar -cf testjnlp.jar *.* this will package all the java's classes into a new jar file, named " testjnlp.jar ". 5. create keystore add a new keystore named " testkeys " keytool -genkey -keystore testkeys -alias jdc it will ask for a keystore password, first name, last name , organization's unit...etc..just fill them all. 6. assign keystore to jar file attached newly generated keystore " testkeys " to your " testjnlp.jar " file jarsigner -keystore testkeys testjnlp.jar jdc it will ask password for your newly created keystore 7. deploy jar it copy " testjnlp.jar " to tomcat's default web server folder, for example, in widnows - c:\program files\apache\tomcat 6.0\webapps\root 8. create jnlp file create a new test.jnlp file, content put this yong mook kim testing testing 9. deploy jnlp file copy test.jnlp to your tomcat default web server folder also. c:\program files\apache\tomcat 6.0\webapps\root 10. start tomcat start tomcat , c:\tomcat folder\bin\tomcat6.exe 11. test it access url http://localhost:8080/test.jnlp , it will prompt you to download the test.jnlp file, just accept and double click on it. if everything went fine, you should see the following output click on the "run" button to lauch the awt program. note if jnlp has no response, put the following code in your web.xml , which is located in the tomcat conf folder. jnlp application/x-java-jnlp-file

create a soap ui project using your wsdl. set the preferences in soap ui for axis2 home directory. right click on the wsdl in soap ui and click generate code. select adb binding and the following settings and click generate following is the directory structure and code files generated. that’s it, you can now use this code from you ide by importing it. ps: you will need to add the axis2 jars to your project class path. for more details visit my blog @ http://nitinaggarwal.wordpress.com/

Spock is an open source testing framework for Java and Groovy that has been attracting a growing following, especially in the Groovy community. It lets you write concise, expressive tests, using a quite readable BDD-style notation. It even comes with its own mocking library built in. Oh. I thought he was a sci-fi character. Can I see an example? Sure. Here's a simple one from a coding kata I did recently: import spock.lang.Specification; class RomanCalculatorSpec extends Specification { def "I plus I should equal II"() { given: def calculator = new RomanCalculator() when: def result = calculator.add("I", "I") then: result == "II" } } In Spock, you don't have tests, you have specifications. These are normal Groovy classes that extend the Specifications class, which is actually a JUnit class. Your class contains a set of specifications, represented by methods with funny-method-names-in-quotes™. The funny-method-names-in-quotes™ take advantage of some Groovy magic to let you express your requirements in a very readable form. And since these classes are derived from JUnit, you can run them from within Eclipse like a normal Groovy unit test, and they produce standard JUnit reports, which is nice for CI servers. Another thing: notice the structure of this test? We are using given:, when: and then: to express actions and expected outcomes. This structure is common in Behaviour-Driven Development, or BDD, frameworks like Cucumber and easyb. Though Spock-style tests are generally more concise more technically-focused than tools like Cucumber and easyb, which are often used for automating acceptance tests. But I digress... Actually, the example I gave earlier was a bit terse. We could make our intent clearer by adding text descriptions after the when: and then: labels, as I've done here: def "I plus I should equal II"() { when: "I add two roman numbers together" def result = calculator.add("I", "I") then: "the result should be the roman number equivalent of their sum" result == "II" } This is an excellent of clarifying your ideas and documenting your API. But where are the AssertEquals statements? Aha! I'm glad you asked! Spock uses a feature called Power Asserts. The statement after the then: is your assert. If this test fails, Spock will display a detailed analysis of what went wrong, along the following lines: I plus I should equal II(com.wakaleo.training.spocktutorial.RomanCalculatorSpec) Time elapsed: 0.33 sec <<< FAILURE! Condition not satisfied: result == "II" | | I false 1 difference (50% similarity) I(-) I(I) at com.wakaleo.training.spocktutorial .RomanCalculatorSpec.I plus I should equal II(RomanCalculatorSpec.groovy:17) Nice! But in JUnit, I have @Before and @After for fixtures. Can I do that in Spock? Sure, but you don't use annotations. Instead you implement setup() and cleanup() methods (which are run before and after each specification). I've added one here to show you what they look like: import spock.lang.Specification; class RomanCalculatorSpec extends Specification { def calculator def setup() { calculator = new RomanCalculator() } def "I plus I should equal II"() { when: def result = calculator.add("I", "I") then: result == "II" } } You can also define a setupSpec() and cleanupSpec(), which are run just before the first test and just after the last one. I'm a big fan of parameterized tests in JUnit 4. Can I do that in Spock! You sure can! In fact it's one of Spock's killer features! def "The lowest number should go at the end"() { when: def result = calculator.add(a, b) then: result == sum where: a | b | sum "X" | "I" | "XI" "I" | "X" | "XI" "XX" | "I" | "XXI" "XX" | "II" | "XXII" "II" | "XX" | "XXII" } This code will run the test 5 times. The variables a, b, and sum are initialized from the rows in the table in the where: clause. And if any of the tests fail, you get That's pretty cool too. What about mocking? Can I use Mockito? Sure, if you want. but Spock actually comes with it's own mocking framework, which is pretty neat. You set up a mock or a stub using the Mock() method. I've shown two possible ways to use this method here: given: Subscriber subscriber1 = Mock() def subscriber2 = Mock(Subscriber) ... You can set these mocks up to behave in certain ways. Here are a few examples. You can say a method should return a certain value using the >> operator: subscriber1.isActive() >> true subscriber2.isActive() >> false Or you could get a method to throw an exception when it is called: subscriber.activate() >> { throw new BlacklistedSubscriberException() } Then you can test outcomes in a few different ways. Here is a more complicated example to show you some of your options: def "Messages published by the publisher should only be received by active subscribers"() { given: "a publisher" def publisher = new Publisher() and: "some active subscribers" Subscriber activeSubscriber1 = Mock() Subscriber activeSubscriber2 = Mock() activeSubscriber1.isActive() >> true activeSubscriber2.isActive() >> true publisher.add activeSubscriber1 publisher.add activeSubscriber2 and: "a deactivated subscriber" Subscriber deactivatedSubscriber = Mock() deactivatedSubscriber.isActive() >> false publisher.add deactivatedSubscriber when: "a message is published" publisher.publishMessage("Hi there") then: "the active subscribers should get the message" 1 * activeSubscriber1.receive("Hi there") 1 * activeSubscriber2.receive({ it.contains "Hi" }) and: "the deactivated subscriber didn't receive anything" 0 * deactivatedSubscriber.receive(_) } That does look neat. So what is the best place to use Spock? Spock is great for unit or integration testing of Groovy or Grails projects. On the other hand, tools like easyb amd cucumber are probably better for automated acceptance tests - the format is less technical and the reporting is more appropriate for non-developers. From http://www.wakaleo.com/blog/303-an-introduction-to-spock

Shutdown Hooks are a special construct that allow developers to plug in a piece of code to be executed when the JVM is shutting down.

this post will walk you through how to implement extjs linked/cascading/nested combo boxes using spring mvc 3 and hibernate 3.5. i am going to use the classic linked combo boxes: state and cities. in this example, i am going to use states and cities from brazil! what is our main goal? when we select a state from the first combo box, the application will load the second combo box with the cities that belong to the selected state. there are two ways to implement it. the first one is to load all the information you need for both combo boxes, and when user selects a state, the application will filter the cities combo box according to the selected state. the second one is to load information only to populate the state combo box. when user selects a state, the application will retrieve all the cities that belong to the selected state from database. which one is best? it depends on the amount of data you have to retrieve from your database. for example: you have a combo box that lists all the countries in the world. and the second combo box represents all the cities in the world. in this case, scenario number 2 is the best option, because you will have to retrieve a large amount of data from the database. ok. let’s get into the code. i’ll show how to implement both scenarios. first, let me explain a little bit of how the project is organized: let’s take a look at the java code. basedao: contains the hibernate template used for citydao and statedao. package com.loiane.dao; import org.hibernate.sessionfactory; import org.springframework.beans.factory.annotation.autowired; import org.springframework.orm.hibernate3.hibernatetemplate; import org.springframework.stereotype.repository; @repository public abstract class basedao { private hibernatetemplate hibernatetemplate; public hibernatetemplate gethibernatetemplate() { return hibernatetemplate; } @autowired public void setsessionfactory(sessionfactory sessionfactory) { hibernatetemplate = new hibernatetemplate(sessionfactory); } } citydao: contains two methods: one to retrieve all cities from database (used in scenario #1), and one method to retrieve all the cities that belong to a state (used in scenario #2). package com.loiane.dao; import java.util.list; import org.hibernate.criterion.detachedcriteria; import org.hibernate.criterion.restrictions; import org.springframework.stereotype.repository; import com.loiane.model.city; @repository public class citydao extends basedao{ public list getcitylistbystate(int stateid) { detachedcriteria criteria = detachedcriteria.forclass(city.class); criteria.add(restrictions.eq("stateid", stateid)); return this.gethibernatetemplate().findbycriteria(criteria); } public list getcitylist() { detachedcriteria criteria = detachedcriteria.forclass(city.class); return this.gethibernatetemplate().findbycriteria(criteria); } } statedao: contains only one method to retrieve all the states from database. package com.loiane.dao; import java.util.list; import org.hibernate.criterion.detachedcriteria; import org.springframework.stereotype.repository; import com.loiane.model.state; @repository public class statedao extends basedao{ public list getstatelist() { detachedcriteria criteria = detachedcriteria.forclass(state.class); return this.gethibernatetemplate().findbycriteria(criteria); } } city: represents the city pojo, represents the city table. package com.loiane.model; import javax.persistence.column; import javax.persistence.entity; import javax.persistence.generatedvalue; import javax.persistence.id; import javax.persistence.table; import org.codehaus.jackson.annotate.jsonautodetect; @jsonautodetect @entity @table(name="city") public class city { private int id; private int stateid; private string name; //getters and setters } state: represents the state pojo, represents the state table. package com.loiane.model; import javax.persistence.column; import javax.persistence.entity; import javax.persistence.generatedvalue; import javax.persistence.id; import javax.persistence.table; import org.codehaus.jackson.annotate.jsonautodetect; @jsonautodetect @entity @table(name="state") public class state { private int id; private int countryid; private string code; private string name; //getters and setters } cityservice: contains two methods: one to retrieve all cities from database (used in scenario #1), and one method to retrieve all the cities that belong to a state (used in scenario #2). only makes a call to citydao class. package com.loiane.service; import java.util.list; import org.springframework.beans.factory.annotation.autowired; import org.springframework.stereotype.service; import com.loiane.dao.citydao; import com.loiane.model.city; @service public class cityservice { private citydao citydao; public list getcitylistbystate(int stateid) { return citydao.getcitylistbystate(stateid); } public list getcitylist() { return citydao.getcitylist(); } @autowired public void setcitydao(citydao citydao) { this.citydao = citydao; } } stateservice: contains only one method to retrieve all the states from database. makes a call to statedao. package com.loiane.service; import java.util.list; import org.springframework.beans.factory.annotation.autowired; import org.springframework.stereotype.service; import com.loiane.dao.statedao; import com.loiane.model.state; @service public class stateservice { private statedao statedao; public list getstatelist() { return statedao.getstatelist(); } @autowired public void setstatedao(statedao statedao) { this.statedao = statedao; } } citycontroller: contains two methods: one to retrieve all cities from database (used in scenario #1), and one method to retrieve all the cities that belong to a state (used in scenario #2). only makes a call to cityservice class. both methods return a json object that looks like this: {"data":[ {"stateid":1,"name":"acrelândia","id":1}, {"stateid":1,"name":"assis brasil","id":2}, {"stateid":1,"name":"brasiléia","id":3}, {"stateid":1,"name":"bujari","id":4}, {"stateid":1,"name":"capixaba","id":5}, {"stateid":1,"name":"cruzeiro do sul","id":6}, {"stateid":1,"name":"epitaciolândia","id":7}, {"stateid":1,"name":"feijó","id":8}, {"stateid":1,"name":"jordão","id":9}, {"stateid":1,"name":"mâncio lima","id":10}, ]} class: package com.loiane.web; import java.util.hashmap; import java.util.map; import org.springframework.beans.factory.annotation.autowired; import org.springframework.stereotype.controller; import org.springframework.web.bind.annotation.requestmapping; import org.springframework.web.bind.annotation.requestparam; import org.springframework.web.bind.annotation.responsebody; import com.loiane.service.cityservice; @controller @requestmapping(value="/city") public class citycontroller { private cityservice cityservice; @requestmapping(value="/getcitiesbystate.action") public @responsebody map getcitiesbystate(@requestparam int stateid) throws exception { map modelmap = new hashmap(3); try{ modelmap.put("data", cityservice.getcitylistbystate(stateid)); return modelmap; } catch (exception e) { e.printstacktrace(); modelmap.put("success", false); return modelmap; } } @requestmapping(value="/getallcities.action") public @responsebody map getallcities() throws exception { map modelmap = new hashmap(3); try{ modelmap.put("data", cityservice.getcitylist()); return modelmap; } catch (exception e) { e.printstacktrace(); modelmap.put("success", false); return modelmap; } } @autowired public void setcityservice(cityservice cityservice) { this.cityservice = cityservice; } } statecontroller: contains only one method to retrieve all the states from database. makes a call to stateservice. the method returns a json object that looks like this: {"data":[ {"countryid":1,"name":"acre","id":1,"code":"ac"}, {"countryid":1,"name":"alagoas","id":2,"code":"al"}, {"countryid":1,"name":"amapá","id":3,"code":"ap"}, {"countryid":1,"name":"amazonas","id":4,"code":"am"}, {"countryid":1,"name":"bahia","id":5,"code":"ba"}, {"countryid":1,"name":"ceará","id":6,"code":"ce"}, {"countryid":1,"name":"distrito federal","id":7,"code":"df"}, {"countryid":1,"name":"espírito santo","id":8,"code":"es"}, {"countryid":1,"name":"goiás","id":9,"code":"go"}, {"countryid":1,"name":"maranhão","id":10,"code":"ma"}, {"countryid":1,"name":"mato grosso","id":11,"code":"mt"}, {"countryid":1,"name":"mato grosso do sul","id":12,"code":"ms"}, {"countryid":1,"name":"minas gerais","id":13,"code":"mg"}, {"countryid":1,"name":"pará","id":14,"code":"pa"}, ]} class: package com.loiane.web; import java.util.hashmap; import java.util.map; import org.springframework.beans.factory.annotation.autowired; import org.springframework.stereotype.controller; import org.springframework.web.bind.annotation.requestmapping; import org.springframework.web.bind.annotation.responsebody; import com.loiane.service.stateservice; @controller @requestmapping(value="/state") public class statecontroller { private stateservice stateservice; @requestmapping(value="/view.action") public @responsebody map view() throws exception { map modelmap = new hashmap(3); try{ modelmap.put("data", stateservice.getstatelist()); return modelmap; } catch (exception e) { e.printstacktrace(); modelmap.put("success", false); return modelmap; } } @autowired public void setstateservice(stateservice stateservice) { this.stateservice = stateservice; } } inside the webcontent folder we have: ext-3.2.1 – contains all extjs files js – contains all javascript files i implemented for this example. liked-comboboxes-local.js contains the combo boxes for scenario #1; liked-comboboxes-remote.js contains the combo boxes for scenario #2; linked-comboboxes.js contains a tab panel that contains both scenarios. now let’s take a look at the extjs code. scenario number 1: retrieve all the data from database to populate both combo boxes. will user filter on cities combo box. liked-comboboxes-local.js var localform = new ext.formpanel({ width: 400 ,height: 300 ,style:'margin:16px' ,bodystyle:'padding:10px' ,title:'linked combos - local filtering' ,defaults: {xtype:'combo'} ,items:[{ fieldlabel:'select state' ,displayfield:'name' ,valuefield:'id' ,store: new ext.data.jsonstore({ url: 'state/view.action', remotesort: false, autoload:true, idproperty: 'id', root: 'data', totalproperty: 'total', fields: ['id','name'] }) ,triggeraction:'all' ,mode:'local' ,listeners:{select:{fn:function(combo, value) { var combocity = ext.getcmp('combo-city-local'); combocity.clearvalue(); combocity.store.filter('stateid', combo.getvalue()); } } },{ fieldlabel:'select city' ,displayfield:'name' ,valuefield:'id' ,id:'combo-city-local' ,store: new ext.data.jsonstore({ url: 'city/getallcities.action', remotesort: false, autoload:true, idproperty: 'id', root: 'data', totalproperty: 'total', fields: ['id','stateid','name'] }) ,triggeraction:'all' ,mode:'local' ,lastquery:'' }] }); the state combo box is declared on lines 9 to 28. the city combo box is declared on lines 31 to 46. note that both stores are loaded when we load the page, as we can see in lines 15 and 38 (autoload:true). the state combo box has select event listener that, when executed, filters the cities combo (the child combo) based on the currently selected state. you can see it on lines 23 to 28. cities combo has lastquery:”" . this is to fool internal combo filtering routines on the first page load. the cities combo just thinks that it has already been expanded once. scenario number 2: retrieve all the state data from database to populate state combo. when user selects a state, application will retrieve from database only selected information. liked-comboboxes-remote.js: var databaseform = new ext.formpanel({ width: 400 ,height: 200 ,style:'margin:16px' ,bodystyle:'padding:10px' ,title:'linked combos - database' ,defaults: {xtype:'combo'} ,items:[{ fieldlabel:'select state' ,displayfield:'name' ,valuefield:'id' ,store: new ext.data.jsonstore({ url: 'state/view.action', remotesort: false, autoload:true, idproperty: 'id', root: 'data', totalproperty: 'total', fields: ['id','name'] }) ,triggeraction:'all' ,mode:'local' ,listeners: { select: { fn:function(combo, value) { var combocity = ext.getcmp('combo-city'); //set and disable cities combocity.setdisabled(true); combocity.setvalue(''); combocity.store.removeall(); //reload city store and enable city combobox combocity.store.reload({ params: { stateid: combo.getvalue() } }); combocity.setdisabled(false); } } } },{ fieldlabel:'select city' ,displayfield:'name' ,valuefield:'id' ,disabled:true ,id:'combo-city' ,store: new ext.data.jsonstore({ url: 'city/getcitiesbystate.action', remotesort: false, idproperty: 'id', root: 'data', totalproperty: 'total', fields: ['id','stateid','name'] }) ,triggeraction:'all' ,mode:'local' ,lastquery:'' }] }); the state combo box is declared on lines 9 to 38. the city combo box is declared on lines 40 to 55. note that only state combo store is loaded when we load the page, as we can see at the line 15 (autoload:true). the state combo box has the select event listener that, when executed, reloads the data for cities store (passes stateid as parameter) based on the currently selected state. you can see it on lines 24 to 38. cities combo has lastquery:”" . this is to fool internal combo filtering routines on the first page load. the cities combo just thinks that it has already been expanded once. you can download the complete project from my github repository: i used eclipse ide + tomcat 7 to develop this sample project. references: http://www.sencha.com/learn/tutorial:linked_combos_tutorial_for_ext_2 from http://loianegroner.com/2010/10/tutorial-linkedcascading-extjs-combo-boxes-using-spring-mvc-3-and-hibernate-3-5/

When I am writing enumerations I very often found myself implementing a static method similar to the standard enum’s valueOf() but based on field rather than name: public static TestOne valueOfDescription(String description) { for (TestOne v : values()) { if (v.description.equals(description)) { return v; } } throw new IllegalArgumentException( "No enum const " + TestOne.class + "@description." + description); } Where “description” is yet another String field in my enum. And I am not alone. See this article for example. Obviously this method is very ineffective. Every time it is invoked it iterates over all members of the enum. Here is the improved version that uses a cache: private static Map map = null; public static TestTwo valueOfDescription(String description) { synchronized(TestTwo.class) { if (map == null) { map = new HashMap(); for (TestTwo v : values()) { map.put(v.description, v); } } } TestTwo result = map.get(description); if (result == null) { throw new IllegalArgumentException( "No enum const " + TestTwo.class + "@description." + description); } return result; } It is fine if we have only one enum and only one custom field that we use to find the enum value. But if we have 20 enums, and each has 3 such fields, then the code will be very verbose. As I dislike copy/paste programming I have implemented a utility that helps to create such methods. I called this utility class ValueOf. It has 2 public methods: public static , V> T valueOf(Class enumType, String fieldName, V value); which finds the required field in specified enum. It is implemented utilizing reflection and uses a hash table initialized during the first call for better performance. The other overridden valueOf() looks like: public static > T valueOf(Class enumType, Comparable comparable); This method does not cache results, so it iterates over enum members on each invocation. But it is more universal: you can implement comparable as you want, so this method may find enum members using more complicated criteria. Full code with examples and JUnit test case are available here. Conclusions Java Enums provide the ability to locate enum members by name. This article describes a utility that makes it easy to locate enum members by any other field.

There is no so-called asynchronous execution support in JDBC mainly because most of the time you want to wait for the result of your DML or DDL, or because there is too much complexity involved between the back-end database and the front end JDBC driver. Some database vendors do provide such support in their native drives. For example Oracle supports non-blocking calls in its native OCI driver. Unfortunately it is based on polling instead of callback or interrupt. Neither Hibernate or Spring supports this feature. But sometimes you do need such a feature. For example some business logic is still implemented using legacy Oracle PL/SQL stored procedures and they run pretty long. The front-end UI doesn't want to wait for its finish and it just needs to check the running result later in a database logging table into which the store procedure will write the execution status. In other cases your front-end application really cares about low latency and doesn't care too much about how individual DML is executed. So you just fire a DML into the database and forget the running status. Nothing can stop you from making asynchronous DB calls using multi-threading in your application. (Actually even Oracle recommends to use multi-thread instead of polling OCI for efficiency). However you must think about how to handle transaction and connection (or Hibernate Session) in threads. Before continuing, let's assume we are only handling local transaction instead of JTA. 1. JDBC It is straightforward. You just create another thread (DB thread hereafter) from the calling thread to make the actual JDBC call. If such a call is frequent, you call use ThreadPoolExecutor to reduce thread's creation and destroy overhead. 2. Hibernate You usually use session context policy "thread" for Hibernate to automatically handle your session and transaction. With this policy, you get one session and transaction per thread. When you commit the transaction, Hibernate automatically closes the session. Again you need to create a DB thread for the actual stored procedure call. Some developer may be wondering whether the new DB thread inherits its parent calling thread's session and transaction. This is an important question. First of all, you usually don't want to share the same transaction between the calling thread and its spawned DB thread because you want to return immediately from the calling thread and if both threads share the same session and transaction, the calling thread can't commit the transaction and long running transaction should be avoided. Secondly Hibernate's "thread" policy doesn't support such inheritance because if you look at Hibernate's corresponding ThreadLocalSessionContext, it is using ThreadLocal class instead of InheritableThreadLocal. Here is a sample code in the DB thread: // Non-managed environment and "thread" policy is in place // gets a session first Session sess = factory.getCurrentSession(); Transaction tx = null; try { tx = sess.beginTransaction(); // call the long running DB stored procedure //Hibernate automatically closes the session tx.commit(); } catch (RuntimeException e) { if (tx != null) tx.rollback(); throw e; } 3.Spring's Declarative Transaction Let's suppose your stored procedure call is included in method: @Transactional(readOnly=false) public void callDBStoredProcedure(); The calling thread has the following method to call the above method asynchronously using Spring's TaskExecutor: @Transactional(readOnly=false) public void asynchCallDBStoredProcedure() { //creates a DB thread pool this.taskExecutor.execute(new Runnable() { @Override public void run() { //call callDBStoredProcedure() } }); } You usually configure Spring's HibernateTransactionManager and the default proxy mode (aspectj is another mode) for declarative transactions. This class binds a transaction and a Hibernate session to each thread and doesn't Inheritance either just like Hibernate's "thread" policy. Where you put the above method callDBStoredProcedure() makes a huge difference. If you put the method in the same class as the calling thread, the declared transaction for callDBStoredProcedure() doesn't take place because in the proxy mode only external or remote method calls coming in through the AOP proxy (an object created by the AOP framework in order to implement the transaction aspect. This object supports your calling thread's class by composing an instance of your calling thread class) will be intercepted. This meas that "self-invocation", i.e. a method within the target object (the composed instance of your calling thread class in the AOP proxy) calling some other method of the target object, won't lead to an actual transaction at runtime even if the invoked method is marked with @Transactional! So you must put callDBStoredProcedure() in a different class as a Spring's bean so that the DB thread in method asynchCallDBStoredProcedure() can load that bean's AOP proxy and call callDBStoredProcedure() through that proxy. About The Author Yongjun Jiao is a technical manager with SunGard Consulting Services. He has been a professional software developer for the past 10 years. His expertise covers Java SE, Java EE, Oracle, application tuning and high performance and low latency computing.

It's been almost 4 years since I wrote a blog post called "EasyMock - Pros, Cons, and Best Practices, and a lot has happened since. You don't hear about EasyMock much any more, and Mockito seems to have replaced it in mindshare. And for good reason: it is better. A Good Humane Interface for Stubbing Just like EasyMock, Mockito allows you to chain method calls together to produce less imperative looking code. Here's how you can make a Stub for the canonical Warehouse object: Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); I know, I like a crazy formatting. Regardless, giving your System Under Test (SUT) indirect input couldn't be easier. There is no big advantage over EasyMock for stubbing behavior and passing a stub off to the SUT. Giving indirect input with mocks and then using standard JUnit asserts afterwards is simple with both tools, and both support the standard Hamcrest matchers. Class (not just Interface) Mocks Mockito allows you to mock out classes as well as interfaces. I know the EasyMock ClassExtensions allowed you to do this as well, but it is a little nicer to have it all in one package with Mockito. Supports Test Spies, not just Mocks There is a difference between spies and mocks. Stubs allow you to give indirect input to a test (the values are read but never written), Spies allow you to gather indirect output from a test (the mock is written to and verified, but does not give the test input), and Mocks are both (your object gives indirect input to your test through Stubbing and gathers indirect output through spying). The difference is illustrated between two code examples. In EasyMock, you only have mocks. You must set all input and output expectations before running the test, then verify afterwards. // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); EasyMock.expect( mock.hasInventory(TALISKER, 50)). andReturn(true).once(); EasyMock.expect( mock.remove(TALISKER, 50)). andReturn(true).once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); That's a lot of code, and not all of it is needed. The arrange section is setting up a stub (the warehouse has inventory) and setting up a mock expectation (the remove method will be called later). The assertion in all this is actually the little verify() method at the end. The main point of this test is that remove() was called, but that information is buried in a nest of expectations. Mockito improves on this by throwing out both the record/playback mode and a generic verify() method. It is shorter and clearer this way: // arrange Warehouse mock = Mockito.mock(Warehouse.class); Mockito.when(mock.hasInventory(TALISKER, 50)). thenReturn(true); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); The verify step with Mockito is spying on the results of the test, not recording and verifying. Less code and a clearer picture of what really is expected. Update: There is a separate Spy API you can use in Mockito as well: http://mockito.googlecode.com/svn/branches/1.8.3/javadoc/org/mockito/Mockito.html#13 Better Void Method Handling Mockito handles void methods better than EasyMock. The fluent API works fine with a void method, but in EasyMock there were some special methods you had to write. First, the Mockito code is fairly simple to read: // arrange Warehouse mock = Mockito.mock(Warehouse.class); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert Mockito.verify(warehouse).remove(TALISKER, 50); Here is the same in EasyMock. Not as good: // arrange Warehouse mock = EasyMock.createMock(Warehouse.class); mock.remove(TALISKER, 50); EasyMock.expectLastMethodCall().once(); EasyMock.replay(mock); //act Order order = new Order(TALISKER, 50); order.fill(warehouse); // assert EasyMock.verify(mock); Mock Object Organization Patterns Both Mockito and EasyMock suffer from difficult maintenance. What I said in my original EasyMock post holds true for Mockito: The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I'm working on fails, it's often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks... This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods. Now, four years later, I have a solution that works well for me. With a little care you can make your mocks reusable, maintainable, and readable. This approach was battle tested over many months in an Enterprise Environment(tm). Create a private static method the first time you need a mock. Any important data needs to be passed in as a parameter. Using constants or "magic" fields hides important information and obfuscates tests. For example: User user = createMockUser("userID", "name"); ... assertEquals("userID", result.id()); assertEquals("name", result.name(); Everything important is visible and in the test, nothing important is hidden. You need to completely hide the replay state behind this factory method if you're still on EasyMock. The Mock framework in use is an implementation detail and try not to let it leak. Next, as your dependencies grow, be sure to always pass them in as factory method parameters. If you need a User and a Role object, then don't create one method that creates both mocks. One method instantiates one object, otherwise it is a parameter and compose your mock objects in the test method: User user = createMockUser( "userID", "name", createMockRole("role1"), createMockRole("role2") ); When each object type has a factory method, then it makes it much easier to compose the different types of objects together. Reuse. But you can only reuse the methods when they are simple and with few dependencies, otherwise they become too specific and difficult to understand. The first time you need to reuse one of these methods, then move the method to a utility class called "*Mocker", like UserMocker or RoleMocker. Follow a naming convention so that they are always easy to find. If you remembered to make the private factory methods static then moving them should be very simple. Your client code ends up looking like this, but you can use static imports to fix that: User user = UserMocker.createMockUser( "userID", "name", RoleMocker.createMockRole("role1"), RoleMocker.createMockRole("role2") ); User overloaded methods liberally. Don't create one giant method with every possible parameter in the parameter list. There are good reasons to avoid overloading in production, but this is test. Use overloading so that the test methods only display data relevant to that test and nothing more. Using Varargs can also help keep a clean test. Lastly, don't use constants. Constants hide the important information out of sight, at the top of the file where you can't see it or in a Mocker class. It's OK to use constants within the test case, but don't define constants in the Mockers, it just hides relevant information and makes the test harder to read later. Avoid Abstract Test Cases Managing mock objects within abstract test cases has been very difficult for me, especially when managing replay and record states. I've given up mixing mock objects and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance. Don't Replace Asserts with Verify My original comments about EasyMock are still relevant for Mockito: The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method's purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don't even allow it! If you're test code contains assert methods then you have a good test. If you're code doesn't contain asserts, and instead contains a long list of verify() calls, then you're relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, "My test method needs to do several things: x, y, and z." The charter and responsibility of the method is no longer clear. This is a candidate for refactoring. No More All or Nothing Testing Mockito's verify() methods are much more flexible than EasyMock's. You can verify that only one or two methods on the mock were called, while EasyMock had just one coarse verify() method. With EasyMock I ended up littering the code with meaningless expectations, but not so in Mockito. This alone is reason enough to switch. Failure: Expected X received X For the most part, Mockito error messages are better than EasyMock's. However, you still sometimes see a failure that reads "Failure. Got X Expected X." Basically, this means that your toString() methods produce the same results but equals() does not. Every user who starts out gets confused by this message at some point. Be Warned. Don't Stop Handrolling Mocks Don't throw out hand-rolled mock objects. They have their place. Subclass and Override is a very useful technique for creating a testing seam, use it. Learn to Write an ArgumentMatcher Learn to write an ArgumentMatcher. There is a learning curve but it's over quickly. This post is long enough, so I won't give an example. That's it. See you again in 4 years when the next framework comes out! From http://hamletdarcy.blogspot.com/2010/09/mockito-pros-cons-and-best-practices.html

I have been working with Maven for 3 years now and over that time I have learned some tips and tricks that help me work faster with Maven. In this article, I am going to talk about 5 of those tips. Maven rf option Most of us work in a multi-module environment and it happens very often that a build fails at some module. It's a big pain to rerun the entire the build. To save you from going through this pain, Maven has an option called rf (i.e. resume) from which you can resume your build from the module where it failed. So, if your build failed at myproject-commons you can run the build from this module by typing: mvn -rf myproject-commons clean install Maven pl option This next Maven option helps you build specified reactor projects instead of building all projects. For example, if you need to build only two of your modules myproject-commons and myproject-service, you can type: mvn -pl myproject-commons,myproject-service clean install This command will only build commons and service projects. Maven Classpath ordering In Maven, dependencies are loaded in the order in which they are declared in the pom.xml. As of version 2.0.9, Maven introduced deterministic ordering of dependencies on the classpath. The ordering is now preserved from your pom, with dependencies added by inheritence added last. Knowing this can really help you while debugging NoClassDefFoundError. Recently, I faced NoClassDefFoundError: org/objectweb/asm/CodeVisitor while working on a story where cglib-2.1_3.jar was getting loaded before cglib-nodep-2.1_3.jar. And as cglib-2.1_3.jar does not have this CodeVisitor class I was getting the error. Maven Classifiers We all know about qualifiers groupId, artifactId, version that we use to describe a dependency but there is fourth qualifier called classifier. It allows distinguishing two artifacts that belong to the same POM but were built differently, and is appended to the filename after the version. For example, if you want to build two separate jars, one for Java 5 and another for Java 6, you can use classifier. It lets you locate your dependencies with the further level of granularity. junitjunittestjdk16 Dependency Version Ranges Have you ever worked with a library that releases too often when you want that you should always work with the latest without changing the pom. It can be done using dependency version changes. So, if you want to specify that you should always work with the version above a specified version, you will write: [1.1.0,) This line means that the version should always be greater than or equal to 1.1.0. You can read more about dependency version ranges at this link. These are my five tips. If you have any tips please share.

Originally Barcodes were 1D representation of data using width and spacing of bars. Common bar code types are UPC barcodes which are seen on product packages. There are 2D barcodes as well (they are still called Barcodes even though they don’t use bars). A common example of 2D bar code is QR code (shown on right) which is commonly used by mobile phone apps. You can read history and more info about Barcodes on Wikipedia. There is an open source Java library called ‘zxing’ (Zebra Crossing) which can read and write many differently types of bar codes formats. I tested zxing and it was able to read a barcode embedded in the middle of a 100 dpi grayscale busy text document! This article demonstrates how to use zxing to read and write bar codes from a Java program. Getting the library It would be nice if the jars where hosted in a maven repo somewhere, but there is no plan to do that (see Issue 88). Since I could not find the binaries available for download, I decided to download the source code and build the binaries, which was actually quite easy. The source code of the library is available on Google Code. At the time of writing, 1.6 is the latest version of zxing. 1. Download the release file ZXing-1.6.zip (which contains of mostly source files) from here. 2. Unzip the file in a local directory 3. You will need to build 2 jar files from the downloaded source: core.jar, javase.jar Building core.jar cd zxing-1.6/core mvn install cd zxing-1.6/core mvn install This will install the jar in your local maven repo. Though not required, you can also deploy it to your company’s private repo by using mvn:deploy or by manually uploading it to your maven repository. There is an ant script to build the jar as well. Building javase.jar Repeat the same procedure to get javase.jar cd zxing-1.6/javase mvn install cd zxing-1.6/javase mvn install Including the libraries in your project If you are using ant, add the core.jar and javase.jar to your project’s classpath. If you are using maven, add the following to your pom.xml. com.google.zxing core 1.6-SNAPSHOT com.google.zxing javase 1.6-SNAPSHOT com.google.zxing core 1.6-SNAPSHOT com.google.zxing javase 1.6-SNAPSHOT Once you have the jars included in your project’s classpath, you are now ready to read and write barcodes from java! Reading a Bar Code from Java You can read the bar code by first loading the image as an input stream and then calling this utility method. InputStream barCodeInputStream = new FileInputStream("file.jpg"); BufferedImage barCodeBufferedImage = ImageIO.read(barCodeInputStream); LuminanceSource source = new BufferedImageLuminanceSource(barCodeBufferedImage); BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source)); Reader reader = new MultiFormatReader(); Result result = reader.decode(bitmap); System.out.println("Barcode text is " + result.getText()); InputStream barCodeInputStream = new FileInputStream("file.jpg"); BufferedImage barCodeBufferedImage = ImageIO.read(barCodeInputStream); LuminanceSource source = new BufferedImageLuminanceSource(barCodeBufferedImage); BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source)); Reader reader = new MultiFormatReader(); Result result = reader.decode(bitmap); System.out.println("Barcode text is " + result.getText()); Writing a Bar Code from Java You can encode a small text string as follows: String text = "98376373783"; // this is the text that we want to encode int width = 400; int height = 300; // change the height and width as per your requirement // (ImageIO.getWriterFormatNames() returns a list of supported formats) String imageFormat = "png"; // could be "gif", "tiff", "jpeg" BitMatrix bitMatrix = new QRCodeWriter().encode(text, BarcodeFormat.QR_CODE, width, height); MatrixToImageWriter.writeToStream(bitMatrix, imageFormat, new FileOutputStream(new File("qrcode_97802017507991.png"))); String text = "98376373783"; // this is the text that we want to encode int width = 400; int height = 300; // change the height and width as per your requirement // (ImageIO.getWriterFormatNames() returns a list of supported formats) String imageFormat = "png"; // could be "gif", "tiff", "jpeg" BitMatrix bitMatrix = new QRCodeWriter().encode(text, BarcodeFormat.QR_CODE, width, height); MatrixToImageWriter.writeToStream(bitMatrix, imageFormat, new FileOutputStream(new File("qrcode_97802017507991.png"))); In the above example, the bar code for “97802017507991″ is written to the file “qrcode_97802017507991.png” (click to see the output). JavaDocs and Documentation The Javadocs are part of the downloaded zip file. You can find a list of supported bar code formats in the Javadocs. Open the following file to see the javadocs. zxing-1.6/docs/javadoc/index.html zxing-1.6/docs/javadoc/index.html From http://www.vineetmanohar.com/2010/09/java-barcode-api/

There isn't a JavaHelp template for Maven-based NetBeans Platform modules, hence you need to set up JavaHelp yourself. It's a bit tricky, involving several different parts that can go wrong easily (which is why a JavaHelp template is so handy). Here's instructions on how to set up JavaHelp in your Maven-based NetBeans Platform modules. Do the following: In each module that needs to provide a JavaHelp set, add this line to the manifest: OpenIDE-Module-Requires: org.netbeans.api.javahelp.Help In the POM of the container project (i.e., NOT the application project and NOT a functionality module project), add the "dependencies" section in the definition of the "org.codehaus.mojo", while setting the "version" to 3.0: org.codehaus.mojo nbm-maven-plugin 3.0 true javax.help javahelp 2.0.02 ${brandingToken} foobar Note that the "version" is set to 3.0. I could not get JavaHelp to work for 3.1 or 3.2 (the default in 6.9+) and had to revert to 3.0. Maybe someone can correct me on this point somehow. Make sure that the POMs in the other projects (i.e., application project and functionality module projects) do not have their own definition of "org.codehaus.mojo". I.e., the container project should handle this definition for the whole application. You have now configured the application to use JavaHelp and have indicated each module that will provide JavaHelp. In each of those modules, you now need to provide JavaHelp content. The simplest approach for this is to create an Ant-based NetBeans Platform module. Then, use the wizard available for Ant-based NetBeans Platform projects for the creation of JavaHelp sets. Then copy the files that are generated into your Maven-based NetBeans Platform project: Note the following in the screenshot above. The "resources" folder contains the "module1-helpset.xml" file, which references the files in the "src/main/javahelp" folder by means of having the following content: And note that the "layer.xml" file above has this folder added: The folder "src/main/javahelp" is new, I created that myself, by copying that folder from the Ant-based NetBeans Platform module. That's the place where you will now create all your help topics within the Mavenized application. The Map file lists the identifying keys of the help topics, the TOC lists those keys that define the table of contents, and the Index does the same to define the index of the helpset. That's all you need. Now you have JavaHelp working in your Mavenized NetBeans Platform application:

sometimes it is useful to display the version of your web application, typically in the footer area of the web page. by displaying the version information, one can quickly determine which version of the app is running or users can use it for error reporting purposes. displaying the version is also useful when there are frequent automated releases using the maven release plugin and it is hard to track which version is actually deployed on a server. here’s an illustration of how version can be displayed in the footer. this article describes how to display the version of your maven project in your web application. step 1: create a view file to display the version first we will create a view file which will display the version. the view file could be a jsp, html, xhtml, a facelet or any format depending on your stack. in this example, we will create a jsp file which will print the version somewhere in the page. however, instead of the hardcoding an actual version, we put a special token ‘project_version’. we don’t want to hard code the version information as it will change from release to release. the special token will be replaced with actual maven project version when the app is packaged. create the following jsp. src/main/webapp/version.jsp here’s a simple file which displays the version information. project version is project_version step 2: add snippet to pom.xml the maven replacer plugin is a simple and useful plugin which can replaces tokens in file of your choice. add the following snippet of code to your pom.xml. com.google.code.maven-replacer-plugin maven-replacer-plugin 1.3.2 prepare-package replace target/myproject/version.jsp false project_version ${project.version} to test the above setup, you can run the following command: mvn prepare-package navigate to the following file. target/myproject/version.jsp the contents of this file will look something like this (assuming that your project version is 1.0-snapshot): project version is 1.0-snapshot when your application ships, the version.jsp file will always have the correct version. you can see the version by invoking the version.jsp page. http://localhost:8080/myproject/version.jsp note: your view file does not have to be a jsp, it can be any a view file suitable to your stack. it could be part of your footer which could be included at the bottom in each page. the special token can be changed from project_version to another value of your choice, just change it in the view file as well as the pom.xml instead of having the version contained in a view file, you could have the version in a properties file or a message bundle and then display a message from the message bundle on the view references maven replace plugin on google code from http://www.vineetmanohar.com/2010/09/how-to-display-maven-project-version-in-your-webapp

a few days back, i was browsing the web and found an interesting open source framework called checkthread . it is a static analysis tool for catching java concurrency bugs at compile time. static analysis tools are used to find out programming error in the code by analyzing their byte code. to me a tool aiming to catch concurrency bugs at compile time was worth spending time on. so, i decided to play with checkthread to find out its capabilities. checkthread requires developers to specify the thread policy in either xml or annotations. thread policy defines whether the piece of code (i.e. a method) is thread safe, not thread safe, or thread confined. thread confined means that this method is confined to a specific runtime thread. for example, the swing api must be invoked on the event-dispatch thread. prerequisite before you start: download eclipse plugin . put plugin in jar in eclipse plugins folder and restart eclipse. for more information refer here . download checkthread annotation jar . this jar is not present in any maven repository, so manually install the jar in your maven repository. mvn install:install-file -dfile=checkthread-annotations-1.0.9.jar -dgroupid=org.checkthread -dartifactid=checkthread-annotations -dversion=1.0.9 -dpackaging=jar checkthread capabilities let's look at an example: public class threadsafetyexample { final map helpermap = new hashmap(); public void addelementtomap() { helpermap.put("name", "shekhar"); } } this is a simple class which puts a key value pair in a map. is this code thread safe? no. lets test this class using multiple threads. in the unit test i am using countdownlatch for producing maximum parallelism. i talked about countdownlatch in an earlier article . @test public void regressiontest() throws exception{ for (int i = 1; i <= 100; i++) { system.out.println("runing "+i); addelementtomapwhenaccessedbymultiplethread(); } } public void addelementtomapwhenaccessedbymultiplethread() throws exception { final countdownlatch latch = new countdownlatch(1); final threadsafetyexample helper = new threadsafetyexample(); class mythread extends thread { @override public void run() { try { latch.await(); } catch (interruptedexception e) { } helper.addelementtomap(); } } int threadcount = 2000; mythread[] mythreads = new mythread[threadcount]; for (int i = 0; i < mythreads.length; i++) { mythreads[i] = new mythread(); mythreads[i].start(); } latch.countdown(); for (mythread mythread : mythreads) { mythread.join(); } assertequals(1, helper.helpermap.size()); } i am calling the method addelementtomapwhenaccessedbymultiplethread in a for loop because if you run the test only once it might not fail (thread timing). so, how can these types of errors be detected at compile time if most of the unit-tests and integration tests that we write do not test the code in multi-threaded environments? checkthread can help you out in such situations. checkthread can only help you if you annotate your method with threadsafe annotation. for example, lets apply @threadsafe annotation to the threadsafetyexample class: public class threadsafetyexample { final map helpermap = new hashmap(); @threadsafe public void addelementtomap() { helpermap.put("name", "shekhar"); } } now when you run the checkthread by pressing the button you will see a compile time error as shown below: as you can see in the above image, the tool shows the code where problems exist and descriptions of errors in the problems section. this can come in handy while writing multi threaded code. you just need to think about your contract, whether the method you have written should be thread safe or not. this was just a simple use case but it can also help you detect race conditions . have a look at this tool, maybe it can help you detect concurrency bugs.

Parameterized types - the <> expressions that can be used in Java as of JDK 5 are not just for collections. I find myself frequently using them in APIs I design. They really do let you write things which are more generic in the non-Java sense of the word - and the result is more reusable code, which means less code overall, which means fewer bugs and things to test. The verbosity, and some of the weirdness of type-erasure are less than ideal, but used right, the benefits are worth the complexity. The standard (and somewhere recommended) naming convention for parameterized types is to use a single-letter name. That works fine in signatures that have only one such type. But in practice, single-letter names make code less self-describing, and if you're defining a class with more than one parameterized type, it can be confusing and hard to read. People other than me will have to call, understand and maintain my code - the more self-describing I can make it, the better. So I am looking for a naming convention that makes it obvious that something is a parameterized type, but allows for descriptive names. I am wondering if anybody else has run into this problem, and if there is any emerging consensus on naming generics. Do you work on a project that uses generics a lot? If so, what do you do? Here's an example. At the moment, I'm writing a generic (in both senses) class which simply limits the number of threads which can access some resource. It's basically a wrapper around a Semaphore which uses a Runnable-like object to ensure that the Semaphore is accessed correctly, and does some non-blocking statistic gathering about thread contention. So to access the scarce resource, you pass in a ResourceAccessor: public interface ResourceAccessor { public Result run (ProtectedResource resource, Argument argument); } The problem is that, when somebody looks at this interface, they will instantly get the idea that there are really classes they need to go find, which are called ProtectedResource, Argument and Result - and of course, no such classes exist - these are just names for generic types. The standard-naming-convention is worse: public interface ResourceAccessor { public S run (T resource, R argument); } Here, nobody could possibly figure out what on earth this class is for without extensive documentation - this is a really horrible idea. So I've concluded that the standard recommendations for generic type names are simply wrong for any non-trivial usage (I.e. Collection is fine, since there is one type and Collections are well-understood). You simply can't do this on a non-collection code structure you have invented, or people will just be confused and not use it. The best suggestion I've heard thus far is using $ as a prefix: public interface ResourceAccessor <$ProtectedResource, $Argument, $Result> { public $Result run ($ProtectedResource resource, $Argument argument); } I don't find this pretty, but I don't have any better ideas, and at least it makes it crystal-clear that there is something different about these names. Any thoughts? What do you do in this situation?