Getting started with CQEngine: LINQ for Java, Only Faster

CQEngine or collection query engine is a library that allows you to build indices over java collections and query them for objects using exposed properties. It offers similar capability to LINQ in .net but is thought to be  faster because it builds indices over collections before querying them and uses set theory instead of iterations.

In this post we will see how to query a simple collection of objects, in our example, a collection of users of a hypothetical system, using CQEngine. In a subsequent post, we will also see how iteratively searching a collection compares to querying via CQEngine.

The first step is to get the CQEengine jar file. Download the jar from the CQEngine website or if you are using Maven, add the following dependency.

	<dependency>
	      <groupId>com.googlecode.cqengine</groupId>
	      <artifactId>cqengine</artifactId>
	      <version>1.0.3</version>
	</dependency>

Next, lets create the Class whose object we will be searching for:

package co.syntx.examples.cqengine;

import com.googlecode.cqengine.attribute.Attribute;
import com.googlecode.cqengine.attribute.SimpleAttribute;

public class User {

	private String username;
	private String password;
	private String fullname;
        private Role role;

	public User(String username, String password, String fullname, Role role) {
		super();
		this.username = username;
		this.password = password;
		this.fullname = fullname;
		this.role = role;
	}

	 public static final Attribute<User, String> FULL_NAME = new SimpleAttribute<User, String>("fullname") {
	        public String getValue(User user) { return user.fullname; }
	 };

	 public static final Attribute<User, String> USERNAME = new SimpleAttribute<User, String>("username") {
	        public String getValue(User user) { return user.username; }
	 };

	public String getUsername() {
		return username;
	}

	public void setUsername(String username) {
		this.username = username;
	}

	public String getPassword() {
		return password;
	}

	public void setPassword(String password) {
		this.password = password;
	}

	public String getFullname() {
		return fullname;
	}

	public void setFullname(String fullname) {
		this.fullname = fullname;
	}

	public Role getRole() {
		return role;
	}

	public void setRole(Role role) {
		this.role = role;
	}
}

Next, we write a class, to perform our searches. I will go function by function.

1. Function to Build a Test Indexed Collection:
In the following function, we build an indexed collection, define indices on attributes, and populate this collection with a certain number of objects. In actual usage, your collection will probably be filled with objects being returned from the DB, read from a file or other similar scenarios.

public void buildIndexedCollection(int size) throws Exception {
	indexedUsers = CQEngine.newInstance();
	indexedUsers.addIndex(HashIndex.onAttribute(User.FULL_NAME));
	indexedUsers.addIndex(SuffixTreeIndex.onAttribute(User.FULL_NAME));

	for (int i = 0; i < size; i++) {
	    String username = RandomStringGenerator.generateRandomString(8,RandomStringGenerator.Mode.ALPHANUMERIC);
	    String password = RandomStringGenerator.generateRandomString(8,RandomStringGenerator.Mode.ALPHANUMERIC);
	    String fullname = RandomStringGenerator.generateRandomString(5,RandomStringGenerator.Mode.ALPHA)
			+ " " + RandomStringGenerator.generateRandomString(5,RandomStringGenerator.Mode.ALPHA);
	    Role role = new Role();
	    role.setName("admin");
	    indexedUsers.add(new User(username, password, fullname, role));
	}
}

• In line 3 we are initializing a new Indexed Collection, a reference of which is stored in the class variable indexedUsers. The reference is of type IndexedCollection<User>
• In lines 4 and 5, we define two indices i) a Hash Index suitable for equal style queries. ii) a Suffix Index suitable for ends with style queried. For the purpose of this example, we are building indices only on the the Full name field.
• In line 8, we use a random string generator to populate dummy objects.
• In line 14 we add our object to our indexed collection.

2. Function to Perform Indexed Search for Exact Matches:
In this function, we are querying for names that exactly match a given name. The equal function takes in the attribute upon which to perform the query and the value to search. The method equal  is statically imported via a

import static com.googlecode.cqengine.query.QueryFactory.*;

In the example below, we are looping the results returned by the retrieve method and not doing anything with it. In your case, you may choose to return the Iterator returned by retrieve.

public void indexedSearchForEquals(String fullname) throws Exception {
       Query query = equal(User.FULL_NAME, fullname);
	   for (User user : indexedUsers.retrieve(query)) {
		// System.out.println(user.getFullname());
	   }
}

3. Function to Perform Indexed Search for Ends With Matches:
In this function, we are querying for names that end with a certain suffix.

public void indexedSearchForEndsWith(String endswith) throws Exception {
       Query query1 = endsWith(User.FULL_NAME, endswith);
	   for (User user : indexedUsers.retrieve(query1)) {
		// System.out.println(user.getFullname());
	   } 
}

4. Function to Perform Indexed Search for Equals or Ends With Matches:
This function is a combination of both queries mentioned below and has an or relation between them.

public void indexedSearchForEqualOrEndsWith(String equals, String ends) throws Exception {
       Query query = or(equal(User.FULL_NAME, equals),endsWith(User.FULL_NAME, ends));
	   for (User user : indexedUsers.retrieve(query)) {
                 // System.out.println(user.getFullname());
	   }
}

5. Putting it together:
All the functions above belong to a class called CQEngineTest. We create a new object, build a test collection, then search for either exact matches, strings that end with a certain suffix or either.

      CQEngineTest test = new CQEngineTest();
      test.buildIndexedCollection(size);
      test.indexedSearchForEqualOrEndsWith("test", "test");

In this example, we have used a Hash Index and a Suffix Tree Index. There are many other types of indices that you can choose depending on the type of query that you want to perform. A list of these indices and when to use them can be found on the cqengine project page. Also, apart from equal  or endswith operations, there are others that you would typically expect to find.

In a subsequent post  we will also see how an indexed search compares with a typical iterative search in terms of times.