Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Sorting by function value in Solr

DZone's Guide to

Sorting by function value in Solr

· Java Zone ·
Free Resource

Verify, standardize, and correct the Big 4 + more– name, email, phone and global addresses – try our Data Quality APIs now at Melissa Developer Portal!

In Solr 3.1 and later we have a very interesting functionality, which enables us to sort by function value. What does that gives us? Actually, a few interesting possibilities.

Let’s start

The first example that comes to mind, perhaps because of the project on which I worked some time ago, it’s sorting on the basis of distance between two geographical points. So far, to implement such functionality was needed changes in Solr (for example, LocalSolr or LocalLucene). Using Solr 3.1 and later, you can sort your search results using the value returned by the defined functions. For example, is Solr, we have the dist function calculating the distance between two points. One variation of the function is a function accepting five parameters: algorithm and two pairs of points. If, using this feature, we would like to sort your search results in ascending order from the point of latitude and longitude 0.0, we should add the following sort parameter to the Solr query:

 
...sort=dist(2, geo_x, geo_y, 0, 0) asc

I suspect that the most commonly used values of the first parameter will be:

  • 1 – calculation based on the Manhattan metrics
  • 2 – calculation of Euclidean distance

A few words about performance

Everything is fine till now, but how it looks like in terms of performance ? I’ve made a two simple tests.

During the first test, I indexed 200 000 documents, every one of them consisted of four fields: identifier (numeric field), description (a text field) and location (two numeric fields). In order not to obscure the test results for sorting, I used one of the simplest functions currently available in the Solr – the sum function which sums two given arguments. I compared the query time of the default sorting (by score) with the ones which used the value of the function. The following table shows the results of the test:

Query Results number Query time Re-query time
q=*:*&sort=score+desc 200.000 31ms 0ms
q=*:*&sort=sum(geo_x,geo_y)+desc 200.000 813ms 0ms
q=opis:ala&sort=score+desc 200.000 47ms 1ms
q=opis:ala&sort=sum(geo_x,geo_y)+desc 200.000 797ms 1ms

Another test was based on a comparison of sorting by a string field to sort using function. The test was almost identical to the first test. I’ve indexed 200,000 documents indexed (with additional field: name_sort – type string) and used the sum function. The following table shows the results of the test:

Query Results number Query time Re-query time
q=*:*&sort=opis_sort+desc 200.000 267ms 0ms
q=*:*&sort=sum(geo_x,geo_y)+desc 200.000 823ms 0ms
q=opis:ala&sort=opis_sort+desc 200.000 266ms 1ms
q=opis:ala&sort=sum(geo_x,geo_y)+desc 200.000 810ms 1ms

Above test shows that sorting using the sort function is much slower than the default sort order (which you’d expect). Sorting on the basis of function value is also slower than sorting with the use of string based field, but the difference is not as significant as in the previous case.

A few words at the end

Of course, the above test just glides through the topic of sorting efficiency using Solr functions, however, shows a direct relationship. Given that, in most cases, this will not be the default sort method and giving us a really powerful tool it seems to me that this is a feature worth remembering. It will definitely be worth using when the requirements says that we have to sort on the value that depends on the query and index values – as in the case of sorting by distance from the point specified by the user.

Developers! Quickly and easily gain access to the tools and information you need! Explore, test and combine our data quality APIs at Melissa Developer Portal – home to tools that save time and boost revenue. Our APIs verify, standardize, and correct the Big 4 + more – name, email, phone and global addresses – to ensure accurate delivery, prevent blacklisting and identify risks in real-time.

Topics:

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}