Solr and Autocomplete (Part 1)

almost everyone has seen how the autocomplete feature looks like. no wonder, then, solr provides mechanisms by which we can build such functionality. in today’s entry i will show you how you can add autocomplete mechanism using faceting.

index

suppose you want to show some hints to the user in the on-line store, for example you want to show products name. suppose that our index is composed of the following fields:

<field name="id" type="string" indexed="true" stored="true" multivalued="false" required="true"/>
<field name="name" type="text" indexed="true" stored="true" multivalued="false" />
<field name="description" type="text" indexed="true" stored="true" multivalued="false" />

a text type is defined as follows:

<fieldtype name="text" class="solr.textfield" positionincrementgap="100">
 <analyzer>
  <tokenizer class="solr.whitespacetokenizerfactory"/>
  <filter class="solr.worddelimiterfilterfactory" generatewordparts="1" generatenumberparts="1" catenatewords="1" catenatenumbers="1" catenateall="0" splitoncasechange="1"/>
  <filter class="solr.lowercasefilterfactory"/>
 </analyzer>
</fieldtype>

configuration

to start, consider what you want to achieve – do we want to suggest only individual words that make up a name, or maybe full names that begin with the letters specified by the user. depending on our choices we have to prepare the appropriate field on which we will build hints.

prompting individual words that make up the name

in the case of single words, we should use a field that is tokenized. in our case, the field named name will be sufficient. however, note that if you want to use for example stemming you should define another type, which do not use stemming because of how this analysis operates on the contents of the field.

prompting full name

for full names of the products suggestions we need a different field configuration – the best for this will be a untokenized field. but we can not use string based field for that. for this purpose, we define the field as follows:

<field name="name_auto" type="text_auto" indexed="true" stored="true" multivalued="false" />

this type is defined as follows:

<fieldtype name="text_auto" class="solr.textfield">
 <analyzer>
  <tokenizer class="solr.keywordtokenizerfactory"/>
  <filter class="solr.lowercasefilterfactory"/>
 </analyzer>
</fieldtype>

to not modify the format of the data we also add the appropriate definition of the copy information :

<copyfield source="name" dest="name_auto" />

how do we use it ?

to use the data we prepared we use a fairly simple query:

q=*:*&facet=true&facet.field=field&facet.mincount=1&facet.prefix=user_query

where:

• field – field on the basis of which we intend to make suggestions. in our case the field named name or name_string .
• user_query - letters entered by the user.

it is worth noting rows=0 parameter is added here to only show the faceting result without the query results. of course, this is not a necessity.

an example query would look like that:

fl=id,name&rows=0&q=*:*&facet=true&facet.field=name_auto&facet.mincount=1&facet.prefix=har

the result of this query might look like this:

<response>
 <lst name="responseheader">
  <int name="status">0</int>
  <int name="qtime">0</int>
 </lst>
 <result name="response" numfound="4" start="0"/>
 <lst name="facet_counts">
  <lst name="facet_queries"/>
  <lst name="facet_fields">
   <lst name="name_auto">
    <int name="hard disk">1</int>
    <int name="hard disk samsung">1</int>
    <int name="hard disk seagate">1</int>
    <int name="hard disk toshiba">1</int>
   </lst>
  </lst>
  <lst name="facet_dates"/></lst>
</response>

additional features

it is worth to mention the additional opportunities which are inherent to this method.

the first possibility is to show the user additional information such as number of results that you get when you select an appropriate hint. if you want to show such information it will certainly be an interesting option.

the next thing is sorting with the use of facet.sort parameter. depending on your needs, we can sort the results by the number of documents (the default behavior, parameter set to true ) or alphabetically (value set to false ).

we may limit the suggestions to those which have more results than a specified number. to take advantage of this opportunity pass in a parameter facet.mincount with the appropriate number.

and as for me the biggest advantage of this method is the possibility of getting only those suggestions that not only match the letters that the user typed but also some other parameters, like category for example. for example, we want to show hints for the user who is in the household section of our store. we suspect that at this moment the user will not be interested in dvd-type products, and therefore we add a parameter fq=department:homeappliances (assuming that we have such a department). after such a modified query, you do not get hints generated from the entire index, we only get those narrowed to the selected department.

a few words at the end

as other method, this one too, have its advantages and disadvantages. the advantage of this solution is its ease of use, no additional components requirement, and that the result hints can be easily narrowed to be generated only from those documents that match the query entered by the user. as a big plus is that the method includes number of result that will be shown after selecting the hint (of course with the same search parameters). for the downside is definitely need to have additional types and fields, quite limited abilities and the load caused by the use of faceting mechanism.

the next entry about the autocomplete will try to expand on and show a further methods of generating hints using solr.