DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • How To Build Self-Hosted RSS Feed Reader Using Spring Boot and Redis
  • Simultaneous, Multiple Proportion Comparisons Using Marascuilo Procedure
  • How To Dockerize Mean Stack App
  • SQL Query Performance Tuning in MySQL

Trending

  • Choosing the Appropriate AWS Load Balancer: ALB vs. NLB
  • Mastering Persistence: Why the Persistence Layer Is Crucial for Modern Java Applications
  • Memory Management in Java: An Introduction
  • Send Your Logs to Loki
  1. DZone
  2. Data Engineering
  3. Databases
  4. Solr and Autocomplete (part 2)

Solr and Autocomplete (part 2)

Rafał Kuć user avatar by
Rafał Kuć
·
Sep. 05, 11 · News
Like (0)
Save
Tweet
Share
17.23K Views

Join the DZone community and get the full member experience.

Join For Free

in the previous part i showed how the faceting mechanism can be used to achieve the autocomplete functionality. today i’ll show you how to use a component called suggester to implement autocomplete functionality.

the begining


there is one thing that you must know – suggest component is not available in solr version 1.4.1 and below. to start using this component you need to download 3_x or trunk version from lucene/solr svn.

configuration

before we get into the index configuration we need to define an search component. so let’s do it:

<searchcomponent name="suggest" class="solr.spellcheckcomponent">
 <lst name="spellchecker">
  <str name="name">suggest</str>
  <str name="classname">org.apache.solr.spelling.suggest.suggester</str>
  <str name="lookupimpl">org.apache.solr.spelling.suggest.tst.tstlookup</str>
  <str name="field">name_autocomplete</str>
 </lst>
</searchcomponent>

it is worth mentioning that suggest component is based on solr.spellcheckcomponent and that’s why we can use the above configuration. we have three important attributes in the configuration:

  • name - name of the component.
  • lookupimpl – an object that will handle the search. at this point we have two possibilities to use – jasperlookup or tstlookup. this second one characterizes greater efficiency.
  • field – the field on the basis of which suggestions are generated.

now let’s add the appropriate handler:

<requesthandler name="/suggest" class="org.apache.solr.handler.component.searchhandler">
 <lst name="defaults">
  <str name="spellcheck">true</str>
  <str name="spellcheck.dictionary">suggest</str>
  <str name="spellcheck.count">10</str>
 </lst>
 <arr name="components">
  <str>suggest</str>
 </arr>
</requesthandler>

quite simple configuration, which defines a handler with an additional search component and tell solr that the maximum number of suggestions returned is 10, this it should use dictionary named suggest (which is actually a suggest component) which is exactly the same as our defined component.

index

let us assume that our document consists of three fields: id, name and description. we want to generate suggestions on the field that hold the name of the product. our index could look like this:

<field name="id" type="string" indexed="true" stored="true" multivalued="false" required="true"/>
<field name="name" type="text" indexed="true" stored="true" multivalued="false" />
<field name="name_autocomplete" type="text_auto" indexed="true" stored="true" multivalued="false" />
<field name="description" type="text" indexed="true" stored="true" multivalued="false" />

in addition, there is the following copy field definition:

<copyfield source="name" dest="name_autocomplete" />

suggesting single words

in order to achieve individual words suggestions text_autocomplete type should be defined as follows:

<fieldtype class="solr.textfield" name="text_auto" positionincrementgap="100">
 <analyzer>
  <tokenizer class="solr.whitespacetokenizerfactory"/>
  <filter class="solr.worddelimiterfilterfactory" generatewordparts="1" generatenumberparts="1" catenatewords="1" catenatenumbers="1" catenateall="0" splitoncasechange="1"/>
  <filter class="solr.lowercasefilterfactory"/>
 </analyzer>
</fieldtype>

suggesting phrases

to implement the entire phrase suggestions our text_autocomplete type should be defined as follows:

<fieldtype class="solr.textfield" name="text_auto">
 <analyzer>
  <tokenizer class="solr.keywordtokenizerfactory"/>
  <filter class="solr.lowercasefilterfactory"/>
 </analyzer>
</fieldtype>

if you want to use phrases you may want to define your own query converter.

dictionary building

before we start using the component, we need to build its index. to this send the following command to solr:

/suggest?spellcheck.build=true

queries

now we come to use of the component. in order to show how the use the component, i decided suggest whole phrases. the example query could look like that:

/suggest?q=har

after running that query i got the following suggestions:

<?xml version="1.0" encoding="utf-8"?>
<response>
 <lst name="responseheader">
  <int name="status">0</int>
  <int name="qtime">0</int>
 </lst>
 <lst name="spellcheck">
  <lst name="suggestions">
   <lst name="dys">
    <int name="numfound">4</int>
    <int name="startoffset">0</int>
    <int name="endoffset">3</int>
    <arr name="suggestion">
     <str>hard drive</str>
     <str>hard drive samsung</str>
     <str>hard drive seagate</str>
     <str>hard drive toshiba</str>
    </arr>
   </lst>
  </lst>
 </lst>
</response>

the end

in the next part of the autocomplete functionality i’ll show how to modify its configuration to use static dictionary into the mechanism and how this can helk you get better suggestions. the last part of the series will be a performance comparison of each method in which i’ll try to diagnose which method is the fastest one in various situations.

Phrase (software) Database Dictionary (software) Efficiency (statistics) Comparison (grammar) Document Trunk (software) Command (computing) Fastest Build (game engine)

Published at DZone with permission of Rafał Kuć, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • How To Build Self-Hosted RSS Feed Reader Using Spring Boot and Redis
  • Simultaneous, Multiple Proportion Comparisons Using Marascuilo Procedure
  • How To Dockerize Mean Stack App
  • SQL Query Performance Tuning in MySQL

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: