Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Our Tryst With Elasticsearch

DZone 's Guide to

Our Tryst With Elasticsearch

Elastisearch for the win!

Free Resource

telescope-looking-out-on-city


Elasticsearch is a popular solution option for searching text data. Recently, we used it to build employee search functionality in one of the web applications on GCP

In short, we wanted to be able to search employees by the initial few characters of the "name" of the employee. Plus:

  1. Show Relevant matches in the order of Manager, City, Country, Sub-segment, Segment. This is in the context of a logged-in user.
  2. Search-as-you-type/autocomplete suggester.
  3. Reusable component.
You may also like: What Is Elasticsearch? (And Why You Need to Use It).

Elasticsearch 

Elasticsearch uses an inverted index for faster searches. Elasticsearch v7.2.0 software is installed on Google VM. Employee data is stored as JSON documents into Elasticsearch for indexing and search purposes. 

The diagram below depicts the overall solution for employee search.

Solution for each employee search








First up, you need to create Index called "employee"

PUT employee
{
"settings": {
"index": {
"max_ngram_diff": "20",
"number_of_shards": "5",
"provided_name": "employee",
"analysis": {
"filter": {
"nGram_filter": {
"token_chars": ["letter", "digit", "punctuation", "symbol"],
"min_gram": "2",
"type": "nGram",
"max_gram": "20"
}
},
"normalizer": {
"lowerasciinormalizer": {
"filter": ["lowercase", "asciifolding"],
"type": "custom"
}
}
}
}
}
}


Ngram (tokens) should be used as an analyzer.

Normalizer filters, such as lowercase and asciifolding, help in case-insensitive search and ignore special characters/spaces respectively.

Then, create a custom mapping for fields on which search needs to be done (i.e. name of employee). Use "lowerasciinormalizer" on "name" field. Default mapping is generated for other fields in JSON document.

PUT /employee/_mapping
{
"properties": {
"name": {
"type": "keyword",
"normalizer": "lowerasciinormalizer"
}
}
}


Employee JSON documents can be imported into Elasticsearch using "_bulk" API.

PUT /employee/_bulk
{"index":{"_id":"02243876"}}
{"empid":"02243876","name":"Peter Scott","lastname":"Scott","firstname":"Peter","managername":"John England","city":"Chicago","country":"USA","subsegment":"Auditing","segment":"Accounts"}


You could also use Kibana GUI to perform a bulk data import from CSV input file into Elasticsearch. Kibana GUI guides you through the process of index setting and mapping before bulk import.

Google persistent disk is used as storage for employee JSON documents.

"Search" Microservice

This is our custom RESTful microservice used to convert input search keyword into Elasticsearch DSL Query clause and return results back to the Angular client application. We built this service in .NET Core 2.2 framework.

NEST is the Elasticsearch client library in .NET. We will install the Nuget package for NEST to interact with Elasticsearch from within our microservice code. 

See the use of Lambda expression to build a Search Query clause using the NEST library. By default, Elasticsearch sorts matching search results by relevance score, which measures how well each document matches a query. 

async Task<IList<Employee>> ReturnElasticSearchResult(Data data) {
  ISearchResponse<Employee> response = null;           
  var elasticSearchUrl = _configuration["elasricsearch:url"];
  var uri = new Uri(elasticSearchUrl);
  var settings = new ConnectionSettings(uri);
  settings.DefaultIndex("employee");                
  ElasticClient client = new ElasticClient(settings);
  Elasticsearch.Net.SearchType searchType = Elasticsearch.Net.SearchType.QueryThenFetch;
  response = client.Search<Employee>(s => s
                .SearchType(searchType)
                .Query(q => (q.Term(t => t.mangername, data.managername) && q.Wildcard(w => w.name, "*" + data.name + "*")) ||
                (q.Term(t2 => t2.city, data.city) && q.Wildcard(w => w.name, "*" + data.name + "*")) ||
                (q.Term(t3 => t3.country, data.country) && q.Wildcard(w => w.name, "*" + data.name + "*")) ||
                (q.Term(t4 => t4.subsegment, data.subsegment) && q.Wildcard(w => w.name, "*" + data.name + "*")) ||
                (q.Term(t5 => t5.segment, data.segment) && q.Wildcard(w => w.name, "*" + data.name + "*")))
                .Sort(sort => sort.Field(f => f.Field("_score").Descending())));
  return response.Documents.ToList();
}


Search service will be exposed as API on API Gateway and invoked by Angular application client. API Gateway can be used to improve search performance by caching search results.

Angular Search GUI Component 

Angular Search component will display input textbox for search purposes in the GUI. This component is exported and imported as an npm component in another application where employee search is required.

The Angular autocomplete feature should be used on UI to prompt/suggest employees matching search keywords. 

 debounceTime(300) is used to wait for 300 ms after the value change event to make the next Elasticsearch API call. Please see code block from component.ts file.

ngOnInit() {
    this.searchEmployeesCtrl.valueChanges.pipe(
        debounceTime(300),
        tap(() => {        
          this.employees = [];
          this.isLoading = true;
        }),

        switchMap(          
          value =>(this.http.post("api/elasticsearch", {'name': value, 
  'managername':this.emp.managername, 
  'city':this.emp.city,                                                                 
  'country':this.emp.country,
                                                  'segment':this.emp.segment,  
  'subsegment':this.emp.subsegment})
          ).pipe(
            finalize(() => {
              this.isLoading = false
            }),
          )
        )
      ).subscribe((data:Employee[]) => {        
          this.employees = data;
        });
  }


Please see the code block from component.html file.

<input matInput placeholder="Employee Search" #Search aria-label="State" [matAutocomplete]="auto" [formControl]="searchEmployeesCtrl">
    <mat-icon matSuffix>search</mat-icon>

    <mat-autocomplete #auto="matAutocomplete" [displayWith]="displayFn" (optionSelected)='getPosts($event.option.value,Search )' >
      <mat-option *ngIf="isLoading" class="is-loading" >Loading...</mat-option>
      <ng-container *ngIf="!isLoading">
        <mat-option *ngFor="let employee of employees" [value]="employee.empid" style="height: 25Px;">
          <span> {{employee.name}}</span>         
        </mat-option>
      </ng-container>
    </mat-autocomplete>
  </mat-form-field>


Related Articles

Topics:
elasticsearch

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}