Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

A Complete And Easy Guide To Check Elasticsearch

DZone's Guide to

A Complete And Easy Guide To Check Elasticsearch

As Ops/DevOps, we are usually more concerned about cluster health and the data inside. This article is a simple step-by-step guide to check Elasticsearch, cluster, nodes, shards, indices, documents, and more.

· Big Data Zone
Free Resource

Need to build an application around your data? Learn more about dataflow programming for rapid development and greater creativity. 

To poll the Elasticsearch DB status, we usually need to learn and try many, many REST APIs. If you're new to this, it can take quite a while to put all the pieces together.

As Ops/DevOps, we are usually more concerned about cluster health and the data inside. Enclosed is a simple step-by-step guide to check Elasticsearch, cluster, nodes, shards, indices, documents, and more.

query_elasticsearch.png

Impatient users can jump to our Elasticsearch Cheat Sheet at the bottom.

1. Basic Intro About Elasticsearch

Generally speaking, Elasticsearch is an efficient and API friendly search engine based on Lucene.

Some key concepts to know before we dive deeper.

  • Cluster: a collection of ES nodes.
  • Nodes: server/instance stores data or coordinate with indexing and search capabilities.
  • Index: a collection of documents. Like tables in MySQL.
  • Type: To ensure optimal performance, we can define mappings for data types.
  • Document: a basic unit of information. Like record in SQL.
  • Shards: It allows us to horizontally split a big index, and store in multiple nodes.

2. GUI Tool: Elasticsearch-Head

elasticsearch-head is a web front-end for browsing and interacting with an Elastic Search cluster1. It is pretty much like phpMyAdmin for mysql2. By default, Elasticsearch doesn't install this plugin.

Though its UI looks a bit primitive and less attracting, it's certainly a useful tool for dummies like me. I frequently use the "Structured Query" menu to grab some random records, in order to understand unfamiliar ES indices.

elasticsearch_head_overview.png

3. Elasticsearch Global Info By API

Time for complicated, yet powerful REST API.

  • Check cluster health.
curl \
 http://localhost:9200/_cluster/health?pretty

es_cluster_health.png

Usually, we pay attention to "status" field, which should be green. When it's yellow, probably we have unassigned shards or unavailable nodes. When it's red, all primary shards of certain indices are inactive. This is truly bad.

  • Check global setting and cluster stats3.
# Get version
curl http://localhost:9200

# Get global settings
curl \
 http://localhost:9200/_cluster/settings

# Poll cluster stats
curl http://localhost:9200/_stats?pretty

Get version by CLI, without starting ES service.

bin/elasticsearch --version
  • List nodes and shards
# List nodes
curl http://localhost:9200/_cat/nodes?v

# List shards
curl http://localhost:9200/_cat/shards?v
  • Show Elasticsearch slow query4: As DevOps/Ops, we definitely want to be on top of this. To enable this, make sure elasticsearch.yml is properly configured. Tips: If you have problems to verify your setting about this, temporarily change thresholds to zero. Then every fetch operation should generate a slow log.

4. Check Elasticsearch Index By API

  • Show Basic Summary Of An Elasticsearch Index
curl \
http://localhost:9200/$index/_count?pretty

es_index_summary.png

"count" indicates how many records in the index. "_shards" shows how the index is horizontally split.

  • Check DB schema of An Index or A field.
# Get Index Mapping
curl \
localhost:9200/$index/_mapping?pretty

# Get Field Mapping
curl \
localhost:9200/$index/_mapping/$type
  • Indices verbose statistics.5.
curl \
http://localhost:9200/$index/_stats

5. Check Documents Inside Indices

  • Get all documents of an index. Be careful with big indices. You might get tons of output running this command.
curl \
http://localhost:9200/$index/_search?pretty
  • Full Text Search
# Get documents contains 50
curl \
http://localhost:9200/$index/_search?q=50
  • URI Search: filter by field.6 URI search indicates parsing request parameters from URI.
# Get documents whose f1 is 50
curl \
http://localhost:9200/$index/_search?q=f1:50
  • Get documents with paging mechanism.7
# Get iterator, fetching 2 records each time
curl \
"localhost:9200/$index/_search?scroll=1m&size=2"\
-d '{
    "query" : {
        "match_all" : {}
    }
}'

# Get scroll id from previous command
# Then keep polling records of next page
curl 'localhost:9200/_search/scroll'  \
-d' {
    "scroll" : "1m", 
    "scroll_id" : "INPUT_YOUR_SCROLL_ID" 
}'
  • Advanced Search With Request Body.8 This would fall into the realm of lucene now.
curl \
http://localhost:9200/$index/_search?pretty\
 -d '{
    "query": {
        "query_string": 
        {"query": "(field1:value1) 
            AND (field2:value2)"}
    }
}'

6. More Tools

  • Elasticdump is the 3rd tool to import and export Elasticsearch indices9. Since it's an npm module, we have to install nodejs/npm to use it. Frankly speaking, this is not my preference. And Elasticsearch has built-in support called snapshot10. However, I would still recommend elasticdump for small indices. Why? Snapshot a bit complicated than I would expect. And we have to configure Elasticsearch.yml and restart es instances to take effect. Too intruding. Compared to elasticdump, I wish Elasticsearch can have more native support in the future version.

7. Recap Today's Discussion: Elasticsearch Cheat Sheet

What To Check Summary
elasticsearch-head1 A web tool
elasticdump7 Import and export tools
Get Version localhost:9200
Version By CLI bin/elasticsearch --version
Cluster health localhost:9200/_cluster/health?pretty
List Nodes localhost:9200/_cat/nodes?v
List shards localhost:9200/_cat/shards?v
List indices localhost:9200/_cat/indices?v
Indice Summary localhost:9200/$index/_count?pretty
Indice Stats localhost:9200/$index/_stats?pretty
Get all docs localhost:9200/$index/_search?pretty
Full Text Search localhost:9200/$index/_search?q=50
Search By field localhost:9200/$index/_search?q=f1:50
Search By 2 fields localhost:9200/$index/_search?q="f1:v1&f2=v2"

More Reading: How To Install ELK Stack With Any Given Version.

Footnotes

1https://mobz.github.io/elasticsearch-head/

2https://www.phpmyadmin.net

3www.elastic.co/guide/en/elasticsearch/reference/current/cluster-stats.html

4www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html

5www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html

6www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html

7www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

8www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html

9https://github.com/taskrabbit/elasticsearch-dump

10www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

Check out the Exaptive data application Studio. Technology agnostic. No glue code. Use what you know and rely on the community for what you don't. Try the community version.

Topics:
devops ,elasticsearch ,guide

Published at DZone with permission of Denny Zhang, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}