There’s nothing like starting a new year with new product-related announcements! In this case, I’m happy to inform our users of a new Search API that will enable them to safely and securely query the data that they are shipping to Logz.io using the Elasticsearch Search API DSL.

This API replaces the older Query API, which was much more limited in scope and capabilities and will be deprecated in a few months.

Why would you use the API to query Elasticsearch instead of Kibana? Well, you could, for example, integrate with the API to create some kind of baseline for monitoring new services. As soon as a new service or host registers, you could use the API to match which of them are logging into Elasticsearch.

Below are some general guidelines on the API along with explanations of existing limitations and one simple example of how to use it.

Licensing and Limitations

For now, only our enterprise users will have access to the Search API. Likewise, we are currently limiting the number of search results to 10,000 (with paging).

Queries can only be performed against data shipped over the prior two days (the current day and the one preceding it).

API Tokens

To use the Search API, you will first need to retrieve an API token. The token can be generated on the User Tokens page in your account’s user settings:

Using the API

Once you have the token, you can begin querying your data.

First, build your query in JSON format in a file. You can refer to the Elasticsearch Search API documentation for guidance on how to build the query.

For example, I’m going to query my account for a simple string and limit the search to one response:

{
 
   "from" : 0, "size" : 1,
    "query": {
        "query_string": {
            "query": "canada"
        }
    }
 }

Then, build your API as follows (replace API-TOKEN with your token and QUERY with the path to your query file):

curl -XPOST 'https://api.logz.io/v1/search' --header "X-USER-TOKEN: <API-TOKEN>" --header "Content-Type: application/json" --data-binary @<QUERY>

And the response:

{  
  "hits":{  
     "total":984,
     "max_score":5.7908382,
     "hits":[  
        {  
"_index":"logz-xmywsrkbovjddxljvcahcxsrbifbccrh-161227_v2",
           "_type":"MyFirewall",
           "_id":"AVk9lI_BMfCTV-I4vosg",
           "_score":5.7908382,
           "_source":{  
              "geoip":{  
                 "timezone":"America/Toronto",
                 "ip":"204.138.58.86",
                 "latitude":43.5924,
                 "continent_code":"NA",
                 "city_name":"Toronto",
                 "country_code2":"CA",
                 "country_name":"Canada",
                 "country_code3":"CAN",
                 "region_name":"ON",
                 "location":[  
                    -79.7611,
                    43.5924
                 ],
                 "real_region_name":"Ontario",
                 "postal_code":"L5N",
                 "longitude":-79.7611
              },
 
"message":"<2016-12-27T20:31:13Z><myfw-dc2>:src_ip=204.138.58.86,sport=4209,dst_ip=87.151.247.146,dport=443",
              "type":"MyFirewall",
              "dst_ip":"87.151.247.146",
              "tags":[  
                 "_logz_http_bulk_json_8070",
                 "geoip"
              ],
              "src_ip":"204.138.58.86",
              "dport":443,
              "hostname":"myfw-dc2",
              "@timestamp":"2016-12-27T20:31:13.000+00:00",
              "sport":4209
           }
        }
     ]
  }

Endnotes

• Be sure to use HTTPS in the request URL. An HTTP request will result in a “403” response.
• The request body needs to comply with the Elasticsearch Search API. There are, however, some limitations that we highly recommend you review before you begin using the API.
• More examples and documentation are available here.

We’d love to get your feedback. If you encounter any bugs, please open an issue on GitHub or send us your comments to: support (at) logz.io.

Happy querying!