DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • A Data-Driven Approach to Application Modernization
  • What Is React? A Complete Guide
  • Introduction To Git
  • Which Is Better for IoT: Azure RTOS or FreeRTOS?

Trending

  • A Data-Driven Approach to Application Modernization
  • What Is React? A Complete Guide
  • Introduction To Git
  • Which Is Better for IoT: Azure RTOS or FreeRTOS?
  1. DZone
  2. Data Engineering
  3. Big Data
  4. Implement a Counter Table in Elasticsearch

Implement a Counter Table in Elasticsearch

Dinesh Agrawal user avatar by
Dinesh Agrawal
·
Updated Dec. 13, 19 · Tutorial
Like (3)
Save
Tweet
Share
16.96K Views

Join the DZone community and get the full member experience.

Join For Free

moving-scale

In your product, you ought to know in-depth information concerning feature usage. With that, you also should be interested in knowing when exactly a particular feature was touched last.In this article, we're going to solve the problem of getting click counts on different entities/pages of your website to find out when a particular page last touched.

To solve this problem, two main solutions come to mind:

  1. Cassandra counter tables — Cassandra supports counter tables, but you can't mix a timestamp column (which cannot be a primary column) with a counter table. And a counter table is allowed only with primary key columns and counter columns. So, with this limitation, we can not proceed with Cassandra counter tables.

  2. Elasticsearch — There is no in-built support for counter tables in Elasticsearch, but we can take advantage of its script functionality to implement them. 

You may also like: Data Modelling: Counter Table.

Solution

Let's assume a user has just seen a video with a unique ID of 88d19b07-86d2-471d-be33-ed10eef5d38e on our website.

POST counter/count/VIDEO_VIEW_88d19b07-86d2-471d-be33-ed10eef5d38e/_update
{
  "script": {
    "source": "ctx._source.counter += params.count;ctx._source.recorded_timestamp = params.recorded_timestamp",
    "lang": "painless",
    "params": {
      "count": 1,
      "recorded_timestamp": "2019-11-19T22:15:30Z"
    }
  },
  "upsert": {
    "entity": "VIDEO",
    "counter": 1,
    "action":"VIEW",
    "uuid":"88d19b07-86d2-471d-be33-ed10eef5d38e",
    "recorded_timestamp": "2019-11-19T22:15:30Z"
  }
}


We have an Elasticsearch index with an index name, "counter," and type, "count." With this information, we're trying to index a document with ID ${entity}_${action}_${UUID} (88d19b07-86d2-471d-be33-ed10eef5d38e in this example.

In case this video is being watched for the first time, the block given the "upsert" key will be executed and will initiate the document with a counter value of 1.

The next time, whenever a user watches this video, the same indexing code will be fired and a document with the given ID will be found with index name, "counter," and type, "count." So, following the script, the counter will be increased by 1 (given under params key), and the updated date will be updated with data given under the params key. Other keys will remain untouched.

Now I have data available that tells me:

  • Which entity is used widely and which particular item of the entity is used widely. In this case, which particular video is most often watched.

  • Which action of which entity is not in use

    • By just querying in Elasticsearch with "give me all the entities where recorded_timestamp  is older than 30 days."

Further Reading

  • MySQL Database Table Data Purge/Removal Using MySQL Event Scheduler.
  • Counting Distinct Users in Real-Time With Redis Using Low Memory.
  • Tuning InnoDB Primary Keys.
Database Elasticsearch

Opinions expressed by DZone contributors are their own.

Trending

  • A Data-Driven Approach to Application Modernization
  • What Is React? A Complete Guide
  • Introduction To Git
  • Which Is Better for IoT: Azure RTOS or FreeRTOS?

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: