Zoomdata Microservices and the Web Application

As we said in our introductory post, we’re going to do several posts about Zoomdata microservices. This one covers a brief overview of microservices and the Zoomdata web application.

Zoomdata Microservices, In General

The Zoomdata platform is architected as a set of loosely-coupled Java microservices. Unlike traditional BI, which is deployed as a monolithic application (or possibly entwined in an old-school enterprise service bus), a microservices architecture allows for:

• Faster deployment of new functionality.
• Optimal resource management.
• Faster recovery in some failure scenarios.
• Greater deployment flexibility.

To optimize memory allocation and runtime attributes, each Zoomdata microservice runs in its own Java Virtual Machine (JVM). Running independently means that a failure in one service doesn’t take down all of Zoomdata, and recovery is faster because only the failed service needs to restart. Deploying new functionality, such as an updated or custom Smart Data Connector, is also faster because it occurs by auto-discovery, without requiring a server restart.

(Want to learn more? This DZone article, Why Use Microservices? is pretty good.)

Zoomdata Microservices: Some Specifics

Zoomdata microservices consist of an HTML5 web application front-end, a query engine, independent Smart Data Connectors for traditional and modern data sources, a stream writer, and complementary services such as a scheduler and screenshot service. Separately, Zoomdata centralizes its metadata store in a relational database and includes an internal messaging queue. Communication protocols include websockets for real-time, bidirectional communication (websockets are important, keep reading!), and HTTP/S.

The Zoomdata Web Application

Zoomdata provides a single web user interface that adapts to the needs of administrators, dashboard developers, analysts, and casual users. There is absolutely no client-side installation to manage. The web application is compatible with modern browsers that support HTML5 and the websocket communication protocol.

The web browser (or any application that connects to Zoomdata with our JavaScript client) is responsible for establishing a two-way websocket communication channel between itself and the Zoomdata query engine. The web browser keeps the websocket connection alive and automatically reconnects to the query engine in case of a network hiccup. Because websockets are “live” two-way connections, data can stream to the dashboard while users issue requests such as creating new charts, changing metrics or groupings, zooming down to lower level detail, and filtering.

The Zoomdata web application delivers end-user functionality such as:

• Data Sharpening™ to stream estimated results when working with very large data sets.
• Timebar slider to simplify time-based filtering.
• Data DVR to pause, rewind, play, and fast forward data streams.
• Live Mode for visualizing near real-time data streams.
• Search box and facets to maximize performance when connecting to search-enabled data platforms such as Apache Solr and Elasticsearch.

A Non-Blocking Data Exploration Experience

There’s more and better BI! The web application offers a “non-blocking” data exploration experience. There are two ways to understand the non-blocking UI and they’re both pretty big deals.

One is that data loads independently in each data visualization, so viewing a complex or dense dashboard is not slowed down by the longest running query.

The second is that, while data rolls in, users can filter, sort, drill down, reformat, change chart types, change and add metrics and attributes, and so on. The ability to change and interact with charts while data loads is significant because it allows users to engage in speed-of-thought analysis against massive and live data sources. When a user takes their data exploration in a different direction, Zoomdata efficiently cancels previously running queries to conserve computing and network resources. For the inquisitive analyst, Zoomdata is a huge time saver and a pleasure to work with.

The value of the Zoomdata non-blocking user experience goes beyond comfort and convenience. Numerous studies report that the human attention span for most tasks is less than 10 seconds (and many suggest the limit is 3 seconds), so it’s important that the user is not left staring at a blank screen or a frustratingly mesmerizing spinner. The best and most effective data exploration happens when users can rapidly and freely explore the data. Traditional BI is not architected for such fluid interactions.

Hopefully, you’re starting to get a feel for why Zoomdata invested so much effort into creating a brand new, modern BI architecture. In our next post, we’ll look at the Zoomdata query engine microservice and push-down processing.