Over a million developers have joined DZone.

What's New in SmartSense 1.3

DZone's Guide to

What's New in SmartSense 1.3

SmartSense upgrade for monitoring, consolidating, and reporting on hadoop clusters.

· Big Data Zone ·
Free Resource

Hortonworks Sandbox for HDP and HDF is your chance to get started on learning, developing, testing and trying out new features. Each download comes preconfigured with interactive tutorials, sample data and developments from the Apache community.

This April, Hortonworks launched a multi-phase initiative to streamline Apache Hadoop operations, and the 1.3 release of SmartSense marks the delivery of the second phase of that initiative, and that is to provide Consolidated Cluster Activity Reporting.

Hortonworks launched SmartSense in 2015 to help customers quickly collect cluster configuration, metrics, and logs to proactively detect issues, and expedite support case resolution. By combining the analytical capabilities of SmartSense, with the interactive reporting capabilities of Apache Zeppelin, SmartSense 1.3 now provides consolidating utilization data reporting across core HDP components in a single dynamic UI available on-premise.

SmartSense mines, consolidates, and stores YARN, MapReduce, Tez, and HDFS utilization data and provides a fully managed Apache Zeppelin instance with pre-built notebooks to analyze, query, and visualize that data. Out of the box notebooks can be extended, and custom notebooks can be created to support customer-specific reporting needs.

This solution allows customers to easily and quickly answer questions like:

  • What are the top ten most active users?
  • Which queues consume the most CPU resources?
  • Which jobs take the longest and consume the most memory on my cluster?
  • Which jobs read and write the most HDFS data?
  • Who’s creating all my small files?
  • How much should I charge my users for their resource utilization?

We believe this capability will be a game changer for our customers and give them the data they need to make better business decisions, and more effectively understand, troubleshoot and manage their cluster.

HDFS Dashboard


MapReduce/Tez Dashboard


YARN Dashboard


Simplified Bundle Upload Options (SFTP and HTTPS)

For many SmartSense users, outbound internet access from the HST server is not readily available. For those users, the HST Gateway has been created to simplify the process of uploading bundles of diagnostic information to Hortonworks Support. In SmartSense 1.3, users have more upload options with SFTP, HTTPS and authenticated HTTPS proxy support.  

On top of that, we’ve embedded an instance of the SmartSense Gateway into the HST Server, so for clusters that have outbound internet access, no additional Gateway setup is required. 

Expanded Anonymization and Redaction Capabilities

Giving customers the flexibility to pre-process cluster diagnostic information before it is encrypted and sent to Hortonworks has always been a priority in SmartSense. All bundles are encrypted and anonymized by default and in SmartSense 1.3 we’ve expanded the number of out-of-the-box Anonymization rules, giving customer more flexibility with how cluster diagnostic information is secured.

Getting Started

We hope you’re as excited about SmartSense 1.3.0 as we are, and here are a few links to get started:

Hortonworks Community Connection (HCC) is an online collaboration destination for developers, DevOps, customers and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.  Join the discussion.

hortonworks ,hadoop ,hdfs ,monitoring

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}