Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

What's New in SmartSense 1.3

DZone's Guide to

What's New in SmartSense 1.3

SmartSense upgrade for monitoring, consolidating, and reporting on hadoop clusters.

· Big Data Zone
Free Resource

See how the beta release of Kubernetes on DC/OS 1.10 delivers the most robust platform for building & operating data-intensive, containerized apps. Register now for tech preview.

This April, Hortonworks launched a multi-phase initiative to streamline Apache Hadoop operations, and the 1.3 release of SmartSense marks the delivery of the second phase of that initiative, and that is to provide Consolidated Cluster Activity Reporting.

Hortonworks launched SmartSense in 2015 to help customers quickly collect cluster configuration, metrics, and logs to proactively detect issues, and expedite support case resolution. By combining the analytical capabilities of SmartSense, with the interactive reporting capabilities of Apache Zeppelin, SmartSense 1.3 now provides consolidating utilization data reporting across core HDP components in a single dynamic UI available on-premise.

SmartSense mines, consolidates, and stores YARN, MapReduce, Tez, and HDFS utilization data and provides a fully managed Apache Zeppelin instance with pre-built notebooks to analyze, query, and visualize that data. Out of the box notebooks can be extended, and custom notebooks can be created to support customer-specific reporting needs.

This solution allows customers to easily and quickly answer questions like:

  • What are the top ten most active users?
  • Which queues consume the most CPU resources?
  • Which jobs take the longest and consume the most memory on my cluster?
  • Which jobs read and write the most HDFS data?
  • Who’s creating all my small files?
  • How much should I charge my users for their resource utilization?

We believe this capability will be a game changer for our customers and give them the data they need to make better business decisions, and more effectively understand, troubleshoot and manage their cluster.

HDFS Dashboard

6Z-HDFS-Dashboard

MapReduce/Tez Dashboard

5Z-MapReduce_Tez-Dashboard

YARN Dashboard

4Z-YARN-Dashboard

Simplified Bundle Upload Options (SFTP and HTTPS)

For many SmartSense users, outbound internet access from the HST server is not readily available. For those users, the HST Gateway has been created to simplify the process of uploading bundles of diagnostic information to Hortonworks Support. In SmartSense 1.3, users have more upload options with SFTP, HTTPS and authenticated HTTPS proxy support.  

On top of that, we’ve embedded an instance of the SmartSense Gateway into the HST Server, so for clusters that have outbound internet access, no additional Gateway setup is required. 

Expanded Anonymization and Redaction Capabilities

Giving customers the flexibility to pre-process cluster diagnostic information before it is encrypted and sent to Hortonworks has always been a priority in SmartSense. All bundles are encrypted and anonymized by default and in SmartSense 1.3 we’ve expanded the number of out-of-the-box Anonymization rules, giving customer more flexibility with how cluster diagnostic information is secured.

Getting Started

We hope you’re as excited about SmartSense 1.3.0 as we are, and here are a few links to get started:

New Mesosphere DC/OS 1.10: Production-proven reliability, security & scalability for fast-data, modern apps. Register now for a live demo.

Topics:
hortonworks ,hadoop ,hdfs ,monitoring

Published at DZone with permission of Paul Codding, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}