Intelligent Application Performance Management (APM) Platform for Big Data

Unravel Data has unveiled a new set of automated actions to improve Big Data operations and performance. The solution was designed with input from more than 100 enterprise customers and prospects to uncover their biggest Big Data challenges and to make DataOps more proactive and productive by automating problem discovery, root-cause analysis, and resolution across the entire Big Data stack, while improving ROI and time to value of Big Data investments.

DataOps is a set of practices and tools used by Big Data teams to increase velocity, reliability, and quality of data analytics. Done right, DataOps fosters a tight collaboration between data engineers/data scientists and IT operations, which in turn leads to faster time to market with Big Data apps that are high performing and reliable.

IDC’s Worldwide Semiannual Big Data and Analytics Spending Guide predicts Big Data and business analytics will grow to $203 billion by 2020. However, running applications on this influx of Big Data systems is becoming more complicated. Operations teams typically leverage multiple tools in order to manage their Big Data stack, such as Application Logs, Cloudera Manager / Ambari UI, MapR Control System, Job History UI, and Spark Web UI. This approach causes bottlenecks and creates delays for users who must turn to short-handed DataOps teams to gain complete diagnostic understanding of the issues, which inevitably results in lost money and time. Unravel mitigates these issues by providing a full-stack performance management software that monitors everything from applications down to infrastructure, all from one place.

“Unravel Data improves the reliability and performance of our Big Data applications and helps us identify bottlenecks and inefficiencies in our Spark, Hadoop, and Oozie workloads,” said Charlie Crocker, Director of Product Analytics at Autodesk. “Unravel also helps us understand how resources are being used on the cluster and forecasts our compute requirements, while enabling us to better scale our cloud infrastructure."

These Big Data challenges faced by the likes of Autodesk (and YP.com) present real-roadblocks for enterprise customers aiming to make their Big Data apps production-ready. Unravel 4.0 addresses these challenges by providing DataOps with an intelligent and automated full-stack APM platform that enhances Big Data operations, makes applications more reliable, and improves overall cluster utilization -- all from a single, connected screen.

“With 4.0, we’re able to address the skills gaps and technical challenges that our Fortune 2000 customers feel by simplifying and automating problem detection, diagnosis and resolution” said Kunal Agarwal, CEO and Co-founder at Unravel Data. “Not only does Unravel simplify DataOps, it helps our customers yield results from their investments by drastically improving the time-to-market for their Big Data apps. Organizations should be able to rely on their Big Data stack. Unravel is instilling the necessary confidence for moving apps from development to production, and guaranteeing that their mission critical apps run fast and error free.” 

Key features of Unravel 4.0 include:

• Runaway applications – detects and diagnoses apps over-allocating resources or under-utilizing containers; provides context of why and re-allocates optimal resources for these apps.

• SLA management – automatically detects and diagnoses why app is slow; recommends ways to speed up application and improve reliability.

• Configuration settings – alerts on bad configurations in the cluster; shows all apps that could run better with new settings, and applies new settings on demand.

• Service degradation/slow-down – shows which service is degraded, e.g., NameNode, MetaStore; provides context of why and which apps/users are affected; removes apps/users causing the issue.

• Storage utilization and caching – alerts users if storage is reaching capacity; shows which tables and files can be removed or cached to get more mileage out of current storage

“Big Data stacks are becoming increasingly complex, and that complexity seems to grow almost geometrically as new apps are added to the stack,” said George Gilbert, Lead Analyst, Big Data and Machine Learning at Wikibon. “In such an environment, running a root cause analysis, as one example, can become unfathomably more challenging and time-intensive. The Unravel platform is built on a recognition of these realities, providing an approach that effectively automates and speeds up problem resolution.”