Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Examination of ManageEngine Applications Manager Through a DevOps Lens

DZone's Guide to

Examination of ManageEngine Applications Manager Through a DevOps Lens

Applications Manager provides a way to provide proactive feedback to developers in a pre-production environment. Read about its pros and cons.

· DevOps Zone
Free Resource

Download the blueprint that can take a company of any maturity level all the way up to enterprise-scale continuous delivery using a combination of Automic Release Automation, Automic’s 20+ years of business automation experience, and the proven tools and practices the company is already leveraging.

In a world where batch sizes are shrinking and the pace of releases is increasing, having automated policy-based control in the pre-production environment proactively reduces the chances of end-user, performance, and operational issues in production.

The ultimate goal is to help enterprises reduce the risk of end users impacting production deployments by monitoring and looping feedback earlier in the application lifecycle. Developers should now be able to immediately see the potential impact of code changes, remediate and fix poor app behavior earlier in the lifecycle. This greatly reduces time spent course-correcting in production and frees up more time for innovation.

This practice of providing early feedback on application behavior enhances product quality and customer experience and give Operations confidence in the Release and environments and deployments fidelity.

Without a doubt, visibility is one of the most important things needed to help development, IT operations and DevOps teams ensure optimal application performance before an app goes into production. Developers must work with IT operations to get maximum performance, but this joint effort requires a single version of truth—a shared view into how an application performs and what the impact is of each element of the application stack. This visibility during development allows developers to see the performance impact of their changes in almost real-time, plus it helps eliminate finger-pointing when something goes wrong. There are various methods, technologies, and tools available to get this view of the code. Two of the most common methods are Log Analysis and Applications Performance Monitoring (APM).

For this article, we will focus on APM as Applications Manager is primarily an APM tool.

Applications Performance Management (APM)

Left shift APM for top application performance: Software is core to business success meaning that a fast app release cadence is a must for all businesses. The result is that the boundaries between what is a test, pre-production and production environment are starting to become blurred. This means that APM solutions should be used earlier in the software development lifecycle to ensure top application performance. Features such as code level analysis and end-user experience monitoring should be utilized in test environments to ensure that performance gremlins are caught before they reach production and real customers.

When operations share real-user monitoring insights with development, apps can be future-proofed to conquer the stresses of production deployment. The successful use of APM in pre-production hinges on whether it is Easy, Proactive, Intelligent and Collaborative or E.P.I.C. This easily remembered checklist helps IT Ops validate whether their APM methods are DevOps ready.

Based on the APM product reviews from real users on IT Central Station, here are the top three ways people use APM before rollout:

  1. Troubleshooting of slow performance in test environments,

  2. Debugging with deep dive diagnostics for collaboration with dev teams, and

  3. Capacity planning.

Abstract

There will always be an open, critical debate over the best APM tool to use before rollout to deliver superior application performance. There are many tools available in market, mostly commercial. We selected Applications Manager as the basic version is free and it provides a decent number of features for a team to use it in pre-production environment to do dynamic code analysis.

ManageEngine Applications Manager is an enterprise-ready, user-friendly, and cost-effective application management platform that gives enterprises the capabilities to track and manage their mission-critical applications and data centers effectively in a pre-production environment.

In this paper, we present our study, the outcome of an evaluation of different features of Applications Manager against The 2017 APM Predictions List.

The key parameters on which we measured Applications Manager are intelligent alerting, code level profiling, troubleshooting of slow transactions and performance, slow database query analysis, log analysis, capacity planning, and support.

Proposed Solution

In order to run our tests and evaluations, we installed Applications Manager on a Windows machine and configured it to monitor a Java application deployed on a cluster of servers running on Linux virtual machines.

We will attempt to subjectively evaluate ManageEngine Applications Manager on the above parameters and present our findings. Please note that we have not done a comparative evaluation of ManageEngine Applications Manager with other APM tools from its competitors, like New Relic and Appdynamics, on quantified parameters, but qualitatively on those parameters, which we feel are "must-have" features where advantage of one over the other will make a significant difference to how we manage our applications and the underlying infrastructures.

One of our objectives is to do an analysis that can help a DevOps team to do an initial level assessment on whether they should give a trial to Applications Manager. In addition, based on this analysis, practitioners will be able to identify some use cases from their work where Applications Manager can be a good/better tool for providing early feedback to developers.

We have analyzed Applications Manager against the 2017 Application Performance Management Predictions by APMdigest and have identified the key predictions which this tool was able to fulfill successfully.

Intelligent Alerting

The event of actively receiving alerts or notifications about application performance degradation before an actual issue or outage happens is termed intelligent alerting. Key features that we leveraged in pre-production include

  1. Dynamic alerting when a preset threshold is approached.

  2.  Menu options like “select time,” “select monitor type,” and “select monitor group,” associated with the “configure alarm” dashboard of Applications Manager.

  3. Proactive alerts on a wide array of error conditions and faults helped developers diagnose and repair Java application performance related issues faster.

  4. The alert notification feature to keep the concerned teams like QA and developers informed through the various supportive services like SMS, Email, Slack, and HipChat.

  5. It helps track alarms generated under a specific monitor group in a particular timeframe and assign it to a proper set of technicians. If the designated technician does not take proper action within the given SLA, then there is a facility to auto-escalate the alert on its own. 

Slow Database Query Analysis

It’s very easy to make any SQL query perform quickly in a dev environment, especially on a developer’s desktop. It’s normally in the test environment where most application scalability and performance limits are found because of load tests and larger data sets.

 In our case, data volume and user transaction concurrency were major factors to impact query performance, both of which typically don’t exist in dev environments.

Applications Manager was able to break down every web transaction to the PostgreSQL database level and track each and every single query like a “SELECT” or “INSERT” statement fired at the database level and their corresponding response times, which in turn helped developers to fine-tune the discrepancies found at the DB level, if any. The team of developers was able to identify just how fast or slow SQL queries actually are, find out about slow database calls, long running queries, and overall performance of the database.

It was a piece of cake to track down the slow response time to the PostgreSQL level and inspect all the queries executed by a web transaction in a test environment.

Applications Diagnostics, Code Level Profiling

The code profiler feature of Applications Manager has the ability which is demonstrated well in the pre-production environment: the benefit of pinpointing erroneous JSP and servlets and slow response times to see the performance impact of specific SQL statements in the application stack.

At present, Applications Manager's capability of code-level diagnostics is limited to Java, .NET, and Ruby on Rails applications. Though the number of applications for which this feature is available might seem low, the amount and level of information about Java application code it yields is worth applause. This way, it gives developers an earlier advantage, in the pre-production environment itself, of drilling down to monitor the JVM's performance and availability, notifies them of health and performance issues of JVM, Java memory allocations, and garbage collection, and early detection of memory leaks. It helps understand how often SQL statements and web service calls are made, and tracks web requests and those transactions to understand the performance of transactions within Java code. Using Applications Manager, we cannot track key methods in the Java code down to each individual line of code.

Log Analysis

We used the capabilities of Applications Manager very well in a QA/STG/SIT environment to enable the various types of applications and File System Monitoring to monitor the changes in the selected files and directories, to provide end-to-end logging to understand our systems’ performance from an end user’s point of view, taking things like network latency, database transaction delays, and page loading times into account. This kind of visibility, when provided to developers, helps them to better deliver a seamless user experience.

We successfully configured Applications Manager to create Event Log rules for generating alarms when a certain error occurred in the system logs, a file crossed a predefined size limit, or to check multiple occurrences of a string containing an exception.

This type of log level monitoring in pre-production environment was used by the developers to further draw conclusions around the applications behavior and fine tune the code as per the results provided.

Right now, the file content level monitoring (wherein a developer can give multiple strings and check for their occurrence) is not available for an application running on Windows servers.

Troubleshooting of Slow Transactions and Performance in the Test Environment

Applications Manager encouraged developers to proactively monitor the availability, response times, and page loading time of the recorded web transactions and the individual URLs in the test environment to troubleshoot issues faster and earlier in the software delivery pipeline, to further enhance customer experience.

Using Applications Manager, developers were able to detect when the response time of Java application was slow, detect slow transactions and find out why they are slow. They then identify each transaction that can cause issues with customer satisfaction, employee productivity, revenue growth, or brand recognition that way locating the exact invocation pattern and details of transactions across Java application environment at an earlier stage.

Capacity Planning

Here capacity planning has been used in the context of estimating in advance the compute resources, computer hardware, software, and connection infrastructure resources in the test environment itself that will be needed at a future time.

Applications Manager has a set record of capacity planning for various Oracle and IBM applications and Unix and Solaris based systems, but it could not be proved useful in the case of Java applications.

Root Cause Analysis

Applications Manager is very good at identifying situations sprawling across multiple event streams, combining the individual events together into the "big picture."  Applications Manager, when used in pre-production for a Java application, tied the seemingly unrelated events together into a form that told a story, what might have happened, and what triggered a hazardous event that caused an application to behave in an unexpected way.

The real-time visibility RCA feature for Java applications running in a pre-production environment enabled easy identification of transaction bottlenecks and performance hiccups, thus saving the time spent on analyzing a problem.

It helped achieve the developer’s goals of hitting the actual pain point and being pointed in the right direction in the application and associated areas before the end user is affected. The ability to restart a Java service associated with a malfunctioning process and how quickly Applications Manager accomplishes this task is what really stands out.

Support

When we shift the actual context of an APM tool targeted for production to pre-production, it is really important to consider

  1. How easy and quick it is to get support from the APM team in case developers, DevOps, or QA faces any challenges or difficulty with the features and setup.

  2. The level at which it can support the huge set of application programming languages and underlying infrastructure and services available on the market.

As per our analysis, Applications Manager has been providing timely support through emails, calls, and personalized meetings whenever developers do not understand how to use a specific feature to get the maximum benefit out of it. They have very good training material on their website and troubleshooting tips for any issue with the tooling.

Not just on the tool functionality support side, Applications Manager also has out-of-the-box support for over 50 different types of applications, databases, servers, and services spanning physical, virtual, and cloud infrastructure, widening the scope of its use by a majority of developer communities.

Summary

The table below gives a brief insight into how well Applications Manager stands out and justifies its various promises/features against our POC, where we installed Applications Manager on a Windows 2008 R2 server platform and then started monitoring the performance of our Java application stack deployed on Centos 7 with Tomcat 7 as the application server and PostgreSQL as  the database.

Features

Advantages

What Can Be Improved

Ease of Use

  • It is easy to install and configure with appealing UI.
  • The entire tool and associated PostgreSQL database setup and configuration took less than an hour.

The tool does not currently support a SAAS based deployment.

Alert

  • Was able to proactively monitor Java application with anomaly detection.
  • We were able to receive application performance degradation alerts like memory leak, longer running components timely and trigger the actions associated.

They can automate and expedite fixing the common performance problems introducing runbooks.

DB Query Analysis

  • We were able to get the actual execution time of MySQL and PostgreSQL database queries with the help of database query monitor


Code Level Profiling

  • It could not live up to the expectations to drill down and see the performance impact of key methods in Java code.
  • Tracks Java memory allocations and garbage collection to help with memory leaks.

Little to no support for tracing each method call and line by line information about application code.

Log Analysis

  • Able to provide file content level monitoring wherein it can identify some error patterns & strings for Java applications running on Centos 7 and Redhat 7 systems.


  • It is not designed to parse over terabytes of log data like log stash, greylog, splunk.
  • Cannot analyze logs.


Slow Transaction Tracing

  • We were able to detect slow java application transactions as well as background transactions and find out why they are slow.


Capacity Planning & Scalability


  • Should provide capacity planning for web based applications.


Root Cause Analysis


  • Detection of the source of performance degradation lacks machine learning.


Support

  • Support for emerging technologies.
  • Provides support through Email, Phone, WebEx, On-premise.
  • The out-of-box support for a Java non-web/web based applications.


  • A long time to market for new features and functions.
  • Feature to import/export the entire set of monitors is not available.


Future Predictions for APM

I see that APM is going to play a pivotal role in providing early feedback in pre-production while leveraging Advanced IT Analytics.

I see the future challenges for Applications Manager to be around IoT. ManageEngine will need to introduce Artificial Intelligence based on machine learning algorithms to help organizations to cope with the incredible complexity of the technology. With this, they will be able to better predict performance issues and their impact. When issues strike, solutions could execute self-healing remediation actions and the best recommendations to restore IT services within the shortest timeframe.

Results/Conclusion

Our analysis concludes that currently, Applications Manager would be a very effective tool for managing your preproduction environments. Setting up Applications Manager is not complex and it has been designed in a way so that configuring monitors on it is very simple. Compared to New Relic & Appdynamics, the time needed to get started with Applications Manager may be a fraction of the time it takes to understand the architecture, setup, hosting, dependency management, and management of their scripts.

With early monitoring, Applications Manager improves performance and product quality for application users throughout the DevOps pipelines and environments. Applications Manager is able to enable organizations to uncover end user, performance, and operational cost impacting issues in cloud environments and enterprise software much earlier in the delivery pipeline.

This tool allows users to detect performance issues earlier in the application lifecycle, have faster remediation and recovery, and prevent outages with its root cause analysis and self-healing techniques.

Applications Manager is an application performance management solution that accelerates innovation, time-to-market, and software quality through fact-based decisions. Users can identify, for example, whether code changes cause any performance, scalability, or architectural regressions. Developers can then stop the build early and allow only quality code in production, resulting in the production of better software faster.

It has redefined how you monitor today's digital ecosystems, full stack and completely automated; it's a one of a kind of solution that provides answers, not just data, based on deep insight into every transaction, across every application.

Appendix A – Scenarios

A year back, when it came to tool selection, the HCL DevOps team prioritized Applications Manager over Appdynamics and New Relic at a time when others were trying to establish whether investing into Applications Manager would be worth it. Our observations and insights in this article have come on the basis of using Applications Manager over the past year in a number of operational scenarios.

Using Applications Manager for Java Application Performance Monitoring on platforms like Azure and VMware right from the monitoring of application response times to track background transactions detects slow transactions with transaction tracing.

Applications Manager can monitor Java based web applications and their components, different platforms like Windows, Linux, and application/web/database servers, specifically for Tomcat and PostgreSQL server.

Appendix B – Options

  • Monitor the entire Java application ecosystem - end-user, applications, and underlying infrastructure components such as application servers, databases, middleware components, web servers, virtual systems on Azure cloud VMs.
  • Get the exact invocation pattern and details of transactions across a Java application environment. Detect slow transactions and find out why they are slow.
  • Track and inspect all SQL queries sent by a Java application. Identify slow database calls, long-running queries, and overall performance of the database.
  • Apply Java runtime monitoring capabilities to manage  Java applications and platform components, ike checking JVM performance, monitoring the threads of the application, garbage collection, heap memory, and keeping performance track of multi-threaded Java applications.

Download the ‘Practical Blueprint to Continuous Delivery’ to learn how Automic Release Automation can help you begin or continue your company’s digital transformation.

Topics:
apm ,performance management ,devops ,manageengine

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}