Database Monitoring

With today's mission-critical 24×7 applications, monitoring your databases can be just as important as the actual database or DBA itself. But how do you know if your company’s current approach to monitoring is the right one?

In my experience, companies tend to select a database monitoring solution based upon a DBA’s personal preference/experience often without a framework or decision process for evaluating which one optimally fits their business. Proactive, always-on and up-to-date monitoring is key to preventing database impacts to the user experience – but the cost for such monitoring should ideally align to your company’s tolerance to fund a solution.

When evaluating whether your monitoring solution is optimal for your business, it is important to consider the overall costs of managing and maintaining your own monitoring “system”.  These costs can include initial software license fees, ongoing software maintenance fees, development/customization expense, hardware costs, as well as ongoing administrative/tool-management resources.  If these costs are material to your budget, it might be worthwhile for you to evaluate new solutions available today that can significantly reduce these costs while also potentially improving your overall database monitoring experience.

Let’s first take a look at the monitoring software itself. Some only check for database state such as up/down or space full. Others check for events, thresholds, error codes, job completions and even complex scenarios with ETL, replication, failover, backups, query /report performance, and security. Some check periodically such as once daily, hourly or every five minutes. Others check in real-time with immediate notification of immediate or pending problems. Like any other IT application, all have a cost/benefit tradeoff to consider.

For manual monitoring and scripting, you need to hire a DBA and give them a pager and time to implement scripts and respond to events. Many IT shops can support this approach only if their databases are very stable, incur nominal growth, receive minimum changes, and can accept two hours or more of downtime.  This approach may appear economical, but organizations should be careful to consider added cost of the time dedicated to creating scripts and reacting to calls. I know several companies where the DBAs just manually run “health check” scripts every morning to see if there are any issues. Otherwise, they just wait for someone to call them, which is not efficient or proactive.

For mission-critical databases, a much more robust monitoring system is required. For the monitoring software, you will need to request the  budget, obtain hardware, negotiate software licenses, conduct training, plan, test, implement and finally, maintain the software all while reacting to database events.  Simply maintaining the monitoring application can become a significant expense over time due to upgrades, bug fixes, configurations, hardware maintenance, storage growth, and administrative expense. Ironically, you will also need to monitor the monitoring system itself in order to confirm that it is actually working.

I know of a Fortune 500 company that spent 18 months and over $1.3M on licenses, infrastructure and DBAs to implement Oracle Enterprise Manager w/Grid Control only to scrap the project and revert back to their 11-year old custom monitoring scripts due to delays, issues, and unplanned costs. These complex scripts, however, are difficult to understand and often fail. I also know another company where the database was not backed up for over three months because the monitoring software’s server had issues and was not running particular monitoring routines, namely the backup checks. This company had an unrecoverable data corruption, no backups to restore from and ended up spending three weeks manually rebuilding the data under the concerned eye of investors and the CEO.

But there might be a better way to approach monitoring. Think of home alarm systems. You could just lock all your doors and listen for sounds. You could install a basic system that sounds an alarm when a door or window opens. You can manually call 911 when a disturbance occurs or even install a system that automatically dials 911. You might even stay up all night with a shotgun sitting on the front porch.  But the winning approach in the market is none of the above. It is Home Alarm Monitoring Services such as ADT. These services provide better peach of mind and offer the equipment, installation, and live 24×7x365 connectivity & response for less.

Like the ADT analogy above, “Monitoring as a Service” is an intriguing monitoring option because it shifts the costs, complexities, and human resources needed to monitor and respond to issues with the database to a monitoring service provider. With Monitoring as a Service, you simply pay a monthly subscription fee and everything is included, even the actual DBAs to immediately respond and resolve issues if desired. There is no upfront license, hardware or installation costs. This approach saves you the time, money and overhead of implementing and maintaining monitoring yourself.  Its benefits are similar to the benefits experienced with “SaaS” or “Software as a Service”.

As an example, Datavail Delta offers Monitoring as a Service which includes a centralized monitoring engine, issue repository and messaging infrastructure along with lightweight database server agents that send secure HTTPS messages back to Datavail for immediate processing.  As a result, the monitoring is always on, always watching and always receiving database alerts that can then be immediately redirected back to your own DBAs or to Datavail’s own 24×7 live support DBA staff. 

While Monitoring as a Service represents a potentially interesting way to address many of the pitfalls of typical monitoring approaches, there is no “right” answer for every company. Your database monitoring solution should match your business and database environment requirements.  There is no one size fits all.