Log management is crucial in identifying performance, and implementing performance enhancements. In the modern world, there’s constant logging: devices, components, and systems are always leaving logs with pertinent information to the health of the whole. This log data is a veritable treasure trove of important metrics. I set about recently to determine how logging, and log management impacts performance. To my great pleasure, I had the honor of chatting with experts Jon Gifford, Founder and CSO at Loggly, and Sven Dummer, Sr. Director of Product Marketing at Loggly.
“This log data is really the only common denominator, the only data format common across all different types of applications, devices, services” – Sven Dummer, Loggly Sr. Director Product Marketing
Thus, we rely on log data for insight into rectifying issues and boosting performance. How often have we heard someone complain “it’s slow” about a website, database, etc.? This is one of the most frustrating pieces of feedback because while it’s really general and vague, often it’s referencing some sort of problem. Enter log data.
However, merely having access to log data isn’t terribly helpful. We need to make sense of this data in some capacity. There’s a recent phenomenon of Franken-monitoring, where we have too many tools and not enough actual, actionable data to allow for an accurate portrait of what’s going on. Similarly, it’s necessary to have analysis for this log data.
“You need to really think about how you’ve captured the log data of all these instances that disappear after they’ve already been used” – Sven Dummer
In the present day, we’ve got an increased reliance on virtual machines with the distributed systems we’ve fostered. Operating in a cloud-environment means there’s an elastic ecosystem, and all that log data isn’t necessarily being captured in full. The solution? Cloud-based log management is one way to aggregate log data, and ensure comprehensive coverage.
“When machines, or applications, or services become ephemeral and elastic, traditional methods of dealing with [log management] tend to fail” – Jon Gifford, Loggly Founder and CSO
With logging, there’s a lot of potential to uncover performance bottlenecks, thereby identifying, and occasionally even prevent, slowdowns. For instance, log data can help us to see correlations between various components, like the database and server. When a website appears slow, there are tons of possible culprits, from your network to the database or server, and the log data is key in exposing the root cause.
So what are the benefits of using log management? Well, logs ultimately help you to understand your system. It’s now possible to compare log data from multiple days, and determine whether anomalies are actually oddities. Using a cloud-based log management system adds the benefits of allowing easier access, and more comprehensive iterations of your log data. However, there are certain cases where keeping log data onsite is advantageous. Notably, the most common instance is where security, regulatory, or both, prevent companies from using cloud services for log data. If you want to keep everything in-house, and have that data available readily for manipulation, then it’s sensible to keep that data on hand. This operation is typically suited to larger organizations (think Google or Yahoo), with the expertise and resources. Usually, your engineers could be better used in other areas.
Have a story to share or thoughts on logging and performance? Comment below, or hit me up on Twitter. Watch out for my next Performance Fundamentals entry, and feel free to make suggestions if there’s a topic you’d like me to cover!
Special thanks to Loggly’s Jon Gifford and Sven Dummer for sharing their knowledge and expertise.