Programs have generated logs for as long as they have existed — but with the advent of modern server and application software, logging has become a part of the IT management and monitoring process. Servers and applications can generate log data on a variety of processes, from simple announcements about healthy checks to detailed information on running processes.
It’d be interesting to look at these tools from a developer’s point of view. In this article, we shall review five of the most popular log management cloud tools of today. They are listed in a random order, not in the order of performance or capabilities.
In LogRhythm, log management and event management are distinct processes. That means purchasing and managing two separate tools. LogRhythm combines both into a single centralized platform to streamline not only log management but log analysis, event management, and reporting as well. However, LogRhythm also provides an Agent that can be installed on a network server. The Agent collects computer’s log data and sends it to the Log Manager.
Intelligent search in real-time.
Collects data from all log sources, including applications and databases (~700 log sources).
Real-time monitoring and flexible, role-based alerts.
One-click correlation from any search.
The console enables users to quickly correlate, search and pivot through their data rapidly.
Big initial cost.
Not transparent user guide and documentation.
LogPacker provides you with two solutions: Standalone and Cloud. The Standalone version illustrates the network with Agents and Servers in the Cluster. There can be any amount of servers in this network. Agents can be installed or integrated into a various amount of platforms such as Unix, Windows, Android, iOs, WP, and even into the website’s JS, while the Server accepts logs from all Agents and saves them into any storage. The main advantage is that LogPacker works just from the box and can find and send to the Cluster all possible logs on the server, grouped and aggregated.
Cloud version (SaaS) provides a full web interface with dashboards and search functions and has disk-based limits.
All LogPacker services are written in Go and created for high performance. Agent instance usually spends around 30-40 MB of memory on default server installation.
Built-in support for more than 100 log sources types.
Multiple storage providers.
Alerting and reporting system by Email, Slack or SMS.
Easy installation from packages.
Wide platforms support (Unix, Windows, Mobile, JS).
REST API to build a custom solution based on saved data.
Events aggregation and security.
Disk-based Cloud dashboard.
The standalone version has no built-in web interface.
The free version has a limit up to five servers in Cluster.
There are few products that Splunk provides: Enterprise, Light, Cloud, and Hunk. Splunk Enterprise helps you to gain valuable Operational Intelligence from your machine-generated data. With a full range of powerful search, visualization, and pre-packaged content for use-cases, any user can quickly discover and share insights. Splunk also has built-in reporting capabilities with advanced charts and dashboards and a pivot interface to generating visual reports with drag-and-drop ease.
Built-in alerting and reporting.
Configurable charts and dashboards.
Scale from a single server to multiple data centers.
Real-time search, analysis, and visualization.
Not easy to setup and add new sources. Each source must be added manually.
Limit of 500MB/day is not enough to use it for free, whereas 1GB/day will cost $2,700 per year.
Scalyr is a server monitoring tool built by ex-Google engineers. It brings together log data, system metrics, website monitoring, and alerting. It just doesn’t scale at all — you must have that data in one place. However, the interesting point here is that they’re also trying to bring in the monitoring components. Other log management tools focus on the logs and gloss over the other parts of application monitoring.
Easy set-up Agent or API.
Security and reliability.
Import logs from Heroku, Amazon RDS, or Amazon CloudTrail.
Doesn’t have a free version.
No Cloud solution.
Logentries automatically collects and centralizes all of your log data in any format into one secure location where you can search, aggregate, and visualize log data to get answers to your questions, in seconds. Logentries offers options for both agent and agentless collection of logs. When problems occur, Logentries provides an aggregated live tail view to see what is happening across your logs in real-time. As your environment dynamically scales, new instances can be easily configured to send all log data in real-time to Logentries.
Works with multiple PaaS and IaaS.
Aggregated live tail search.
Custom tags of logs.
SQL-Like Query Language (LEQL) for searching.
Supports a diverse set of programming languages.
Free up to 5 GB.
Manual installation and manual log sources management.
Can't track the source of errors in third-party libraries.
There's a limit of 100 logs per server.
Insufficiently secure web client logger.
Logscape is an Enterprise ready big-data analytics tool built for time-series data visualization of machine data. Whether it is data generated by your systems or extracted from an external source, any form of machine generated data can easily be searched, filtered, and placed onto interactive dashboards.
5 GB daily data limit.
LDAP for user and role-based access control.
Custom dashboards, workspaces, and reports.
Indexers in place processing on remote files.
Scale on existing or dedicated infrastructure.
Supports only data from network input or syslog.
Loggly is a robust log analyzer, focusing on simplicity and ease of use for a DevOps audience. It’s targeted for developers and DevOps – making it less enterprise-focused. When to use it: Primary use cases are for troubleshooting and customer support scenarios. It’s a good tool for a DevOps team.
Unlimited custom dashboards based on any search.
Full-system RESTful API to integrate with other applications.
Text-based logs from any source.
Adaptable interface with multiple views, pages, and workspaces.
Free lite version.
Not transparent configuration and sources configuration.
Not good in cloud infrastructures.
All these tools and services are different. Some solutions are a component of a larger SIEM platform, providing even more sophisticated analytics capabilities, long-term storage, and data encryption, while others can be used with any SIEM solution or even as stand-alone appliances. Depending on your business, you can choose a suitable log management system for your purposes.