Software usage analytics enables you to collect, analyze, visualize, and integrate data on how users use software applications. Gartner predicts that by 2021, 75% of software providers will rely on insights from embedded software usage analytics to inform product management decisions and measure customer health.

Event-based metrics provide real-time insights from actual customer usage, highlighting which features are being adopted and which are being ignored, revealing problems in the UX or UI, helping to accelerate trial periods, improving user experience for particular OS or version builds, prioritizing feature enhancements, and tracking license compliance. And, when product usage analytics is combined with targeted in-application messaging, it provides direct communication with users to provide real-time user assistance, gather valuable feedback, or shape behavior. Product teams with these insights experience a sharp increase in development efficiency and shorter release iterations, resulting in optimized resource investment.

This Refcard gives an overview of software usage analytics, highlights specific metrics you can track, and compares software usage analytics to other developer tools including application lifecycle management and application planning solutions.

Traditional software vendors rely on conventional methods to collect and analyze information regarding customer usage of their product. These methods (questionnaires, surveys, support calls, customer interviews, observations, focus groups, and sales feedback) all come with their respective limitations, including differences in understanding and interpreting questions and incomplete responses. It may be difficult to create a concrete roadmap, including UX/UI direction, from responses that contain unreliable replies.

Embedded analytics gives you the information you need to help your product teams better follow Agile development principles. Direct insight into how users interact with your application allows you to plan new and old feature support with instant data from active users. This data helps you to optimize your Sprint planning to use developer resources only where they are needed. Software usage analytics is available for SaaS, desktop, and server applications alike.

While certain capabilities of application lifecycle management and application performance management methodologies provide tools to assist in the design, development, and testing of software applications, they miss out on direct metrics that dig deep to reveal how users are engaging with your applications, and fail to pinpoint when and why issues occur.

With APM, ALM, and software usage analytics methodologies in place, organizations can significantly improve the way they design, build, test, and adapt their software applications to best meet customer needs. Below is a summary of the methodologies and their key features.

Software usage analytics expose the depth of customer behavior through event-based user data, which can be used in addition to APM and ALM to build better applications.

In short, APM details the actual performance of features while ALM helps you roll out all of the changes you need, and helps you roll them back again, whereas software usage analytics expose the depth of customer behavior through event-based user data. Software usage analytics combined with APM and ALM helps you build the perfect app.

In addition to collecting data across different reports, software usage analytics gives the ability to identify types of users or segment by the different pieces of information across any report. For instance, you can track total feature usage, or you can drill down and analyze feature usage segmented by people running a particular OS or hardware, which makes the findings more interesting and actionable.

Version, Edition, and Build

By tracking how many users are running each version and build, you can plan and prioritize support activity and upgrades by region, user group, or support plan. Additionally, you can evaluate the need to sunset low-use editions.

Example: Software Usage Analytics may reveal that a large number of users are experiencing a bug for a specific feature with a recent build in Brazil.

By knowing how many users are running your application by platform, you can analyze the most critical OS metrics — version, platform, language, and architecture size and support features that work only on specific OS versions.

Example: Software usage analytics may reveal that all of the users experiencing a particular bug are running macOS Mojave version 10.14.

Hardware Architecture

Knowing which hardware your customers use to run your application allows you to replicate how your customers interact with the product, optimize for the device type down to the type of chips and graphics cards, and evaluate the performance of the software on the most used CPU core, memory, and graphics card configurations. This overarching view gives unprecedented visibility into the entire end-user experience while allowing you to understand environments running your software.

Example: Software usage analytics may reveal that few users have sufficient memory specs needed for an upgraded version and that it’s worth offering them extended support for their older version.

Geography

By knowing where your users are active, you can compare adoption rates across regions, analyze trends by region, country, or state, and utilize heat maps to compare churn rates across countries. Heat maps reveal how users with different attributes are distributed globally, highlighting a specific version or license type.

Example: Software usage analytics may reveal that trial users running version 3.0 on Windows in Germany have a low conversion rate.

License Key

Knowing the license keys of each respective user allows you to easily locate and delete user information (GDPR compliance), filter and view user activity based on license key status, and verify blacklisted, expired, or abused keys.

Example: Software usage analytics reveals which license keys have surpassed the install quota. Use in-application messaging to reach out to these users and offer an upgrade path or discount.

By analyzing application engagement metrics, you can collect product insights on new, lost, or active user types, report on runtime frequency by the number of days, sessions, and hours, and prioritize development and support versions by user behavior and other properties.

With this breadth of information, product teams can run a user engagement report on new, active and lost users by geography, version edition and build, OS, etc. From these reports, you can understand user behavior and analyze how new releases impact global usage. The ability to closely follow the user experience and gather targeted feedback allows you to create a strategic, data-driven application roadmap.

Example: Need to know when your users uninstall? Is it after two months of installation? After downloading the new version? The churn and engagement report allows you to determine precisely where along the customer lifetime are users churning.

By tracking core usage statistics, you are able to identify features that may require or deserve the most attention. Collect aggregate numerical values when events occur, group similar events, identify user-wide shifts in behavior, and monitor high-level feature usage and their trends. Event tracking allows you to quickly identify behavioral changes across your entire install base that warrants further analysis.

Example: See how many users utilize the save button vs. the menu item. Filter by product version, product build, and license type to make data-driven UX/UI decisions in the future.

Custom Event Tracking

Answer the unique company-specific business questions by configuring custom event tracking across your install base. Tagging and monitoring custom events allow you to follow, timestamp, and analyze specific custom data values during any date range, collect details from any data type, and analyze any combination of data points to determine valuable relationships, outliers, and key user characteristics.

Example: Evaluate if users follow your ideal workflow through an application, or if a better UX or help system is needed to guide them through.

Having detailed insights for particular clients allows you to manage GDPR requests for data access, segment data sets of Client IDs that match specific criteria. Integrate data into third-party systems to deliver more powerful analyses .

Example: Segment and analyze the data sets for all client IDs that have high activity rates versus lower activity rates in the same region, language, build, and OS.

By knowing the stack trace and software exceptions associated, you can fix software bugs before users encounter them, react to errors more quickly, collect full or truncated stack trace, quantify particular exceptions to assess the required level of response and reduce revision cycles.

Example: 30% of users are experiencing software bugs with the latest version. Software usage analytics indicate that the affected users span multiple languages and locations, and are running Windows 7 build 7601 with a firewall installed. A product manager devises a plan to react to this error quickly and send out an update before most users re-open the apps.

Know which users churn by region, language, build. Compare usage trends of active vs. less active users, pinpoint customer drop-off points, evaluate how quickly users drop from trial versions of your application, and optimize marketing campaigns and product messaging according to group attributes.

Example:

You notice that 85% of customers using the trial version use Feature A and Feature B the most. Reduce the number of trial users from churning by offering a version with only Feature A and Feature B at a lower price point while opting for in-app upgrades for the remaining features.

Conversion metrics allow you to understand what drives purchase intention, compare user behavior across products, chart conversion funnel analysis trends, and identify where and why users are not converting.

Example: Usage analytics reveal where users drop off in the conversion funnel segmented by region, platform, build, product version, and language.

With integrated in-application messaging, product teams can directly contact users with relevant messaging to introduce new features and functionality, raise awareness of training resources to educate end users, upsell or cross-sell, promote company user events, and inform users about important updates or offers before they churn. You can target users with high usage for feedback, or ensure that users interact with a feature a certain amount of times before requesting feedback.

Example: Use in-application messaging coupled with software usage analytics to identify and notify segments of users that have not discovered the most valuable features in your product. Prompt them to take advantage of features with relevant “how-to” tips. With increased feature adoption, your customers are delighted, and you create long-term product affinity.

With software usage analytics, you can do the following.

Support Agile Development

Software usage analytics provide the concrete evidence needed to make a change or improvement, or even to retire legacy code during an upcoming development cycle. They can help you prioritize features for an anticipated update and pinpoint where more user feedback is needed in the development cycle. Software usage analytics can also be used to send in-application messages to solicit, receive, and analyze the aforementioned feedback quickly.

Accelerate Product Delivery

Shortening software development lifecycles is possible with software usage analytics — they help you to know precisely what the end-user is experiencing and which steps to take to alleviate issues according to version and build, geography, and tech stack. They can also help measure effectiveness in adoption in a live install base and gain integral feedback in real-time. With software usage analytics, we can segment data by version and build, OSS, hardware architecture, language, etc., allowing product managers to create a concrete roadmap for the next Sprint. They also help us use actionable intelligence to make insight-driven development decisions.

Minimize QA Time and Resources

Knowing the full spectrum about the users, including license key, OS, geographical location, stack trace, and their hardware environment can pinpoint issues clearer than surveys — and software usage analytics can be a great help in doing this. They also replace costly and inaccurate surveys with custom event reporting.

Understand End-User Behavior and Environments for Better Decision-Making

Software usage analytics can help analyze the behavior of power users vs. casual users, as well as the total number of times an event has been generated by all users. From this information, you will know which features to prioritize and which to depreciate. They can also pinpoint which tools are inappropriately hidden or hard to reach and help provide an understanding of the behavior of users that have churned or are about to churn and take steps to stop it happening.

Communicate Effectively, Directly With Users

In-application messaging allows developers and product managers to do the following:

• Segment, reach, and educate end-users with fully targeted, event-driven, in-application messaging and surveys.
• Deliver contextually relevant messages based on defined rules (such as feature usage, geographic profiles, software version, or license status) to send the right message at the right time.
• Announce new product features and offers, push upgrade notifications, communicate security and patch alerts, and post helpful tips all within the application itself.
• Create messages and deliver them to users, customized for different user groups, segmented by region, platform, etc. to relay the right message to the right user at the optimal time.
• Send direct-to-desktop surveys to users based on specific behavior.

Reducing Product UI Redesign Time by 50%

A major software company used software usage analytics to dramatically reduce their product redesign time. Over 225,000 manufactures rely on the software, in industries ranging from automobile manufacturing to consumer goods to medicine. As a pioneer in the industry, the company knew that it needed detailed product usage data to accelerate development, improve focus on the customer, and innovate more effectively. After many failed attempts at DIY analytics (painstakingly tracking interesting statistics for its 1,200 functions and collecting them in a scary excel sheet), they looked into other options.

After implementing a major software usage analytics tool provider on a production product, the company was able to track adoption rates, identifying different usage patterns for over 15 languages that it supports. With the same product interface for 10 years, the company embarked on a daunting task: updating the interface without losing quality. With strong insight into the needs of their 225,000 users, and prioritizing features with data, the entire redesign took half as long as it would have without software usage analytics.

Lowering the Cost of QA

Over 100,000 professionals and 5,000 organizations rely on a major software vendor to increase their ROI from fonts, digital assets, and images. The vendor offers a full suite of tools for large creative organizations down to freelance creative professionals. The product team wanted to bring the voice of the customer into its Agile development process, but without software usage analytics, it was impossible to understand true customer needs.

The software supports the manipulation of many file types, which made it difficult to pinpoint which to support. Previously relying on surveys, the company figured out that when they sent out discovery surveys, it was often the sysadmins who answered, not the creative users. The gap between IT’s perception and the end-users needs was extensive.

After implementing a software usage analytics solution, the company received up-to-date, correct product usage data from actual users. They were able to focus R&D investment, and direct QA resources effectively, translating to a savings of $10,000-$20,000 per release in QA time alone.

Building Innovative Features That Customers Are Technologically Ready For

A BIM software company had been relying on focus groups, sales meetings, in-person conversations with customers, and competitive intelligence to plan features for its latest product release. However, these methods tended to deliver anecdotal rather than systematic data, and couldn’t provide fast answers for timely decision-making. Product teams wanted quick input from a broad cross-section of customers who already relied on their promising new feature.

After implementing a Software Usage Analytics Tool, product teams could identify power users of the new feature and directly ask them for feature feedback, send event-based messages, ask users to vote on possible improvements and receive immediate answers – while users were in the product. These answers were directly linked to data containing hardware and software environments, which identified differences in viewpoints amongst different kinds of customers. As a result, the company now experiences a significantly tighter alignment between R&D investment and product value to actual customers.

When searching for an analytics platform, these are the top criteria to consider:

• Does the platform collect and present meaningful data that deliver actionable insight (for example, metrics related to user engagement, feature usage, and purchases versus simplistic log files)?
• Does the platform deliver real insights in a highly graphical format that’s easy to access, digest, and share across all stakeholders?
• Can the platform present high-level information with the option to drill down interactively when you need to learn more?
• Can the platform provide out-of-the-box reporting to help quickly understand and answer both basic and complex questions for rapid return on investment (ROI)?
• Does the platform allow a user to customize dashboards and reports to meet company-specific requirements easily?
• Does the platform allow data to integrate seamlessly with third-party business intelligence and CRM tools?
• Can the platform adapt and scale to meet evolving business requirements?
• Does the storage capability scale to meet growing business requirements while maintaining an affordable cost structure?
• Does the solution include the ability to easily build and execute in-application messaging campaigns and end user surveys to help guide behavior and gather real-time feedback?