DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

The software you build is only as secure as the code that powers it. Learn how malicious code creeps into your software supply chain.

Apache Cassandra combines the benefits of major NoSQL databases to support data management needs not covered by traditional RDBMS vendors.

Generative AI has transformed nearly every industry. How can you leverage GenAI to improve your productivity and efficiency?

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

  1. DZone
  2. Refcards
  3. Software Usage Analytics for Data-Driven Development
refcard cover
Refcard #286

Software Usage Analytics for Data-Driven Development

Software usage analytics can help you uncover UI/UX problems, optimize trial periods, prioritize feature enhancements, measure user adoption, and more. Gartner predicts that by 2021, 75% of software providers will rely on insights from embedded software usage analytics to inform major product decisions and measure customer happiness. In this Refcard, you’ll learn about the challenges that software usage analytics can help developers solve, see how software usage analytics can track product and conversion metrics, look at specific software usage analytics use cases, and more.

Download Refcard
Free PDF for Easy Reference
refcard cover

Written By

author avatar Chris Ward
Zone Leader, DZone
author avatar Lesa Mone
Writer, Lesa Mone
Table of Contents
► Software Usage Analytics Addresses Developer Challenges ► Software Usage Analytics vs. APM and ALM ► What Software Usage Analytics Can Track ► Product and Environment Metrics ► Operating System ► User Activity and Engagement ► Event Tracking ► Client Profile Reporting ► Identify Software Exceptions and Stack Trace ► Retention and Churn ► Conversion Metrics ► Bonus: In-Application Messaging ► Software Usage Analytics Benefits ► Use Cases ► Conclusion
Section 1

Software Usage Analytics Addresses Developer Challenges

Software usage analytics enables you to collect, analyze, visualize, and integrate data on how users use software applications. Gartner predicts that by 2021, 75% of software providers will rely on insights from embedded software usage analytics to inform product management decisions and measure customer health.

Event-based metrics provide real-time insights from actual customer usage, highlighting which features are being adopted and which are being ignored, revealing problems in the UX or UI, helping to accelerate trial periods, improving user experience for particular OS or version builds, prioritizing feature enhancements, and tracking license compliance. And, when product usage analytics is combined with targeted in-application messaging, it provides direct communication with users to provide real-time user assistance, gather valuable feedback, or shape behavior. Product teams with these insights experience a sharp increase in development efficiency and shorter release iterations, resulting in optimized resource investment.

This Refcard gives an overview of software usage analytics, highlights specific metrics you can track, and compares software usage analytics to other developer tools including application lifecycle management and application planning solutions.

Traditional software vendors rely on conventional methods to collect and analyze information regarding customer usage of their product. These methods (questionnaires, surveys, support calls, customer interviews, observations, focus groups, and sales feedback) all come with their respective limitations, including differences in understanding and interpreting questions and incomplete responses. It may be difficult to create a concrete roadmap, including UX/UI direction, from responses that contain unreliable replies.

Embedded analytics gives you the information you need to help your product teams better follow Agile development principles. Direct insight into how users interact with your application allows you to plan new and old feature support with instant data from active users. This data helps you to optimize your Sprint planning to use developer resources only where they are needed. Software usage analytics is available for SaaS, desktop, and server applications alike.

Section 2

Software Usage Analytics vs. APM and ALM

While certain capabilities of application lifecycle management and application performance management methodologies provide tools to assist in the design, development, and testing of software applications, they miss out on direct metrics that dig deep to reveal how users are engaging with your applications, and fail to pinpoint when and why issues occur.

With APM, ALM, and software usage analytics methodologies in place, organizations can significantly improve the way they design, build, test, and adapt their software applications to best meet customer needs. Below is a summary of the methodologies and their key features.

Software usage analytics expose the depth of customer behavior through event-based user data, which can be used in addition to APM and ALM to build better applications.

Software Usage Analytics Application lifecycle management (ALM) Application performance management (APM)
Definition Software usage analytics gives powerful insight into how users are engaging with an application at all stages in the lifecycle. Combined with in-app messaging, contextually relevant messages or requests for feedback can be sent to highly targeted users. ALM is an integrated system of people, processes, and tools that manage the life of an application from concept to retirement. APM helps you detect and diagnose application performance problems to maintain an expected level of service.
Capabilities
  • Capture metrics on hardware, software environments
  • Track feature usage and adoption
  • Segment users based on metrics or usage patterns, including location or version
  • Identify reasons for user churn
  • In-app messaging to specific user segments/behavioral profiles
  • Requirements management
  • Estimation and planning
  • Testing and QA measurement
  • Maintenance and support
  • Real-time planning and team communication
  • Digital experience monitoring
  • Application discovery, tracing, and diagnostics
  • Application topology (runtime architecture)
  • Discovery and visualization
  • User-defined transaction profiling
  • Application component deep-dive
In-application messaging Yes No No
Event-based tracking Yes No No

In short, APM details the actual performance of features while ALM helps you roll out all of the changes you need, and helps you roll them back again, whereas software usage analytics expose the depth of customer behavior through event-based user data. Software usage analytics combined with APM and ALM helps you build the perfect app.

Section 3

What Software Usage Analytics Can Track

In addition to collecting data across different reports, software usage analytics gives the ability to identify types of users or segment by the different pieces of information across any report. For instance, you can track total feature usage, or you can drill down and analyze feature usage segmented by people running a particular OS or hardware, which makes the findings more interesting and actionable.

Section 4

Product and Environment Metrics

Version, Edition, and Build

By tracking how many users are running each version and build, you can plan and prioritize support activity and upgrades by region, user group, or support plan. Additionally, you can evaluate the need to sunset low-use editions.

Example: Software Usage Analytics may reveal that a large number of users are experiencing a bug for a specific feature with a recent build in Brazil.

Section 5

Operating System

By knowing how many users are running your application by platform, you can analyze the most critical OS metrics — version, platform, language, and architecture size and support features that work only on specific OS versions.

Example: Software usage analytics may reveal that all of the users experiencing a particular bug are running macOS Mojave version 10.14.

Hardware Architecture

Knowing which hardware your customers use to run your application allows you to replicate how your customers interact with the product, optimize for the device type down to the type of chips and graphics cards, and evaluate the performance of the software on the most used CPU core, memory, and graphics card configurations. This overarching view gives unprecedented visibility into the entire end-user experience while allowing you to understand environments running your software.

Example: Software usage analytics may reveal that few users have sufficient memory specs needed for an upgraded version and that it’s worth offering them extended support for their older version.

Geography

By knowing where your users are active, you can compare adoption rates across regions, analyze trends by region, country, or state, and utilize heat maps to compare churn rates across countries. Heat maps reveal how users with different attributes are distributed globally, highlighting a specific version or license type.

Example: Software usage analytics may reveal that trial users running version 3.0 on Windows in Germany have a low conversion rate.

License Key

Knowing the license keys of each respective user allows you to easily locate and delete user information (GDPR compliance), filter and view user activity based on license key status, and verify blacklisted, expired, or abused keys.

Example: Software usage analytics reveals which license keys have surpassed the install quota. Use in-application messaging to reach out to these users and offer an upgrade path or discount.

Section 6

User Activity and Engagement

By analyzing application engagement metrics, you can collect product insights on new, lost, or active user types, report on runtime frequency by the number of days, sessions, and hours, and prioritize development and support versions by user behavior and other properties.

With this breadth of information, product teams can run a user engagement report on new, active and lost users by geography, version edition and build, OS, etc. From these reports, you can understand user behavior and analyze how new releases impact global usage. The ability to closely follow the user experience and gather targeted feedback allows you to create a strategic, data-driven application roadmap.

Example: Need to know when your users uninstall? Is it after two months of installation? After downloading the new version? The churn and engagement report allows you to determine precisely where along the customer lifetime are users churning.

Section 7

Event Tracking

By tracking core usage statistics, you are able to identify features that may require or deserve the most attention. Collect aggregate numerical values when events occur, group similar events, identify user-wide shifts in behavior, and monitor high-level feature usage and their trends. Event tracking allows you to quickly identify behavioral changes across your entire install base that warrants further analysis.

Example: See how many users utilize the save button vs. the menu item. Filter by product version, product build, and license type to make data-driven UX/UI decisions in the future.

Custom Event Tracking

Answer the unique company-specific business questions by configuring custom event tracking across your install base. Tagging and monitoring custom events allow you to follow, timestamp, and analyze specific custom data values during any date range, collect details from any data type, and analyze any combination of data points to determine valuable relationships, outliers, and key user characteristics.

Example: Evaluate if users follow your ideal workflow through an application, or if a better UX or help system is needed to guide them through.

Section 8

Client Profile Reporting

Having detailed insights for particular clients allows you to manage GDPR requests for data access, segment data sets of Client IDs that match specific criteria. Integrate data into third-party systems to deliver more powerful analyses .

Example: Segment and analyze the data sets for all client IDs that have high activity rates versus lower activity rates in the same region, language, build, and OS.

Section 9

Identify Software Exceptions and Stack Trace

By knowing the stack trace and software exceptions associated, you can fix software bugs before users encounter them, react to errors more quickly, collect full or truncated stack trace, quantify particular exceptions to assess the required level of response and reduce revision cycles.

Example: 30% of users are experiencing software bugs with the latest version. Software usage analytics indicate that the affected users span multiple languages and locations, and are running Windows 7 build 7601 with a firewall installed. A product manager devises a plan to react to this error quickly and send out an update before most users re-open the apps.

Section 10

Retention and Churn

Know which users churn by region, language, build. Compare usage trends of active vs. less active users, pinpoint customer drop-off points, evaluate how quickly users drop from trial versions of your application, and optimize marketing campaigns and product messaging according to group attributes.

Example:

You notice that 85% of customers using the trial version use Feature A and Feature B the most. Reduce the number of trial users from churning by offering a version with only Feature A and Feature B at a lower price point while opting for in-app upgrades for the remaining features.

Section 11

Conversion Metrics

Conversion metrics allow you to understand what drives purchase intention, compare user behavior across products, chart conversion funnel analysis trends, and identify where and why users are not converting.

Example: Usage analytics reveal where users drop off in the conversion funnel segmented by region, platform, build, product version, and language.

Section 12

Bonus: In-Application Messaging

With integrated in-application messaging, product teams can directly contact users with relevant messaging to introduce new features and functionality, raise awareness of training resources to educate end users, upsell or cross-sell, promote company user events, and inform users about important updates or offers before they churn. You can target users with high usage for feedback, or ensure that users interact with a feature a certain amount of times before requesting feedback.

Example: Use in-application messaging coupled with software usage analytics to identify and notify segments of users that have not discovered the most valuable features in your product. Prompt them to take advantage of features with relevant “how-to” tips. With increased feature adoption, your customers are delighted, and you create long-term product affinity.

Section 13

Software Usage Analytics Benefits

With software usage analytics, you can do the following.

Support Agile Development

Software usage analytics provide the concrete evidence needed to make a change or improvement, or even to retire legacy code during an upcoming development cycle. They can help you prioritize features for an anticipated update and pinpoint where more user feedback is needed in the development cycle. Software usage analytics can also be used to send in-application messages to solicit, receive, and analyze the aforementioned feedback quickly.

Accelerate Product Delivery

Shortening software development lifecycles is possible with software usage analytics — they help you to know precisely what the end-user is experiencing and which steps to take to alleviate issues according to version and build, geography, and tech stack. They can also help measure effectiveness in adoption in a live install base and gain integral feedback in real-time. With software usage analytics, we can segment data by version and build, OSS, hardware architecture, language, etc., allowing product managers to create a concrete roadmap for the next Sprint. They also help us use actionable intelligence to make insight-driven development decisions.

Minimize QA Time and Resources

Knowing the full spectrum about the users, including license key, OS, geographical location, stack trace, and their hardware environment can pinpoint issues clearer than surveys — and software usage analytics can be a great help in doing this. They also replace costly and inaccurate surveys with custom event reporting.

Understand End-User Behavior and Environments for Better Decision-Making

Software usage analytics can help analyze the behavior of power users vs. casual users, as well as the total number of times an event has been generated by all users. From this information, you will know which features to prioritize and which to depreciate. They can also pinpoint which tools are inappropriately hidden or hard to reach and help provide an understanding of the behavior of users that have churned or are about to churn and take steps to stop it happening.

Communicate Effectively, Directly With Users

In-application messaging allows developers and product managers to do the following:

  • Segment, reach, and educate end-users with fully targeted, event-driven, in-application messaging and surveys.
  • Deliver contextually relevant messages based on defined rules (such as feature usage, geographic profiles, software version, or license status) to send the right message at the right time.
  • Announce new product features and offers, push upgrade notifications, communicate security and patch alerts, and post helpful tips all within the application itself.
  • Create messages and deliver them to users, customized for different user groups, segmented by region, platform, etc. to relay the right message to the right user at the optimal time.
  • Send direct-to-desktop surveys to users based on specific behavior.
Section 14

Use Cases

Reducing Product UI Redesign Time by 50%

A major software company used software usage analytics to dramatically reduce their product redesign time. Over 225,000 manufactures rely on the software, in industries ranging from automobile manufacturing to consumer goods to medicine. As a pioneer in the industry, the company knew that it needed detailed product usage data to accelerate development, improve focus on the customer, and innovate more effectively. After many failed attempts at DIY analytics (painstakingly tracking interesting statistics for its 1,200 functions and collecting them in a scary excel sheet), they looked into other options.

After implementing a major software usage analytics tool provider on a production product, the company was able to track adoption rates, identifying different usage patterns for over 15 languages that it supports. With the same product interface for 10 years, the company embarked on a daunting task: updating the interface without losing quality. With strong insight into the needs of their 225,000 users, and prioritizing features with data, the entire redesign took half as long as it would have without software usage analytics.

Lowering the Cost of QA

Over 100,000 professionals and 5,000 organizations rely on a major software vendor to increase their ROI from fonts, digital assets, and images. The vendor offers a full suite of tools for large creative organizations down to freelance creative professionals. The product team wanted to bring the voice of the customer into its Agile development process, but without software usage analytics, it was impossible to understand true customer needs.

The software supports the manipulation of many file types, which made it difficult to pinpoint which to support. Previously relying on surveys, the company figured out that when they sent out discovery surveys, it was often the sysadmins who answered, not the creative users. The gap between IT’s perception and the end-users needs was extensive.

After implementing a software usage analytics solution, the company received up-to-date, correct product usage data from actual users. They were able to focus R&D investment, and direct QA resources effectively, translating to a savings of $10,000-$20,000 per release in QA time alone.

Building Innovative Features That Customers Are Technologically Ready For

A BIM software company had been relying on focus groups, sales meetings, in-person conversations with customers, and competitive intelligence to plan features for its latest product release. However, these methods tended to deliver anecdotal rather than systematic data, and couldn’t provide fast answers for timely decision-making. Product teams wanted quick input from a broad cross-section of customers who already relied on their promising new feature.

After implementing a Software Usage Analytics Tool, product teams could identify power users of the new feature and directly ask them for feature feedback, send event-based messages, ask users to vote on possible improvements and receive immediate answers – while users were in the product. These answers were directly linked to data containing hardware and software environments, which identified differences in viewpoints amongst different kinds of customers. As a result, the company now experiences a significantly tighter alignment between R&D investment and product value to actual customers.

Section 15

Conclusion

When searching for an analytics platform, these are the top criteria to consider:

  • Does the platform collect and present meaningful data that deliver actionable insight (for example, metrics related to user engagement, feature usage, and purchases versus simplistic log files)?
  • Does the platform deliver real insights in a highly graphical format that’s easy to access, digest, and share across all stakeholders?
  • Can the platform present high-level information with the option to drill down interactively when you need to learn more?
  • Can the platform provide out-of-the-box reporting to help quickly understand and answer both basic and complex questions for rapid return on investment (ROI)?
  • Does the platform allow a user to customize dashboards and reports to meet company-specific requirements easily?
  • Does the platform allow data to integrate seamlessly with third-party business intelligence and CRM tools?
  • Can the platform adapt and scale to meet evolving business requirements?
  • Does the storage capability scale to meet growing business requirements while maintaining an affordable cost structure?
  • Does the solution include the ability to easily build and execute in-application messaging campaigns and end user surveys to help guide behavior and gather real-time feedback?

Like This Refcard? Read More From DZone

related article thumbnail

DZone Article

How to Merge HTML Documents in Java
related article thumbnail

DZone Article

Enforcing Architecture With ArchUnit in Java
related article thumbnail

DZone Article

Creating a Web Project: Caching for Performance Optimization
related article thumbnail

DZone Article

Exploring Intercooler.js: Simplify AJAX With HTML Attributes
related refcard thumbnail

Free DZone Refcard

Open-Source Data Management Practices and Patterns
related refcard thumbnail

Free DZone Refcard

Real-Time Data Architecture Patterns
related refcard thumbnail

Free DZone Refcard

Getting Started With Real-Time Analytics
related refcard thumbnail

Free DZone Refcard

Getting Started With Apache Iceberg

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: