API Developer Outreach Research for the Department of Veterans Affairs: Part 5
Check out API developer outreach research for the Department of Veterans Affairs. Explore the types of metrics for measuring adoption and making decisions.
Join the DZone community and get the full member experience.Join For Free
This is Part 5 (you can find Part 4 here) of a series on a write-up for research I conducted with my partner Skylight Digital. The team conducted a series of interviews with leading public and private sector API platforms regarding how they approached developer outreach, and then I wrote it up as a formal report, which the Skylight Digital team then edited and polished. We are looking to provide as much information as possible regarding how the VA and other federal agencies should consider crafting their API outreach efforts.
In the pages below, you will find a large number of specific suggestions culled from extensive interviews and our collective personal experience. All of these specific techniques are in service to the idea of designing the API program with the programmers who will use the API in mind at all times.
Types of Metrics for Measuring Adoption and Making Decisions
This area of our research overlaps somewhat with the earlier section on measuring success, but here, we provide a more precise look at what can be measured to help quantify success while also ensuring that findings are used as part of the decision-making process throughout the API journey.
What We Learned
Our interviews reminded us that it is useful to consider that not all API providers have a fully fleshed out strategy for measuring activity across their platforms. However, we did come away with some interesting lessons from those providers that were using metrics to drive API decision making.
- Look at it as a funnel: Treat API outreach and engagement as a sales and marketing funnel. Attract as many new users as you can, but then profile the target demographic to try to understand who they are and what their working objectives comprise. From there, devote efforts to incentivizing users to “move down the funnel” through the sandbox environment and eventually to production status. In short, treat API operations like a business and platform users like they are customers.
- Do not have formal metrics: It was illuminating to also learn that some providers felt that having an overly formal metrics strategy might constrain developer outreach and engagement. Providing words of caution when it comes to measuring too much, as well as only examining data when it comes to making critical decisions, can keep outreach efforts more constructively interacting with API consumers regarding their needs.
- API keys registered: For data accessibility purposes, it is worthwhile to ensure that all developers have an application and API key before they can access any API resources. Requiring all internal, partner, and eternal developers to pass along their API key with each API call allows all platform activity to be measured, tracked, and used as part of the wider platform decision-making process.
- Small percentage of users: We also heard that it is common for a small percentage of the overall platform users to generate the majority of API calls to the platform. This makes it important to measure activity on the platform in terms of which users are the most salient (and thereby driving the majority of value on a platform).
- Amount of investment: Importantly, the usage rates of a platform’s resources can provide a strong justification for investing more resources into the platform’s success, making tracking that data of paramount importance. This transforms investment into a data-driven decision that responds to the actual needs of the platform.
The interview portion of our research provided a valuable look at how API providers are measuring activity across their platforms. Data and metrics are not only being used to define success but are also used as part of the ongoing decision making process around the direction API providers take their platform, particularly when it comes to measuring adoption of API resources across a platform.
What Our Thoughts Are
When it comes to measuring adoption and understanding how API consumers are putting resources to work, we recommend starting small. Activity tracking is something that will change and evolve as the organization develops a better understanding of platform resources and the interests of internal stakeholders, partners, and 3rd party consumers. These are just a handful of areas we recommend collecting data on to begin with.
- Traffic: Measure and understand traffic (network and otherwise) across the platform developer portal.
- New accounts: Track and profile all new accounts signing up for API access.
- New applications: Track and profile all new applications registered for access.
- Active applications: Measure and track the usage of the active platform applications.
- Number of API calls: Understand how many APIs are being called in different dimensions.
- Conversation: Measure all conversation happening across the platform and use these estimates to develop awareness.
- Support: Measure all support activity in order to pinpoint the needs of API consumers.
- Personas: Quantify the different types of consumers who are putting a platform to use.
Begin here when it comes to tracking: develop an awareness of the community and get to know what is important. Then expand from there. Add and remove the metrics that make sense for your organization without tracking metrics for no reason (i.e. all tracking should add value to the API platform or its decision-making process).
Opinions expressed by DZone contributors are their own.