API Developer Outreach Research for the Department of Veterans Affairs: Part 2
Let's take a look at part 2 of this series on API developer outreach research for the department of veterans affairs. Explore how success is defined.
Join the DZone community and get the full member experience.Join For Free
This is Part 2 (you can find Part 1 here) of a series on a write-up for research I conducted with my partner Skylight Digital. The team conducted a series of interviews with leading public and private sector API platforms regarding how they approached developer outreach, and then I wrote it up as a formal report, which the Skylight Digital team then edited and polished. We are looking to provide as much information as possible regarding how the VA and other federal agencies should consider crafting their API outreach efforts.
In the pages below, you will find a large number of specific suggestions culled from extensive interviews and our collective personal experience. All of these specific techniques are in service to the idea of designing the API program with the programmers who will use the API in mind at all times.
How Success Is Defined
With a variety of possible purposes, scopes, and approaches, an API’s success can be defined in a myriad of ways. Depending on the motivations behind the API, and the investment made, the measure of success will depend on what matters to the organization. However, there are some common patterns to defining success, which we extracted from both the interviews we conducted and the research we performed as part of this outreach study.
What We Learned
Along with what we learned about the purpose and scope of API platforms, we discovered more about the different ways in which API providers are defining success. We have collected the highlights among these metrics below.
- Building awareness: API success revolves around building awareness that an API exists, as well as awareness of the value of the API resources that are being made available. Awareness is not simply a consumer side consideration, though; providers, too, must possess an awareness of the value of their resources in relation to both other API developers and consumers alike.
- Attracting new users: Bringing attention to an API and attracting new users is one of the most common ways of measuring the success of API operations. While new users won’t always immediately become active users, their interest and involvement will bring attention to and awareness of what the API platform can deliver. Attracting new users is one of the easiest and most accessible ways to measure the success of any API, according to our interviews, but importantly, none of the platform providers we spoke to recommended that an organization should stop there.
- Incentivizing active users: While attracting new users is easy, producing active users is much harder. The easier it is to onboard with an API and make the first series of API calls, the greater the likelihood that API consumers will integrate the platform into their own resources and work, which is a critical metric for any API provider.
- Applications: Applications and development are the cornerstones that incentivize API providers to invest in APIs, and across the board, our interviewees cited application integration and involvement as a prime candidate for determining an API’s success. This could be quantified both in terms of new applications relying on the API platform, as well as active application processes that integrate with the API’s platform. In either case, measuring usage was considered an excellent means to justify the existence and growth of an API platform.
- Entities: Getting the attention of companies, organizations, institutions, and other government agencies is an important part of the API journey. In particular, developing an awareness of and encouraging the usage and adoption of APIs among groups already leveraging the technology is an important metric by which success can be determined.
- End users: Of course, API providers articulated the importance of serving end-users. Besides serving an organization’s mission, the true purpose of an API is to satisfy an end-user, and so measuring success based upon how much value is created and delivered to these users and customers, and even the public at large, can directly verify that an API is living up to its billing.
- Stakeholders: Further discussions with API providers implied that success is also defined in terms of involvement with other stakeholders. Ensuring that the definition of success was crafted in an inclusive way allows everyone involved and impacted by the project to input their voice. This widens the target audience to make sure success is a large umbrella that covers as many individuals within an organization as possible.
- New resources: An additional area that was used to define success was the number of new API resources added to a platform. If an organization is currently in the development phase and deploying APIs into production, it is likely that a platform already has a handle on what it takes to successfully deliver APIs throughout their lifecycle. Making new APIs a great way to understand the velocity of any platform, and how well it is ultimately doing.
Measuring the success of an API platform is a much more ambiguous goal than determining scope, purpose, and investment. Our interviews revealed that success often means different things to different providers. Moreover, an organization’s understanding of success is also something that will evolve over time. We learned a lot from API providers about pragmatic views on what API success looks like, and now we would like to translate that into some basic guidance on how to help ensure the success of providing APIs.
What Our Thoughts Are
Defining, measuring, and quantifying the success of API operations is not easy. As discussed above, measuring success functionally amounts to hitting a moving target. It is important to start with the basics, be realistic, and define success in a meaningful way. Adding to what has been gathered from interviews with API providers, we recommend a consideration of the following factors when it comes to defining just exactly what success is.
- Know your resources: Understand the resources you are making available via an API. Ensure that they are well defined and made accessible in ways that consider security, privacy, and the interests of all stakeholders. Do not just open up APIs for the sake of delivering APIs — make sure the resources are well defined, designed, and serve a purpose.
- Manage your APIs: API management is essential to measuring success. It is extremely difficult to define success without requiring all developers to authenticate, log, quantify, and analyze their consumption. Measuring these kinds of consumption activities helps to quantify the value produced by the API and its related platform, and an understanding of this value serves as the foundation for any API’s future success.
- Have a plan: There is no success without a plan. A set of plans are required to apply at the management level in order to quantify the addition of new accounts, define whether they are active or not, and understand how applications are putting resources to work. Providing a plan for how resources are made available, and how they are consumed, generates a framework to think about and measure what API success means.
- Measure portal activity: Treat your API developer portals as you would any other web property and actively measure its traffic. Apply data-analytic solutions to track sessions, time, and visitors, and use this information to contribute to a sales and marketing funnel that can be used to understand how developers are using portal resources. Importantly, this kind of analysis can also discover points of friction that developers may be encountering when trying to use your API platform.
- Analyze and report: Produce weekly, monthly, quarterly, and annual reports from data gathered across the API stack, API portal, and from social media. Developing an understanding of what is happening based upon actual data, and consistently reporting upon findings with all stakeholders in API operations, ensures both transparency of API knowledge and information access to formulate plans for growth.
- Discuss and evangelize: Have a strategy for taking any analysis or reporting from API operations and disseminating it amongst stakeholders. With these resources distributed, consider conducting regular on- and off-line discussions around what they mean. Work with everyone involved to understand the activity across a platform, and use these discussions to transform the understanding of success as platform awareness grows.
- Make things observable: Make every building block of API operations observable. Ensure that everything has well-defined inputs and outputs, and consider how these can be used to better understand whether the platform is working or not. Allowing every single aspect of the platform to be able to contribute to an overall definition of what success means by providing real-time and historic data around how resources are being used can signal important insights about an API and how it might be improved.
The success of an API platform will mean different things to different groups and will evolve over time as awareness around an organization’s resources grows. Know your resources, properly manage your APIs, and have a plan, but make sure you are constantly reassessing exactly what success means while having ongoing conversations with stakeholders. With more experience, you will find that API platform success becomes much more nuanced, but importantly, it also becomes easier to define once you know what it is that you want.
Opinions expressed by DZone contributors are their own.