REST-API Versioning Strategies

Abstract

The approach to managing Application Programming Interfaces (APIs) for distributed heterogeneous systems differs from the classic tools as offered by Apache Maven, which is designed to simplify build processes for the assembly of technologically and homogeneously similar artifacts. The limitations of such a classic dependency management tool become evident in the context of managing APIs between technologically different and independent systems, where APIs such as the Representational State Transfer (REST-APIs) ensures their networking capabilities, for instance.

The mutual contract between a service provider, which might offer a few API versions implemented, and its consumers, plays a critical role and thus demands another approach to guarantee the contract by design in operation for a prolonged period. Moreover, the semantic versioning as used by Apache Maven by differentiating three digits would be unrealistic for REST-APIs.

Consequently, the applied semantic scheme deliberately distinguishes only major releases in which each increment stands for breaking changes. In this position paper the author emphasizes, for the versioning concepts of the REST-API to be effective and sustainable, software engineers must understand capabilities and limitations of the different approaches, and which of them is most effectively aligned with the functions of the underlying transport technologies.

The author provides a set of principles, for the protection of an organization from technical debts while a prolonged period of the coexistence API versions increases its magnitude, to which API owners, as well as engineers, must adhere when introducing a new version of an API.

You may also like:  Versioning a REST API With Spring Boot and Swagger

Introduction

Application Programming Interfaces (APIs) have helped companies to integrate applications or even build new ecosystems. There is a trend to foster innovation and monetization of services through APIs, which is recognizable in many sectors. Enhanced systems through interfaces not only play a decisive role in industries, which determines a part of Industry 4.0, in particular, the smart factories but also in the financial world to adhere to the Open Banking directive for instance. [1][2]

Advances towards the Representational State Transfer (REST)-APIs have paved the way for communication across heterogeneous systems, where most of the applications are from different nature. However, the view over versioning REST-APIs differs and constitutes a rarely written paper, not only from a scientific point of view. In this paper, the term API is taken to mean the REST-API in general. While advocates believe that any API should adopt a versioning identifier to be prepared for changes with the potential of backward incompatibility, what describes the notion of breaking change, the opponents claim that the vast majority of applications are not even subject to such a degree of change and thus, does not need versioning at all. 

Nevertheless, the design of maintainable APIs has presented numerous challenges to many software architects and developers. Even though many strategies have been proposed to find effective ways to deal with breaking changes, all of them have revealed a common burden to organizations when the number of coexistence APIs is increasing. A brief introduction to each of the technologies involved in state transmissions, such as Domain Name System (DNS), Uniform Resource Locator (URL), Uniform Resource Identifier (URI), and Hypertext Transfer Protocol (HTTP) is a necessity to understand their primary function, as each of them builds a critical element of the various methods.

DNS

The widely used translation service named DNS to resolve Internet Protocol (IP) addresses to human-friendly names, and vice versa offers global uniqueness, robustness, scalability, and discovery functionality by leveraging the hierarchical arrangement of separable names. DNS has continuously been evolving in terms of supporting protocols and scalability since its launch about 30 years ago.

While names still help to remember objects more quickly than numerical addresses, the identifiable resources and information behind have become more versatile, not least due to the significant growth of application interfaces in recent years. However, emerging technologies such as the ubiquitous Internet of Things (IoT) pose challenges to this global standard concerning the original characteristics.

According to reference [3], DNS is not designed for mobility, efficiency and security aspects of today’s requirements in an IoT-influenced world. The resolution mechanism is subject to an increased delay, the more deeply a hierarchy becomes. Another contradicting constellation is given between robustness and agility, while the former is needed to manage the vast namespace, agility on subdomains to identify resources on demand.

URL and URI

Though the clear distinction between the two terms URI and URL, they are often considered equivalent, however, each one is devised to a particular kind of resource management. While the URI deals with the identification of an object, the URL describes the location of an identifiable element. However, since the URL also identifies objects, it shares identical characteristics as the URI, but the opposite is not the case. [4] In other words, the URI is about identifying resources whose existence, the location, is unimportant and thus, stands in contrast to URL. [5]

Berners-Lee, the inventor of the Web and director of the World Wide Web Consortium (W3C), writes about the importance of developing strategies aimed to achieve unchanged URIs. [6] Moreover, the concept of the cool URI intends to prevent an increase in the number of identifiers during technological and organizational changes to comply with the architectural recommendations of the World Wide Web. [7] [8] In contrast, if allowing underlying technologies to influence naming schemes, any change in technology could then adversely alter the critical concept of the URL, resulting in an arbitrary evolution. [4]

HTTP

HTTP refers to a transmission protocol at the application layer as defined in the ISO/OSI model and has been used by the WWW initiative since 1990. [9] [10] Apart from HTTP's secured version with Transport Layer Security (TLS) namely HTTPS, an HTTP message comprises a header and a body whilst the former aims to separate metadata from the payload.

More precisely, the header consists of a set of value pairs to bind metadata to the payload, which is being processed by applications. Thus, HTTP headers enable for both client and server to pass additional information in a key-value manner to its request, response respectively.

For instance, the negotiation over the data representation is a critical feature of HTTP to give the clients certain flexibility on it, such as web browsers. In essence, an application expresses its preferences on data representation alongside the following dimensions: media type, language, compression, and character set.

Position: Capabilities and Opportunities of Technologies for API Versioning

Semantic Versioning

The term semantic versioning (semver) describes a schema of differently weighted elements to inform developers about the degree of changes among the different versions that affect the interfaces invoked from applications. One way is to have three different version numbers for the major, minor, and patch-related scope of changes, as devised by [11], the founder of GitHub.

While an increment of the major digit indicates an incompatible change, both the minor and patch refers to backward-compatible changes, with the difference of added features of the minor and applied bug fixes of the patch level. [11] Therefore, semantic versioning helps in estimating the rework needs to be done when upgrading to a new version while breaking changes are only permitted in the most significant element, the major digit in the above example.

Versioning With DNS

Following the DNS characteristics, the resulting possibilities of DNS appear to be very modest. The utilization of subdomains, one for each version, allows the support of having multiple versions in coexistence. The general form, excluding the parts that make up the path, is given by https://<version>.<host>. An example would be:

myapi-v1.fictious.com

and 

myapi-v2.fictious.com

However, there are significant drawbacks in the use of subdomains to achieve versioned APIs, not least because DNS lags behind agility, but also the need to change firewall rules to ensure connectivity between client and server again. Another drawback is the scope affected, as it usually affects one or more different endpoints that are always subject to the same release cycle, which in many cases proves unnecessary.

Versioning With URL and URI

In contrast to the DNS-based approach, where the identifier usually is beyond the possibilities of capturing it through a path-based pattern, the URI approach offers significantly more flexible ways through the handling of version identifiers at the various stages on the backend side, in Java, the class or method for instance.

Depending on its position alongside paths, the level determines the granularity of the scope of what falls under the same release. Undisputedly, the URI-based approach proves to be by far advantageous compared to DNS. The following example enables an intermediate granularity, so everything behind my API becomes part of the same release cycle:

https://<host>/my-api/{versionIdentifier}/members

When it comes to the decision making, the benefits of fine-grained API versions with a minimum of unchanged or uncritical parts included and its administrative costs needs to be weighed against the consequences of coarse cut releases with a higher amount of unchanged parts and the associated administrative costs, whereby a significant increase in administrative effort may lead to an increase of scope.

In contrast, reference [7] point out that any identifiable entity should be obtained through a version-less URI by following the WWW recommendations which, portraits some challenges, while finding a suitable solution for the navigation between versions forms the main priority. However, since URL refers to objects in the most literal sense of the word, URL in connection with REST appears misleading due to its representative nature. [5]

Nevertheless, given the above example about members, from an ethical point of view, it is highly questionable to talk about the different versions of the same human being. Alternatively, passing an API key to the backend in the form of a query parameter can mitigate the demerits discussed of the path-based approach, while a correlation between API key and version on the backend side becomes a necessity. For example, each application identifies itself in the form of:

https://<host>/api/<my-endpoint>?key=<APIkey>

Although there is no reason against adding such an identifier to each request, which is already common practice, it might not be a suitable solution for APIs when it comes to changes that affect paths.

Versioning With HTTP: Content Negotiation

An emerging approach to version-less URI is known as Memento and describes ”time travel for the Web” by utilizing the HTTP protocol to navigate among versions without changing the URI at all. The notion of transparent content negotiation describes data retrieval from a ”transparently negotiable resource,” available in multiple variations and versions. [7]

In particular, content negotiation aims to describe the specific form of data to be transferred between client and server, back and forth and is defined through the Content-Type header typically. For example, the client with the header ”Accept: application/json” accepts application data in general, narrows down it to JSON in particular. Depending on the information provided by the header, the server can then respond either with “HTTP 200 OK” and “Content-Type: application/json”, or “HTTP 302 Found” by offering the client a choice of URIs each of which for an alternative form. Otherwise, the server responses with “HTTP 406 Not Acceptable” and again provides a list of acceptable representations. 

Leveraging the HTTP features for data representation is semantically the wisest since it sounds more natural for dealing with entities in their various representations, including their versions. Moreover, the utilization of the Content-Type enables the introduction of domain-specific media types with version. However, some of the existing types come already with version support.

According to [5], the introduction of a new type is preferred over an existing one, unless the most appropriate type is meaningful enough and supports versioning. An example gives the following headers from the client and its corresponding response from the server:

Accept: application/vnd.myname.v1+json

The server’s response:

Content-Type: application/vnd.myname.v1+json

As a consequence of version-less URL might reveal simple URL-based caching mechanisms, when the version-determining element within the header is not considered, resulting in arbitrary responses.

Versioning API

The author vouches for developers and architects to understand the different concepts of REST-API versioning, including their merits and demerits. Such an understanding reduces the risk of uncertainty about the extent to which the future will prove the viability of a chosen strategy, which in turn hinders the formation of a fruitful ground from which unacceptability might develop.

The natural behavior in response to uncertainty and increased pressure to determine a solution from the outset would not foresee an implementation to support differentiated API versions. In fact, such a viewpoint would potentially not turn out to be failing when considering the study by [12] based on 150,000 jar files with 22,000 different open source libraries published to the Maven Central over seven years, concluded that merely a minority of versions, accounting to one third, were subject to breaking changes.

Despite the vast majority of different versions of libraries were backwards compatible and underly a sophisticated mechanism for versioning, it may also support the position of the author that developers favor avoiding introducing breaking changes to libraries, which does not appear unnatural. Nevertheless, some thoughts still need to be given on how to handle a blue/green deployment with coexisting versions of an API during development phases. 

Some companies tend to increase the version identifier even though the API fulfills the notion of backward compatibility, which undoubtedly leads to an avoidable excessive increase in mutual maintenance by API providers and consumers. Since short release cycles are not unusual in the Agile-driven world in which each Sprint lasts between one and four weeks, for instance, those frequencies may act as a vulgar accelerator.

Undoubtedly, content negotiation seems to be the most natural and sophisticated approach, not least because it applies to the application layer of the ISO/OSI model and thus should be aspired whenever possible. Even if an architect feels convinced by another strategy, the chosen concept should not only convey the developer's conveniences but rather be in line with the concepts of the underlying technology.

Switching Versions

Apart from a particular approach, actively pushing customers to upgrade them to the latest version is very critical, since each inferior version is an increase of the risk bringing a company into technical debts, of which small and medium-sized enterprises may be endangered. [13] Besides a plannable migration phase, low efforts in the maintenance of multiple version should then always be the primary goal.

Prior to each introduction of a new API version demands an organization to carefully compiling release notes to highlight the relevant differences to its inferior one. Depending on the extent of a change to an API, it can reach a severity up to the level of breaking change. Aside from a particular motivation to increase such an identifier, the author outlines a set of four principles designed to help companies to counteract a subtle increase in the number of coexisting API versions as well as the associated costs.

Principle 1: Map a subordinate version to its new one

Attempt to forward each incoming request from an inferior version to its directly superior one, perhaps by applying stepwise data transformations between them, which can also include data enrichment. By following this principle, code redundancy between the different versions can be substantially reduced, which in turn has a positive effect on operation. 

Principle 2: Advertise release notes and shutdown schedules

Update the end of the support and availability period of the previous API version and promote release notes to customers. Release notes help developers to roughly estimate the amount of rework required for an upgrade, which together with the shutdown date helps them to plan such an activity.

Principle 3: Stop issuing API keys

Stop issuing API keys for an inferior version. [5] This principle helps to prevent obtaining API keys from obsolete APIs and simultaneously reduces the number of applications to migrate. Such a measure stands in sharp contrast to the fashion of how libraries are treated within Maven Central.

Principle 4: Push clients to upgrade

Remind customers frequently about a new version of an API and the shutdown date of their used ones. Gathering the details necessary about API consumers here is vital to communicate about changes, which is one of the duties of an API management system.

Further Reading

REST API Best Practices — Design Examples From Java and Spring Web Services

Best Practice: API Versioning for HTTP REST Interfaces

References

1. Marr, B., Data Strategy: How to profit from a world of big data, analytics, and the internet of things, 2017, Kogan Page, Kindle Books
2. Passi, L. F., ‘An open banking ecosystem to survive the revised Payment Services Directive: Connecting international banks and FinTechs with the CBI Globe platform’, Journal of Payments Strategy & Systems, 12(4), pp. 335–345. 2018. Available: https://search-ebscohost-com.liverpool.idm.oclc.org/login.aspx?direct=true&db=bth&AN=135550741&site=eds-live&scope=site, -Accessed on: June 23, 2019
3. Yan, Z., Li, H., Zeadally, S., Zeng, Y. & Geng, G., Is DNS Ready for Ubiquitous Internet of Things?, IEEE Access, Access, IEEE, p. 28835. 2018. doi: 10.1109/ACCESS.2019.2901801, Accessed on: June 10, 2019
4. Verborgh, R., Mannens, E., Van De Walle, R., Van Hooland, S., Cope, A.S. & Chan, S., ‘The fallacy of the multi-API culture conceptual and practical benefits of representational state transfer ( )’, Journal of Documentation, 71(2), pp. 233–252. 2013. doi: 10.1108/JD-07-2013-0098, Accessed on: June 12, 2019
5. Varga, E., ‘Versioning REST APIs’, Creating Maintainable APIs, 2016. [Online]. Available: https://search-ebscohost-com.liverpool.idm.oclc.org/login.aspx?direct=true&db=edb&AN=120919656&site=eds-live&scope=site Accessed on: June 9, 2019
6. Hardin, S. ‘Tim Berners-Lee: The Semantic Web - Web of Machine-Processable Data’, Bulletin of the American Society for Information Science & Technology, 31(3), pp. 12–13. 2005. doi: 10.1002/bult.1720310305, Accessed on: June 14, 2019
7. Van de Sompel, H., Nelson, M.L., Sanderson, R., Balakireva., L.L., Ainsworth, S. & Shankar, H., ‘Memento: Time Travel for the Web’, 2009. [Online]. Available: https://search-ebscohost-com.liverpool.idm.oclc.org/login.aspx?direct=true&db=edsarx&AN=edsarx.0911.1112&site=eds-live&scope=site, Accessed on: June 13, 2019
8. Sauermann, L., Cyganiak, R. & Völkel, M., Cool URIs for the Semantic Web, 2006. [Online]. Available: https://pdfs.semanticscholar.org/0db4/92409aa2047142184fb4d2a1dd889cef074f.pdf, Accessed on: June 12, 2019
9. Melendez, W. A., ‘The upper layers of the ISO/OSI Reference Model (Part II)’, Computer Standards and Interfaces, 20(4–5), pp. 185–199. 1999. Available at: https://search-ebscohost-com.liverpool.idm.oclc.org/login.aspx?direct=true&db=edselc&AN=edselc.2-52.0-21844469752&site=eds-live&scope=site, Accessed on: June 16, 2019
10. W3C, Hypertext Transfer Protocol -- HTTP/1.1, 1999. [Online]. Available: https://www.w3.org/Protocols/rfc2616/rfc2616.html, Accessed on: July 16, 2019
11. Preston-Werner, T., Semantic Versioning 2.0.0, [Online]. Available: https://semver.org, Accessed on: June 26, 2019
12. Raemaekers, S., van Deursen, A. & Visser, J., ‘Semantic Versioning versus Breaking Changes: A Study of the Maven Repository’, 2014 IEEE 14th International Working Conference on Source Code Analysis and Manipulation, Source Code Analysis and Manipulation (SCAM), 2014 IEEE 14th International Working Conference on, Source Code Analysis and Manipulation (SCAM), 2013 IEEE 13th International Working Conference on, p. 215. doi: 10.1109/SCAM.2014.30, Accessed on: June 25, 2019
13. Reimanis, D. & Izurieta, C., 'Towards Assessing the Technical Debt of Undesired Software Behaviors in Design Patterns', (2016) 2016 IEEE 8Th International Workshop On Managing Technical Debt (MTD), Managing Technical Debt (MTD), 2016 IEEE 8Th International Workshop On, p.24, IEEE Xplore Digital Library, EBSCOhost, Accessed on: June 21, 2019