GraphQL Revisited: Adoption in Blockchains

GraphQL Revisited

Key Takeaways

1. GraphQL is an open-source data query and modification language for APIs leveraged by several prominent tech shops such as Facebook (original creator), Github, Pinterest, Intuit, Coursera, Paypal, Yelp, and Shopify to name a few. It has gotten a lot of positive attention from the engineering community and supporters have termed it as 'Better REST,' asserting a range of benefits over traditional REST.
2. With all the buzz around GraphQL, do developers need to make an active shift towards it and deprecate REST endpoints completely? It depends on many factors, which are discussed in this article. While GraphQL is an extremely powerful and flexible API strategy, it is not a silver bullet for all your data CRUD needs. You should evaluate your application needs and developer skills to make the right call (by using a skills assessment like this one: https://pluralsight.pxf.io/doLnDj).  
3. GraphQL adoption (switching from REST) usually requires a major rewrite of the API and Client layer for your application. While there are material benefits to make the switch, depending on the size and complexity of your app, this can be a massive undertaking in terms of time and resources. There is also a learning curve with GraphQL and its best practices which should be taken into account before taking the leap.
4. At the same time, GraphQL can remarkably simplify/optimize your data access and modification needs for both client and server-side engineers, regardless of the languages or environment you’re in. If you’re writing an app from scratch and/or not afraid to try something new, GraphQL presents itself as a great option with many compelling reasons to use it. It can vastly improve the performance of your app and developer experience.
5. If you decide to leverage GraphQL, don't rush on your graph schema design. Measure twice and cut once. Mapping a good GraphQL schema is a non-trivial task. So take your time and try to get it right the first time around; it will save you from a lot of inconvenience down the line.
6. GraphQL is seeing increased adoption in the Blockchain and Cryptocurrency space. Due to the distributed nature of blockchains, querying on-chain data is a hard problem and it is being simplified using GraphQL. With Graph APIs, viewing data across different crypto exchanges, markets, assets, and on-chain data can all be done in a single query.

Introduction

GraphQL is a data-query language created by Facebook that went open source in 2015. It provides a completely understandable description of the data in the API and enables clients to function in a declarative style to fetch exactly what they need — nothing less, nothing more. In this article, we will summarize how it works, why you should use it, what are some of its drawbacks and how it is increasingly being adopted in the Crypto space.

How Does It Work?

GraphQL enables you to fetch (or modify) all the data on a server in one go. Your client can make HTTP requests to the /graphql endpoint by providing query, variables, and operationName. For example, the request would look something like this:

xxxxxxxxxx

{

        “operationName”: “operation name”,

        “query”: “query string”,

        “variables”: “variables”

}

GraphQL allows for three kinds of operations:

• Query
• Mutation
• Subscription

GraphQL operations are either a query (read), mutation (write), or subscription (continuous read). Each of these operations is only a string that needs to be constructed according to the GraphQL specification. Once this operation reaches the backend application, it can be interpreted against the entire GraphQL schema there and resolved with data for the frontend application.

1. Query

Query enables the client to fetch data from the server. They are comparable to the GET calls in traditional REST. A GraphQL query is a string that is sent to a server to be interpreted and fulfilled, which then returns JSON back to the client. At a very basic level, each query contains fields and arguments. The shape of the query is of the same shape as the result. This is a key feature in GraphQL because you always get back what you expect, and the server knows exactly what fields the client is asking for. Let’s take the example of the ‘Ticket’ resource given below. The query call is made from a web IDE called Graphiql that helps you to test and structure your queries against your server. We have defined a query that pulls ticket fields by passing in the ticket ID as an identifier argument.

Queries can also parse through dynamic arguments which can be passed in JSON format as variables. In the example above, instead of hardcoding the ticket ID, you could pass in a variable $id. Given below is the query network call as seen in the browser dev tools.

2. Mutation

Just reading data from the server is not enough. We also need the ability to create, update and delete data from the server. In REST, this is accomplished by POST with a payload (usually JSON) that is passed to the server. In GraphQL this is solved through Mutation. Mutations enable the client to modify data on the server-side. In our example around 'Ticket,' we will replace the query with the mutation keyword. The corresponding mutation type also needs to exist on the server-side. This is an example:

In our example, the createTicket mutation accepts two arguments for creating a ticket: title and author_id. On ticket creation, we return the $id of the newly created ticket. Just like the query, the mutation is a root object type. Mutations, for the most part, are very flexible and can return whatever you desire: scalars such as int, string, bool, and core types like the Ticket, or even custom response objects. Similar to queries, if the mutation field returns an object type, you can ask for nested fields. 

3. Subscription

Subscription enables the client to fetch real-time updates from the server. You can think of subscriptions as analogous to continuous polling mechanisms. It makes it possible for the server to stream data to all the clients that are listening or 'subscribed' to it. Just like queries, subscriptions allow you to read data. Unlike queries, subscriptions maintain an active connection to your GraphQL server, most commonly via WebSocket. This enables your server to push updates to the client over time. Executing a subscription creates a persistent function on the server that maps an underlying Source Stream to a returned Response Stream. You can define available subscriptions in your GraphQL schema as fields of the Subscription type.

So when should you use subscriptions? In most use cases, you should not require subscriptions. Your client can stay consistent with the backend by querying intermittently, or re-execute queries on-demand based on triggers/actions from the user. Subscriptions are a great for:

1. Small incremental changes to large objects: If your backend object is continuously being updated, re-querying constantly from the client can get expensive and slow. For example, consider the stock trading apps — Robinhood, WeBull, Fidelity, etc. The ticker price for a given stock is constantly fluctuating. Rather than persistently querying the server, subscriptions would provide a much more elegant solution. You can fetch the initial state of the object with a query, and your server can proactively push updates to individual fields as they occur.
2. Low-latency, real-time updates: A chat application is a good example where all the chat participants need to receive messages as soon as they are posted. 

Why Use GraphQL?

GraphQL may be a good candidate for your API layer if your application has a data domain with highly interrelated, nested, or traversable concepts. These are the types of relationships that are difficult to model with a RESTful API and usually result in repeated round trips to the backend. Other reasons to consider GraphQL are: You want to give clients control over what data (fields) are returned, you want to reduce the amount of data sent per request (related to clients asking for what they want), leveraging information about what the client requests (query) allows you to more efficiently load and serve the data.

Following are the benefits of using GraphQL:

• Exact data fetching — GraphQL minimizes the data that needs to be transferred over the network. Your client can specify exactly what resource properties need to be pulled from the server, reducing the overall payload and complexity of the call.
• A single request can fetch multiple resources — Unlike traditional REST, a single GraphQL query call can fetch data across multiple backend resources. This reduces the over-sprawl of endpoints on the server, which can be a big issue with REST.
• More power to the client — One of the key benefits of GraphQL is that the client has complete control of over-data fetching. Rather than the server responding with static payloads, the client can dynamically request necessary data points.
• Schema stitching — Great fit for complex systems with microservices. By integrating multiple systems behind its API, GraphQL unifies them and hides their complexity.
• A highly reusable code means that everyone benefits from everyone else's work. When types (schema objects) get added, everyone gets to use them without writing a single line of new code.
• Enables parallel processing — GraphQL allows for loading fields at the same level in parallel. For example, for a given user if you want to fetch medical history and employment history, you can load those in parallel.
• Discovery of types, i.e., figuring out what other people have already made — is easy when using tools like Voyager, GraphQL Playground, and Graphiql. This helps in reducing duplicate code and helps achieve the DRY principle.
• First-class server-side rendering support lets you get stuff done faster. See Server-side rendering with GraphQL for detailed explanations and examples.
• Amazing tooling and community — GraphQL Playground and Graphiql are just some of the many open-source things available for use. Also, check out IDE integrations like JS graphql.
• Deferred Resolvers give us a real way to attack the n+1 problem, which can be hard to solve in traditional REST.
• Deprecate API's on a field level — The client receives a deprecation whenever a field is marked to be deprecated. Once all the client dependencies are updated/removed the field can be safely deprecated.
• Great ergonomics — Write all your code in the same file; queries go right next to your markup. Need more data? Add it to your query and you're done.
• Very rapid prototyping and iteration when working with types (schema objects) that already exist in the GraphQL ecosystem.
• Support for serverless applications — Running the GraphQL backend (except Subscriptions) on a serverless cloud function works really well. Since GraphQL exposes a single endpoint, you can run your entire GraphQL server off a single cloud function.

Why Not Use GraphQL?

There are many use cases where it's quite appropriate to GraphQL, but here are some instances where it's not:

• Authentication is usually handled by headers/hashing (stateless) or by specific service endpoints that then set up authenticated sessions or provide tokens for use with the API.
• File Uploads have been a major pain point in GraphQL. You could send a base64 string with a mutation, but large files result in large (and therefore unwieldy, slow to process) strings. A dedicated endpoint for uploads is more practical. Another option can be multipart request extension GraphQL mutations as explained here.
• Dynamic Connections. This can happen with generic key/value pairs where the value amounts to a foreign key. The nature of those connections will vary from query to query.
• RPC-Style Operations. Mutations are expected to execute a query and return those results. This can be an unnecessary overhead for some asynchronous RPC-style operations.
• Caching can get difficult since the requested fields with each query can change; it uses a single endpoint that can return different kinds of payloads vs REST where there are multiple endpoints, and the response payload remains constant. The problem is partially solved by using persisted GraphQL queries that assist in producing a JSON file for mapping queries and identifiers.
• Potential for a single point of failure. Since the entire resource (and its fields) are defined on a single endpoint, any breaking change on the type/endpoint will negatively affect multiple client apps consuming it.
• Since GraphQL is not a versioned API the process for designing additions to the graph is more rigorous. GraphQL server implementations do support @deprecated directive OOB which helps.
• Overkill for small apps. GraphQL is a good fit for complex systems with multiple microservices, but for simple apps, it might be better to go with the tried and tested REST architecture.

One important thing to keep in mind is that by opening up the ability for clients to query across the domain space and relationships, you add the possibility of very complex data loading. GraphQL gives a client complete freedom to request whatever it needs. This can get contentious since the client can potentially request many fields across multiple resources and thus causing sluggish network calls. This is similar to how queries in SQL can get very complex and expensive. Add in the recursive relationships that can exist in the graph and some queries can tax your system.

GraphQL Adoption in Blockchains

In today’s age, our data is stored on private servers, data centers, or on cloud infrastructure from trusted institutions. The world wide web, in its current state, is based on client-server architecture and continues to practice centralized data management.

These servers are protected by firewalls and unauthorized data access on a server resembles breaking into a house, where security is provided by a fence and an alarm system. The drawbacks of this approach are single points of failure, which has been exemplified by recent data breaches across online platforms and service providers over the last decade.

This collective of the world wide web along with current data-exchange protocols has been termed as Web2. The Web2 brought producers and consumers of goods, information, and services together and has enabled global P2P interactions, but always with a middle entity: a platform that acts as a trusted intermediary. While these platforms have been critical in the creation of the P2P economy towards content management and value settlement layer, they also decide the transaction rules and have access to data being exchanged.

What are Blockchains/Web3?

Blockchain is a data store that is fundamentally distributed in nature. A blockchain, at its core, is an append-only data structure of immutable values, i.e., a chain of blocks. It provides a unique set of data (a universal state layer) and value settlement layer that is collectively managed. This allows for safe immutable data exchange with true P2P transactions without intermediaries or a middleman. This next-generation distributed model of the internet based on blockchains is also referred to as Web3.

From a security point of view, blockchain protocols are designed in a way that you would need to break into multiple houses around the globe simultaneously, which each have their own fence and alarm system, making it unreasonably expensive (read 51% attack). Within Web3, data is replicated across multiple copies of a P2P network, also known as a public ledger. The management rules for this public ledger are formalized in the protocol and secured by the majority consensus of all network participants or voters, who are incentivized with a native network token for their activities. Blockchain, as the backbone of Web3, redefines the data structures in the backend of the Web. By introducing a democratic governance layer two people who do not know or trust each other can transact without an explicit need of an intermediary.

Each block in a blockchain is immutable and unchanging. We achieve mutability by having variables that differ in value block to block such as smart contract or account balance. By tracing the value of a given variable through the blocks, we get audit logs for free. This allows external processes to directly depend on and trust this data with confidence.

Blockchain technology is extensively used in the financial industry with primary application in the creation of cryptocurrencies, record keeping, digital notary, and smart contracts. As of writing this article, there are 8000+ Cryptocurrencies in circulation with a total market cap of more than a trillion dollars.

Ok, I'll Bite. What Does GraphQL Have to Do With Blockchains?

While conceptually similar to databases, querying blockchains is a much harder problem to solve. This is because data is stored on a network of nodes and each node has a full copy of the ledger (sequential storage of chain data) and each transaction has to be signed off by every other node in the network. Due to the distributed nature of blockchains, the consensus protocols, and cryptography/data opacity, there is a higher performance cost when it comes to storing and querying data. Performance is only part of the issue with retrieving data from blockchains. To make matters worse, there is no standard query language and developers have to resort to writing custom code.

In traditional web architectures, GraphQL runs over a remote server's dataset. But in the world of blockchain, there is no centralized server. It is a distributed ledger where data is replicated across many nodes. Client schema definition enables the use of GraphQL as an abstraction layer over a collection of contracts. Using this approach we can resolve queries against the blockchain as if it was a traditional server. By defining a standardized way of doing this indexing and serving queries deterministically, developers will be able to run their indexing logic on public open infrastructure where security can be enforced.

Following are some promising applications of GraphQL in blockchains and cryptocurrency:

EIP-1767

EIP-1767 (Ethereum Improvement Proposal) is one of the early specifications of a GraphQL schema for accessing data stored on an Ethereum node. It aims to provide a complete replacement to the read-only information exposed via the present JSON-RPC APIs provided by all Ethereum clients while improving usability, consistency, efficiency, and future-proofing.

It is based upon the prior work on EthQL — a server that exposes a GraphQL endpoint to the public Ethereum ledger. EthQL is built in TypeScript, and thus leverages the vast ecosystem of GraphQL tooling while preserving compile-time type safety.

Advantages of EIP-1767

• Streamlines complex reads — Depending on the use case, it is easy to consolidate queries on different contract function calls, different contract addresses for different wallet addresses into one single GraphQL query.
• With GraphQL alias, one can efficiently aggregate multiple RPC calls for different data into one GraphQL query and let it do the heavy lifting behind the scenes.
• Not reliant on dependencies — No additional libraries are necessary to utilize GraphQL, i.e., code will be more stable and less vulnerable to security loopholes from expired dependencies.

TheGraph 

The Graph is a protocol for building decentralized applications (dApps) quickly on Ethereum and IPFS using GraphQL. The Graph also has an open source implementation, which can be used to build and publish open APIs called Subgraphs, making data easily accessible. All data is stored and processed on open networks with verifiable integrity. The Graph makes querying this data fast, reliable, and secure.

Running an Ethereum Node can take a few hundred gigabytes of storage on the device. Having said that, Ethereum Smart contracts can take thousands of gigabytes of storage, making API management crucial to adoption. While blockchains and storage networks are critical components of the stack, data is rarely stored in a format that can be consumed directly in applications. By using The Graph, developers can query a peer-to-peer network of indexing nodes using GraphQL and verify the results on the client. This allows teams to invest in their dApps’ core functionality.

The Graph has also open sourced all their main components: 

1. Graph Node — An implementation of an indexing node built-in Rust that event sources the Ethereum blockchain to deterministically update a data store that can be queried via the GraphQL endpoint.
2. Graph TS — TypeScript/AssemblyScript library for writing subgraph mappings to be deployed to TheGraph.
3. Graph CLI — Command-line tools for speeding up development.

How does the EIP-1767 specification compare to The Graph?

The Graph enables the development of custom ETL schemas over Ethereum events. This makes it possible to efficiently index and query contract-specific data.

In contrast, this EIP-1767 specifies a GraphQL schema for presenting the node data that’s already stored and indexed by nodes and provided via the JSON-RPC interface, with the ultimate goal of replacing JSON-RPC. This is particularly useful for tools that consume blockchain data en-masse, like The Graph.

BitQuery

BitQuery is a blockchain data service that provides actionable GraphQL APIs for 30+ blockchains. Their unified Graph schema makes it easier to aggregate data and build solutions that need to support multiple blockchains.

Following are some of its key features:

• Single endpoint for 30+ blockchain datasets.
• Unified schema helps scale to multiple blockchains.
• Support for smart contracts and tokens, DApps, DEX, and DeFi applications data.
• Supports smart contracts and token implementations, including NFTs on Ethereum, Tron, EOS, etc.
• Simple GraphQL schema as API documentation.
• Subscription capability for application monitoring.
• Powered by big data warehouse technology and robust ETL pipeline.

Using the Bitquery editor, developers can write GraphQL queries and also share queries with anyone; for example, this query returns the top 10 DEX trades based on USD amount. The editor and queries are powered by BitQuery Infra, where more than 160 TB of blockchain data is currently processed. BitQuery has also enabled GraphQL charting for visualizing results in the IDE directly as seen in this query.

Blocktap

Blocktap uses GraphQL to query digital assets. Their APIs enable viewing data across exchanges, markets, assets, and Bitcoin on-chain data from a single query. 

• Price History
• Time Series
• Bitcoin On-Chain Metrics 
• Tickers

Blocktap queries are separated into the following high-level object types.

• Asset: A coin, token, or another asset.
  • Bitcoin: Additional information for Bitcoin.
• Assets: An array of asset objects.
• Exchange: An organization that allows trading via markets.
• Exchanges: An array of exchange objects.
• Market: A single trading pair on an exchange.
  • OHLCV: An array of OHLCV data belonging to a market.
• Markets: An array of market objects.
• Time Series: Any data that changes over time. Includes market, asset, and exchange metrics.
  • Bitcoin Time Series: Additional time series data for Bitcoin.

Blocktap.io aims at providing institutional-grade cryptocurrency data. At present:

1. ~54k collections per second.
2. ~25 Billion total traded collected.
3. ~60 Billion total time-series data points collected.
4. 1222 digital assets available to query.
5. 7423 markets available.
6. ~39 billion queries per month.

Closing Thoughts

GraphQL shifts your focus, from thinking of data in terms of resource URLs and traditional REST endpoints, into a graph of objects and the models used in apps.

GraphQL provides some significant benefits over RESTful architecture. It can substantially simplify and optimize your data retrieval and modification requirements while allowing engineers to deliver faster. Having said that, it is not a panacea for your data access needs and the devil is in the detail. If chosen for the wrong reasons or not implemented correctly, it can negatively affect your application and developer experience which is why its imperative to up-level you team before departing on that voyage.

Finally, GraphQL is seeing significant adoption in blockchain technology. Decentralized Apps (dApps) are client-heavy, i.e., client browser apps interfacing with smart contracts over a blockchain. In such an environment, the catalyst for developing the technology stack of Web3 will be the needs and conventions of front-end engineers. The primary benefit of GraphQL is to give the client complete control, making it very well positioned to be the API interface of choice for blockchains. This is further exemplified by initiatives such as EIP-1767, TheGraph, BitQuery, Blocktap, and others which are already available for commercial use.