Evolution of Consensus Mechanisms: From Classical Methods to Local Consensus
From classical to local — learn more about the evolution of consensus mechanisms and distributed technology.
Join the DZone community and get the full member experience.Join For Free
The consensus mechanism is the cornerstone of distributed technology, such as the blockchain. Since the decentralized nature of systems based on such technologies
The consensus mechanism is the cornerstone of distributed technology like blockchain. Since the decentralized nature of systems based on such technologies does not provide a single center to validate transactions, we must have a mechanism that, on the one hand, allows us to reliably perform this function; and on the other, it needs to have a distributed nature. In other words, such a mechanism should provide confirmation of each transaction’s validity by reaching an agreement between all network participants (or rather all active participants – full nodes) on this issue, providing protection against a possible double-spend problem, and ideally providing the immutability of blockchain’s distributed ledger.
There are many variations of such mechanisms all of which have both their strong sides and disadvantages, and they have also evolved in some ways. The main stages of this evolution include classical methods of achieving consensus in distributed databases; Nakamoto consensus mechanisms used in modern decentralized blockchain systems; and finally to sharding and local consensus, which becomes the foundation principle of some Layer 2 and Layer 3 multi-hop projects working on the basis of state channels and trustlines.
Classical Consensus Methods
Classical consensus methods emerged long before the advent of blockchain technology in its modern form back in the 1980-90s. Since inception, these methods have mainly been used in distributed databases of varying degrees of decentralization.
- Byzantine Fault Tolerance (BFT)
- Practical Byzantine Fault Tolerance (pBFT)
- Other variations of the BFT (Q/U, HQ, ABsTRACT, Zyzzyva, Aardvark, RBFT, A2M-PBFT-EA, MinBFT)
Nakamoto Consensus Mechanisms
There are different ways to achieve consensus in modern blockchain systems. Their main shared feature is the presence of some kind of “difficulty” (work, resource, asset, etc.), which a network member must “invest” in order to be able to create a new block. At the same time, other members of the network should be able to easily verify that the work was actually done. The presence of this “difficulty” or spent resource, as well as its verifiability, must guarantee the security of the network by increasing the price of the possible attacks.
For now, various approaches to solving this problem have been developed:
- Proof of Work (PoW)
- Proof of Stake (PoS)
- Proof of Importance (PoI)
- Delegated Proof of Stake (DPoS)
- Proof of Authority (PoA)
- Proof of Burn (PoB)
- Proof of Capacity (PoC)
- Proof of Elapsed Time (PoET)
The General Consensus Against Local Consensus
As we see, despite a diversity of approaches the ideal method of achieving consensus does not exist (yet?). At the same time, most of these methods differ only in what is considered to constitute “complexity” and “work” in each of them. In other elements, they are quite similar: all the consensus mechanisms listed above set the task of reaching agreement on the state of the network, ideally, between all of its participants (or at least of all active ones). The key word here in this is the word “all.”
Precisely herein lies the reason for one of the main challenges of all existing blockchains: namely the scalability and speed of transactions. No matter what consensus algorithm — PoW, DPoS, pBFT or other — a project uses, having a commonly distributed registry, as well as the need to achieve consensus among all active network participants (full nodes), will always impose certain limits on the number of transactions per second.
Is There a Way to Overcome This Basic Constraint?
This is quite hard to imagine from the point of view of classic blockchain projects, like Bitcoin or Ethereum. After all, as we said at the very beginning, universal consensus is one of the basic principles on which all distributed blockchain technology rests, and if we move away from it, will we not actually return to centralized systems?
It turns out – no. There are ways to do this by moving away from the principle of universal consensus. One such method is the transition to sharding (network segmentation).
Sharding aims to solve the problems of scalability, delays, and throughput of transactions. This is a concept widely used in databases to increase their effectiveness. A segment (shard) is a horizontal part of a database, each of which is stored in a separate server. This distributes the load and makes the database more efficient. In the case of blockchain systems, with the implementation of the sharding solution, each node will have only a portion of the blockchain data, and not all the information of the entire blockchain. Nodes that support a segment store information only about that segment in a distributed form, so decentralization is still maintained within the segment. However, each node does not store complete information on the whole blockchain, and that contributes to scalability.
This leads to the fact that the Proof of Work algorithm cannot be used in conjunction with segmentation, because how can all participating nodes be involved in checking a transaction if each of them has information only on the segment to which this particular node belongs? Therefore, blockchains that implement sharding use the Proof of Stake consensus algorithm.
However, in general, sharding is a method more suitable for directly updating the blockchain systems of the basic Layer 1 level. But there is a more radical way to solve scalability problems, and also there is the interoperability issue of different blockchain ecosystems. This means that the implementation of off-chain solutions, i.e. a departure from the need to have a single distributed registry, makes it possible to move to a local consensus instead of a global one.
The essence of a local consensus is that agreement about a particular transaction is not achieved by the entire network, but only between specific participants in this particular transaction, through direct communication between them.
It somewhat resembles a strongly simplified version of pBFT, which we mentioned above, in which consensus is also achieved through communication between the nodes. But unlike pBFT, the local consensus involves essentially only the nodes that participate in a specific transaction. And only the final result of all transactions between these nodes (if necessary) is transmitted for writing to the usual blockchain.
But, of course, we are not talking about the transfer of value only between two neighboring nodes. Indeed, if necessary, you can create chains of such nodes, making multi-hop payments possible.
On such principles and technologies, various multi-hop projects of the Layer 2 and Layer 3 class are built, i.e. special decentralized networks that are overlays over basic blockchain ecosystems (Layer 1), which are designed to solve primarily the above-mentioned scalability problem (Layer 2 – Lightning Network, Raiden, Celer), as well as the interoperability problem (Layer 3 – GEO Protocol, Interledger). Payments in such systems are performed through a chain of state channels and/or trustlines opened between their members.
In order to understand how this actually works, let's turn to the example of the GEO Protocol. All information about the state of the opened channel between two nodes, as well as the history of all transactions between them, is stored locally on these two nodes and, if necessary, is coordinated between them each time by exchanging the one-time keys from the pre-generated key pool.
Thus, nodes can repeatedly change the balance of their composite channel (in the GEO Protocol, a state channel and trustline combination is used) without needing to refer to other nodes, not to mention all the other nodes of the network. If a problem occurs, there are local conflict resolution mechanisms, but if for some reason they have failed, the nodes can apply for arbitration to special service nodes (the so-called observers).
In the case of channel closure, its final balance is fixed by both nodes and, if necessary (in the case of the use of the state channel), is passed to be written to an external blockchain (or blockchains, if operations with several assets took place).
The nature of the first-level blockchain systems (Layer 1) is that in order for them to function properly, they are forced to put up with a multitude of design constraints, many of which cannot be overcome. In a sense, these restrictions are not only natural but also necessary. The need to reach a general consensus refers precisely to such necessary restrictions.
At the same time, to improve usability, as well as market adoption of blockchain technology, there is a need to create special overlays over basic ecosystems (Layer 2 and 3) whose task is primarily to overcome the problems of scalability and interoperability of various blockchain systems, as well as their integration with existing solutions of the real economy.
The architecture of Layer 2 and 3 systems allow the use of technologies other than those used in conventional blockchain systems. These technologies include the local consensus method.
Opinions expressed by DZone contributors are their own.