Apache Ignite Transactions Architecture: Concurrency Modes and Isolation Levels

In the previous article in this series, we looked at the two-phase commit protocol and how this worked with various types of cluster nodes in Apache Ignite. Here are topics we will cover in the rest of this series:

• Concurrency modes and isolation levels.
• Failover and recovery.
• Transaction handling at the level of Ignite persistence (WAL, checkpointing, and more).
• Transaction handling at the level of 3rd party persistence.

In this article, we will focus on concurrency modes and isolation levels.

Most modern multi-user applications allow concurrent data access and modification. To manage this capability and ensure that the system moves from one consistent state to another, the concept of transactions is used. Transactions rely upon locks, which can be acquired at the beginning of a transaction (pessimistic locking) or at the end of a transaction (optimistic locking) before work is committed.

There are two concurrency modes supported by Apache Ignite: pessimistic and optimistic. Let’s begin with pessimistic concurrency.

Pessimistic Concurrency

An example of pessimistic concurrency is transferring funds from one bank account to another. We need to ensure that one bank account is correctly debited and another bank account correctly credited. Locks would be acquired on the two accounts to ensure that updates are completed successfully and reflect the new balances of both accounts.

In pessimistic concurrency, applications acquire locks for all the data that need to be read, written or modified at the beginning of the transaction. Apache Ignite also supports a number of isolation levels with pessimistic concurrency, which provide flexibility when reading and writing data:

• Read Committed
• Repeatable Read
• Serializable

In Read Committed mode, the locks are acquired before any changes to the data brought by write operations, such as put() or putAll().The Repeatable Read and Serializable modes are used for situations where locks need to be acquired for both read and write operations. Apache Ignite also has built-in functionality that makes it easier to debug and fix distributed deadlocks.

The following Java code shows an example of a pessimistic transaction with Repeatable Read, as the application needs to perform both read and write operations on a particular bank account.

try (Transaction tx = Ignition.ignite().transactions().txStart(PESSIMISTIC, REPEATABLE_READ)) {
    Account acct = cache.get(acctId);

    assert acct != null;

    ...

    // Deposit into account.
    acct.update(amount);

    // Store updated account in cache.
    cache.put(acctId, acct);

    tx.commit();
}

In this example, we have the txStart() and tx.commit() methods to start and commit the transaction, respectively. The txStart() method takes PESSIMISTIC and REPEATABLE READ as parameters. Within the body of the try block, the code performs a cache.get() on the “acctId” key. Further along, some funds are deposited into the account and the cache is updated using cache.put().

The following Java code shows an example of a pessimistic transaction with Read Committed and deadlock handling.

try (Transaction tx = ignite.transactions().txStart(TransactionConcurrency.PESSIMISTIC, TransactionIsolation.READ_COMMITTED, TX_TIMEOUT, 0)) {

    // More code here.

    tx.commit();
} catch (CacheException e) {
    if (e.getCause() instanceof TransactionTimeoutException &&
        e.getCause().getCause() instanceof TransactionDeadlockException)

        System.out.println(e.getCause().getCause().getMessage());
}

In this example, the code shows how to use the deadlock detection mechanism in Apache Ignite. This simplifies the debugging of distributed deadlocks that may be caused by application code. To enable this feature, we need to start an Ignite transaction with a non-zero timeout (TX_TIMEOUT > 0) and catch the TransactionDeadlockException that will contain the deadlock details.

Let’s now look at the message flows for the different isolation levels. We will start with Read Committed, as shown in Figure 1. In this isolation mode, Apache Ignite does not obtain locks for read operations, such as get() or getAll(), which may better suit some use cases.

Figure 1: Read Committed

1. A transaction is started (1 tx.Start).
2. The Transaction coordinator manages the transaction request internally (2 IgniteInternalTx).
3. The application writes keys K1 and K2 (3 tx.putAll(K1-V1, K2-V2)).
4. The Transaction coordinator writes K1 to the local transaction map (4 Put(K1)).
5. The Transaction coordinator initiates a lock request to the Primary Node where K1 is stored (5 lock(K1)).
6. The Primary Node manages the transaction request internally (6 IgniteInternalTx).
7. The Primary Node sends an acknowledgment to the Transaction coordinator (7 ACK) that it is ready.
8. Steps 4 to 7 are repeated for K2, as shown in Figure 1.
9. A transaction commit is requested (12 tx.commit).
10. K1 and K2 are written to their respective Primary Nodes (13 Write(K1) and 13 Write(K2)).
11. The Primary Nodes confirm that the transaction has been committed (14 ACK).

Next, let’s look at the message flows for Repeatable Read and Serializable, as shown in Figure 2.

Figure 2: Repeatable Read and Serializable

1. A transaction is started (1 tx.Start).
2. The Transaction coordinator manages the transaction request internally (2 IgniteInternalTx).
3. The application reads keys K1 and K2 (3 tx.getAll(K1-V1, K2-V2)).
4. The Transaction coordinator starts processing the K1 read request (4 Get(K1)).
5. The Transaction coordinator initiates a lock request to the Primary Node where K1 is stored (5 lock(K1)).
6. The Primary Node manages the transaction request internally (6 IgniteInternalTx).
7. The Primary Node sends an acknowledgment to the Transaction coordinator (7 ACK) that it is ready and transfers the requested value for K1.
8. Steps 4 to 7 are repeated for K2, as shown in Figure 2.
9. The application writes keys K1 and K2 (12 tx.putAll(K1-V2, K2-V2)).
10. The Transaction coordinator writes the K1 update to the local transaction map (13 Put(K1)).
11. The Transaction coordinator writes the K2 update to the local transaction map (14 Put(K2)).
12. A transaction commit is requested (15 tx.commit).
13. K1 and K2 are written to their respective Primary Nodes (16 Write(K1) and 16 Write(K2)).
14. The Primary Nodes confirm that the transaction has been committed (17 ACK).

To summarize, in pessimistic mode, locks are held until a transaction is finished and the locks prevent access to data by other transactions.

Next, let’s look at optimistic concurrency.

Optimistic Concurrency

An example of optimistic concurrency is in Computer Aided Design (CAD), where a designer is working on part of a design and typically checks-out the design from a central repository into a local workstation, then does some updates and checks that design back into the central repository. Since the designer is responsible for a part of the overall design, it is unlikely that there are any update conflicts with other parts of the design.

In contrast to pessimistic concurrency, optimistic concurrency delays lock acquisition. This may be better suited to applications where there is less contention, as in the CAD example described above. Apache Ignite also supports a number of isolation levels with optimistic concurrency, which provide flexibility when reading and writing data:

• Read Committed
• Repeatable Read
• Serializable (deadlock-free)

Recall the discussion from the previous article on the various phases in the two-phase commit protocol. When using optimistic concurrency, during the prepare phase, lock acquisition takes place on the Primary Nodes. When using Serializable mode, if data has changed since it was requested by a transaction, the transaction will fail at the prepare phase. In this situation, the developer must code the application behavior on whether or not it should restart the transaction. The two other modes, Repeatable Read and Read Committed, never check if data has changed. Whilst these modes may bring performance benefits, there are no data atomicity guarantees and, therefore, these two modes are rarely used in production applications.

The following Java code shows an example of an optimistic transaction with Serializable, as the application needs to perform both read and write operations on a particular bank account.

while (true) {
    try (Transaction tx = ignite.transactions().txStart(TransactionConcurrency.OPTIMISTIC, TransactionIsolation.SERIALIZABLE)) {

        Account acct = cache.get(acctId);

        assert acct != null;

        ...

        // Deposit into account.
        acct.update(amount);

        // Store updated account in cache.
        cache.put(acctId, acct);

        tx.commit();

        // Transaction succeeded. Exiting the loop.
        break;
    } catch (TransactionOptimisticException e) {
        // Transaction has failed. Retry.
    }
}

In this example, we have an outer while loop so that if there is a transaction failure, it can be retried. Next, we have the txStart() and tx.commit() methods to start and commit the transaction, respectively. The txStart() method takes OPTIMISTIC and SERIALIZABLE as parameters. Within the body of the try block, the code performs a cache.get() on the “acctId” key. Further along, some funds are deposited into the account and the cache is updated using cache.put(). If the transaction is successful, the code will break from the loop. If the transaction is unsuccessful, an exception occurs and the transaction is retried. For OPTIMISTIC and SERIALIZABLE transactions, keys can be accessed in any order because transaction locks are acquired in parallel with an additional check allowing Apache Ignite to avoid deadlocks.

Let’s now look at the message flows for the different isolation levels. We will start with Serializable, as shown in Figure 3.

Figure 3: Serializable

1. A transaction is started (1 tx.Start).
2. The Transaction coordinator manages the transaction request internally (2 IgniteInternalTx).
3. The application writes key K1 (3 tx.put(K1-V1)).
4. The Transaction coordinator writes K1 to the local transaction map (4 Put(K1)).
5. The application writes key K2 (5 tx.put(K2-V2)).
6. The Transaction coordinator writes K2 to the local transaction map (6 Put(K2)).
7. A transaction commit is requested (7 tx.commit).
8. The Transaction coordinator initiates lock requests to the Primary Nodes where K1 and K2 are stored (8 lock(K1, TV1) and 8 lock(K2, TV1)).
9. The Primary Nodes manage the transaction requests internally (9 IgniteInternalTx).
10. The Primary Nodes send acknowledgments to the Transaction coordinator (10 ACK) that they are ready.
11. K1 and K2 are written to their respective Primary Nodes (11 Write(K1) and 11 Write(K2)).
12. If there are no data conflicts (that is, K1 and K2 were not updated by another application), the Primary Nodes confirm that the transaction has been committed (12 ACK).

Finally, let’s look at the message flows for Repeatable Read and Read Committed, as shown in Figure 4.

Figure 4: Read Committed and Repeatable Read

1. A transaction is started (1 tx.Start).
2. The Transaction coordinator manages the transaction request internally (2 IgniteInternalTx).
3. The application writes key K1 (3 tx.put(K1-V1)).
4. The Transaction coordinator writes K1 to the local transaction map (4 Put(K1)).
5. The application writes key K2 (5 tx.put(K2-V2)).
6. The Transaction coordinator writes K2 to the local transaction map (6 Put(K2)).
7. A transaction commit is requested (7 tx.commit).
8. The Transaction coordinator initiates lock requests to the Primary Nodes where K1 and K2 are stored (8 lock(K1) and 8 lock(K2)).
9. The Primary Nodes send an acknowledgment to the Transaction coordinator that they are ready (9 ACK).
10. K1 and K2 are written to their respective Primary Nodes (10 Write(K1) and 10 Write(K2)).
11. The Primary Nodes manage the transaction requests internally (11 IgniteInternalTx).
12. The Primary Nodes confirm that the transaction has been committed (12 ACK).

Summary

In this second article, we have looked at the main locking modes and isolation levels supported by Apache Ignite. We can see that there is considerable flexibility and choice available. In the next part of this series, we will look at failover and recovery.