Unit Testing Ethereum Smart Contract In Solidity: Tips and Tricks

Unlike other software programs, smart contact, if deployed once into a specific address, can not be modified or removed. This unique constraint make the vulnerability in smart contact far more dangerous than others. So more exhaustive testing is required.

Currently the most well-known unit testing tools for Solidity unit testing are like followings

• Truffle smart contract test framework
• Remix-IDE unit testing plugin
• OpenZeppelin test environment

OpenZeppelin test environment has recently appeared, so doesn't seems verified enough, although it looks promising. Remix is one of the most powerful in editing tools. But unit testing is more proper in command-line mode than in GUI. So as of now Truffle smart contract test framework is mostly recommended.

The below image link is a Truffle unit test program for ERC-20 smart contract.

In the test program, a few tactics not contained in Truffle documentation are used. They can make test programs more effective and efficient and will be explained below.
If you are not familiar with the Truffle test framework, it's better to read the official documentation^[1] first.

[1] Truffle - Writing tests in JavaScript

Grouping Test-Cases

 Truffle test framework adopted well-known Mocha^[1]. Twisting the basic structure of Mocha test a little, Truffle test program starts with contract() function and contains it() functions as test-cases inside it^[2].

contract("ERC20Regular Contract Test Suite", async accounts => {
  
  it("Should have 'name' and 'symbol' specified at the constructor ...", async() => {
    ...
  });

  it("Should have 0 supply, not be paused, and set 0 balances for all accounts at start.", async() => {
    ...
  });

  it("Can mint tokens increasing the owners balance and total supply as much", async() => {
    ...
  });

  it("Can mint token only by minters(accounts granted minter role).", async() =>{
    ...
  });

  ...
  it("Can transfer decreasing sender's balance and increasing recipient's balance as much.", async() => {
    ...
  });

  it("Should not change balances of irrelative accounts(neither sender nor recipient).", async() => {
    ...
  });

  ...
});    

If a smart contract has tens of unit tests, such a linear structure like above becomes difficult to read, maintain and update. Mocha allows test-cases to be nested into intermediate describe() functions, so Truffle test case can use nested describe() functions to group test-cases for more readability.
In the below test program, test-cases are grouped into Initial State, Minting, Transfer, Approval, Delegated Transfer, Burning, and Circuit Breaker according to the top-level concepts of token.  These groups are describe() functions and they contains it() functions as test-cases underneath.

contract("ERC20Regular Contract Test Suite", async accounts => {
  describe("Initial State", () => {
    it("Should have 'name' and 'symbol' specified at the constructor and ...", async() => {
      ...
    });

    it("Should have ZERO supply, not be paused, and set ZERO balances for all ...", async() => {
      ...
    });
  });

  describe("Minting", () => {
    it("Can mint tokens increasing the owners balance and total supply as much", async() => {
      ...
    });

    it("Can mint token only by minters(accounts granted minter role).", async() =>{
      ...
    });

    it("Should fire 'Transfer' event after minting.", async() => {
      ...
    });
  });

  describe("Transfer", () => {
    it("Can transfer decreasing sender's balance and increasing recipient's balance as much.", async() => {
      ...
    });
    
    ...
    it("Should not change balances of irrelative accounts(neither sender nor recipient).", async() => {
      ...
    });

    it("Should not change total supply at all after transfers.", async() => {
      ...
    });

    ...
  });

  describe("Approval", () => {
    ...
  });

  describe("Delegated Transfer", () => {
    ...
  });

  describe("Burning", () => {
    ...
  });

  describe("Circuit Breaker", () => {
    ...
  });
});      

With lots of test-cases, testing newly added test-cases could be very inefficient if all the existing test-cases are also executed every time. Separating test-cases into several test programs to avoid this would cause other concerns in incoherency and maintainability. You can use only() function to run only selected test-cases you want to test with Mocha framework^[3].
If test program below be executed, only test-cases under Transfer category, of which describe() function is marked with only() , will be run.

contract("ERC20Regular Contract Test Suite", async accounts => {
  describe("Initial State", () => {
    ...
  });

  describe("Minting", () => {
    ...
  });

  describe.only("Transfer", () => {
    ...
  });

  describe("Approval", () => {
    ...
  });

  describe("Delegated Transfer", () => {
    ...
  });

  ...
});      

You can apply only() to it() functions to narrow the execution scope more. Multiple describe() or it() functions can be marked only() in a test program.
Executing the test program below, only 2 test cases starting with it.only would be run skipping all other test-cases.

contract("ERC20Regular Contract Test Suite", async accounts => {

  describe("Initial State", () => {
    ...
  });

  describe("Minting", () => {
    ...
  });

  describe("Transfer", () => {
    it("Can transfer decreasing sender's balance and increasing recipient's balance as much.", async() => {
      ...
    });

    ...
    it.only("Should not change balances of irrelative accounts(neither sender nor recipient).", async() => {
      ...
    });

    it.only("Should not change total supply at all after transfers.", async() => {
      ...
    });

    ...
  });

  describe("Approval", () => {
    ...
  });

  ...
});      

[1] Mocha : a feature-rich JavaScript test framework
[2] Truffle test in JavaScript
[3] Mocha / Exclusive Tests

Big Number

Ethereum prefers large numbers. Implicit unit in Ethereum is wei and the most representative ether is 10¹⁸ wei^[1]. 

In JavaScript, the maximum integer value for the primitive number type is about 2⁵³ (~ 10¹⁶)^[2]. So, to handle Ethereum with JavaScript, another number type for huge numbers is necessary. web3.js^[3], one of the most fundamental gadgets for Ethereum uses bn.js^[4] and bignumber.js^[5]. Datatypes of value parameter and gasPrice parameter in web3.eth.sendTransaction() function shows this. Although it is not clear why web3.js supports two different types for huge numbers, considering web3.utils.BN() and web3.utils.toBN(), it seems that BN(bn.js) is preferred.

To handle big numbers more easily inside test program, define a reference to web3.utils.toBN() function at the beginning.

const toBN = web3.utils.toBN;

BN(bn.js) API contains various functions including arithmetic operations, comparison operations, and bitwise operations. Some functions are named with postfix n meaning the operand is expected to be primitive number type.
In the following sample code, functions with normal names such as add(), div(), sub(), eq() take BN type operand and functions named with postfix n such as addn(), divn(), muln(), eqn() have primitive number type operand.

const toBN = web3.utils.toBN;
 
toBN(1E19).add(toBN(1E19));      // add BN type operand 
toBN(1E19).addn(1E5);            // add primitive number type operand 
toBN(1E19).div(toBN(1E16));      // divide by BN type operand 
toBN(1E19).divn(1E6);            // divide by primitive number type operand 
toBN(2E19).sub(toBN(1E19)).eq(toBN(1E19));  // equal to BN type operand 
toBN(2E19).muln(0).eqn(0);       // equal to primitive number type operand

[1] Ether
[2] Number.MAX_SAFE_INTEGER
[3] web3.js : Ethereum JavaScript API
[4] bn.js : BigNum in pure javascript
[5] bignumber.js : A JavaScript library for arbitrary-precision arithmetic

Random Test Data

One of the easiest ways to increase test coverage and avoid accidental test result is using random test data^[1][2][3]. 

Below is test-case to check name and symbol specified at the constructor are correctly set up and queried for a token contract. If the token contract setups the name field with hard-coded value of "RGB", although it is definitely defect, the following test case couldn't find it. The test-case uses the same literal "RGB" as test data accidentally, the test will not fail.

it("Should have 'name' and 'symbol' specified at the constructor and ...", async() => {
  const name = 'Color Token';  
  const symbol = 'RGB';
  const admin = accounts[0];
  const token = await Token.new(name, symbol, {from: admin});
  assert.equal(await token.name(), name);
  assert.equal(await token.symbol(), symbol);
  assert.isTrue((await token.decimals()).eqn(18));
});

To prevent such an accidental result, we can use random test data like the sample below. Chance^[4] is a JavaScript library to generate random data in various formats and constraints. A few functions of Chance are used to produce random sentence and word or chose an element from array.

it("Should have 'name' and 'symbol' specified at the constructor and ...", async() => {
  const chance = new Chance();
  const name = chance.sentence({words: 3});
  const symbol = chance.word({length: chance.natural({min: 1, max: 5})}).toUpperCase();
  const admin = chance.pickone(accounts);
  const token = await Token.new(name, symbol, {from: admin});
  console.debug(`New token contract deployed - name: ${name}, symbol: ${symbol}, address: ${token.address}`);

  // inquire and verify token's name, symbol and decimals
  assert.equal(await token.name(), name);
  assert.equal(await token.symbol(), symbol);
  assert.isTrue((await token.decimals()).eqn(18));
});

In the next sample, random generation is used to set amount to mint( balance), amount to transfer(delta), and accounts for sender and recipient (sender, recipient). In case of amount to transfer, chance.bool({likelihood: 10}) is used for boundary condition(zero amount) to be tried in about 10 percent.

it("Can transfer decreasing sender's balance and increasing recipient's balance as much.", async() => {
  const chance = new Chance();
  const admin = chance.pickone(accounts);
  const token = await Token.new('Color Token', 'RGB', {from: admin});
  console.debug(`New token contract deployed - address: ${token.address}`);

  // mint initial balances to all accounts
  let balance = 0;
  for(const acct of accounts){
    balance = toBN(1E19).muln(chance.natural({min: 1, max: 100}));
    await token.mint(acct, balance, {from: admin});
  }
  
  ...
  for(let i = 0; i < loops; i++){
    [sender, recipient] = chance.pickset(accounts, 2);
    senderBal1 = await token.balanceOf(sender);
    recipientBal1 = await token.balanceOf(recipient);
    
    delta = chance.bool({likelihood: 10}) ?
       toBN(0) : toBN(1E10).muln(chance.natural({min: 1, max: 1000000}));
    await token.transfer(recipient, delta, {from: sender});

    senderBal2 = await token.balanceOf(sender);
    recipientBal2 = await token.balanceOf(recipient);
    assert.isTrue(senderBal2.eq(senderBal1.sub(delta)));
    assert.isTrue(recipientBal2.eq(recipientBal1.add(delta)));
  }
});

Chance provides more than 80 functions for diverse  types or formats including number, text, date-time, location and so on. In each function, detailed facets or constraints can be set via option.

chance.bool({likelihood: 30});          // 'true' in 30% probability
chance.character({alpha: true, numeric: true, symbol: false, casing: 'lower'});
                                        // a single character among lower-case alphabet and numbers
chance.integer({min: -273, max: 10000});   // an integer between -273 and 10,000
chance.natural({max: 2048});            // a natural number between 0 and 2,048
chance.prime({min: 1E5, max: 1E5});     // a prime number between 100,000 and 1,000,000
chance.word({length: 5});               // a word in 5 length
chance.sentence({word: 7});             // a sentence with 7 words
chance.color({format: 'hex', casing: 'upper'});   // a RGB color code like '#2F3AE7'
chance.email();                         // an e-mail address
chance.ip();                            // an IPv4 address
chance.country();                       // a 2-letter country code in ISO 3166-1 alpha-2 (KR, US)
chance.locale();                        // a 2-letter locale code in ISO 639-1 (ko, en, es, pt)
chance.date({year: 2020});              // an Date object in 2020 year
chance.timestamp();                     // any UNIX Epoch time
chance.guid();                          // an UUID(GUID) - https://en.wikipedia.org/wiki/UUID
chance.pickone(['MON', 'TUE', 'WED', 'TUR', 'FRI']);   // any element from the given array
chance.pickset([1, 2, 3, 4, 5], 3);     // 3 distinct elements from the given array

[1] Random Test Data (MSDN)
[2] Random testing (Wikipedia)
[3] Unit Testing Guidelines
[4] Chance : a minimalist generator of random strings, numbers, etc. 

Revert, Event

Smart contract interacts with external systems asynchronously, so the result of transaction can not be delivered by return value. Instead, events are fired and transaction receipt is issued. To confirm more thoroughly whether smart contract behaves as expected or not, smart contract test should verify fired events. Smart contract test should also check designed or intended reverts in fail cases such as invalid input value, insufficient privilege, insufficient balance and so on.  

To check revert or events, processing transaction receipt is required after transaction is sent^[1][2]. The code may be a little verbose, so it would be useful there is convenience function. To my surprise, Truffle test framework doesn't provide anyone. But we can use the following libraries.

• truffle-assertions
• OpenZeppelin Test Helpers

Currently the 2 libraries provide similar features, the latter is preferred due to the name value of OpenZeppelin.

To use OpenZeppelin test helpers^[3][4], it is necessary to import @openzeppelin/test-helpers module.

const Token = artifacts.require("ERC20Regular");
const Chance = require('chance');
const toBN = web3.utils.toBN;
const { constants, expectEvent, expectRevert } = require('@openzeppelin/test-helpers');

To confirm the transaction has been reverted with fail test case, use expectRevert.unspecified() function.

it("Can mint token only by minters(accounts granted minter role).", async() =>{
  const chance = new Chance();
  const admin = chance.pickone(accounts);
  const token = await Token.new('Color Token', 'RGB', {from: admin});
  console.debug(`New token contract deployed - address: ${token.address}`);

  let tryer = null;
  // select any account other than admin
  do{ tryer = chance.pickone(accounts); }while(tryer == admin)

  let amt = 0;
  for(const acct of accounts){
    amt = toBN(1E17).muln(chance.natural({min: 1, max: 100}));
    await expectRevert.unspecified(token.mint(acct, amt, {from: tryer}));
  }
});

it("Can't transfer to ZERO address from any account", async() => {
  const chance = new Chance();
  const admin = chance.pickone(accounts);
  const token = await Token.new('Color Token', 'RGB', {from: admin});
  console.debug(`New token contract deployed - address: ${token.address}`);

  let balance = 0;
  for(const acct of accounts){
    balance = toBN(1E9).muln(chance.natural({min: 1, max: 100}));
    await token.mint(acct, balance, {from: admin});
  }

  let delta = 0;
  for(const acct of accounts){
    delta = toBN(1E3).muln(chance.natural({min: 0, max: 100}));
    await expectRevert.unspecified(token.transfer(constants.ZERO_ADDRESS, delta, {from: acct}));
  }
});

To verify events with pass test case, use expectEvent() function. Event arguments including parameter names and values can be confirmed.

it("Should fire 'Approval' event after approval.", async() => {
  const chance = new Chance();
  const admin = chance.pickone(accounts);
  const token = await Token.new('Color Token', 'RGB', {from: admin});
  console.debug(`New token contract deployed - address: ${token.address}`);

  const loops = 10;
  let owner = 0, spender = 0, allowance = 0;
  for(let i = 0; i < loops; i++){
    owner = chance.pickone(accounts);
    spender = chance.pickone(accounts);
    allowance = chance.bool({likelihood: 10}) ?
       toBN(0) : toBN(1E5).muln(chance.natural({min: 1, max: 1000000}));

    expectEvent(await token.approve(spender, allowance, {from: owner}),
      'Approval', {owner: owner, spender: spender, value: allowance.toString()});
  }
});

Instead of event parameter names, indexes can be specified.

expectEvent(await token.approve(spender, allowance, {from: owner}),  
  'Approval', {0: owner, 1: spender, 2: allowance.toString()});

[1] web3.eth.getTransactionReceipt
[2] Deep dive into Ethereum logs
[3] OpenZeppelin Test Helpers source project
[4] OpenZeppelin Test Helpers API reference

ECMAScript 8 (2017)

It has been quite some time since JavaScript was born, it is still evolving rapidly. JavaScript was standardized by ECMAScript and new version of specification has been announced every year since 2015^[1][2].

For more efficient working on Truffle test programs, it is important to select proper version of JavaScript that contains useful features for test. 

JavaScript is asynchronous by nature. Most frameworks and libraries including web3.js and Truffle contract abstraction run asynchronously. Programming flows with async processes handling callbacks or promises may be beneficial for performance, but it can be more complex and difficult. For test program, readability and easiness can be more important than optimization or performance.
async^[3]/ await^[4] statements enables synchronous flows of asynchronous functions, so the code can avoid callback stacking.
await statement is valid only in async block. In case of truffle test, the test function, 2nd argument of it() function, would be async function and then await call would be made inside test function.
Below sample shows all calls to token contract( Token.new, token.mint, token.totalSupply, token.transfer) are await inside a async function (async() => {}).

it("...", async() => {
  const chance = new Chance();
  const admin = chance.pickone(accounts);
  const token = await Token.new('Color Token', 'RGB', {from: admin});

  let balance = 0;
  for(const acct of accounts){
    balance = toBN(1E19).muln(chance.natural({min:1,max:100}));
    await token.mint(acct, balance, {from: admin});
  }
  const total = await token.totalSupply();
  
  const loops = 20;
  let sender = 0, recipient = 0, delta = 0;
  for(let i = 0; i < loops; i++){
    sender = chance.pickone(accounts);
    recipient = chance.pickone(accounts);
    delta = toBN(1E13).muln(chance.natural({min:0,max:100}));
    await token.transfer(recipient, delta, {from: sender});
    assert.isTrue((await token.totalSupply()).eq(total));
  }
});

In JavaScript, variable declaration using var keyword has unusual semantics such as function scoping and hoisting^[5]. This is not common feature in other programming languages and can make non native JavaScript programmers frustrated even with simple codes. ECMAScript 6 unveiled in 2015 introduced const^[6] and let^[7] statements to compensate those unexpected effect of var. Variable declaration using const and let has block scoping and no hoisting^[8]. Hoisting can actually be more complex behind scenes^[9]. But virtually there is no hoisting for const and let when compared with var.
So, it is strongly recommended to use const and let, if function scoping or hoisting is intended.
To make variable declaration and usage even less error-prone, using strict mode^[10] is also recommended. It is enough to add 'use strict' literal at the beginning line of contract() function. This simple line will remove many of error-prone old features and make you feel more comfortable.

contract("ERC20Regular Contract Test Suite", async accounts => {
  
  "use strict";
  
  if(accounts.length > 8){  // avoid too many accounts
    accounts = (new Chance()).pickset(accounts, 8);
  }
  ...
)};

async/await statements were added in ECMAScript 8, and const/let in ECMAScript 6. So, Node.js 9.11.2 or higher supporting ECMAScript 8 almost fully^[11] is recommended to utilize those statements.

[1] ECMAScript versions
[2] JavaScript versions
[3] async statement
[4] await statement
[5] JavaScript Scoping and Hoisting
[6] const statement
[7] let statement
[8] Differences Between var and let
[9] Hoisting in Modern JavaScript — let, const, and var
[10] strict mode
[11] Node.js ECMAScript compatibility tables

Ganache CLI

Unit testing can be quite tedious, so a test environment with quick execution is necessary.  Especially, with Ethereum, an environment mostly identical with mainnet as possible except PoW consensus algorithm.  Smart contract unit tests are basically independent of consensus algorithm.
So, as a test environment, testnets with PoA such as Rinkeby or Kovan or local standalone Ethereum client(node) implementations such as Ganache or Ganache CLI are preferred.

Ganache CLI is long developed, runs fast and provides various configurable options^[1] which make it useful even test environment.
The following command-line will launch Ganache CLI instance proper to smart contract unit testing.

ganache-cli --networkId 31 \
    --host '127.0.0.1' --port 8545 \
    --gasPrice 2.5E10 --gasLimit 4E8 \
    --deterministic \
    --defaultBalanceEther 10000 --accounts 10 --secure \
    --unlock 0 --unlock 1 --unlock 2 --unlock 3 --unlock 4 \
    --hardfork 'petersburg' \
    --blockTime 0 \
    --db '/var/lib/ganache-cli/data' >> /var/log/ganache.log 2>&1

[1] Ganache CLI Options