The Invariant test suite we created for the security audit mentioned in this article is available on GitHub. To learn more about smart contract security and audits, visit Cyfrin.io. A list of our public audit reports can be found here.

Neo taught us that there are two realities:

1. Base Reality — where we live, and;

2. The Matrix — a simulation of Base Reality.

Consider Ethereum Mainnet as Base Reality: the real world. No one has root access here, and the things that happen have consequences. It would be madness to YOLO deploy contracts to this environment for fear of what the consequences might be. Therefore, we must test them in a simulation before letting them roam free.

Enter the Invariant Test Suite. The Matrix/real-world simulator.

No one has root access to Base Reality. However, we do have root access to The Matrix. We get to define everything that can happen and everything that must remain true, no matter what. By classifying these things, we can start to reason about them better:

• Actions — A set of everything that can happen during a simulation.

• Invariants — A set of truths that should always hold no matter which Action or sequence of Actions is performed.

By applying sets of Actions and Invariants, we create a simulated world that leverages Foundry to sniff out crazy edge cases and hard-to-find bugs before we go anywhere near Base Reality.

The Silver Bullet?

Broken record: Audits are not a silver bullet, and no tool is either. And although AI will likely get there someday, tools can’t replace the auditing process, but they can certainly help. We use several tools to aid us when performing security audits. Recently, the most fruitful of these has been utilizing Foundry to create Invariant Tests Suites that simulate real-world conditions.

We think about Invariant Test Suites by first making some surface-level assumptions. We assume that the low-hanging fruit (access control, correct modifiers, reentrancy, typos, etc.) are covered in unit tests (We are assuming a basic level of test coverage here, by which we mean 100% of code paths are covered by unit tests). We do this because we want our suite to try to break the contract’s Invariants using only valid Actions. If we can demonstrate that a system’s truth is not true in all possible valid scenarios, we have to assume that someone will take advantage of that on mainnet, especially if they can profit from it.

Writing Effective Invariant Tests

Fuzzers are dumb, and computation is bounded. If we define a bunch of Actions, the fuzzer will randomly choose and perform them. What if it randomly picks the wrong Action at the wrong time or is pranked by an incorrect address, and the contract reverts as expected? This is low-hanging fruit. We know it should revert, and the unit tests should cover this, so ideally, we want our invariant tests to avoid calling these Actions because it wastes precious fuzz calls that could be better used on valid ones.

Therefore, one of the challenges of writing Invariant Tests is getting the most value out of them, even if the Foundry fuzzer picks an invalid Action. Let’s consider an example from Invariant testing the Chainlink Staking v0.1 contracts.

In Chainlink Staking v0.1, different states allow a specific set of actions. The state flow looks like this: Closed → Open → Closed. Before the pool can accept LINK to be staked, the pool has to be opened by the admin address. If we make the stake() Action available to the fuzzer before the pool has been opened; that Action will likely be called by the fuzzer and revert. This is expected, but as far as we’re concerned, it is a wasted fuzz call. It has reduced the overall value of the simulation since the fuzzer will not go as deep as it could have.The same goes for onlyOwner functions. If a randomly seeded address calls one of them, it will revert, wasting another call. If an Invariant Test Suite has too many of these, most calls will be wasted by needless reverts that we already know about, rendering the suite almost useless. Hence, some architectural decisions must be made to ensure these cases do not occur and that the suite is as valuable as possible.

Luckily, our space is full of pseudonymous legends who take it upon themselves to contribute ideas and level up the space. Recently, horsefacts wrote an article using WETH9 as the test target, describing how to structure an invariant test suite using a Handler pattern (since adopted into the official Foundry docs). He introduces the concept of “ghost variables” to keep track of the assumed state inside a target contract.

This is also where the Actions and Invariants we defined earlier can be described more concretely. He defines two contracts, the Handler, and the Invariants.Handler — defines the Actions.Invariants — defines the Invariants..

Writing Invariants in a Security Audit

We wrote an Invariant Test Suite during an audit of Beanstalk Farms’ Wells protocol. The Wells protocol is a framework for deploying constant function automated market maker liquidity pools, which can be configured to use any number of tokens and any function (constant product being the base case). The pool and the function are separated into different contracts, with the aim of being composable pieces in a larger ecosystem.

Luckily, the Wells protocol already used Foundry for its tests so that we could piggyback off helper functions they had already written (setupWell() and several overloaded versions for more granular setups) to deploy a set of contracts we could test.

The First Invariant

The Well (pool) uses the outsourced function ([ConstantProduct2 in this case](https://github.com/BeanstalkFarms/Wells/blob/master/src/functions/ConstantProduct2.sol)) to calculate the total supply and reserves when a shift in reserves occurs, whether from providing or removing liquidity or swapping from one token to another. The first Invariant we identified was this:

 well.totalSupply() == wellFunction.calcTotalLpSupply()

This Invariant tests that the total supply of LP tokens should always equal the calculated total supply of the wellFunction. Pretty simple. Step one is complete, but we are yet to define any Actions, so our Invariant Suite does nothing but deploy the Well so far.

The First Action — Adding Liquidity

Next, we began adding Actions to our handler. First up: adding liquidity. Following horsefacts’ WETH9 example, we have to do a few things here:

1. Seed a random address, token0 amount, and token1 amount.

2. Prank as the seeded address.

3. Bound the amounts to reasonable values and mint to the seeded address.

4. Approve the Well to spend our tokens.

5. Add the liquidity.

6. Add the address to our “ghost variable” LP list.

The function (full file here) looks like this:

/// @dev addLiquidity

function addLiquidity(uint addressSeed, uint token0AmountIn, uint token1AmountIn) public {

  // bound address seed

  address msgSender = _seedToAddress(addressSeed);

  changePrank(msgSender);

  // bound token amounts

  token0AmountIn = bound(token0AmountIn, 1, type(uint96).max);

  token1AmountIn = bound(token1AmountIn, 1, type(uint96).max);



  uint[] memory tokenAmountsIn = new uint[](2);

  tokenAmountsIn[0] = token0AmountIn;

  tokenAmountsIn[1] = token1AmountIn;

  // mint tokens to the sender

  IERC20[] memory mockTokens = s_well.tokens();

  for (uint i = 0; i < mockTokens.length; i++) {

    MockToken(address(mockTokens[i])).mint(msgSender, tokenAmountsIn[i]);

    // approve the well

    mockTokens[i].approve(address(s_well), tokenAmountsIn[i]);

  }



  // add liquidity

  uint minLpAmountOut = s_well.getAddLiquidityOut(tokenAmountsIn);

  uint lpAmountOut = s_well.addLiquidity(tokenAmountsIn, minLpAmountOut, msgSender, block.timestamp);

  assertGe(lpAmountOut, minLpAmountOut);

  // add the LP to the set

  s_LPs.add(msgSender);

}

Why “Step 6: Add the address to our “ghost variable” LP list”? This is the first example of preparing to avoid those dreaded reverts that waste fuzz calls. That ghost variable is not needed yet, but it will be when the removeLiquidity Action is implemented. This is because, for an address to remove liquidity, it must own LP tokens. Otherwise, it will revert, wasting the call. More on this later…

Running forge test — match-contract Invariants we see that so far, all is green; we have a clean bill of health. At this stage, this is what our simulation does:

1. Deploys a Well and WellFunction.

2. Adds liquidity from hundreds of randomly seeded addresses.

The first Invariant holds.

The Second Action — Removing Liquidity

Next up, we added the Action for removing liquidity:

1. Check if our ghost variable LP list is not length zero.

2. Take a seeded LP address index and an LP token amount.

3. Bound those to reasonable values.

4. Prank as the LP address obtained from the ghost variable s_LPs list.

5. Remove the liquidity.

6. Remove the address from the ghost variable list if it holds zero LP balance (if all of its liquidity is removed).

It looks like this:

/// @dev removeLiquidity

function removeLiquidity(uint addressIndex, uint lpAmountIn) public {

  if (s_LPs.length() == 0) {

    return;

  }

  // bound address index

  address msgSender = _indexToLpAddress(addressIndex);

  changePrank(msgSender);

  // bound lp amount

  lpAmountIn = bound(lpAmountIn, 0, s_well.balanceOf(msgSender));



  // remove liquidity

  uint[] memory minTokenAmountsOut = s_well.getRemoveLiquidityOut(lpAmountIn);

  uint[] memory tokenAmountsOut =

    s_well.removeLiquidity(lpAmountIn, minTokenAmountsOut, msgSender, block.timestamp);



  assertGe(tokenAmountsOut[0], minTokenAmountsOut[0]);

  assertGe(tokenAmountsOut[1], minTokenAmountsOut[1]);



  // remove the LP from the set if they have no more LP

  if (s_well.balanceOf(msgSender) == 0) {

    s_LPs.remove(msgSender);

  }

}

Note that this function returns if the ghost variable list of LPs is empty. If true, this is a wasted fuzz call to this action. This can be mitigated in several ways:Add liquidity from several addresses during the setUp() function, reducing the likelihood that the fuzzer will remove all of the liquidity before adding some more.Redirecting the call to the addLiquidity() Action in the case of a zero-length LP list.

At this stage, this is what our simulation does:

1. Deploys a Well and WellFunction.

2. Adds liquidity from hundreds of randomly seeded addresses.

3. Randomly removes liquidity from addresses with LP tokens.

Running the tests this time, we hit errors.

Our initial Invariant failed… By running the test command above with -vvv Foundry gives us a full stack trace of every Action performed, the values those Actions used, and the Invariant that failed. Very useful.

We used this to create a unit test that recreated the broken Invariant to include in the final audit report and find ways to mitigate it.

The Security Onion

This broken Invariant led us down a rabbit hole for several days, resulting in a high-severity finding that touched on the underlying math, rounding directions, and architectural quirks. As we built out the Invariant Suite parallel to our manual audit, we uncovered several more issues that we made available to Beanstalk Farms in our report. We have also open-sourced the Invariant Test Suite so that any future changes to the Wells protocol can be tested against this extra layer of defense.

After all, security is not binary; it’s fuzzy (pun intended). It comprises layers of techniques, strategies, best practices, tests, reviews, and more that will never amount to a silver bullet but build an onion. The more layers of security it has, the more likely it is to protect the core.

At Cyfrin, we believe in leveling up the security of the Web3 space. We will continue to provide Invariant Test Suites alongside our audits so that developers can learn and apply them to every smart contract system they create.

The Invariant test suite we created for the security audit mentioned in this article is available on GitHub. To learn more about smart contract security and audits, visit Cyfrin.io. A list of our public audit reports can be found here.