...
./gradlew referenceTest
The system tests can be triggered with:
./gradlew smokeTest
The acceptance tests can be triggered with:
...
Known Failures
Currently there are 16 consensus tests, 3 sync tests and 1 graphql tests is 1 consensus test and 4 sync tests that are failing. The consensus tests are test is believed to be performance related and the sync tests involve aleth and nethermind clients which are not properly configured within the test suite.
- failing consensus tests
- modexp_37120_37111_37111_25000_d0g0v0_Istanbul
- modexp_9_3711_37111_25000_d0g1v0_Istanbul
- CallEcrecover0_NoGas_d0g0v0_Istanbul sstore_0toXtoX_d1g0v0_Istanbul
- static_callcallcodecall_010_2_d1g0v0_Istanbul
- CALLBlake2f_MaxRounds_d0g0v0_Istanbul
- sstore_combinations_initial00_d26g0v0_Istanbul
- sstore_combinations_initial11_2_d280g0v0_Istanbul
- sstore_combinations_initial11_2_d36g0v0_Istanbul
- sstore_combinations_initial20_d373g0v0_Istanbul
- StoreClearsAndInternlCallStoreClearsSuccess_d0g0v0_Istanbul
- ecmul_1-3_0_21000_80_d0g3v0_Istanbul
- pointMulAdd_d5g2v0_Istanbul
- pointMulAdd2_d25g3v0_Istanbul
- ecmul_0-3_340282366920938463463374607431768211456_28000_80_d0g3v0_Istanbul
- SimpleTxCosts20000_Frontier
- failing sync tests
- fastsync besu -sync aleth_nightly > openethereum
- sync besu -> besu_latesttrinity
- fast-sync besu _latest -> aleth_nightly
- fast-sync nethermind _latest -> besu_latest-> besu
Enterprise oriented testing
Currently the acceptance tests cover both public chain and enterprise/permissioned chains to some degree. For example "BftMiningAcceptanceTest" tests both IBFT and QBFT with basic single-node and 4-node chains, mainly submitting a small number of transactions to transfer value between accounts. The soak-testing that takes place on every Besu release does not exercise BFT chains any more than that acceptance tests that run on each PR, since the soak testing focuses on syncing with public chains.
It would be useful to have an easy way for Besu to perform more exhaustive testing of enterprise/permissioned chains on every release, and on an ad-hoc basis.
As a starting point, the following is a list of test scenarios that would be useful in validating longer running BFT chains:
- High throughput transaction tests using the sequenced transaction pool, running at high TPS for a minimum of a few hours
- "High throughput" in this context would typically mean 400+ TPS with 2-second block periods
- Single validator to prove basic performance hasn't been regressed
- Multi validator to prove any changes in the consensus code haven't degraded overall performance
- Chain & state integrity tests when upgrading between versions of Besu
- Validating that the DB/storage layer retains state correctly during an upgrade
- Chain downgrade testing
- Either validating that an accidental downgrade is prevented from occurring, or that a downgrade works correctly without loss of state
- Long running medium-TPS (e.g. 200TPS) soak test, configurable to run 1→N hours with periodic transaction submissions & state checks incorporated into the test
- Should be easy to run without complicated setup (e.g. a new gradle task that isn't included in the current build tasks but can be manually run)
- Transition between milestones over a period of time, e.g.
- Start with berlin
- Run N hours
- Transition to london
- Run N hours
- Transition to shanghai
- Run N hours
- eth_syncing