Context

Its dawned on us that Besu now represents upward of ~$5B on Mainnet in stake !! Great job for everyone https://supermajority.info/

This is a big responsibility, not just on Mainnet, but other chains that use Besu too. We are representing more and more value and reputation for a ton of organizations. The Consensys team is thinking on how we may be more careful in the release process (specifically regressions). As we approach the Cancun fork, is it OK to delay less-tested PRs on Main to avoid releasing any issues (especially in 24.1.2)? The problem is, users may not be able to safely downgrade after the fork without a manual intervention / new release. Is anything going into this next release that may be less tested? or a default that should be hidden behind a feature flag? Additionally - I wanted to crowd source ideation about regressions. We have had a handful of interventions on releases recently. Instead of just entirely slowing down releases, any ideas from maintainers? The Consensys team is scheduling a sprint to look at testing gaps and improvements that can be made in the codebase. If anyone would like to help in this, let me know! Thanks all!

Ideation

  • integrate Hive testing into CI/CD pipeline.
  • We should measure code coverage and make it a condition for merging a PR
  • benchmark metrics - sync time, block import time etc and test PRs for regression
    • some of these metrics (sync time, peering) may require long-running tests so may not be practical to run on individual PRs - could be on a nightly cadence
  • Ensure test coverage for key features eg bonsai
  • tests that run on mainnet config aren't testing upcoming hard fork features - to avoid finding bugs at the last minute, change this to holesky/sepolia (or make it configurable?)
  • burn-in on existing (synced) nodes vs syncing from scratch
    • backward sync is triggered when you upgrade
    • it would be awesome to make the (Consensys internal) burn-in process less manual.
  • don't want to make the release process more painful
  • No labels

1 Comment

  1. It would be great to have a test suite to detect performance regression of critical features. I think at the moment some features rely on extracting and comparing these metrics "manually". E.g A new PR needs to go through this suite to ensure it hasn't degraded the block import performance.