Modularization of Hyperledger Besu - can we make Besu more flexible by factoring it into decoupled components which can be exchanged for alternate implementations.?

Goal of this document: 

• Starting a conversation about modularizing Besu.

• Keeping track of the discussions.

General context

We are getting various signals that the future of blockchain technologies is all about modularity.  If L2 chains on top of L1 chains are the future, how can we make an L1 client that can be composed from various implementations of sub-components.?  We also see evidence of this elsewhere as well - The Merge separated consensus from execution. MEV actors like Flashbots separate proposing a block from building it. Even within clients, we see teams like Erigon re-writing their client in different languages, and combining the best performing subcomponents regardless of language.

Apart from the general direction of blockchain, software has been trending away from monolithic implementations, in order to maximize developer efficiency, and reduce change fatigue. Smaller components can reach stability more easily than large monoliths can. 

Current monolith structure - password: hyperledger

Goals 

The goals of this work are to expand the contexts in which Besu can be valuable to users and operators while reducing tech debt in maintenance of the code base and its release process. New use-cases require client modification (customization of Besu's rules). New use-cases may also want some, but not all of the functionality that Besu provides. These use-cases may also want an easy way to package and distribute their work with Besu's permissive licensing.

To support flexibility and development of Besu in novel context's going forward, the client needs a new approach to its existing monolithic architecture. Today, the protocol schedule defines how the monolith operates, with different sets of rules, but is unwieldy and hard to modify. We need a new approach that allows for the evolution of the client without the baggage of the monolithic approach. 

Enter modularity.

The modularity work can be largely set against three goals:

1. Resolution of tech debt - to support today's existing use-cases in Besu, we have an unwieldy monolithic approach. This is becoming hard to manage and should be addressed (with the added benefit of the two below goals).
   1. Incremental, mergeable, no big bangs, review cycle to warrant inclusion  
2. Better Distribution - Tailoring the code-base by customizing a set of modules of "Besu" to provide users exactly what they need in context. I.e. I need a private network distribution of Besu, so I need PoA consensus, but not PoW validation rules. These distributions can also have their own CI/release process to adjust testing definition and quality standards. These can also be for individual modules like the EVM. 
3. Better Client Modification - Tailoring the run-time to suit user/developer needs, allowing for deep customization of the client, its rules, and components. Here we have some approaches:
   1. Plug-ins and the plug-in API - Using the existing plug-in API, developers can inject modifications into the Besu code at start up that replaces the "vanilla" rule-sets and functionality. It also allows for new components to interface with Besu via the API, but does not allow for whole-sale swapping of components.
   2. Modules - Creating boundaries and interfaces in the code-base so Besu's existing components can stand-alone and/or be replaced with like modules. This opens up the client to flexibility of its modules, like replacing the EVM or consensus mechanisms with novel ones (or a new storage/peering stack, etc.). This can help with tech debt, but is a heavy handed approach to customization.
   3. Remote APIs - It is currently possible to drive a blockchain purely over the rpc-apis, using the Engine API developed to facilitate Proof of Stake. This approach could be expanded to allow for other use cases that want to interact with the blockchain.

The latter two goals are somewhat linked. Distributions can be tackled with any client modification approach. APIs vs. modules, however, should be considered as differing tracks. 

Potential Benefits

With the above in mind - we outline some potential benefits:

1. Releases - finer grained components could have a finer grained release process, speeding up the release cycle.

   ...

   1. 1. Distributions can be cut more easily for specific use-cases, based on what components and customizations to the base client are needed (Mainnet, ETC, Private nets, standalone EVM)
   2. Customization - Modular components with plug-ins enable customization of the client to fit the needs of different use-cases in a clean way, with well defined APIs and module boundaries
      1. Linea Rollup definition, using plug-ins and novel components to allow Besu to operate a L2 network. Distributing these changes in a repeatable, reliable way to users and node operators.
   3. Increases pace of innovation - experiments and prototypes become much easier, faster, and lower risk to pursue.
      1. New use-cases for Besu can be piloted quickly, without maintaining complex forks of the Besu client 
   4. End User Control - software modularity should lend itself easily to greater customizability for the end user

   ...

   1. . 
      1. Client modification can be done easily by swapping or altering components. Altered components are Besu at their core, but the plug-in system changes their behavior. Modular components may be Besu components or completely novel components that work with Besu via documented interfaces (i.e. a Rust EVM).
   2. Reduces cognitive complexity - better defined scope for contributors to target a specific part of the codebase.  New developers can focus more narrowly, and get up to speed faster with fewer distractions.

   General Concerns and Challenges, Possible Mitigations

   1. Engineering effort around Besu
      1. Large engineering effort  effort - we will need to always prefer incremental delivery over greenfield or big-bang approaches.
      2. Series of workshops to define the work
   2. Technical project organization
      1. Communication planning
         1. Internal - how do we make sure all Besu contributors can keep their finger on the pulse of this initiative.
         2. External - do we need to convey this to external users or interested parties. If so, how?
      2. multi stakeholders discussion, federating people around modular besu. Examples of stakeholders this would benefit:
         1. MEV searchers
         2. rollup implementers
         3. infrastructure providers like Infura or Alchemy
         4. developers

   Step 1: What are Besu Minimum viable components ? Or should we think about it as a combination of modules (brainstorming / workshop session)

   Situations to assess:

   Besu Minimum Useful Components

   Hypothetical situations that would benefit from component composition:

   • Isolatable execution engine
     • EVM and state are
     Situation where only the EVM is
     • needed and not the Consensus. Ex:
     Rollup
     • Rollups, Hedera Hashgraph
     • , EVM testing tools
   • Transaction pool, transaction validation, and block gossip needed. MEV searchers. 
     • Possibly EVM needed to for gas use analysis.
   • Use case specific builds
     • All-in-one mainnet client that provides ethereum proof-of-stake as its only consensus mechanism.
   • State Synch Testbed, rapid prototyping for data stores which can be populated with state changes from a moving chain.

   Potential First Steps

   • Catalog all components 
   • Set up call with Revenant / Chupa
   • Ask folks to bring ideas around modularization
   • Find minimum viable components 
   • Catalog all components 
   • Scope MVP (minimum viable platform)
   • Test approach on one or more modules (see the timing and scoping/lessons learned)situation listed above. 
   • Extrapolate out rough timeline on MVP scope and modules timing vs the catalog 

   ...

   • catalog.
   • Scope MVP (minimum viable platform)

   Questions

   1. Plug-ins vs. modules - will we expand the plug-in API to be unwieldy and exposing too much? 

   ...

   
   

   ...

   Debrief of meeting with Erigon

   Meeting #1 - 9/14/21

   Participants: Alexey, Madeline, Sajida

   ...