Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

fabric-test repository


Reports

Test Strategy Presentations

During the lifecycle of the hyperledger fabric project the test strategy and process changes to allow for improvements with each release. The following presentations have outlined the different strategies at different points during the project.

Attachments


Disk Space and RAM Requirements

Handling Large Transactions

The default largest blocksize is 100MB, due to a limitation in GRPC. Remember that each block on the chain is duplicated in every peer and orderer, so using such large transactions would consume diskspace quickly! It is preferable to store smaller transactions on the chain, possibly including a hash of an object stored off-chain.

...

Note: to process multiple huge msgs, it is recommended to allocate more GB for the pods, to avoid causing the peers or orderers to panic due to resource limitations.

Note: in raft networks, the orderers update the Write Ahead Log (WAL) periodically. If disk-write speeds are too slow (eg. slower than 10 IOPS/GB), then you may experience seconds of delays when processing huge messages - and possibly even dropped transactions if a leadership change is triggered due to the slow WAL disk write (which may block the raft orderer channel leader from sending heartbeats to other orderers).

How much disk space is required for a blockchain?

  • First, be aware that, in general terms, a full copy of a blockchain channel ledger is stored on each peer that joins the channel, and on each orderer, and (when using Kafka) on each kafka broker in the RF set. Also, system managers could set up storage systems to store multiple copies of the peer ledger.

  • The ones and zeroes stored in each location are not identical; the transactions in the peer are modified to contain the VSCC result (TxValidationCode code: valid, or reason for being invalid), which differentiates it from that of the orderer. And the KBs store transactions and markers, not blocks, so one estimate puts the size requirement in the KB as 10% less than the orderer and peer ledger size.

  • Beyond that, the answer is different for each blockchain. To devise an accurate number for disk space requirements, system managers must determine the average size of transactions. Multiple things impact the size of each transaction, including:

    • the endorsement policies used in the channels - which affect the number of endorser signatures attached to each transaction;

    • the average size of each signature attached by each requested endorser peer and by the orderer that delivers the transaction block (an estimate for one signature with identity and root cert is 1KB size);

    • the average size of data stored for each transaction of the specific chaincode (ranging from a few bytes to MegaBytes or larger, depending on the documents or whatever is processed by the business app chaincode).

    • And to a lesser factor, a larger batchsize for the orderer service would mean fewer blocks, which reduces overhead (note a simple block header could be 2KB - 4KB).

  • Of course, to compute space requirements, multiplication factors include the average size of each transaction, and the number of transactions you expect to process and store on the blockchain forever.

...