Page History

...

Performance Monitoring and Observability

As figure below in performance sandbox meet up, today, from traditional monitoring as metrics only, we added more features/concepts at observability level. As distributed tracing and logging, to know what's happened among distributed system.

Monitoring, been given with more human insight, and means chart ops/ai opts there, basing on data collected by observability to interacted with systemtodo Yi Yuangive a short brief here.

Performance with k8s

Learn from CNCF paper, we can give a steps for blockchain adopt with cloud native and with observability's help to improve performance or handling anything related with performance.

...

Performance Improvements with Custom Hardware

TODO Haris Javaid

There are various ways to improve the hardware infrastructure for better performance of a blockchain system and application. Here, we provide a brief description of one such work called Blockchain Machine, done by a PSWG member (Haris Javaid) and his team at Xilinx/AMD. The Blockchain Machine explores the use of network-attached hardware acceleration for Hyperledger Fabric to improve its performance beyond what is achievable by software-only implementation on a multi-core server. It leverages FPGA accelerator cards (such as AMD Alveo), which are being increasingly adopted for accelerating cloud workloads and are also available from major public cloud providers such as AWS and Microsoft Azure.

The scalability and peak performance of Fabric is primarily limited by the bottlenecks present in its block validation/commit phase. The validation phase is run by either an endorser peer (which also endorses transactions) or validator peer. The Blockchain Machine is a hardware accelerator which is coupled with a hardware-friendly communication protocol to act as a validator peer in Hyperledger Fabric network. Hence, it is targeted for a server with a network-attached FPGA card in contrast to existing validator peers which run Hyperledger Fabric software on just a multi-core server. The Blockchain Machine peer receives blocks from the orderer through a hardware-friendly protocol, and the block data is retrieved in FPGA without any involvement of the host CPU. The extracted block and its transactions are then passed through an efficient block-level and transaction-level pipeline in FPGA, which implements the bottleneck operations of the validation phase. Finally, Hyperledger Fabric software running on the host CPU accesses the block validation results from hardware, and then commits the block to disk-based ledger just like the software-only validator peer. Overall, a Blockchain Machine peer is a hardware/software co-designed peer, leveraging both CPUs and FPGA-based accelerator cards to deliver significantly better performance than just using CPUs in a multi-core server.

For more technical details about Blockchain Machine, check out the following paper and the open-source repo:

• H. Javaid, J. Yang, N. Santoso, M. Upadhyay, S. Mohan, C. Hu, G. Brebner. 2022. Blockchain Machine: A Network-Attached Hardware Accelerator for Hyperledger Fabric. International Conference on Distributed Computing Systems (ICDCS).
• AMD. Fabric Machine Repo. Available at https://github.com/hyperledger-labs/fabric-machine

For a case study on how AMD used this technology to scale its supply chain, listen to the following talk:

• M. Kumaraswamy, H. Javaid. 2022. Accelerated Hyperledger Fabric for Supply Chain Applications in Semiconductor Industry. Hyperledger Global Forum.

How to Scale Performance Monitoring and Improvements

Just answer one single question, how to testing a system with thousands of nodes?
It seems that we got one answer learn from CNCF on a recording for kubecon.
As we test scaling with mock system, for detail:
When we deep dive blockchain node, we are able to split features or modules inside a node.
internal features
In this area, when we scale up size of blockchain system, it just linear increasing.
for example, an engine to running a specific chain code. For most blockchain system, the node should be able to running chaincode for execution and business logic.
However, for scale up, it just linear increasing the resources.
Unfortunate our research environment is resource limited, hence to do the research or just simulate a large size of network, we need to mock this part of features in our mock system for example making it always return success for any response.
external communication features
In this area, when we scale up size of blockchain system, it is the key features been impacted by size.
for example, most of blockchain system relay on p2p network or gossip network.
With scale up the sizing of blockchain network, the performance of p2p network will impact the performance of blockchain system.