About this paper
This is offline discussed via PSWG members, as long term goals PSWG starting from 2nd half 2022. Welcome any kind of contribution to this paper and any project mentioned in this paper. We will use this paper as roadmap for our sample project as performance sandbox.
In future, the Working Group plans to discuss technological in greater detail and to offer additional guidance on standard procedures and emerging best practices for evaluating blockchain performance. To provide your feedback and stay informed about subsequent versions of this paper, please join us in the Performance and Scale Working Group.
Introduction
Infra
We found k8s is becoming a standard infrastructure for software and which means we are able to have a vertical digram for how people deploy blockchain system.
As figure above, in the following parts of this article, we are going to discuss blockchain performance and scale related question base on k8s and hardware area. As most k8s based tool as observability related tools also support no k8s deployment, hence we don't specific case for non k8s deployment. You are free and able to use those tools to adopt with your no k8s deployment.
Performance
Learn from CNCF paper, we can give a steps for blockchain adopt with cloud native and with observability's help to improve performance or handling anything related with performance.
Steps to adopt the observability
1. making blockchain able to deploy on k8s.
many blockchain systems supports docker based deployment and for this is the 1st step here to move your blockchain from local container runtime to k8s.
2. making blockchain able to deploy on k8s with observability related tools.
there two things need to be done in this step. 1st is to implemented observability api basing on your blockchain network. For example metric and distributed tracing. 2nd is to update your blockchain deployment yaml to support observability injection for example, sidecar based/operator based observability tools as prometheus operator.
3.a making performance testing tools able to deploy on k8s in a distributed way.
In short, both blockchain system and test harness should be deployed on k8s and integrated with observability.
3.b display metrics on monitoring system for example grafana.
Once finished with dashboard setting, then start with correlating-observability-signals
Best practices for adopt with observability
Taking Hyperledger Fabric as sample.
1. Hyperledger Fabric on k8s(a short description for fabric operators)
There lots of project deploy Hyperledger Fabric on k8s, for example, https://github.com/hyperledger-labs/fabric-operator, even so far performance sandbox still working with shell script based k8s-test-network, but as a long term plan, we will adopt performance sand box with fabric-operator.
2. Hyperledger Fabric with observability
Hyperledger Fabric makes metrics long before, as ref
now a days, with rfc community schedules supporting opentelemetry.
3. Hyperledger Fabric performance testing tools on k8s with observability.
Here we take Tape as sample.
Tape is a Simple Traffic Generator for Hyperledger Fabric, it takes 3 steps from zero to integrated deployment on k8s with observability.
3.1 distributed
To make test harness support distributed, you need to split traffic generator and monitoring client. Making them able to run in split mode.
3.2 metric
For test harness, we are able to expose metrics for performance research usage. For example, when we want to calculate latency, there is one way to make a latency report at test harness instead of tracing metrics for each node.
3.3 tracing
You are able to use labels and grpc options to support distributed tracing among test harness.
4. dashboard
Transaction throughput for Hyperledger Fabric, is the rate at which valid transactions are committed by the peers in a defined time period. Note that this is not the rate at a single node, but across the entire SUT, i.e. committed at all nodes of the network. This rate is expressed as transactions per second (TPS) at a network size. So which means, at least, peers from each organization participate in specific channel as the block broadcasting.
ref to https://www.hyperledger.org/learn/publications/blockchain-performance-metrics
Transaction Throughput = Total committed transactions / total time in seconds @ #committed nodes
In the test data with about 10k transaction, in 1k block.(as 10 transactions in one block following configuration.)
We can find the ledger blockchain height rate as block height/time(as rate) and among the duration in time zone. Some how it shows the throughput for transactions base on block height. Or we are able to use transactions in specific channel. As Business transaction per channel, chaincode shows. How the transactions increase during the time.
Hardware
to be finish by Haris
Scale
Just answer one single question, how to testing a system with thousands of nodes?
It seems that we got one answer learn from CNCF on a recording for kubecon.
As we test scaling with mock system, for detail:
When we deep dive blockchain node, we are able to split features or modules inside a node.
- internal features
In this area, when we scale up size of blockchain system, it just linear increasing.
for example, an engine to running a specific chain code. For most blockchain system, the node should be able to running chaincode for execution and business logic.
However, for scale up, it just linear increasing the resources.
Unfortunate our research environment is resource limited, hence to do the research or just simulate a large size of network, we need to mock this part of features in our mock system for example making it always return success for any response.
- external communication features
In this area, when we scale up size of blockchain system, it is the key features been impacted by size.
for example, most of blockchain system relay on p2p network or gossip network.
With scale up the sizing of blockchain network, the performance of p2p network will impact the performance of blockchain system.
Auth
Sam Yuan(Yi Yuan )
to be update by Haris