Project Health
- Two maintainers and one significant contributor (soon to be maintainer) work on the project.
- An ongoing mentorship project targets a new feature that will lower the entry barrier for defining workloads.
- Rocket.chat activity is mainly about the configuration and setup of Caliper.
Required Information
- Have you switched from master to main in all your repos? Yes
- Have you implemented the Common Repository Structure in all your repos? Partially
Questions/Issues for the TSC
No issues at this time.
Releases
v0.4.x patch releases.
Overall Activity in the Past Quarter
- Identifying limitations that could hinder the implementation of complex, large-scale benchmarks.
- HL Mentorship contributions: gradually implementing parts of a built-in, declarative workload module.
Current Plans
- Finish, integrate, and document the HL Mentorship project. Reach out to the community for feedback.
- Implement the common repo requirements.
- Migrate the CI/CD pipeline from Azure to GitHub Actions.
- Make the orchestration, communication, and round execution of worker services more flexible to allow for more complex benchmark implementations.
Maintainer Diversity
Attila Klenik: Maintainer
Nick Lincoln: Maintainer
Dave Kelsey: Contributor (soon to be maintainer)
Contributor Diversity
Primary activity from IBM contributions, smaller PRs being raised by the community.
Additional Information
Reviewed By
- Angelo De Caro
- Arnaud J Le Hors
- Artem Barger
- Arun S M
- Bobbi Muscara
- Danno Ferrin
- David Enyeart
- Grace Hartley
- Hart Montgomery
- Jim Zhang
- Kamlesh Nagware
- Nathan George
- Peter Somogyvari
- Tracy Kuhrt
- Troy Ronda
Submission date
$action.dateFormatter.formatGivenString("dd-MMM-yyyy", $content.getCreationDate())
5 Comments
Arun S M
Re: mentorship project, please feel free to make use of TSC meeting as well. Updating the mentorship project might bring in interest from other project maintainers.
Tracy Kuhrt
Attila Klenik : can you comment on what remains to be completed in the common repository structure?
Attila Klenik
Technically, the required files are present (pending a minor link fix), but some of them need a content update (e.g., contributing, maintainers). So no major issues, just some polishing.
Artem Barger
> Identifying limitations that could hinder the implementation of complex, large-scale benchmarks.
Is there a folder/list with suggested benchmarks or workloads that could evaluate the performance of different platforms to provide the baseline?
Attila Klenik
Artem Barger I would use the word "baseline" carefully when talking about comparing different platforms. The applied workload characteristics (its attributes and rate) can significantly influence the effective performance of a platform. So I would define a performance baseline in the scope of a specific workload using a "default" system setting (whatever that means). Then you can start tweaking the system settings and scale to up the performance in the context of the selected workload. There won't be a one-fits-all setup due to the diversity of possible workloads.
To answer the actual question: I'd suggest taking a look at the SmallBank workload in the Caliper benchmarks repo. The workload's attributes are relatively complex and well defined, and you can tune the workload rate through Caliper. It has a corresponding Fabric Golang chaincode implementation, and it probably wouldn't be difficult to create a Solidity-based implementation too (e.g., for HL Besu, or other private Ethereum client implementation, supported by Caliper).
Currently (but hopefully not for long, spoiler alert
), SmallBank is the most representative and openly available benchmark (regarding its complexity) we have for DLTs. But if you have any information about other complex DLT benchmarks, we'd be happy to take a look at and include them in the repository in a Caliper-compatible format.