Project

Hyperledger Caliper

Project Health

Caliper continues to grow and mature.In the past quarter, one of many contributors has grown the maintainer according to his great efforts. There were still many consulting questions in rocket chat and issues,most of which were well answered.Besides that, Caliper intern also participated in the project development and is in charge of Caliper visualization.

Issues

There are no major issues at this time.

Releases

Caliper npm packages and fixed API is on the way. After that, Caliper plans to release alpha version.

Overall Activity in the Past Quarter

During the past Quarter, 30 issues have been closed, 41 PRs have been reviewed and merged. Now the project has 290+ stars and 180+ forks.Thanks every contributor's efforts. Below are some main updates in the past 3 months:

  • Support the latest iroha;
  • Support the Ethereum benchmark;
  • Refactored Caliper code,split Caliper code base into modules.
  • Created CLI and sample package.
  • Added Fabric kafka example and raft example;
  • Enhanced abilities, such as automatically generate crypto config, a license check process and Caliper running environment check.
  • Improved documentation, e.g.Ethereum adaptor configuration doc,docs for Caliper CLI and runtime configuration doc. 

Current Plans

Real time charts

high

Various reports generation

medium

Integrate with Grafana

High

Metrics extension

Low

Docker image

Medium

Maintainer Diversity

Now the maintainers are from Huawei,Budapest University and IBM.

  • Attila Klenik (Budapest University) 
  • Nick Lincoln (IBM)
  • Yu Pan (Huawei)
  • Feihu Jiang (Huawei)

Contributor Diversity

There are 25 contributors from various companies/organisations such as Huawei, IBM, Intel, Soramitsu, Budapest University, Monax and the university of Oregon. And we are connecting with several companies now to try to attract more contributors.

Additional Information

N/A

Reviewed by

  • Arnaud Le Hors
  • Baohua Yang
  • Binh Nguyen
  • Christopher Ferris
  • Dan Middleton
  • Hart Montgomery
  • Kelly Olson
  • Mark Wagner
  • Mic Bowman
  • Nathan George
  • Silas Davis


  • No labels

6 Comments

  1. Dan Middleton

    Do you feel that Caliper is aligned with the PSWG whitepaper in terms of metric definitions and reporting?

    1. Attila Klenik

      Hi Dan! Yes, the TX data gathered by Caliper is sufficient to produce the high-level client-side (end-to-end) metrics defined by the whitepaper. The generated report will be refined in the coming weeks, the Caliper UI internship project will shed some light on how to present the data in the best way (and to select the interesting views/visuals/numbers).

      1. Dan Middleton

        That sounds good. I'm particularly interested that the reporting reflects network size, which was one of the key definitions of the whitepaper.

        As you mention UI, please consider incorporating accessibility requirements into the UI if those requirements don't already exist.

        1. Attila Klenik

          Well, network size is a tricky question. Caliper doesn't necessarily need to know about the entire network (especially after we add the discovery feature for Fabric, for example). So we can't really "infer the truth" about the whole network topology from client-side, we rely on the user-provided settings.

          But users can include arbitrary metadata about the test in the test configuration, and those metadata are included in the report. This is the current way to specify the network size in the tests. But we'll think about how to automatically derive the (know) network topology and include it in the report.

          Can you elaborate on the accessibility requirements? Do you mean simple authentication features, like for a grafana server? Or more exotic data access control, like with Hyperledger Explorer?  

  2. Raghunathan P

    Hi i am trying to implement a blockchain application for my research. Where can I find step by step tutorial to integrate Caliper with Ethereum for benchmarking the implementation?

    1. David Boswell

      Raghunathan P – You can find some tutorials for Caliper at: https://www.hyperledger.org/use/tutorials

      And for additional questions about Caliper, I recommend asking on the Caliper chat channels, mailing list or in our of their community calls.  Links to those are at: Hyperledger Caliper