...

Description 

TODO

Additional Information

TODO:

Performance evaluation of DLTs is a complex process stemming from the inherent complexity of distributed systems. One way to mitigate such complexity is the separation of concerns: use task-oriented solutions for different aspects of the process:

• The System Under Test (SUT) is deployed in a representative operating environment.
• A dedicated, purpose-specific load generator tool submits representative requests to the SUT.
• The SUT is closely monitored during the load generation process, and the measured data is typically stored for post-mortem analysis. 
• A detailed analysis is performed on the measured data using dedicated data analysis techniques. A summarizing report is the typical final output of the process, containing insights about the SUT.

Even though the above concerns should be separated, it becomes cumbersome to provide a one-click ("cloud-like") experience for running a performance analysis, mainly because the last step does not build on common solutions and contains ad-hoc/in-house solutions.

Hyperledger Caliper provides capabilities to integrate lightweight components that can retrieve aggregated results from external components and incorporate them into the generated report (e.g., the Prometheus monitor).

The goal of the project is to provide a single entry/exit point to the performance analysis (once the SUT and its monitoring are configured) by:

• Providing an open, well-designed, and thoroughly documented side-car service for the detailed performance analysis of distributed Fabric transaction traces.
• And integrating it into the Caliper load generation and response measurement process as part of the final report.

The project will heavily build on the PSWG's Performance Sandbox, aiming to "standardize" its flow and methodologies independently of the applied technologies.

Additional Information

• A systematic performance analysis approach for Hyperledger Fabric:
  • Adding Semantics to Measurements: Ontology-Guided, Systematic Performance Analysis (highly recommended, lighter reading)
  • Measurement-based performance evaluation of distributed ledger technologies (more general, advanced material)
• Related PSWG activities:
  • Hyperledger Blockchain Performance Metrics White Paper
  • PSWG Performance Sandbox: HL Fabric deployment and monitoring
• Hyperledger Caliper, the scalable and extendable workload generator of Hyperledger: https://www.hyperledger.org/use/caliper
• Hyperledger Fabric, a modular and scalable distributed ledger implementation of Hyperledger: https://www.hyperledger.org/use/
• dissertation
• pswg
• sandbox
• caliper
• fabric

Learning Objectives

• Mastering the open-source workflow
  • Managing issues, pull requests, development branches, and repository content
• Being part of an open community, navigating and utilizing different forums
• Working in a team
• Covering a broad spectrum of software development skills
  
  • Designing/implementing functionally rich, testable, maintainable software components
  • Introduction to service-oriented software engineering approaches
  • Documentation and presentation skills
    • Developer documentation
    • User documentation
    • To-the-point, coherent presentation

Expected Outcome

1. Detailed, technology-agnostic documentation about the observability of Hyperledger Fabric, integrated with its (semi-)formal consensus model.
2. A transaction/event-level monitoring solution (preferably building on the existing Performance Sandbox capabilities) for tunneling distributed traces into a single processing component.
3. A self-contained, side-car service that systematically aggregates, validates, and analyses the incoming distributed traces and exposes the results in a simple format.
4. An enhanced "Monitoring & Reporting" pipeline in Hyperledger Caliper that facilitates the integration of custom monitors and reporters.
5. A custom Caliper monitor integration that pulls the results of the side-car service and a custom Caliper reporter integration that displays the detailed performance results in a self-contained format.

...

• Hyperledger Fabric: the project aims to provide a canonical methodology for conducting such a performance analysis of the platform.
• Hyperledger Caliper: the project aims to extend Caliper with the capability of easily integrating external monitoring/measurement data, thus keeping the concerns separated (Caliper is primarily a workload generator).
• PSWG: the project is the first step toward a unified performance analysis approach for Hyperledger DLT solutions, specializing (along a common methodology) the previous PSWG whitepaper for each platform.

Mentee Skills

Education Level

At least an ongoing M.Sc. study (or strong design skills) in software engineering is recommended.

Technical Skills

Required skills:

• Basic understanding of version control and git
• Experience with JS (Node.JS)
• Basic data analysis skills (e.g., pandas in python)
• Intermediate verbal and writing skills in English
• A firm understanding of the Hyperledger Fabric consensus

Nice-to-have skills (the mentee can learn these during the internship):

• Advanced git usage (upstream repositories, branching, rebasing, etc)
• Familiarity with Visual Studio Code or similar IDE
• Experience in service-oriented system design
• Experience with monitoring and log processing solutions
• Writing documentation in markdown
• Capability for unsupervised learning

Mentee Open Source Contribution Experience

...

Attila Klenik, research fellow, attila.klenik@vik.bme.hu, Budapest University of Technology and Economics, Dept. of Measurement and Inf. Systems, Critical Systems Research Group (ftsrg)

...