TitleAnalyzing Hyperledger Fabric Ledger, Transactions, and Logs using Elasticsearch and Kibana
Status

Difficulty

   Select a difficulty level and delete the others (e.g. code development would be considered less challenging than research-focused)


Description


Each blockchain platform, including Hyperledger Fabric, provide a way to record information on blockchain in an immutable manner. In the case of Hyperledger Fabric, information is recorded as a `key-value` pair. All previous updates to a `key` are recorded in the ledger, but only the latest value of a `key` can be easily queried using CouchDB; the previous updates are only available in ledger files. This mechanism makes it challenging to perform analysis of updates to a `key`, a necessary requirement for information provenance.

The goal of this project is to

Time permitting, the dashboards can be extended to analyze Fabric logs and in-progress transaction data, as well as creating dashboards similar to Hyperledger Explorer.

Of course, a blockchain solution can track information provenance in multiple ways. In one such mechanism, a solution may always write new key-value pairs to blockchain, and maintain the relationship among key-value pairs within the solution (off-chain), instead of blockchain. This project does not concern itself on how a solution manages relationship among key-value pairs.


Additional Information

Learning Objectives


Expected Outcome

A open source implementation, eventually available as Hyperledger Labs, containing:

Relation to Hyperledger 

Education Level

Undergraduate or graduate

Skills

Future plans

Create a Hyperledger Lab or incubation project for analyzing ledgers.

Preferred Hours and Length of Internship

Full-time (40 hours a week for 12 weeks during the summer) 

Mentor(s) Names and Contact Info

Salman Baset

salman.a.baset@gmail.com

Rocketchat id: salmanbaset

Mentee Name and Contact Info

Balazs Prehoda

prehoda.balazs@gmail.com

Rocketchat id: balazsprehoda

Project Deliverables

Overall goals of the project:

  1. write an Elastic beats module (in Go), that will ship ledger data to Elasticsearch instance
  2. create generic Kibana dashboards that will show both the operational and data aspects of ledger data. Basically, allow selection of a particular key or a channel configuration, and visualize updates to it (channel, id, timestamp etc)


Week 1

  • Exploration. Setup Hyperledger Fabric network, connect Hyperledger Explorer to Fabric. Use Filebeat to send data to Kibana.

Week 2

  • Exploration. Extend Fabric network (Fabric-ca, binary data and json chaincode). Dump ledger data from HL Explorer and visualize it in Kibana.

Week 3

  • Write Beats agent with configuration that sends data to Elasticsearch.

Week 4

  • Create operational dashboards similar to HL Explorer. Create data query dashboards.

Week 5

  • Refine data flow, send every block and transaction data to Elasticsearch. Make keys indexable.

Week 6

  • Refactor code, prepare the system to receive data from various peers in separate or similar indices. Add peer selection functionality. Add which user issued query in beats agent and in dashboard. Modify the application such that: 1) it only writes the most recently added keys to ledger 2) adds a previous key in the data schema, which can be added to key value in addition to hash.

Week 7

  • Test agent with multiple peers, multiple channels. Each channel may have its own zero or more chaincodes and data schema. It should be possible to specify per channel chaincode data schema in beats agent.

Week 8

  • Create example HL Fabric network setups and dashboards for different topics and use-cases (supply chain, medicine provenance, etc.)

Week 9

  • Refine the examples and prepare for submission as Hyperledger Lab. Evaluate how to read data directly from ledger file instead of using peer APIs.

Week 10

  • Submit the project as Hyperledger Lab.

Week 11

  • Create program that dumps data into custom output (default implementation is json, but can be implemented for any databases) for exploring analysis possibilities aside from Elasticsearch.

Week 12

  • Refine documentation, evaluate how to replace the ledger file with custom database (CouchDB, MongoDB).

Project Milestones

Project Plan

Summary Report

Slides

Two short demo videos (without audio):

multichannel_demo.mp4

applechain_demo.mp4