|Jason T Clarkfirstname.lastname@example.org|
The Hyperledger Fabric (and associated) projects utilize various tools and workflows for continuous project development. This documentation will assist you in getting started with using these tools and understanding the workflow(s) our contributors use while working with the Fabric CI infrastructure.
Welcome to the Hyperledger Fabric community! We are excited that you want to contribute to the Continuous Integration/Release Engineering efforts. You're in the right place to get started. In the event that you need additional assistance, We encourage you to engage with our CI contributors via the following channels:
The Fabric CI environment is comprised of two parallel deployments of the Jenkins CI system:
Note: You will notice that both the Jenkins production instance and the Jenkins sandbox instance are both accessible from the same URL, though the sandbox instance is accessed at /sandbox.
For more information on how to use the sandbox, use the documentation in https://github.com/hyperledger/ci-management/blob/master/Sandbox_Setup.md
The table below indicates all of the Hyperledger Projects that are built, tested, verified, and released using the Jenkins CI infrastructure:
On the Jenkins dashboard, a list of all Jenkins jobs required to build, test, verify and release each of the Hyperledger Fabric projects is organized and displayed in separate views. For example, the Fabric view displays each of the jobs related to the fabric project:
Here, all of the Jenkins jobs for testing the Fabric project are displayed. The Jenkins jobs are displayed in a table, and indicates:
Continuing with the Fabric project example, the jobs list contains multiple jobs required to accurately build, test, and verify the Hyperledger Fabric project. The job types for the Fabric project are as follows
* Verify * End-to-End (performs e2e_cli, sdk node e2e, java sdk e2e tests) * Unit Tests (performs make linter, make unit-tests) * Documentation changes (Builds the documentation and ship it to nexus log server.
You can check the RTD format of you patch set documentation build here https://logs.hyperledger.org/production/vex-yul-hyp-jenkins-3/fabric-rtd-verify-master/<build_number>/html
IMPORTANT NOTE: Although many of the job types listed above are common across all Hyperledger Fabric projects, the list indicated above is specific to the Hyperledger Fabric project. Different projects will have different job types based on their individual build/test/release requirements.
There are several Jenkins job types that are common across most Hyperledger Fabric projects. In some cases, you may/may not see all of the common job types in every project. This is heavily dependent on the needs of the Hyperledger Fabric project. That said, let's take a look at each of the most common jobs:
Verify jobs are triggers when a patchset-created-event is triggered in gerrit. All verify jobs are depends on the patchset parent commit and patchset commit. That tells, verify jobs run on the parent commit on which developer submitted patch not on the latest master code.
Merge jobs triggers when a “change-merged-event” event is triggered in gerrit. In all the merge jobs, Jenkins clones the latest master code and perform the tests unlike verify jobs.
Release jobs are triggered after a “ref-updated-event” is created in each of the Fabric repositories targeted for release. Release jobs are intended to create to publish docker images, binaries and publish npm modules. For more information regarding the release process, refer to this Release Process Document.
With most job types, the Jenkins build jobs are broken down further to build, test, and release the Hyperledger Fabric projects with support for varying CPU architectures. They include:
Additionally with most job types, you will notice the Jenkins jobs are further isolated to include a number of test types. Those include:
For more information on what each of these test types are used for, refer to the Fabric CI Process documentation.
The ci-management repo hosts the CI job definitions for all of the Hyperledger Fabric projects integrated with Jenkins. Each job configuration is written in YAML format, and built with the Jenkins Job Builder (JJB) tool.
Much of the work inside of Fabric CI involves the creation or modification of Jenkins job definitions. To get a better understanding of how to write Jenkins job definitions, start with reading through the JJB Job Definitions documentation.
When creating or modifying Jenkins job definitions, it is important to know whether or not your job definitions work prior to adding them to the production Jenkins instance. This is why Hyperledger Fabric has provided a Jenkins Sandbox instance, allowing you to test your Jenkins job definitions. To use the Jenkins sandbox, read through the Sandbox Setup process, which explains how to install Jenkins Job Builder (JJB), as well as testing, updating, and triggering jobs on the Jenkins Sandbox.
Contributing to the Fabric CI process starts with identifying tasks to work on, and bugs to be fixed. The JIRA site provided by the Hyperledger Community is where to find these items. To narrow down tasks and bugs that are directly related to Fabric CI, use the following URLs:
Trigger builds through comments to a commit/change:
Re-verification/trigger of builds is possible in Jenkins by entering any of the below comment phrases as a comment to the Gerrit patch set. To do so, follow the below process:
Step 1: Open the failed Gerrit patch set
Step 2: Click on Reply and type the below comment phrases based on the project and click Post
CI jobs listens on the below comment phrases, when you type a comment as reverify-x in fabric project, this triggers fabric-verify-x86_64 CI job.
All other repositories use any of the below comment phrases
For an overview of CI Processes, refer to the FAQ, below.
If you are a developer having difficulty obtaining that green checkmark consistently from hyperledger-jobbuilder because your merge job or verify job fails, then *create a bug* in jira to start the process rolling to fix the problem for everyone in the community!
Below is a table that shows you where to find the most informative resources for the most frequently asked questions about Fabric CI.
|Which Fabric projects are currently using CI?||hyperledger_fabric_projects_on_ci|
|What is the process used to build, test, and release Fabric projects?||https://github.com/hyperledger/ci-management/blob/master/docs/fabric_ci_process.md|
|Where is the Jenkins CI for Hyperledger Fabric?||https://jenkins.hyperledger.org/|
|Where is the sandbox installation of Jenkins CI for Hyperledger Fabric?||https://jenkins.hyperledger.org/sandbox|
|Where can I found out more about each of the Jenkins jobs used to build, test, and release Hyperledger Fabric projects?||https://github.com/hyperledger/ci-management|
|How do I write Jenkins job definitions for use with Hyperledger Fabric projects?||https://docs.openstack.org/infra/jenkins-job-builder/definition.html|
|How should I test my Jenkins job definitions for use by the Fabric CI process?||https://github.com/hyperledger/ci-management/blob/master/Sandbox_Setup.md|
|I want to find Fabric CI related tasks to work on. Where should I go?||https://jira.hyperledger.org/issues/?filter=11500|
|I have questions about Fabric CI. Where should I go for help?||https://chat.hyperledger.org/channel/ci-pipeline|