What are the criteria for a "First Major Release"? How does Dave's proposed Project Readiness fit in? FWIW, I created this set of criteria for the Fabric 1.0 release.

Approved Resolution 2 (TSC 09/05/2019):

Prior to bringing a project to the TSC for a vote on granting a First Major Release a project must fulfill the following criteria: 

  1. Project has reached active status (the processes and community are mature)
  2. The release features substantially fulfill the project's charter
  3. The release satisfies criteria defined by the project itself. All projects should meet some internally defined non-functional release criteria (bug counts, performance, etc.)
    1. Test coverage (unit and functional/integration) is sufficient to have confidence that the release supports the non-functional criteria
    2. Maintainers have reviewed all remaining open bugs and agree on severity/priority
    3. Zero high or highest priority bugs remain open
    4. The preceding release candidate has been published for a minimum period of time of 2 weeks
    5. The release should not require users/operators to compile code to operate the base project on supported platforms
    6. The documentation is sufficient to ensure that users/operators have clear guidance on how to get started and how to configure and operate.
  4. The project has met all technical criteria for the release.
    1. The project has met all the criteria for CII Best Practices Badge Best Practices Badge,
    2. A security audit provided by Hyperledger was completed
      1. no bugs that affect security of the system remain open unless they have mitigating workarounds published in release notes
      2. crypto code included in the release has been audited for crypto export compliance
    3. A license scan was completed and all issues were resolved, including receiving an exception from the legal committee.
    4. Changelog of all commits is generated and published.
    5. Release notes generated and published.


  • No labels

19 Comments

  1. Arnaud J Le Hors

    Proposed/Draft Resolution 2:

    <insert here any criteria from Dave's Project Readiness proposal that do not pertain to what would be considered for graduating to "Active" status, which is handled separately.>

    Arnaud J Le Hors

    It would actually seem reasonable that there is an overlap.

    1. Vipin Bharathan


      Let us not a create a draft that is dependent on another draft. According to me that (project readiness proposal) needs some serious rework mostly for prolixity. Also for a lot of soft and possibly irrelevant criteria for readiness.


    2. Arnaud J Le Hors


      I agree with the sentiment Vipin. For expediency I merely put a reference when I did this yesterday but "we" should simply incorporate whatever is relevant into this very document.


    3. Vipin Bharathan

      Proposed/Draft Resolution 2:

      Can we just point to a doc (that is not yet ready but draft) which will be a form of the project readiness document.

  2. Arnaud J Le Hors

    Vipin, are you proposing to do exactly what you said earlier we shouldn't do? (smile)

    1. Vipin Bharathan

      No, I am saying work on the project readiness document first. It will subsume the criteria for 1.0

  3. Dan Middleton

    The point of this event is that Hyperledger gets behind the release with marketing and brand. To qualify for that a project should 

    1. Have reached active status (the processes and community are mature)
    2. Completed a security audit provided by HL
    3. Completed a license scan
    4. The release features substantially fulfill the project's charter
    5. The release satisfies criteria defined by the project itself. All projects should meet some internally defined non-functional release criteria (bug counts, performance, etc.)

    The first three are largely objective. #4 is clearly subjective but says that if a project is chartered to fill some role at Hyperledger a major release should show that functionality. 

    #5 nudges projects to establish their own criteria without dictating to them things that are project-specific.

    This  Project Readiness doc additionally mentions use of HL infrastructure (wiki, chat, etc.). We should make that part of active status requirements.

  4. Shawn Amundson

    If we follow semver (which we do, generally), the version number is primarily an indicator of a commitment to some level of backward compatibility support. This may fit under Dan's (#5) but may be worth calling out specifically so projects spend some time considering what it means.

  5. Christopher Ferris

    Adding to Dan's recommendation above, I would cite the Fabric release criteria we used for 1.0 Release Exit Criteria

    1. Dan Middleton

      Good idea. I did a best effort to fold those two together and set that as the resolution. We can iterate on this a bit.

  6. Arnaud J Le Hors

    Thanks for taking a crack at putting a proposal together Dan. It looks good to me but I think we need to specify what "a minimum period of time" for the candidate release is. I realize this may be a bit arbitrary but without any specification at all the requirement becomes meaningless. Fabric set it to 2 weeks. You don't think that's good enough?

    1. Dan Middleton

      I don't have a strong opinion on the duration. I think it's some combination of the time necessary for availability testing (sawtooth uses a 7 day burn-in for example) and user feedback. If say RC1 was up for a month and had a bug, I would think that RC2 would only need to be up for long enough to QA (1 week in ST case) because other user acceptance aspects would not have changed from RC1 to RC2.

    2. Binh Nguyen

      I agreed we should specify "a minimum period of 2 weeks" Two weeks would give most willing people time to arrange to take a dive into the new release. I would personally prefer 4 weeks, so that's why "minimum period"

  7. Christopher Ferris

    "d. the preceding release candidate will have been published for a minimum period of time" - should we stipulate a duration? Otherwise, this could be nanoseconds;-)

    1. Arnaud J Le Hors

      I added a minimun of 2 weeks.

  8. Christopher Ferris

    Unless we are building for every platform, this seems unreasonable:

    "e. release will not require users/operators to compile code to operate the base project"

    Maybe should not.

    1. Arnaud J Le Hors

      As a middle ground I changed this to: "should not ... on supported platforms."

  9. Christopher Ferris

    "documentation sufficient to ensure that users/operators have clear guidance on how to get started and how to configure and operate. The operational aspects need to be correct and independently user tested"

    This is rather subjective. I think that yes, the documentation should be complete and current, but I am not sure how you have it independently user tested. That would be quite an undertaking. Documentation is almost always insufficient;-) Continuous improvement and cleaning up technical debt as the project evolves are critical. As long as there is a plan and process in place that should be sufficient.

    1. Arnaud J Le Hors

      I dropped the last sentence.

  10. Christopher Ferris

    If we really aren't issuing code to be compiled, it is unclear why we need a signed source release tarball unless Hyperledger is going to provide the tools to create that from the git repos.

    1. Dan Middleton

      Yeah we might want to split some of these into a different topic. The ability to produce signed audited releases might be a topic for the ci/cd committee. 

    2. Arnaud J Le Hors

      I dropped that requirement.