| Badge Type | Qualification Criteria | Monitoring Method | Monitoring Frequency | Requirement for Stages | Penalty for Non-Compliance | Notes |
|---|---|---|---|---|---|---|
| Legal | Project Badging Proposal | Declarations in repo and in files, GitHub Action to monitor | Continuous | All | (To be determined by LF legal staff) (TOC would have to judge this on a case-by-case basis, depending on the precise nature of non-compliance) (Ask maintainers' to take action. If they don't act within a given time period, say a month, move it to EOL. What do we do for a multi-repo project? Should we move all repos to EOL or just the ones not in compliance?) | Ask Ry (or various project maintainers) about using GitHub action to track this Peter: keep dependency pinned to last version of license (that meets legal requirements)? Jim: does badge apply to releases or latest project snapshot? Ry: reasonable to have badges tied to release as opposed to latest snapshot. (no need to add noise, i.e., multiple badges for releases). Tracy: hard to track whether a project is continuing to be on track or going downhill. We need a good way to track the project health. Bobbi: pizza pie metaphor for project health Tracy: prevent rather than penalize. Test for compliance in every PR. |
| Diversity/Decentralized | Project Badging Proposal | Look at (or parse programmatically) MAINTAINERS.md (possibly with additional org field), check commit activity and dashboard over a time period | Continuous | All | Need at least 3 independent (i.e., from different orgs) regular committers for the projct to be deemed healthy from the community's perspective. (This is specified in the Incubation exit criteria). Ensure that project is not dominated by one particular org, even if superficially the project has a diversity of maintainers. Check if there's a programmatic way to find out how diverse the contributions have been. Ry: can add a company affiliation in Maintainers.md. Tracy: determine xyz% maintainers from each company? This check can also subsume a check for the presence of a MAINTAINERS.md file. | |
| Release | Project Badging Proposal | Scan the GitHub repo to measure frequency of new releases, check ROADMAP.md, decent amount of activity on public channels (Discord and mailing lists), existence of public RFCs | It's easy to evaluate what's in the repo, but hard to track how much the project's maintenance has occurred publicly with community visibility. Perhaps we should just mandate the first and rely on self-declaration for the second. | |||
| Testing | Project Badging Proposal | Ensure that processes for unit and integration testing are in place and run periodically. Test coverage MD file; ask maintainers to self-report this. | (Evaluate this badge vs "Production" badge) Exact criteria to use is hard to determine. Collect coverage stats from various GitHub actions, check report badges on README? It may be enough to have projects report their test coverage results in a well-known location, as this will inspire confidence in users (the exact criteria and numbers are less important). | |||
| Documentation | Project Badging Proposal | Inspection, compliance with look-and-feel, fulfil feature set defined in the 'mkdocs' template (and whatever set of topis the Documentation Task Force comes up with) | Can we automate this inspection using GitHub actions? How do we handle empty files in the docs (if it's just a skeleton website)? | |||
| Project Badging Proposal | We probably don't need this one as these criteria are assumed to be fulfilled if a project proposal is accepted. | |||||
Accepted (as Labs or as Top-Level Project) | Project Badging Proposal | (Evaluate the need for this against "Responsiveness" and "Conformity" badges). Project should already have the various features set up. What we need to infer are activity and engagement levels. Any potential for automation here? (Add Discord link or mailing list link in the main README. Also add invite link to Hyperledger Discord server.) Note: Labs' Discord channels are optional. Does Accepted (Project) take precedence over Accepted (Labs)? Does Accepted go away with EOL? (We can go with "yes") | ||||
OpenSSF | Project Badging Proposal | Needs to be "Passing" for Graduated stage. | CII's new incarnation is OpenSSF. (Check what OpenSSF recommends.) (Check if this can ever be downgraded, after someone fills out a questionnaire.) Compare and contrast this list with the HL Best Practices guidelines. Borrow badge ideas from the latter. | |||
| Security? | https://github.com/arsulegai/tsc/blob/security-gov/governing-documents/security.md (PR under review, but approved) | All |
(Track security GitHub issues to see if they have been resolved within a given time period, like 90 days. Otherwise, flag as out-of-compliance; badge can be revoked if maintainers don't take action within a prescribed review period.) Badges may help during annual review, gives people an opportunity to examine the projects and see if they deserve the badges they currently possess. | |||
(Maybe revisit this later, if we can define the criteria more specifically.) | Should reuse components from other Hyperledger projects as needed and not reinvent the wheel | Q. Would this be useful to the end user? What is the right level of looking at this uniqueness? This criterion is not concrete enough yet. | ||||
(Revisit this when the Automated Pipelines Task Force has made more progress) | Best Practices for Automated Pipelines | Question is to what extent does a project meet these criteria. | ||||
| Automation/Usability? | Can users easily launch and test the software using automated scripts or do they have to jump through hoops to get started? | |||||
| https://github.com/hyperledger/toc/blob/gh-pages/guidelines/project-best-practices.md | ||||||
| Conformity? | Does the look and feel of the repo structure and the documentation match that of other projects (according to Hyperledger's recommended practices)? This may overlap with "documentation". Need suggestions from documentation task force. | Use action to track whether the repo has all the required files (maintainer, security policy, security SPOC, rfcs, etc.; list TBD). | ||||
| Responsiveness/Engagement? | Are maintainers engaging with the community and responding reasonably promptly to questions and solicitations for involvement? Need suggestions from onboarding task force. The number of summer mentees may also factor into this. | |||||
| Production? | Is the software currently being used in at least one production-grade system outside of Hyperledger (publicly verifiable or certified by third party consumer)? | Evaluate adopters and see what stage of their projects they are in | This may be a more useful criterion and badge than "Testing". Maybe recommend an ADOPTERS.md file (Gives people an idea of who are using it.) |
Overview
Content Tools
Activity