Best Practices Criteria for Free/Libre and Open Source Software (FLOSS)

Introduction

This is a set of best practices for Free/Libre and Open Source Software (FLOSS) projects. Projects that follow these best practices will be able to voluntarily self-certify and show that they've achieved a Core Infrastructure Initiative (CII) badge. Projects can do this, at no cost, by using a web application (BadgeApp) to explain how they meet these practices and their detailed criteria.

There is no set of practices that can guarantee that software will never have defects or vulnerabilities; even formal methods can fail if the specifications or assumptions are wrong. Nor is there any set of practices that can guarantee that a project will sustain a healthy and well-functioning development community. However, following best practices can help improve the results of projects. For example, some practices enable multi-person review before release, which can both help find otherwise hard-to-find technical vulnerabilities and help build trust and a desire for repeated interaction among developers from different organizations.

These best practices have been created to:

  1. encourage projects to follow best practices,
  2. help new projects discover what those practices are, and
  3. help users know which projects are following best practices (so users can prefer such projects).

The idiom "best practices" means "a procedure or set of procedures that is preferred or considered standard within an organization, industry, etc." (source: Dictionary.com). These criteria are what we believe are widely "preferred or considered standard" in the wider FLOSS community.

The "passing" criteria listed here focus on identifying best practices that well-run FLOSS projects typically already follow. The criteria are inspired by a variety of sources; see the separate "background" page for more information.

The criteria for higher/more advanced badges describe the criteria for the higher-level badges. These are known as the "silver" and "gold" levels, and sometimes also described as "other" criteria. You must achieve the "passing" criteria before you can achieve silver or gold.

The Linux Foundation also sponsors the OpenChain Project, which identifies criteria for a "high quality Free and Open Source Software (FOSS) compliance program." OpenChain focuses on how organizations can best use FLOSS and contribute back to FLOSS projects, while the CII Best Practices badge focuses on the FLOSS projects themselves. The CII Best Practices badge and OpenChain work together to help improve FLOSS and how FLOSS is used.

We expect that these practices and their detailed criteria will be updated, even after badges are released. Thus, criteria (and badges) probably will have a year identifier and will phase out after a year or two. We expect it will be easy to update the information, so this relatively short badge life should not be a barrier. We plan to add new criteria but mark them as "future" criteria, so that projects can add that information and maintain their badge.

Feedback is very welcome via the GitHub site as issues or pull requests. There is also a mailing list for general discussion.

Below are the current criteria, along with and where to get more information. The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as described in RFC 2119. The additional term SUGGESTED is added, as follows:

We assume that you are already familiar with software development and running a FLOSS project; if not, see introductory materials such as Producing Open Source Software by Karl Fogel.

Terminology

A project is an active entity that has project member(s) and produces project result(s). Its member(s) use project sites to coordinate and disseminate result(s). A project does not need to be a formal legal entity. Key terms relating to project are:

Current criteria: Best Practices for FLOSS

Here are the current criteria. Note that:

In some cases we automatically test and fill in information if the project follows standard conventions and is hosted on a site (e.g., GitHub) with decent API support. We intend to improve this automation in the future (improvements welcome!).

The actual criteria are stored in the file "criteria/criteria.yml", including details, rationale, and how it could be automated.

There is an implied criterion that we should mention here:

Basics

Basic project website content

OSS License

Documentation

Other

Change Control

Public version-controlled source repository

Unique version numbering

Release notes

Reporting

Bug-reporting process

Vulnerability report process

Quality

Working build system

Automated test suite

New functionality testing

Warning flags

Security

Secure development knowledge

Use basic good cryptographic practices

Secured delivery against man-in-the-middle (MITM) attacks

Publicly known vulnerabilities fixed

Other security issues

Analysis

Static code analysis

Dynamic code analysis

A note on good cryptographic practices

Note: These criteria do not always apply because some software has no need to directly use cryptographic capabilities. A "project security mechanism" is a security mechanism provided by the delivered project's software.

Non-criteria

We do not require any specific products or services, and in general do not require any particular technology. In particular, we do not require proprietary tools, services, or technology, since many free software developers would reject such criteria. For example, we intentionally do not require git or GitHub. We also do not require or forbid any particular programming language. We do require that additional measures be taken for certain kinds of programming languages, but that is different. This means that as new tools and capabilities become available, projects can quickly switch to them without failing to meet any criteria.

We do provide guidance and help for common cases. The criteria do sometimes identify common methods or ways of doing something (especially if they are FLOSS), since that information can help people understand and meet the criteria. We also created an "easy on-ramp" for projects using git on GitHub, since that is a common case. But note that nothing requires git or GitHub. We would welcome good patches that help provide an "easy on-ramp" for projects on other repository platforms; GitLab was one of the first projects with a badge.

We avoid requiring, at the passing level, criteria that would be impractical for a single-person project, e.g., something that requires a significant amount of money. Many FLOSS projects are small, and we do not want to disenfranchise them.

We do not plan to require active user discussion within a project. Some highly mature projects rarely change and thus may have little activity. We do, however, require that the project be responsive if vulnerabilities are reported to the project (see above).

Uniquely identifying a project

One challenge is uniquely identifying a project. Our Rails application gives a unique id to each new project, so we can use that id to uniquely identify project entries. However, that doesn't help people who searching for the project and do not already know that id.

The real name of a project, for our purposes, is the URL for its repository, and where that is not available, the project "front page" URL can help find it. Most projects have a human-readable name, and we provide a search mechanims, but these names are not enough to uniquely identify a project. The same human-readable name can be used for many different projects (including project forks), and the same project may go by many different names. In many cases it will be useful to point to other names for the project (e.g., the source package name in Debian, the package name in some language-specific repository, or its name in OpenHub).

In the future we may try to check more carefully that a user can legitimately represent a project. For the moment, we primarily focus on checking if GitHub repositories are involved; there are ways to do this for other situations if that becomes important.

Non-admin users cannot edit the repo URL once one is entered. (Exception: they can upgrade http to https.) If they could change the repo URL, they might fool people into thinking they controlled a project that they did not. That said, creating a bogus row entry does not really help someone very much; what matters to the software is the id used by the project when it refers to its badge, and the project determines that.

Why have criteria?

The paper Open badges for education: what are the implications at the intersection of open systems and badging?identifies three general reasons for badging systems (all are valid for this):

  1. Badges as a motivator of behavior. We hope that by identifying best practices, we'll encourage projects to implement those best practices if they do not do them already.
  2. Badges as a pedagogical tool. Some projects may not be aware of some of the best practices applied by others, or how they can be practically applied. The badge will help them become aware of them and ways to implement them.
  3. Badges as a signifier or credential. Potential users want to use projects that are applying best practices to consistently produce good results; badges make it easy for projects to signify that they are following best practices, and make it easy for users to see which projects are doing so.

We have chosen to use self-certification, because this makes it possible for a large number of projects (even small ones) to participate. There's a risk that projects may make false claims, but we think the risk is small, and users can check the claims for themselves.

Improving the criteria

We are hoping to get good suggestions and feedback from the public; please contribute!

We launched with a single badge level called passing. For higher level badges, see other.

You may also want to see the "background" file for more information about these criteria, and the "implementation" notes about the BadgeApp application.

See also

Project participation and interface:

Criteria:

Development processes and security: