Participants:

Objective:

Define a standard set of expected files and their location, and if relevant, the specified content, for all Hyperledger Incubating and Active projects. The TODO Group has defined a repo linter tool that can check for expected content. We might want to emulate some subset of their expected content.

Proposals:

  1. RECOMMENDATION: adopt repolinter with a policy as in this PR. Each repo would add the same repolint.json file and the CA team (or preferably CI) could periodically run a check to ensure projects are in compliance. Additionally, we should establish a template repository that has LICENSE, NOTICE, CoC and Security default files.

    $ npx repolinter ../hyperledger/fabric
    npx: installed 60 in 3.726s
    Target directory: /Users/cbf/dev/hyperledger/fabric
    Ruleset: repolint.json
    Linguist Axiom: Linguist not found in path, only running language-independent rules
    Target directory: /Users/cbf/dev/hyperledger/fabric
    ✔ notice-file-exists: found (NOTICE)
    ✔ license-file-exists: found (LICENSE)
    ✔ readme-file-exists: found (README.md)
    ✔ contributing-file-exists: found (CONTRIBUTING.md)
    ✔ code-of-conduct-file-exists: found (CODE_OF_CONDUCT.md)
    ✔ changelog-file-exists: found (CHANGELOG.md)
    ✔ security-file-exists: found (SECURITY.md)
    ⚠ support-file-exists: not found: ({docs/,.github/,}SUPPORT*)
    ✔ readme-references-license: File README.md contains license
    ✔ binaries-not-present: Excluded file type doesn't exist (**/*.exe,**/*.dll,!node_modules/**)
    ✔ test-directory-exists: found (test-pyramid.png)
    ✔ integrates-with-ci: found (.github/workflows/trigger.yml)
    ⚠ code-of-conduct-file-contains-email: File CODE_OF_CONDUCT.md doesn't contain email address
    ⚠ github-issue-template-exists: not found: (ISSUE_TEMPLATE*, .github/ISSUE_TEMPLATE*)
    ✔ github-pull-request-template-exists: found (.github/PULL_REQUEST_TEMPLATE.md)
    ✔ license-detectable-by-licensee: Licensee identified the license for project: Apache-2.0

Discussion:

Please see the discussion in the TSC pending topic page.

  • No labels

29 Comments

  1. I'll send an email to the TSC soliciting a meeting time. Will need to accommodate EST and PST so far.

  2. I really like having a tool that can be used to check that the repo has the files that Hyperledger expects.

    1. agreed, but neither Ry nor I could get it working. When I get some cycles I may try again and ask around upstream.

      1. Hmmm...This is what I got when I ran it against a Cello directory I had. Not sure if that is what you meant.

        $ git clone https://github.com/todogroup/repolinter.git
        Cloning into 'repolinter'...
        remote: Enumerating objects: 33, done.
        remote: Counting objects: 100% (33/33), done.
        remote: Compressing objects: 100% (28/28), done.
        remote: Total 1393 (delta 14), reused 12 (delta 5), pack-reused 1360
        Receiving objects: 100% (1393/1393), 433.06 KiB | 2.21 MiB/s, done.
        Resolving deltas: 100% (795/795), done.

        $ cd repolinter/

        $ npx repolinter ../cello
        npx: installed 60 in 4.615s
        Target directory: /Users/tracy.a.kuhrt/hyperledger/cello
        Ruleset: ../../.npm/_npx/49618/lib/node_modules/repolinter/rulesets/default.json
        Linguist Axiom: Linguist not found in path, only running language-independent rules
        Licensee Axiom: Licensee not found in path, only running license-independent rules
        ✔ license-file-exists: found (LICENSE)
        ✔ readme-file-exists: found (README.md)
        ✖ contributing-file-exists: not found: (CONTRIB*, .github/CONTRIB*)
        ✖ code-of-conduct-file-exists: not found: (CODEOFCONDUCT*, CODE-OF-CONDUCT*, CODE_OF_CONDUCT*, .github/CODEOFCONDUCT*, .github/CODE-OF-CONDUCT*, .github/CODE_OF_CONDUCT*)
        ✔ changelog-file-exists: found (CHANGELOG.md)
        ✖ security-file-exists: not found: (SECURITY.md)
        ✖ support-file-exists: not found: ({docs/,.github/,}SUPPORT*)
        ✔ readme-references-license: File README.md contains license
        ✔ binaries-not-present: Excluded file type doesn't exist (**/*.exe,**/*.dll,!node_modules/**)
        ✔ test-directory-exists: found (src/api-engine/api/tests.py)
        ✖ integrates-with-ci: not found: (.gitlab-ci.yml, .travis.yml, appveyor.yml, .appveyor.yml, circle.yml, .circleci/config.yml, Jenkinsfile, .drone.yml, .github/workflows/*, azure-pipelines.yml)
        ⚠ source-license-headers-exist: The first 5 lines of 'src/dashboard/config/config.js' do not contain the patterns:
        Copyright
        License

        [snip a bunch of source-license-headers-exists issues]

        ✔ source-license-headers-exist: The first 5 lines of 'src/dashboard/functions/api.js' contain all of the requested patterns.
        ✖ github-issue-template-exists: not found: (ISSUE_TEMPLATE*, .github/ISSUE_TEMPLATE*)
        ✖ github-pull-request-template-exists: not found: (PULL_REQUEST_TEMPLATE*, .github/PULL_REQUEST_TEMPLATE*)
        ✖ license-detectable-by-licensee: Licensee not installed


        1. You need to install npm and hence npx other dependencies in the readme file in repolinter.

          You can run against the github repo directly: For example I ran against fabric:

          See below:

          npx repolinter --git https://github.com/hyperledger/fabric
          Target directory: <mypath hidden for privacy>\repolinter\tmp\944fe967-bdc0-481d-a21a-5a6ea811c08a
          Ruleset: ..\..\..\..\..\AppData\Roaming\npm-cache\_npx\2712\node_modules\repolinter\rulesets\default.json
          Linguist Axiom: Linguist not found in path, only running language-independent rules
          Licensee Axiom: Licensee not found in path, only running license-independent rules
          √ license-file-exists: found (LICENSE)
          √ readme-file-exists: found (README.md)
          √ contributing-file-exists: found (CONTRIBUTING.md)
          √ code-of-conduct-file-exists: found (CODE_OF_CONDUCT.md)
          √ changelog-file-exists: found (CHANGELOG.md)
          √ security-file-exists: found (SECURITY.md)
          × support-file-exists: not found: ({docs/,.github/,}SUPPORT*)
          √ readme-references-license: File README.md contains license
          √ binaries-not-present: Excluded file type doesn't exist (**/*.exe,**/*.dll,!node_modules/**)
          √ test-directory-exists: found (test-pyramid.png)
          × integrates-with-ci: not found: (.gitlab-ci.yml, .travis.yml, appveyor.yml, .appveyor.yml, circle.yml, .circleci/config.yml, Jenkinsfile, .drone.yml, .github/workflows/*, azure-pipelines.yml)
          × code-of-conduct-file-contains-email: File CODE_OF_CONDUCT.md doesn't contain email address
          × github-issue-template-exists: not found: (ISSUE_TEMPLATE*, .github/ISSUE_TEMPLATE*)
          × github-pull-request-template-exists: not found: (PULL_REQUEST_TEMPLATE*, .github/PULL_REQUEST_TEMPLATE*)
          × license-detectable-by-licensee: Licensee not installed 

  3. Looks like it wanted a very recent version of Node.js and npm, and I also had to upgrade gawk for Catalina. Works now, looks like we need to define a set of rules.

  4. I had a long write-up of some analysis I did of repolinter tool but the WIKI logged me out and I lost it.

    tl;dr:

    RECOMMENDATION: adopt repolinter with a policy that omits checking the NOTICES and CoC email. Fix the rule for Azure pipelines and omit the SUPPORT file check and PR/Issues templates. Everything else in the default policy should be required.

    1. Chris, I believe if you go to either the "Recently Worked On" or the "Drafts", you should be able to find your work. I have had this happen to me in the past too.

      1. Hmmm, don't see either of those options. Confluence might be powerful but the UX isn't great;-)


        1. Agree. Both would be under your profile picture in the upper right corner. Click on pick, and a menu shows up with these options.

          1. Ahh...looks like it is under Drafts: https://wiki.hyperledger.org/users/viewmydrafts.action

            When you edit a page but don't publish the changes, a draft is created. HitResumeto continue editing or publish the changes.

            1. Thanks... but I was editing a comment and no draft was saved.

  5. All rule sets can be changed in default.json or in repolinter.json. For checking repos, it should be done via local workspace files NOT the target, since the repolinter can be gamed by changing the rule sets in the target. The whole purpose of checking the target for compliance will be lost. Unless of course the target rule set file also contains explanations of why the rule set has been modified from our recommendations.

    SUPPORT.md

    I think SUPPORT.md should be required. For example repolinter SUPPORT.md says:

    "If you are having trouble running repolinter, make sure you have the latest npm with npx support before running npx repolinter.

    If you find a bug, please submit it to our issue tracker at https://github.com/todogroup/repolinter/issues and label it as a bug"
    This is extremely useful, a problem of not able to run repolinter was brought up before- A look at SUPPORT.md tells you what to do. I have heard a lot of people complaining about the lack of support or response in Hyperledger frameworks. Having a SUPPORT.md makes it possible to point to it or to quote it for a first level support. It does not have to be elaborate, just a few lines to help you get started on a journey.


    PR and ISSUE_TEMPLATEs
    Should be required, for ease of reporting issues and putting together PRs. This will help more collaborators be productive.

    ABOUT the emails for CoC or NOTICES: I do not have a problem with these being taken out of the rule set. Unless of course the email is the general support email which has a chance of being answered.


    If you look above, at the screen dumps of the runs that Tracy Kuhrt did or I did, you wll notice that linguistics and license libraries are missing, those are gem components that need ruby; of course for complete coverage you probably need those libraries- I have not yet run with them so I do not know what the results of such a run would be.

    1. I would prefer that the items one would describe in SUPPORT or the PR templates are just in contributing.md. Too many files is clutter which can make it harder to find what you are looking for.

  6. Note, that the NOTICES file may be required by some compatible licenses. Most of our larger projects probably need this. It's worth discussing whether requiring this file would stimulate some awareness from projects that do need it.

  7. Change log feels optional to me. Some projects publish release notes that uplevel the commit history which is otherwise available through the repo directly.

  8. The SECURITY.md and CODE_OF_CONDUCT.md should be standard across projects. I think there was some talk of a repo for these things. Maybe when we reorg the governing docs?

    The content of the other files should be flexible to each project's needs.

  9. We could indeed establish a template repository for creating new repositories. That won't help with ones we already have. It would simplify the work of the CA team, though. I submitted a PR to fabric that includes a modified policy file for repolinter that makes the changes I had been suggesting. https://github.com/hyperledger/fabric/pull/630

    I believe that all repos now have the common SECURITY.md file. That should also be added to any template repo we create. The Fabric repo uses the following to refer to the CODE_OF_CONDUCT in the wiki

    Code of Conduct Guidelines
    ==========================
    
    Please review the Hyperledger [Code of
    Conduct](https://wiki.hyperledger.org/community/hyperledger-project-code-of-conduct)
    before participating. It is important that we keep things civil.
    
    <a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.
    

    I think that someone suggested we cut-n-paste from the wiki into the CoC file. I could live with that, but it would make for a chore if we ever change it.

  10. Here's a gist of all the repos under the hyperledger GH org repolinted:


    https://gist.github.com/christo4ferris/13df399dff3613d405e8a60f23847385

  11. Why is there a github-issue-template check if most (all?) HL projects use Jira?

  12. https://gist.github.com/christo4ferris/13df399dff3613d405e8a60f23847385#file-gistfile1-txt-L7446

    I believe these lines here will always clash with our tests because of how truffle migration files are structured.


    Danno Ferrin suggests not requiring copyright headers in javascripts as another tweak that should be considered.

    1. Most of these are warnings. The overarching idea to have a lint is good. We can always work out the details.

  13. I believe ROADMAP.md isn't being checked. What's the status of that file? 

  14. No hyperledger projects have a NOTICE file (the only exception is when go code is vendoring in libraries).  I propose it be deleted from the lint check.  Alternately we should define what goes in it.

      1. My github search was for whole words only.  Missed that one.

        1. For besu we've been checking the license but not aggregating the notices since we've been bringing them in as fully compiled libraries and not building from source.

          Getting a common way that HLP projects compile and report this would be good.  This is reasonable to accomplish with our gradle build.

          1. The licenses on code still apply to binaries, so it's perhaps even more important to add that to a NOTICE file. I presume you're already documenting the source of those binaries elsewhere.

            1. Maven central has a requirement that all published artifacts have a license in the pom.xml metadata.  We compile that data and refuse to produce a build artifact if we see a license we have not whitelisted.  The whitelist only consists of permissive licenses like the various BSD types and Apache 2.0.