PHEP 4: PyHC Package Tiering

pheps/phep-9999.md

@@ -0,0 +1,174 @@
+```
+PHEP: 9999
+Title: PyHC Package Tiering
+Author: Julie Barnum <julie.barnum@lasp.colorado.edu> <https://orcid.org/0000-0001-8755-0694>
+Discussions-To: https://github.com/heliophysicsPy/standards/pull/31
+Revision: 1
+Status: Draft
+Type: Process
+Content-Type: text/markdown; charset=UTF-8; variant=CommonMark
+Created: 
+Post-History: 09-July-2024 
+```
+
+# Abstract
+<a name="abstract"></a>
+This PHEP establishes a new tiering structure to PyHC projects, which will automatically affect PyHC packages once it goes into effect. Included herein is information on requirements for each of the new four tiers of PyHC projects (Gold, Silver, Bronze, and Copper), as well as benefits accrued at each tier. 
+
+# Motivation
+<a name="motivation"></a>
+Currently, PyHC is at a crossroads for how to push forward as a community. There are two main schools of thought—originating from bi-annual meeting discussions, telecon chats, and further sidebar converstaion—with regards to what PyHC is and should be:  
+
+- a basic interpretation where PyHC is a collection, and listing, of open-soure Python packages with a relevance to Heliophysics and space physics, or
+- a standards-based interpretation where PyHC strives for compliance with our set standards, package interoperability, and standardization around one or more tools.   
+
+There is utility and validity to both approaches. A new PyHC package tiering system is intended to find a "best of both worlds" with the two ideas. Older, out-of-date, unmaintained, or publication-specific code could still have a place for listing and findability, while also allowing nuance between other packages that are more robust, trustworthy, maintained, and work toward the standards-based interpretation of being a PyHC package. 
+
+Further, this tiering system also allows users to get a clearer picture on what each PyHC package has to offer, and the state of the package's condition and development. Creation of a PyHC package tiering system also allows for justification for a myriad of benefits, for example, consideration for funding from a community travel fund, or extra help with improving a standards grouping grade.
+
+# Rationale
+<a name="rationale"></a>
+Decisions for tiering levels, requirements for each tier, and benefits accrued at each tier are based on conversations with the community (bi-annual meetings, telecons, etc.), and are listed here as a starting point for more discussion, likely to be refined in the future. Initially, ideas were presented to the community in a pyramid format. To make the differences between tiers more visible and understandable, it has been transformed into a spreadsheet format. 
+
+# PyHC Package Tiering Specifications
+<a name="specification"></a>
+There are four tiers proposed in this PHEP: Copper, Bronze, Silver, and Gold. These would replace the current structure of "Core Packages", "Other Packages", and "Un-evaluated Packages". See the table below for requirements associated with each tier:
+
+|       | **Gold** | **Silver** | **Bronze** | **Copper** | 
+| :-: | :----------: | :-----------: | :------------: | :-------------: |
+| Self-Evaluation Status | Completed | Completed | Completed | Not Done |
+| PyHC Standards Grading | Complies with all "must" and many "should" requirements from applicable standards-track PHEPs | Complies with most "must" requirements from applicable standards-track PHEPs | Complies with some "must" requirements from applicable standards-track PHEPs | N/A |
+| pyOpenSci Review Status | Completed | In progress or Completed | Not Started | Not Started |
+| Installable in PyHC environment? | Yes | Yes | Yes | No |
+| HSSI Metadata Compliant? | Silver + some optional fields and controlled vocabulary contribution\* | Bronze + all recommended fields\* | Copper + some recommended fields\* | All mandatory fields\* |
+| Software License | Fully compliant, doesn't allow NOSA/GPL license | Fully compliant, allows NOSA/GPL license | Has non-recommended license | Has non open-source license | 
+| DOI | Required | Required | Required | Not Required |
+| pip installable? | Yes | Yes | Yes | No |
+| conda installable? | Yes | No | No | No |
+
+\* = See HSSI metadata schema for details
+
+Descriptions for each heading are as follows:
+   - Self Evaluation Status: indicates whether a package has completed a self evaluation against PyHC's standards
+   - PyHC Standards Grading: indicates status of compliance with each standard or standards-track PHEP replacement for a standard within a package's self evaluation
+   - pyOpenSci Review Status: indicates status of a PyHC-pyOpenSci review
+      - The exact specifications of this review process to come in a later PHEP defining the PyHC-pyOpenSci review process
+   - Installable in PyHC Environment?: indicates state of installation conflicts within the PyHC software environment  
+   - HSSI Metadata Compliant?: indicates state of compliance with the new Heliophysics Software Search Interface's (HSSI) metadata requirements 
+      - The full metadata standards to be defined in a future PHEP for the HSSI and PyHC connection.
+      - For Gold level packages, contributing to the controlled vocabularies can mean, for example, adding missing vocabularies
+   - Software License: indicates level of software license compliance with the [PyHC Standards Document](https://doi.org/10.5281/zenodo.2529131) 
+   - DOI: indicates whether or not a package's software repository has a DOI (e.g., from Zenodo) 
+   - pip installable?: indicates whether a package is pip installable
+   - conda installable?: indicates whether a package is conda installable
+
+The following table shows the benefits that are associated with each tier:  
+
+|       | **Gold** | **Silver** | **Bronze** | **Copper** | 
+| :-: | :----------: | :-----------: | :------------: | :-------------: |
+| Summer School Inclusion \* | Yes | Yes** | No | No |
+| PyHC-top-tier Env Inclusion \* | Yes | Yes ** | No | No | 
+| PyHC-all Env Inclusion \* | Yes | Yes | Yes | No |
+| PyHC-Chat Bot Inclusion \* | Yes | Yes ** | No | No |
+| Standards Compliance Assistance \* | Yes *** | Yes | Yes | Yes |
+| Main Page Listing | Yes | Yes | Yes | No |
+| Secondary Page Listing | No | No | No | Yes |
+| HSSI Inclusion | Yes | Yes | Yes | Yes | 
+| Conference Travel Funding | Yes | Yes | Yes | No |  
+
+\*  = can opt-in or opt-out of this  
+
+\** = conditional upon justification on importance/use of package (e.g. number of users, level of commitment within PyHC activities, or effort made for those items but gold level not yet achieved)  
+
+\*** = conditional upon justification that package is in danger of dropping down to a Silver tier 
+
+Descriptions for each heading are as follows:
+   - Summer School Inclusion: indicates whether a package will be included in summer school teaching materials
+   - PyHC-top-tier Env Inclusion: indicates whether a package will be included within the current PyHC software environment used at the summer school  
+   - PyHC-all Env Inclusion: indicates whether a package will be included within the current PyHC software environment containing all packages Bronze and higher.
+   - PyHC-Chat Bot Inclusion: indicates whether a packages will have up-to-date information included within the ChatGPT4-powered PyHC-Chat bot
+   - Standards Compliance Assistance: indicates whether a package will receive extra help and/or funding (dependent on available PyHC resources) for conforming to the PyHC standards
+   - Main Page Listing: indictes whether a package's information will be displayed on a new, main PyHC Project page
+   - Secondary Page Listing: indicates whetheer a package's information will be displayed on a new, secondary PyHC Project page 
+   - HSSI Inclusion: indicates whether a package will be included within the Heliophysics Software Search Interface, further:
+      - Gold and Silver level packages have priority consideration in contributing to the controlled vocabularies used by the HSSI
+      - Gold and Silver level packages are also eligible, subject to review, to manage controlled vocabularies in a rotating fashion under HSSI metadata leadership
+   - Conference Travel Funding: indicates whether developers from a package will be considered for PyHC travel funding assistance to relevant science conferences (e.g. SHINE, CEDAR, or GEM)
+
+Packages are evaluated (as described in 'Implementation Process' below) against level of compliance with each requirement, as shown in the PyHC tiering chart. To be accepted for a tier, a package must meet **all** the requirements for said tier. Therefore, should a package fall into different tiers depending on the specific requirement, the package will be accepted at the lowest tier of requirements it meets.  
+
+For example, if a package meets some requirements for the Silver tier, but other requirements only meet the Bronze tier, the package will be considered a Bronze tier package.   
+
+
+# Package Tiering Implementation Process
+
+Following acceptance of this PHEP, the PyHC website will be updated to reflect the new tiers. The process is then as follows:
+- Within six months of PyHC website update (written communication of this provided in email form), packages must submit a PR to [the PyHC website GitHub](https://github.com/heliophysicsPy/heliophysicsPy.github.io) indicating what level they believe their package is
+    - need to include written justification on the PR for chosen tier 
+- Within three months of PR publish, a member of the to-be-established technical steering committee (TSC; including the PyHC Tech Lead and volunteer members of the PyHC not involved with the submitted package) reviews the package and asserts if self-evaluated level matches expectations
+- PyHC Lead reviews decision of TSC member and gives final vote of approval on tier selection
+- PyHC Lead then either:
+    - 1) merges the PR if in agreement with the TSC member, or 
+    - 2) meets with technical steering committee member to discuss dissenting opinion
+        - if consensus reached for previos tier selection, PyHC Lead then merges the PR, or
+        - PyHC Lead provides written justification on the PR if a tier upgrade is deemed necessary  
+
+General comments:  
+
+If a package does not submit a self evaluation within six months of PyHC website update, they are automatically placed into the Copper tier. They will be able to apply for a tier regrade if this occurs, using the same process listed above.  
+
+Packages are allowed to move between tiers. If a package upgrades their status to match that of Silver, instead of Bronze, tier, for example, they can submit a new PR to have their tier regraded.   
+
+The flip side is also true; should a package become defunct or drop in status, they may be downgraded to a lower tier. The TSC shall meet on an annual basis (likely tied to a bi-annual meeting) to review package tier status and determine if any are in danger of a tier downgrade. A package downgrade requires consensus of the technical steering committee. 
+
+Should any package be flagged for downgrade, the follow will occur:  
+- Packages will receive six months' notification before a tier downgrade is to take place through written e-mail communication (from the PyHC Lead)  
+- Packages then have the opportunity to rectify issues within the six-month period, through changes submitted in a PR  
+    - If changes result in continued compliance with current tier (as deemed by the PyHC Tech Lead and PyHC Lead), no tier regrade happens  
+    - If changes do not result in continued compliance with current tier, a tier regrade happens; packages may later on apply for a tier regrade once more  
+
+# Backwards Compatibility
+<a name="backwards-compatibility"></a>
+This PHEP does not propose a direct change to PyHC package code, simply the inclusion or not of packages within the various tiers, thus it introduces no compatibility concerns.
+
+# Security Implications
+<a name="security-implications"></a>
+This PHEP raises no security implications as it does not interact with any executing code.
+
+# How to Teach This
+<a name="how-to-teach-this"></a>
+This PHEP's contents and changes will be presented on, discussed, and hacked at various PyHC bi-weekly telecons and PyHC bi-annual meetings. Additionally, explanations for tiering, the process of obtaining a PyHC tier, etc. will be posted on the new main Projects page, as well as communicated within a blog post under the PyHC Blog page.   
+
+Further, links to pages (and potentially recordings) that explain how to satisfy each requirement for each tier will be made and communicated through a PyHC blog post (link then emailed to the community and presented at either a telecon or bi-annual meeting, whichever comes first).  
+
+Please note that the PyHC project submission process used thus far will be superseded by this PHEP's PyHC tier request submission. Further, the self evaluations relied upon previously will now serve as the 'starting point' for a package; Technical steering committee evaluation is required to fully verify a package's tier. Also, there are now additional requirement for package tiers that exist apart from the main PyHC standards (see this PHEP in its entirey for more information). PyHC leadership commits to updating the submission process to match this PHEP's specifications.
+
+# Rejected Ideas
+<a name="rejected-ideas"></a>
+None yet to note.
+
+# Open Issues
+<a name="open-issues"></a>
+None yet to note.
+
+# Footnotes
+<a name="footnotes"></a>
+None yet to note.
+
+# Revisions
+<a name="revisions"></a>
+Revision 1 (pending): Initial draft.
+
+# Copyright
+<a name="copyright"></a>
+This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. It should be cited as:
+```
+@techreport(phep2,
+  author = {Julie I. Barnum},
+  title  = {PHEP Package Tiering},
+    year = {2024},
+    type = {PHEP},
+  number = {9999},
+     doi = {10.5281/zenodo.xxxxxxx}
+)
+```