diff --git a/requirements.txt b/requirements.txt index 125f0cf33..17492e03c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,3 +4,4 @@ sphinx-inline-tabs==2021.4.11b9 python-docs-theme==2022.1 sphinx-copybutton==0.5.0 pypa-docs-theme @ git+https://github.com/pypa/pypa-docs-theme.git +sphinx-toolbox==3.5.0 diff --git a/source/conf.py b/source/conf.py index 7ff60c18c..9113b8c68 100644 --- a/source/conf.py +++ b/source/conf.py @@ -36,6 +36,7 @@ 'sphinx.ext.todo', 'sphinx_inline_tabs', 'sphinx_copybutton', + 'sphinx_toolbox.collapse', ] # config for copy button diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 35a2f653d..20ed37740 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -1,14 +1,14 @@ -name: Publish Python 🐍 distributions 📦 to PyPI and TestPyPI +name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI on: push jobs: - build-n-publish: - name: Build and publish Python 🐍 distributions 📦 to PyPI and TestPyPI + build: + name: Build distribution 📦 runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v4 with: @@ -20,21 +20,90 @@ jobs: build --user - name: Build a binary wheel and a source tarball - run: >- - python3 -m - build - --sdist - --wheel - --outdir dist/ - . - # Actually publish to PyPI/TestPyPI - - name: Publish distribution 📦 to Test PyPI - uses: pypa/gh-action-pypi-publish@release/v1 + run: python3 -m build + - name: Store the distribution packages + uses: actions/upload-artifact@v3 with: - password: ${{ secrets.TEST_PYPI_API_TOKEN }} - repository-url: https://test.pypi.org/legacy/ + name: python-package-distributions + path: dist/ + + publish-to-pypi: + name: >- + Publish Python 🐍 distribution 📦 to PyPI + if: startsWith(github.ref, 'refs/tags/') # only publish to PyPI on tag pushes + needs: + - build + runs-on: ubuntu-latest + environment: + name: pypi + url: https://pypi.org/p/ # Replace with your PyPI project name + permissions: + id-token: write # IMPORTANT: mandatory for trusted publishing + + steps: + - name: Download all the dists + uses: actions/download-artifact@v3 + with: + name: python-package-distributions + path: dist/ - name: Publish distribution 📦 to PyPI - if: startsWith(github.ref, 'refs/tags') uses: pypa/gh-action-pypi-publish@release/v1 + + github-release: + name: >- + Sign the Python 🐍 distribution 📦 with Sigstore + and upload them to GitHub Release + needs: + - publish-to-pypi + runs-on: ubuntu-latest + + permissions: + contents: write # IMPORTANT: mandatory for making GitHub Releases + id-token: write # IMPORTANT: mandatory for sigstore + + steps: + - name: Download all the dists + uses: actions/download-artifact@v3 + with: + name: python-package-distributions + path: dist/ + - name: Sign the dists with Sigstore + uses: sigstore/gh-action-sigstore-python@v1.2.3 with: - password: ${{ secrets.PYPI_API_TOKEN }} + inputs: >- + ./dist/*.tar.gz + ./dist/*.whl + - name: Upload artifact signatures to GitHub Release + env: + GITHUB_TOKEN: ${{ github.token }} + # Upload to GitHub Release using the `gh` CLI. + # `dist/` contains the built packages, and the + # sigstore-produced signatures and certificates. + run: >- + gh release upload + '${{ github.ref_name }}' dist/** + --repo '${{ github.repository }}' + + publish-to-testpypi: + name: Publish Python 🐍 distribution 📦 to TestPyPI + needs: + - build + runs-on: ubuntu-latest + + environment: + name: testpypi + url: https://test.pypi.org/p/ + + permissions: + id-token: write # IMPORTANT: mandatory for trusted publishing + + steps: + - name: Download all the dists + uses: actions/download-artifact@v3 + with: + name: python-package-distributions + path: dist/ + - name: Publish distribution 📦 to TestPyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 382fb9fd8..f5ba918db 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -8,7 +8,9 @@ popular choice is having a workflow that's triggered by a ``push`` event. This guide shows you how to publish a Python distribution whenever a tagged commit is pushed. -It will use the `pypa/gh-action-pypi-publish GitHub Action`_. +It will use the `pypa/gh-action-pypi-publish GitHub Action`_ for +publishing. It also uses GitHub's `upload-artifact`_ and `download-artifact`_ actions +for temporarily storing and downloading the source packages. .. attention:: @@ -17,40 +19,66 @@ It will use the `pypa/gh-action-pypi-publish GitHub Action`_. details of building platform specific projects. If you have binary components, check out :ref:`cibuildwheel`'s GitHub Action examples. -Saving credentials on GitHub -============================ +Configuring trusted publishing +============================== -In this guide, we'll demonstrate uploading to both -PyPI and TestPyPI, meaning that we'll have two separate sets -of credentials. And we'll need to save them in the GitHub repository -settings. +This guide relies on PyPI's `trusted publishing`_ implementation to connect +to `GitHub Actions CI/CD`_. This is recommended for security reasons, since +the generated tokens are created for each of your projects +individually and expire automatically. Otherwise, you'll need to generate an +`API token`_ for both PyPI and TestPyPI. In case of publishing to third-party +indexes like :doc:`devpi `, you may need to provide a +username/password combination. + +Since this guide will demonstrate uploading to both +PyPI and TestPyPI, we'll need two trusted publishers configured. +The following steps will lead you through creating the "pending" publishers +for your new :term:`PyPI project `. +However it is also possible to add `trusted publishing`_ to any +pre-existing project, if you are its owner. -Let's begin! 🚀 + .. attention:: -1. Go to https://pypi.org/manage/account/#api-tokens and - create a new `API token`_. If you have the project on PyPI - already, limit the token scope to just that project. - You can call it something like - ``GitHub Actions CI/CD — project-org/project-repo`` - in order for it to be easily distinguishable in the token - list. - **Don't close the page just yet — you won't see that token - again.** -2. In a separate browser tab or window, go to the ``Settings`` - tab of your target repository and then click on `Secrets`_ - in the left sidebar. -3. Create a new secret called ``PYPI_API_TOKEN`` and copy-paste - the token from the first step. -4. Now, go to https://test.pypi.org/manage/account/#api-tokens - and repeat the steps. Save that TestPyPI token on GitHub - as ``TEST_PYPI_API_TOKEN``. + If you followed earlier versions of this guide, you + have created the secrets ``PYPI_API_TOKEN`` and ``TEST_PYPI_API_TOKEN`` + for direct PyPI and TestPyPI access. These are obsolete now and + you should remove them from your GitHub repository and revoke + them in your PyPI and TestPyPI account settings in case you are replacing your old setup with the new one. - .. attention:: + +Let's begin! 🚀 + +1. Go to https://pypi.org/manage/account/publishing/. +2. Fill in the name you wish to publish your new + :term:`PyPI project ` under + (the ``name`` value in your ``setup.cfg`` or ``pyproject.toml``), + the GitHub repository owner's name (org or user), + and repository name, and the name of the release workflow file under + the ``.github/`` folder, see :ref:`workflow-definition`. + Finally, add the name of the GitHub Environment + (``pypi``) we're going set up under your repository. + Register the trusted publisher. +3. Now, go to https://test.pypi.org/manage/account/publishing/ and repeat + the second step, but this time, enter ``testpypi`` as the name of the + GitHub Environment. +4. Your "pending" publishers are now ready for their first use and will + create your projects automatically once you use them + for the first time. + + .. note:: If you don't have a TestPyPI account, you'll need to create it. It's not the same as a regular PyPI account. + .. attention:: + + For security reasons, you must require `manual approval `_ + on each run for the ``pypi`` environment. + + +.. _workflow-definition: + Creating a workflow definition ============================== @@ -67,70 +95,116 @@ should make GitHub run this workflow: :language: yaml :end-before: jobs: +Checking out the project and building distributions +=================================================== + +We will have to define two jobs to publish to PyPI +and TestPyPI respectively, and an additional job to +build the distribution packages. + +First, we'll define the job for building the dist packages of +your project and storing them for later use: + +.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml + :language: yaml + :start-at: jobs: + :end-before: Install pypa/build + +This will download your repository into the CI runner and then +install and activate the newest available Python 3 release. + +And now we can build the dists from source and store them. +In this example, we'll use the ``build`` package. +So add this to the steps list: + +.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml + :language: yaml + :start-at: Install pypa/build + :end-before: publish-to-pypi Defining a workflow job environment =================================== -Now, let's add initial setup for our job. It's a process that -will execute commands that we'll define later. +Now, let's add initial setup for our job that will publish to PyPI. +It's a process that will execute commands that we'll define later. In this guide, we'll use the latest stable Ubuntu LTS version -provided by GitHub Actions: +provided by GitHub Actions. This also defines a GitHub Environment +for the job to run in its context and a URL to be displayed in GitHub's +UI nicely. Additionally, it allows aqcuiring an OpenID Connect token +that the ``pypi-publish`` actions needs to implement secretless +trusted publishing to PyPI. .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml - :start-after: on: + :start-after: path: dist/ :end-before: steps: +This will also ensure that the PyPI publishing workflow is only triggered +if the current commit is tagged. -Checking out the project and building distributions -=================================================== +Publishing the distribution to PyPI +=================================== -Then, add the following under the ``build-n-publish`` section: +Finally, add the following steps at the end: .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml - :start-after: runs-on: - :end-before: Install pypa/build - -This will download your repository into the CI runner and then -install and activate the newest available Python 3 release. + :start-after: id-token: write + :end-before: github-release: -And now we can build dists from source. In this example, we'll -use ``build`` package. +This step uses the `pypa/gh-action-pypi-publish`_ GitHub +Action: after the stored distribution package has been +downloaded by the `download-artifact`_ action, it uploads +the contents of the ``dist/`` folder into PyPI unconditionally. -.. tip:: +Signing the distribution packages +================================= - You can use any other method for building distributions as long as - it produces ready-to-upload artifacts saved into the - ``dist/`` folder. You can even use ``actions/upload-artifact`` and - ``actions/download-artifact`` to transfer files between jobs or make them - accessible for download from the web CI interface. +The following job signs the distribution packages with `Sigstore`_, +the same artifact signing system `used to sign CPython `_. -So add this to the steps list: +It uses the `sigstore/gh-action-sigstore-python GitHub Action`_, +and then uploads them to a GitHub Release. .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml - :start-after: version: "3.x" - :end-before: Actually publish to PyPI/TestPyPI + :start-at: github-release: + :end-before: publish-to-testpypi -Publishing the distribution to PyPI and TestPyPI -================================================ +.. note:: -Finally, add the following steps at the end: + This is a replacement for GPG signatures, for which support has been + `removed from PyPI `_. + However, this job is not mandatory for uploading to PyPI and can be omitted. + + +Separate workflow for publishing to TestPyPI +============================================ + +Now, repeat these steps and create another job for +publishing to the TestPyPI package index under the ``jobs`` +section: .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml - :start-after: Actually publish to PyPI/TestPyPI + :start-at: publish-to-testpypi + +.. tip:: -These two steps use the `pypa/gh-action-pypi-publish`_ GitHub -Action: the first one uploads contents of the ``dist/`` folder -into TestPyPI unconditionally and the second does that to -PyPI, but only if the current commit is tagged. It is recommended -you use the latest release tag; a tool like GitHub's dependabot can keep -these updated regularly. + Requiring manual approvals in the ``testpypi`` GitHub Environment is typically unnecessary as it's designed to run on each commit to the main branch and is often used to indicate a healthy release publishing pipeline. +The whole CI/CD workflow +======================== + +This paragraph showcases the whole workflow after following the above guide. + +.. collapse:: Click here to display the entire GitHub Actions CI/CD workflow definition + + .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml + :language: yaml + That's all, folks! ================== @@ -140,6 +214,11 @@ And it'll publish any push to TestPyPI which is useful for providing test builds to your alpha users as well as making sure that your release pipeline remains healthy! +.. note:: + + It is recommended to keep the integrated GitHub Actions at their latest + versions, updating them frequently. + .. _API token: https://pypi.org/help/#apitoken .. _GitHub Actions CI/CD: https://github.com/features/actions @@ -148,5 +227,13 @@ sure that your release pipeline remains healthy! https://github.com/pypa/gh-action-pypi-publish .. _`pypa/gh-action-pypi-publish GitHub Action`: https://github.com/marketplace/actions/pypi-publish +.. _`download-artifact`: + https://github.com/actions/download-artifact +.. _`upload-artifact`: + https://github.com/actions/upload-artifact +.. _Sigstore: https://www.sigstore.dev/ +.. _`sigstore/gh-action-sigstore-python GitHub Action`: + https://github.com/marketplace/actions/gh-action-sigstore-python .. _Secrets: https://docs.github.com/en/actions/reference/encrypted-secrets +.. _trusted publishing: https://docs.pypi.org/trusted-publishers/