github-actions

Template repo that holds Github actions that can be reused by other repositories.

Highlights

• If a new repo needs to have actions set up for the first time, check out Set Up Actions and Rules.
• To understand how a particular file works, check out the relevant link here:
  • Build .NET and Publish to NuGet.
• In everyday use:
  • Open a PR to run checks for 1) semantic version increase compared to the most recent release, and 2) building in debug/release for both Ubuntu and Windows systems.
  • Once all checks pass, merge the PR to build the project from the main branch and publish a pre-release package to the Github package manager.
  • Create a new release that is tagged with the current version in main to publish a full package.
    • This will build what is in the main branch, publish a full package to Github package manager, and also publish the full package to NuGet.
    • Additionally, this will announce the release on Discord.
• Workflow calling and secrets.
  • When another repo calls the workflows in this repo, it acts as if the workflow was copied and pasted into the other repo; that is, the "checkout" action will checkout the calling repo and not github-actions, even though this is the repo where the workflow exists.
  • To fully understand how secrets work with reusable workflows, check out this link.

Set Up Actions and Rules

• Copy the example_caller_workflow.txt and place it in the .github/workflows folder of the repository that needs to run the actions. As an example, this file will be placed in the .github/workflows folder for bonsai-onix1.

  • Note that the actions will not run unless the file is renamed. Specifically, it should be renamed so that the extension is .yml (e.g., build_and_publish.yml). The name in front of the extension is not critical, but it will show up in the Actions tab of the repository, so using a meaningful name is always better. The name will also appear during status checks for open PRs.

• Ensure that the calling repository (in this case, bonsai-onix1) has a secret named NUGET_APIKEY that contains the NuGet API key generated by the open-ephys organization on NuGet.

  • To add this secret so that it can be passed to the github-actions repo, navigate to the calling repository (bonsai-onix1) and navigate to -> Settings -> Secrets and Variables -> Actions -> New repository secret -> Name: NUGET_APIKEY, Secret: paste the api key generated on NuGet with the proper permissions.

• Create rulesets to ensure that the actions are being used correctly as a check during PRs and commits. To do so, navigate to the calling repository, then go to Settings -> Rules -> Rulesets -> New ruleset -> New branch ruleset.

  • The name can be anything, but "Main" is a good one.
  • Leave the bypass list empty.
  • Click Add target -> Include default branch to ensure that the main branch is protected.
  • Check the following options:
    • Restrict deletions.
    • Require a pull request before merging, with these additional settings.
      • Required approvals: 1
      • Require conversation resolution before merging
    • Require status checks to pass by hitting Add checks and search for the following actions.
      • build_and_publish / check-semver
      • build_and_publish / build (debug, ubuntu-latest)
      • build_and_publish / build (debug, windows-latest)
      • build_and_publish / build (release, ubuntu-latest)
      • build_and_publish / build (release, windows-latest)
        • Note: these checks might not appear if no actions have been run. In that case, ensure that the actions are in the correct folder and named appropriately, then open a PR or push a new commit to an existing PR to trigger actions to run, then come back to this step.
        • Typing in build or check_semver and then selecting the checks is often sufficient to find them; the full name does not need to be typed in.
        • Another thing to note is that the exact wording for these checks will vary depending on the names chosen. Specifically, build_and_publish is the name of the job in the calling workflow (see example_caller_workflow.txt). These names will also be prepended by the name: property of the calling workflow when viewed from an active PR and looking at the status checks.
    • Block force pushes.
  • Save the changes.

Create a Release

• From the main page of the repository, click on the Releases header on the right side of the page.
• Click on Draft a new release.
• Click Choose a tag.
  • Type out vX.Y.Z, where X is the major version, Y is the minor version, and Z is the patch version for this release.
  • Select the Create new tag option at the bottom of the dropdown.
• Hit Generate release notes to automatically create release notes based on all of the PRs that have been merged since the last release.
• Manually move around or remove PRs into sections depending on what was done. Sometimes this will just highlight bug fixes if this is a patch version increase, but if this is a major version release there will probably be breaking changes that need to be highlighted.
• Select Publish Release when it is ready, or hit Save draft to come back to it later.
  • As soon as the release is published, a new action will start running that will go through all the steps listed below Steps which will eventually publish the package to NuGet.

Link Github to Discord via Webhook

For more information on the individual webhooks for Github and Discord, check out this Github page and this Discord page. Much of the inspiration for how to accomplish this integration is also due to this tutorial.

The purpose of this section is to automate the spreading of information when a new release is published. By following the steps below, anytime a new release is created (which triggers the workflows here to run and publish a full package to NuGet) the webhook will run and send a message that there is a new release available for the given repo.

Get Discord ready

• Navigate to the #announcements channel in Discord.
• Click the gear icon to edit the channel.
• Click on Integrations, then click on the Webhooks option.
• If there are no webhooks already, click on New Webhook.
  • Rename the webhook to something meaningful (e.g., Github Releases if this will be a Github webhook that focuses solely on Releases).
• Expand the options for a webhook if they are not already expanded, and click on Copy Webhook URL.

Integrate with Github

• Navigate to the repository that will send the webhook payload.
• Go to the settings, click on Webhooks, and Add webhook.
• Paste the copied webhook URL from Discord in the Payload URL box.
  • At the end of the URL, add /github.
• Change the content type to application/json.
• Leave Secret blank, and leave SSL verification enabled.
• Choose Let me select individual events.
  • Deselect Pushes, which is enabled by default.
  • Select Releases.
• Scroll to the bottom and hit Add webhook.

Verifying connections

Once the webhook is created in Github, it will say that there was a ping payload sent to test it out. This should not trigger an actual message in Discord, but the webhook can be clicked on to see what the HTTP response was to the ping. At the top of the Manage Webhook page, go to Recent Deliveries and click on the ping payload. The response should say 204 to indicate a successful ping.

Once a release is created for this repo, it should send a message to the Discord channel. If there is no message, check the Recent Deliveries page to see if there is a reason given for why it failed.

Build .NET and Publish to NuGet

The following is a breakdown of the lines in the build_dotnet_publish_nuget.yml file, with comments to describe what is happening.

Name

name: Build .NET Project and Publish to NuGet

Defines an easy to read name for the workflow.

On

on:
  workflow_call:
    secrets: 
      NUGET_APIKEY:
        required: true

This workflow will only run whenever it is called from another workflow. It also requires a NUGET_APIKEY secret to be transferred from the calling workflow to this one.

Env

env:
  DOTNET_NOLOGO: true
  DOTNET_CLI_TELEMETRY_OPTOUT: true
  DOTNET_GENERATE_ASPNET_CERTIFICATE: false
  ContinuousIntegrationBuild: true
  CiRunNumber: ${{ github.run_number }}
  CiRunPushSuffix: ${{ github.ref_name }}-ci${{ github.run_number }}
  CiRunPullSuffix: pull-${{ github.event.number }}-ci${{ github.run_number }}

Define environment variables.

• Some of them (DOTNET*) relate to how the dotnet commands will operate, while the continuous integration variables are used to create unique names for each version created, even if it is a pre-release version.

Jobs

jobs:

This sections describes the specific jobs that will be run during the workflow.

Setup

setup:
    runs-on: ubuntu-latest
    outputs:
      build-suffix: ${{ steps.setup-build.outputs.build-suffix }}
    steps:
      - name: Setup Build
        id: setup-build
        run: echo "build-suffix=${{ github.event_name == 'push' && env.CiRunPushSuffix || github.event_name == 'pull_request' && env.CiRunPullSuffix || null }}" >> "$GITHUB_OUTPUT"

Creates the suffix for this run. In terms of SemVer, this defines the pre-release version. For example, if this is a push to main it will generate main-ciXXX as the build-suffix, where XXX is the run number for the workflow. This is an incrementing variable that should guarantee a unique name for each pre-release version.

Note that there is support for creating a pre-release version name during PRs, but this is not uploaded to the Github package manager. If needed, this package can be manually downloaded from the Actions tab for a particular run, and it will contain the build-suffix pull-YYY-ciXXX, where YYY is the event.number property, again an incrementing variable, and ciXXX is the same as above, this is the run number for the workflow.

Check SemVer

Check that the semantic version of the project is increasing in some way compared to the most recent release. For example, if the most recent release is 0.4.0, then the version number be increase the major / minor / patch version. If the current version is still 0.4.0, this check will fail, and if this is a PR it will prevent the PR from merging. Note that while the tag name should match the current version (i.e., v0.4.0 should correspond to version 0.4.0), this step downloads the previous release and compares the VersionPrefix property to ensure that the version is truly increasing.

  check-semver:
    runs-on: ubuntu-latest
    if: github.event_name != 'release'

This step only runs if this is not a release event. Any pushes to main, or any PRs, will always trigger this check.

steps:
      - name: Checkout current version
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

Checkout the code for the current version of the calling repository, whether this is main or a PR.

      - name: Find previous release
        id: previous-release
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          echo "RELEASE_NAME=$(gh release list -L 1 --json tagName -q .[0].tagName)" >> $GITHUB_OUTPUT

Using Github CLI, grab the most recent release in the calling repository.

      - name: Checkout last release version
        uses: actions/checkout@v4
        with:
          ref: ${{ steps.previous-release.outputs.RELEASE_NAME }}
          path: last-release
          sparse-checkout: |
            Directory.Build.props
          sparse-checkout-cone-mode: false

Grab the Directory.Build.props file from the most recent release.

      - name: Extract Versions
        id: extract-versions
        run: |
          echo "CURRENT_VERSION=$(cat ./Directory.Build.props | grep -Po '(?<=VersionPrefix>).*(?=</VersionPrefix>)')" >> $GITHUB_OUTPUT
          echo "PREVIOUS_VERSION=$(cat ./last-release/Directory.Build.props | grep -Po '(?<=VersionPrefix>).*(?=</VersionPrefix>)')" >> $GITHUB_OUTPUT
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.10.12"

Find the VersionPrefix value for the release (PREVIOUS_VERSION) and the current code (CURRENT_VERSION) to compare semantic versions.

      - name: Install semver
        run: python -m pip install semver

      - name: Compare Versions
        run: |
         version_current=$(cat ./Directory.Build.props | grep -Po '(?<=VersionPrefix>).*(?=</VersionPrefix>)')
         version_release=$(cat ./last-release/Directory.Build.props | grep -Po '(?<=VersionPrefix>).*(?=</VersionPrefix>)')
 
         echo "Current Version: $version_current"
         echo "Release Version: $version_release"
         
         if [ ! $(python -c "import semver; print(semver.compare(\"$version_current\", \"$version_release\"))") == 1 ]; then
           echo "::error title=Invalid version number::Version number must be increased"
           exit 1
         fi

Install the Python package semver, and compare the versions. If the CURRENT_VERSION is not incremented in some way, throw an error message and exit 1 to stop the workflow from progressing.

Build

This section builds, packages, and uploads the NuGet package and symbols package for all projects in this solution.

  build:
    needs: [setup, check-semver]                                      # Only run this job if both `setup` and `check-semver` completed successfully
    if: always() && !failure() && !cancelled()                        # Always run this job only if both previous jobs completed successfully, and were not cancelled
    strategy:
      fail-fast: false
      matrix:                                                         # Create a matrix to build in all configurations
        configuration: [debug, release]
        os: [ubuntu-latest, windows-latest]
        include:
          - os: windows-latest
            configuration: release
            collect-packages: true                                    # For the combination of windows-latest and release, collect the packages too
    runs-on: ${{ matrix.os }}
    env:
      CiBuildVersionSuffix: ${{ needs.setup.outputs.build-suffix }}   # Grab the build-suffix created above

This is the first job that is dependent on other jobs; if either of the previous jobs were cancelled or failed in some way (for example, the semantic version did not increase), this job will be skipped.

steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: 8.x
      
      - name: Restore
        run: dotnet restore

Checkout the code for the calling repository, and set up the necessary .NET configurations. Restore any needed packages.

      - name: Build
        run: dotnet build --no-restore --configuration ${{ matrix.configuration }}

For each configuration, build the projects. If this step fails due to a build error, this specific step in the matrix will fail.

      - name: Pack
        id: pack
        if: matrix.collect-packages
        run: dotnet pack --no-build --configuration ${{ matrix.configuration }}

For the Windows Release configuration, also pack the project into a NuGet package.

      - name: Collect packages
        uses: actions/upload-artifact@v4
        if: matrix.collect-packages && steps.pack.outcome == 'success' && always()
        with:
          name: Packages
          if-no-files-found: error
          path: artifacts/package/${{matrix.configuration}}/**

Upload the packages created as an artifact. This allows for the packages to be used in future jobs, and also allows the user to manually download this package by navigating to the Actions tab, and choosing the desired workflow run. At the bottom of the page, there will be an Artifacts section, with a Packages name that can be downloaded containing the NuGet and symbols packages for all projects.

Publish Github

This section will publish to the Github package manager if the event that triggered it is a push to the main branch, or a release was created.

  publish-github:
    runs-on: ubuntu-latest
    permissions:
      packages: write
    needs: [build]
    if: (github.event_name == 'push' || github.event_name == 'release') && always() && !failure() && !cancelled()

This only runs if the build step completed successfully for all configurations.

    steps:
      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: 8.x

      - name: Download packages
        uses: actions/download-artifact@v4
        with:
          name: Packages
          path: Packages

Set up dotnet again, since this is a new job the environment needs to be reinitialized. Download the artifacts created during the build step.

      - name: Push to GitHub Packages
        run: dotnet nuget push "Packages/*.nupkg" --skip-duplicate --no-symbols --api-key ${{secrets.GITHUB_TOKEN}} --source https://nuget.pkg.github.com/${{github.repository_owner}}
        env:
          # This is a workaround for https://github.com/NuGet/Home/issues/9775
          DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER: 0

Publish the package to the Github package manager. Currently, this only accepts the Nuget package itself and not the symbols.

Deploy

For releases, this will push the NuGet package and symbols to NuGet, publishing the full version. Note that this will only occur for full releases, since a pre-release name is not defined for a release event.

  deploy:
    name: Deploy Package
    runs-on: ubuntu-latest
    needs: [build]
    if: github.event_name == 'release' && always() && !failure() && !cancelled()

Deploy the package if the build job completed successfully, and this is a release event. There is an implicit dependency on setup and check-semver completing successfully as well, since build will only run if those jobs were successful.

    steps:
      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: 8.x

      - name: Download packages
        uses: actions/download-artifact@v4
        with:
          name: Packages
          path: Packages

Setup .NET again, and download the packages.

      - name: Publish NuGet Package
        run: dotnet nuget push "Packages/*.nupkg" --skip-duplicate --api-key "${{ secrets.NUGET_APIKEY }}" --source https://api.nuget.org/v3/index.json

Push the packages to NuGet automatically. This will push the package as well as the symbols.

This is the step that requires the NUGET_APIKEY to be passed to the workflow with the correct permissions.