Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert Azure Pipeline to Github Actions #543

Merged
merged 50 commits into from
Nov 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
9954644
add test python requirements
lawrence-mbf Sep 8, 2023
de7671a
migrate azure pipeline to github actions
lawrence-mbf Sep 8, 2023
39f6acd
Fix test workflow syntax
lawrence-mbf Sep 8, 2023
2089bac
attempt to fix python setup
lawrence-mbf Sep 8, 2023
c0806bb
switch run-test to run-command
lawrence-mbf Sep 8, 2023
b516aac
try to upload to codecov
lawrence-mbf Sep 8, 2023
45fac03
fix matlab syntax error
lawrence-mbf Sep 8, 2023
83e4cdd
retab functions
lawrence-mbf Sep 8, 2023
4ccde42
Refactor add column function pull out functions
lawrence-mbf Sep 8, 2023
cffd360
Fix CI testing
lawrence-mbf Sep 8, 2023
ef4832c
Refactor addTableColumn
lawrence-mbf Sep 8, 2023
28d9bf3
Fix addColumn usage in tests and tutorials
lawrence-mbf Sep 8, 2023
84ba20b
update error id chekcing
lawrence-mbf Sep 8, 2023
7f92bbe
check ci env setting
lawrence-mbf Sep 8, 2023
4a0f268
fix debug pipeline
lawrence-mbf Sep 8, 2023
fbe0ffc
more env fixing
lawrence-mbf Sep 8, 2023
51d194d
fix expected error test id
lawrence-mbf Sep 8, 2023
c3fbce7
remove debug prints from ci
lawrence-mbf Sep 8, 2023
e1d091a
add source files to download repo
lawrence-mbf Sep 8, 2023
f43c703
try different test suite
lawrence-mbf Sep 8, 2023
1923456
try different test suite (again)
lawrence-mbf Sep 8, 2023
ddac00c
Merge branch 'master' into convert-ci-to-actions
ehennestad Sep 30, 2024
5cfc738
Try using MATLAB R2024a for tests in workflow
ehennestad Sep 30, 2024
dd60ea2
Give test workflow a shorter name
ehennestad Sep 30, 2024
6d3d5d7
Add codecov settings file
ehennestad Sep 30, 2024
d4b9f94
Merge branch 'master' into convert-ci-to-actions
bendichter Sep 30, 2024
23c3933
Merge branch 'convert-ci-to-actions' of https://github.com/NeurodataW…
ehennestad Sep 30, 2024
026b5ad
Update tests.yml
ehennestad Sep 30, 2024
9365c27
Merge master into branch
ehennestad Nov 1, 2024
b4421e5
Merge branch 'add-code-section-read-remote-tutorial' into convert-ci-…
ehennestad Nov 1, 2024
6ba4cc8
Update versions of dependent actions to latest versions
ehennestad Nov 1, 2024
51f68be
Merge branch 'master' into convert-ci-to-actions
ehennestad Nov 1, 2024
3a5c9ea
Update test badge in README.md
ehennestad Nov 1, 2024
7ad1642
Merge branch 'convert-ci-to-actions' of https://github.com/NeurodataW…
ehennestad Nov 1, 2024
4825605
Merge branch 'master' into convert-ci-to-actions
ehennestad Nov 1, 2024
533fce8
Reset / remove files related to dynamic table refactor PR #612
ehennestad Nov 2, 2024
a3063e4
Rename utilities folder to tools
ehennestad Nov 2, 2024
eef5de4
Separate test workflows for master branch and development
ehennestad Nov 2, 2024
766d120
Add codespell workflow
ehennestad Nov 2, 2024
0dfe3e5
Ignore changes to markdown files or github workflows in workflow for …
ehennestad Nov 2, 2024
36a55ef
Update codespell workflow
ehennestad Nov 2, 2024
841c548
Fix wrong casing for options name
ehennestad Nov 2, 2024
b11d945
provoke codespell error
ehennestad Nov 2, 2024
0de1cf5
Fix codespell config
ehennestad Nov 2, 2024
27b7968
Update README.md
ehennestad Nov 2, 2024
50fa17c
Undo create separate test workflows for master and dev
ehennestad Nov 2, 2024
8bc3f61
Merge branch 'convert-ci-to-actions' of https://github.com/NeurodataW…
ehennestad Nov 2, 2024
dff6655
Update validateCodecovSettings.m
ehennestad Nov 2, 2024
bce141c
Fix yaml syntax issue
ehennestad Nov 2, 2024
6ef9d12
Fix readme and paths-ignore in run_tests workflow
ehennestad Nov 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions +tests/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
pynwb
hdf5plugin
2 changes: 1 addition & 1 deletion .codespellrc
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[codespell]
skip = *.html,*logo_matnwb.svg,*fastsearch.m,*.yaml,*UpdateThirdPartyFromUpstream.sh,*testResults.xml
skip = *.html,*logo_matnwb.svg,*fastsearch.m,*.yaml,*testResults.xml
ignore-words-list = DNE,nd,whos
12 changes: 12 additions & 0 deletions .github/.codecov.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
coverage:
range: "90...100"
status:
project:
default:
target: 90 # Set the desired coverage target as 90%
threshold: 1 # Allowable drop in coverage
patch:
default:
# 75% of the changed code must be covered by tests
threshold: 25
only_pulls: true
46 changes: 46 additions & 0 deletions .github/workflows/run_codespell.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Note: This workflow allows specifying a custom location for the Codespell
# configuration file by defining CONFIG_FILE as an environment variable.
# A defined subset of options is extracted from this file and passed to the
# Codespell action. This also ensures that the output of the codespell action
# prints out the values of these options.
# Todo: Generalize the extraction of codespell input arguments/options.

name: Codespell

on:
pull_request:
branches:
- master
push:
branches-ignore:
- master

jobs:
codespell:
name: Check for spelling errors
runs-on: ubuntu-latest
env:
CONFIG_FILE: .codespellrc

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Extract codespell configurations from configuration file
id: config
run: |
# Extract 'skip' value from the config file, excluding 'skip = ' part
skip=$(grep -E '^skip' "$CONFIG_FILE" | sed 's/^skip *= *//')

# Extract 'ignore-words-list' value from the config file, excluding 'ignore-words-list = ' part
ignore_words=$(grep -E '^ignore-words-list' "$CONFIG_FILE" | sed 's/^ignore-words-list *= *//')

# Export values as environment variables
echo "SKIP=$skip" >> $GITHUB_ENV
echo "IGNORE_WORDS_LIST=$ignore_words" >> $GITHUB_ENV

- name: Codespell
uses: codespell-project/actions-codespell@v2
with:
skip: "${{ env.SKIP }}"
ignore_words_list: "${{ env.IGNORE_WORDS_LIST }}"
82 changes: 82 additions & 0 deletions .github/workflows/run_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Run and publish MATLAB tests with coverage
name: Run tests

on:
pull_request:
branches:
- master
paths-ignore:
- "*.md"
- "*.codespellrc"
- ".github/**"
push:
branches:
- master

jobs:
run_tests:
name: Run MATLAB tests
runs-on: ubuntu-latest
steps:
- name: check out repository
uses: actions/checkout@v4
- name: install python
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: configure python env
run: |
python -m pip install -U pip
pip install -r +tests/requirements.txt
echo "HDF5_PLUGIN_PATH=$(python -c "import hdf5plugin; print(hdf5plugin.PLUGINS_PATH)")" >> "$GITHUB_ENV"
- name: install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a # this is necessary to test dynamic filters
- name: run tests
uses: matlab-actions/run-command@v2
with:
command: results = assertSuccess(nwbtest); assert(~isempty(results), 'No tests ran');
- name: upload JUnit results
uses: actions/upload-artifact@v4
with:
name: test-results
path: testResults.xml
retention-days: 1
- name: upload coverage results
uses: actions/upload-artifact@v4
with:
name: test-coverage
path: ./coverage.xml
publish_junit:
name: Publish JUnit Test Results
runs-on: ubuntu-latest
if: ${{ always() }}
needs: [run_tests]
steps:
- name: retrieve result files
uses: actions/download-artifact@v4
with:
name: test-results
- name: publish results
uses: mikepenz/action-junit-report@v4
with:
report_paths: 'testResults.xml'
publish_coverage:
name: Publish Cobertura Test Coverage
runs-on: ubuntu-latest
needs: [run_tests]
steps:
- name: check out repository
uses: actions/checkout@v4
- name: retrieve code coverage files
uses: actions/download-artifact@v4
with:
name: test-coverage
- name: publish on Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.xml
name: codecov-matnwb
verbose: true
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
![MatNWB Logo](logo/logo_matnwb_small.png)

[![Open in MATLAB Online](https://www.mathworks.com/images/responsive/global/open-in-matlab-online.svg)](https://matlab.mathworks.com/open/github/v1?repo=NeurodataWithoutBorders/matnwb&file=tutorials/basicUsage.mlx)
[![codecov](https://codecov.io/gh/NeurodataWithoutBorders/matnwb/branch/master/graph/badge.svg?token=apA7F24NsO)](https://codecov.io/gh/NeurodataWithoutBorders/matnwb) ![Azure DevOps tests](https://img.shields.io/azure-devops/tests/NeurodataWithoutBorders/matnwb/4)
[![codecov](https://codecov.io/gh/NeurodataWithoutBorders/matnwb/branch/master/graph/badge.svg?token=apA7F24NsO)](https://codecov.io/gh/NeurodataWithoutBorders/matnwb)
[![Run tests](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_tests.yml/badge.svg)](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_tests.yml?query=event%3Apush+branch%3Amaster)
[![Codespell](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_codespell.yml/badge.svg?branch=master)](https://github.com/NeurodataWithoutBorders/matnwb/actions/workflows/run_codespell.yml?query=event%3Apush+branch%3Amaster)

MatNWB is a Matlab interface for reading and writing Neurodata Without Borders (NWB) 2.x files.

Expand Down
48 changes: 0 additions & 48 deletions azure-pipelines.yml

This file was deleted.

16 changes: 16 additions & 0 deletions tools/maintenance/validateCodecovSettings.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
function validateCodecovSettings()
% validateCodecovSettings Validate a codecov settings file.
%
% Note: This is a utility function developer's can use to check the
% codecov settings file in .github/.codecov.yaml

sysCommand = sprintf("curl -X POST --data-binary @%s https://codecov.io/validate", ...
fullfile(misc.getMatnwbDir, '.github', '.codecov.yaml'));

[status, message] = system(sysCommand);

assert(status == 0, 'Curl command failed')

assert(contains(message, 'Valid!'), ...
'Codecov settings file is invalid')
end