Skip to content

Commit

Permalink
Merge branch 'feature/add_transient_noise_mask' into merge/dev
Browse files Browse the repository at this point in the history
  • Loading branch information
ruxandra-valcu authored Nov 7, 2023
2 parents f98e825 + 5249398 commit 9d526eb
Show file tree
Hide file tree
Showing 26 changed files with 1,572 additions and 139 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,17 @@ jobs:
echo "IMAGE_SPEC=${IMAGE_SPEC}" >> $GITHUB_ENV
echo "DATE_TAG=${DATE_TAG}" >> $GITHUB_ENV
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_TOKEN }}
- name: Build and push
id: docker_build_push
uses: docker/build-push-action@v4
uses: docker/build-push-action@v5
with:
context: ./
file: ./.ci_helpers/docker/${{ matrix.image_name }}.dockerfile
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/packit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v4.7.0
uses: actions/setup-python@v4.7.1
with:
python-version: 3.9

Expand Down Expand Up @@ -52,7 +52,7 @@ jobs:
needs: build-artifact
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-python@v4.7.0
- uses: actions/setup-python@v4.7.1
name: Install Python
with:
python-version: 3.9
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pypi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v4.7.0
uses: actions/setup-python@v4.7.1
with:
python-version: 3.9

Expand Down Expand Up @@ -56,7 +56,7 @@ jobs:
needs: build-artifact
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-python@v4.7.0
- uses: actions/setup-python@v4.7.1
name: Install Python
with:
python-version: 3.9
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/windows.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,12 @@ jobs:
# Check data endpoint
curl http://localhost:8080/data/
- name: Setup Python
uses: actions/setup-python@v4.7.0
uses: actions/setup-python@v4.7.1
with:
python-version: ${{ matrix.python-version }}
architecture: x64
- name: Cache conda
uses: actions/cache@v3.3.1
uses: actions/cache@v3.3.2
env:
# Increase this value to reset cache if '.ci_helpers/py{0}.yaml' has not changed
CACHE_NUMBER: 0
Expand Down
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ exclude: |
)
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
Expand All @@ -24,7 +24,7 @@ repos:
args: ["--profile", "black", "--filter-files"]

- repo: https://github.com/psf/black
rev: 23.7.0
rev: 23.10.1
hooks:
- id: black

Expand All @@ -34,7 +34,7 @@ repos:
- id: flake8

- repo: https://github.com/codespell-project/codespell
rev: v2.2.5
rev: v2.2.6
hooks:
- id: codespell
# Checks spelling in `docs/source` and `echopype` dirs ONLY
Expand Down
4 changes: 2 additions & 2 deletions docs/source/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ This diagram depicts the complete workflow we use in the source GitHub repositor
- ``doc patch``: Updates to the documentation that refer to the current ``echopype``
release can be pushed out immediately to the
`echopype documentation site <https://echopype.readthedocs.io>`_
by contibuting patches (PRs) to the ``stable`` branch. See `Documentation development`_
by contributing patches (PRs) to the ``stable`` branch. See `Documentation development`_
below for more details.
- ``code patch``: Code development is carried out as patches (PRs) to the ``dev``
branch; changes in the documentation corresponding to changes in the code can be
Expand Down Expand Up @@ -231,7 +231,7 @@ To view the HTML files generated by Jupyter Book, open the
``docs/_build/html/index.html`` in your browser.

Jupyter Book `configurations <https://jupyterbook.org/en/stable/customize/config.html>`_ can be found in the ``docs/source/_config.yml`` file.
The `table of contents <https://jupyterbook.org/en/stable/structure/toc.html>`_ arragements for the sidebar can be found in ``docs/source/_toc.yml`` file.
The `table of contents <https://jupyterbook.org/en/stable/structure/toc.html>`_ arrangements for the sidebar can be found in ``docs/source/_toc.yml`` file.

When ready to commit your changes, please pull request your changes to the `stable` branch. Once the PR is submitted, the `pre-commit` CI will run for basic spelling and formatting check (See the `pre-commit hooks section <contributing.html#pre-commit-hooks>`_ for more details). Any changes from the `pre-commit` check have to be pulled to your branch (via `git pull`) before your push further commits. You will also be able to view the newly built doc in the PR via the "docs/readthedocs.org:echopype" entry shown below.

Expand Down
4 changes: 2 additions & 2 deletions docs/source/data-proc.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Data processing

Echopype data processing funcionalities are structured into different subpackages with expandability and a series of [data processing levels](processing-levels) in mind. Once the data is converted from the raw instrument data files to standardized [`EchoData` objects](data-format:echodata-object) (or stored in `.zarr` or `.nc` format) and calibrated, the core input and output of most subsequent functions are generic [xarray `Datasets`](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html). This design allows new processing functions be easily added without needing to understand specialized objects, other than functions needing access of data stored only in the raw-converted `EchoData` objects.
Echopype data processing functionalities are structured into different subpackages with expandability and a series of [data processing levels](processing-levels) in mind. Once the data is converted from the raw instrument data files to standardized [`EchoData` objects](data-format:echodata-object) (or stored in `.zarr` or `.nc` format) and calibrated, the core input and output of most subsequent functions are generic [xarray `Datasets`](https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html). This design allows new processing functions be easily added without needing to understand specialized objects, other than functions needing access of data stored only in the raw-converted `EchoData` objects.

The section [**Data processing functionalities**](data-proc:functions) provides information for current processing functions and their usage.

Expand All @@ -9,6 +9,6 @@ The section [**Additional information for processed data**](data-proc:additional
(data-proc:format)=
## Format of processed data

Once raw data (represented by the `EchoData` objects) are calibrated (via [`compute_Sv`](echopype.calibrate.compute_Sv)), the calibrated data and the outputs of all subsequent [processing functions](data-process:funcionalities) are generic [xarray Datasets](https://docs.xarray.dev/en/stable/user-guide/data-structures.html#dataset).
Once raw data (represented by the `EchoData` objects) are calibrated (via [`compute_Sv`](echopype.calibrate.compute_Sv)), the calibrated data and the outputs of all subsequent [processing functions](data-process:functionalities) are generic [xarray Datasets](https://docs.xarray.dev/en/stable/user-guide/data-structures.html#dataset).
We currently do not follow any specific conventions for processed data, but we retain provenance information in the dataset, including the [data processing levels](./processing-levels.md).
However, whether and how data variables used in the processing will be stored remain to be determined.
11 changes: 10 additions & 1 deletion echopype/clean/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
from .api import estimate_noise, remove_noise
from .api import (
estimate_noise,
get_attenuation_mask,
get_impulse_noise_mask,
get_transient_noise_mask,
remove_noise,
)

__all__ = [
"estimate_noise",
"remove_noise",
"get_impulse_noise_mask",
"get_transient_noise_mask",
"get_attenuation_mask",
]
Loading

0 comments on commit 9d526eb

Please sign in to comment.