Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/annotation tests #82

Merged
merged 26 commits into from
May 13, 2024
Merged

Feature/annotation tests #82

merged 26 commits into from
May 13, 2024

Conversation

Marcel-Mueck
Copy link
Collaborator

What

The changes for this PR are threefold

Testing

Testing is performed by github actions

@Marcel-Mueck Marcel-Mueck requested a review from endast May 8, 2024 13:49
Copy link
Collaborator

@endast endast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, i just saw a few small things that we might want to address.

pipelines/resources/absplice_splicing_pred_DNA.snakefile Outdated Show resolved Hide resolved
pipelines/resources/environment_spliceai_rocksdb.yaml Outdated Show resolved Hide resolved
tests/annotations/test_annotations.py Outdated Show resolved Hide resolved
pipelines/resources/absplice.yaml Outdated Show resolved Hide resolved
@endast endast self-requested a review May 13, 2024 11:04
Copy link
Collaborator

@endast endast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!
image

@Marcel-Mueck Marcel-Mueck merged commit 99ed88e into main May 13, 2024
13 checks passed
@Marcel-Mueck Marcel-Mueck deleted the feature/annotation-tests branch May 13, 2024 11:14
endast added a commit that referenced this pull request May 13, 2024
commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>
endast added a commit that referenced this pull request May 13, 2024
commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>
endast added a commit that referenced this pull request May 14, 2024
commit 5715e99
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Tue May 14 09:30:02 2024 +0200

    Fix omitted variants, correct test cases.  (#84)

    * addert assert to make sure no variants are omitted when merging annotations

    * changed expected files, sorts input files in aggregate_abscores

    * fixup! Format Python code with psf/black pull_request

    ---------

    Co-authored-by: “Marcel-Mueck” <“mueckm1@gmail.com”>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

commit 6f04801
Author: Kayla Meyer <129152803+meyerkm@users.noreply.github.com>
Date:   Mon May 13 14:49:59 2024 +0200

    Revert "Feature streamline cv training" (#79)

    * Revert "Feature streamline cv training (#71)"

    This reverts commit d9ede28.

    * Add in association-only flag in rule config. Add in rule evaluate to complete cv-training-association-testing pipeline.

    * n_avg_chunks = 1

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Eva Holtkamp <eva.holtkamp@gmx.de>

commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>
endast added a commit that referenced this pull request May 14, 2024
commit 5715e99
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Tue May 14 09:30:02 2024 +0200

    Fix omitted variants, correct test cases.  (#84)

    * addert assert to make sure no variants are omitted when merging annotations

    * changed expected files, sorts input files in aggregate_abscores

    * fixup! Format Python code with psf/black pull_request

    ---------

    Co-authored-by: “Marcel-Mueck” <“mueckm1@gmail.com”>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

commit 6f04801
Author: Kayla Meyer <129152803+meyerkm@users.noreply.github.com>
Date:   Mon May 13 14:49:59 2024 +0200

    Revert "Feature streamline cv training" (#79)

    * Revert "Feature streamline cv training (#71)"

    This reverts commit d9ede28.

    * Add in association-only flag in rule config. Add in rule evaluate to complete cv-training-association-testing pipeline.

    * n_avg_chunks = 1

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Eva Holtkamp <eva.holtkamp@gmx.de>

commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>
endast added a commit that referenced this pull request May 14, 2024
* Use dynamic paths from rules for preprocessing pipeline

* Rename

* Use black dag

* Update preprocessing.md

Fix link

* Snakefmt pipelines

* Squashed commit of the following:

commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

* Revert "Squashed commit of the following:"

This reverts commit 011273c.

* Squashed commit of the following:

commit 5715e99
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Tue May 14 09:30:02 2024 +0200

    Fix omitted variants, correct test cases.  (#84)

    * addert assert to make sure no variants are omitted when merging annotations

    * changed expected files, sorts input files in aggregate_abscores

    * fixup! Format Python code with psf/black pull_request

    ---------

    Co-authored-by: “Marcel-Mueck” <“mueckm1@gmail.com”>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

commit 6f04801
Author: Kayla Meyer <129152803+meyerkm@users.noreply.github.com>
Date:   Mon May 13 14:49:59 2024 +0200

    Revert "Feature streamline cv training" (#79)

    * Revert "Feature streamline cv training (#71)"

    This reverts commit d9ede28.

    * Add in association-only flag in rule config. Add in rule evaluate to complete cv-training-association-testing pipeline.

    * n_avg_chunks = 1

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Eva Holtkamp <eva.holtkamp@gmx.de>

commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

* Revert "Squashed commit of the following:"

This reverts commit 8ad84ea.
endast added a commit that referenced this pull request May 14, 2024
* add pytest to env

* Set workdir to repo

So we can run the tests from anyhwere

* Add string sample

Update test data

* Fix sample names

* Add SAM prefix

* add xdist run

* don't cast samples to ints

* fixup! Format Python code with psf/black pull_request

* Add sample str test to process_sparse_gt test

* Remove used import and variable

* update pytest

* update gh runner

* Fix xdist version

* Downgrade xdist

* Squashed commit of the following:

commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

* Revert "Squashed commit of the following:"

This reverts commit 5ce3e77.

* Squashed commit of the following:

commit 5715e99
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Tue May 14 09:30:02 2024 +0200

    Fix omitted variants, correct test cases.  (#84)

    * addert assert to make sure no variants are omitted when merging annotations

    * changed expected files, sorts input files in aggregate_abscores

    * fixup! Format Python code with psf/black pull_request

    ---------

    Co-authored-by: “Marcel-Mueck” <“mueckm1@gmail.com”>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

commit 6f04801
Author: Kayla Meyer <129152803+meyerkm@users.noreply.github.com>
Date:   Mon May 13 14:49:59 2024 +0200

    Revert "Feature streamline cv training" (#79)

    * Revert "Feature streamline cv training (#71)"

    This reverts commit d9ede28.

    * Add in association-only flag in rule config. Add in rule evaluate to complete cv-training-association-testing pipeline.

    * n_avg_chunks = 1

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Eva Holtkamp <eva.holtkamp@gmx.de>

commit 99ed88e
Author: Marcel Mück <mueckm1@gmail.com>
Date:   Mon May 13 13:14:16 2024 +0200

    Feature/annotation tests (#82)

    * Create test skeleton for annotations

    * added annotation script to deeprvat setup, updated docs to reflect that change, added first test for annotation pipeline.

    * added tests for annotation pipeline, variant file now parquet

    * added data for test

    * added data for tests

    * Test for merge_deeprsea_pcas function

    * added test for absplice score aggregation

    * added robustness for mixed entry types in ID column of input vcf, created test case

    * added further tests

    * added pyranges to environment

    * Update absplice.yaml

    * Update environment_spliceai_rocksdb.yaml

    ---------

    Co-authored-by: Magnus Wahlberg <endast@gmail.com>
    Co-authored-by: Mück <m991k@b260-pc003.inet.dkfz-heidelberg.de>
    Co-authored-by: PMBio <PMBio@users.noreply.github.com>

* Revert "Squashed commit of the following:"

This reverts commit 1d0bd6d.

* Update micromamba

* Use xdist for annotation tests

* Move above comment

* Lock ubuntu version

* Cleanup yaml

* Lock down ubuntu

---------

Co-authored-by: PMBio <PMBio@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants