Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix – Inferred cell type field not in SDRF breaks experiment design #69

Conversation

alfonsomunozpomer
Copy link
Member

@alfonsomunozpomer alfonsomunozpomer commented Sep 28, 2023

If inferred cell type fields are not present in the SDRF file (e.g. E-CURD-4) then the script assigns a null value to the experiment design column ordinal.

In such cases, the inferred cell type column should be placed in the last position (highest ordinal plus one).

Things I did here:

  • Utility/Common functions were spread between two scripts, I consolidated into just common_routines.sh
  • I extracted the AWK code into a separate file that’s easier to understand and work with (maybe even test!)
  • Rather than piping each SQL INSERT to psql all the statements are written to a file, enclosed in a transaction block and then have psql process it
  • Moved all the fixtures to the tests directory (remember: the fixtures directory is for the scripts that we use to generate fixtures for atlas-web-single-cell)
  • Replaced the condensed SDRF to test the experiment design load to use a file where a field isn’t present in the SDRF file
  • Lastly, I updated the image used in the tests, which is a story for another time

@alfonsomunozpomer alfonsomunozpomer marked this pull request as ready for review October 6, 2023 13:05
@alfonsomunozpomer alfonsomunozpomer added the bug Something isn't working label Oct 6, 2023
Copy link
Contributor

@ke4 ke4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine with me 👍

@ke4 ke4 requested review from anilthanki and pmb59 October 9, 2023 13:43
@ke4 ke4 requested a review from anilthanki October 9, 2023 14:53
Copy link

@anilthanki anilthanki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pmb59
Copy link
Member

pmb59 commented Oct 10, 2023

Hi @alfonsomunozpomer can you please rebase to include this #68

Copy link
Member

@pmb59 pmb59 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @alfonsomunozpomer !

I have a small request related to

Replaced the condensed SDRF to test the experiment design load to use a file where a field isn’t present in the SDRF file

New experiment for test is E-CURD-4 (Arabidopsis). Could we still keep the previous one E-MTAB-2983 (human) as a separate test in the .bats file?

@@ -1,4 +1,4 @@
FROM quay.io/ebigxa/atlas-db-scxa-base:0.1.0
FROM quay.io/ebigxa/atlas-db-scxa-base:0.15.0.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tag doesn't have the best naming - but it does exist in quay so let's leave it as it is

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to keep it in sync with major releases of Single Cell Expression Atlas (so that v15 corresponds to 0.15.0.0).

bin/add_exps_to_collection.sh Show resolved Hide resolved
…ll-type-field-not-in-sdrf-breaks-experiment-design

# Conflicts:
#	bin/add_exps_to_collection.sh
#	bin/create_collection.sh
#	bin/delete_collection.sh
#	bin/delete_exp_from_collection.sh
#	bin/get_experiment_info.sh
#	bin/load_db_scxa_analytics.sh
#	bin/load_db_scxa_analytics_pg9.sh
#	bin/load_db_scxa_cell_clusters.sh
#	bin/load_db_scxa_dimred.sh
#	bin/load_db_scxa_marker_genes.sh
#	bin/load_exp_design.sh
#	bin/modify_collection.sh
@alfonsomunozpomer
Copy link
Member Author

alfonsomunozpomer commented Oct 10, 2023

LGTM @alfonsomunozpomer !

I have a small request related to

Replaced the condensed SDRF to test the experiment design load to use a file where a field isn’t present in the SDRF file

New experiment for test is E-CURD-4 (Arabidopsis). Could we still keep the previous one E-MTAB-2983 (human) as a separate test in the .bats file?

Done.

@alfonsomunozpomer alfonsomunozpomer merged commit 908307e into develop Oct 10, 2023
1 check passed
@alfonsomunozpomer alfonsomunozpomer deleted the bugfix/inferred-cell-type-field-not-in-sdrf-breaks-experiment-design branch October 10, 2023 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants