Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start extracting a cellranger-only configuration subset #421

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
f3fd110
Start extracting cellranger-related args.
macklin-10x Apr 5, 2024
7a216f2
Start extracting the CR subset of args into a type.
macklin-10x Apr 7, 2024
e7b2789
Remove unused bug reports config.
macklin-10x Apr 7, 2024
59629ad
Move the nopager config option.
macklin-10x Apr 7, 2024
1af6d20
Remove dead NOPRETTY arg and fix up NOPAGER.
macklin-10x Apr 7, 2024
f992b77
Remove pager function from enclone_ranger.
macklin-10x Apr 7, 2024
a25e9ac
Restore NOPAGER proc.
macklin-10x Apr 7, 2024
9fe7093
Remove NOPAGER from required args.
macklin-10x Apr 7, 2024
1d7ccbc
Extract the PROTO arg.
macklin-10x Apr 7, 2024
53c577a
Extract the PROTO_METADATA arg.
macklin-10x Apr 7, 2024
935beee
Mark proto/etc as done.
macklin-10x Apr 16, 2024
0f7d07c
Move NUMI into cr_opts.
macklin-10x Apr 16, 2024
96a71e1
Move NUMI_RATIO into cr_opts.
macklin-10x Apr 16, 2024
2282461
Move NUMI_RATIO into cr_opts.
macklin-10x Apr 16, 2024
14e09a4
Make a note that NOPRINT is unused in enclone_ranger.
macklin-10x Apr 16, 2024
76c1345
Move REF into cr_opts.
macklin-10x Apr 16, 2024
2076750
Move PRE= into cr_opts.
macklin-10x Apr 16, 2024
d8f0a06
Make a note that internal_run is hardcoded in enclone_ranger.
macklin-10x Apr 17, 2024
3f56dbe
Make a note about refactoring how we handle MAX_CORES.
macklin-10x Apr 17, 2024
b460362
Mark NUMI and NUMI_RATIO as complete.
macklin-10x Apr 17, 2024
43da902
Move FATE_FILE into cr_opts.
macklin-10x Apr 17, 2024
698019f
Move GAMMA_DELTA into cr_opts.
macklin-10x Apr 17, 2024
4f9b063
Move NGRAPH_FILTER into cr_opts.
macklin-10x Apr 17, 2024
6e623dc
Move NWEAK_CHAINS into cr_opts.
macklin-10x Apr 17, 2024
f8ce1ed
Move NFOURSIE_KILL into cr_opts.
macklin-10x Apr 17, 2024
840ab1f
Move NDOUBLET into cr_opts.
macklin-10x Apr 17, 2024
fa80625
Move NSIG into cr_opts.
macklin-10x Apr 17, 2024
786eebc
Move META into cr_opts.
macklin-10x Apr 17, 2024
f6c0a9a
Tweak docstring.
macklin-10x Apr 17, 2024
e4a7e86
Delete unused cr_version param.
macklin-10x Apr 18, 2024
cbcca5e
Remove dead WEAK option.
macklin-10x Apr 18, 2024
93d0a0f
Remove unused EXP argument.
macklin-10x Apr 18, 2024
6e8cc48
Remove never-read extc param.
macklin-10x Apr 18, 2024
cca0fc1
Remove never-completed EXT functionality.
macklin-10x Apr 18, 2024
b99ee83
Delete dead FB_SHOW arg.
macklin-10x Apr 18, 2024
917a6a7
Remove several more dead args.
macklin-10x Apr 18, 2024
f230ed2
Delete dead condition field in PlotOpt.
macklin-10x Apr 18, 2024
fc30b6a
Delete PLOT2 arg which has identical function as PLOT.
macklin-10x Apr 18, 2024
5ed6099
Delete unused fields from AlleleData.
macklin-10x Apr 18, 2024
653db37
Delete unused JoinAlgOpt field.
macklin-10x Apr 18, 2024
028b066
Delete always-false/dead fails_only param and branch.
macklin-10x Apr 18, 2024
5a4f932
Delete dead chain_brief option.
macklin-10x Apr 18, 2024
7bd3de7
Remove unused AG_DIST_FORMULA parameter.
macklin-10x Apr 18, 2024
c15f9b0
Remove dead vj_refname_strong parameter.
macklin-10x Apr 18, 2024
34c79dd
Move nogray into clono_print_opts.
macklin-10x Apr 18, 2024
aaec17b
Move more printing options into ClonoPrintOpt.
macklin-10x Apr 18, 2024
a9033ac
Remove never-read pathlist and last_modified params.
macklin-10x Apr 18, 2024
9f5584f
Remove all code that populated dead pathlist.
macklin-10x Apr 18, 2024
319e8a8
Remove unused loading/storing of feature_refs.
macklin-10x Apr 18, 2024
ec6e76a
Rip out entirely-unused metrics loading code.
macklin-10x Apr 18, 2024
7c4df1f
Add validation for readability/writability.
macklin-10x Apr 18, 2024
1f4f0d7
Restore handling of FORCE_EXTERNAL.
macklin-10x Apr 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 1 addition & 11 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 0 additions & 3 deletions enclone/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,6 @@ string_utils = { path = "../string_utils" }
vdj_ann = { path = "../vdj_ann" }
vector_utils = { path = "../vector_utils" }

[target.'cfg(not(windows))'.dependencies]
pager = "0.16"

[target.'cfg(not(windows))'.dependencies.hdf5]
git = "https://github.com/10XGenomics/hdf5-rust.git"
branch = "conda_nov2021"
Expand Down
11 changes: 0 additions & 11 deletions enclone/src/UNDOC_OPTIONS
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,13 @@ Optional arguments governing input and output files:
Optional arguments that control printing of individual clonotypes:

- white = percent of sequences implicated in whitelist expansion.
- CHAIN_BRIEF: show abbreviated chain column headers
- DEBUG_TABLE_PRINTING: add print lines to help debug printing of tables.
- NOTE_SIMPLE: note if the first sequence for the chain is simple, in the sense that it exactly
equals the concatenation of the right-truncated V with the full J segment.

Other optional arguments:

- FORCE: make joins even if redundant
- EXP: exploratory code for exact clonotyping on
- WEAK: for EXP, print all and show weaks
- GRAPH: show logging from light-heavy graph construction
- UTR_CON: run experimental UTR consensus code
- CON_CON: run experimental constant region consensus code
Expand All @@ -67,7 +64,6 @@ Other optional arguments:
2. You want to see the effect of changed annotation code.
- NPLAIN: reverses PLAIN
- INDELS: search for and list CDR3s from clonotypes with possible SHM indels (exploratory)
- NOPRETTY: turn off pretty trace entirely
- HEAVY_CHAIN_REUSE: look for instances of heavy chain reuse
- BINARY=filename: generate binary output file
- PROTO=filename: generate proto output file
Expand All @@ -83,13 +79,6 @@ expanded out.
CELLRANGER: for use if called from cellranger -- changes failure message and prevents exit
upon normal completion

EXT=filename:
Given output of an external clonotyping algorithm which took as inputs the pipeline outputs
for the lenas in enclone.testdata, for each exact subclonotype found by enclone, report its
composition in the external clonotyping, as clonotype_id[count], ...
The input file should have lines of the form:
sample barcode clonotype_id.

SUMMARY_CLEAN: if SUMMARY specified, don't show computational performance stats, so
we can regress on output

Expand Down
2 changes: 1 addition & 1 deletion enclone/src/graph_filter.rs
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ pub fn graph_filter(
.insert(tig_bc[i][0].barcode.clone(), BarcodeFate::GraphFilter);
}
}
if !ctl.gen_opt.ngraph_filter {
if !ctl.cr_opt.ngraph_filter {
erase_if(tig_bc, &to_delete);
}
if graph {
Expand Down
4 changes: 0 additions & 4 deletions enclone/src/info.rs
Original file line number Diff line number Diff line change
Expand Up @@ -222,10 +222,6 @@ pub fn build_info(
} else {
// maybe can't happen
vs.push(rt.clone());
// At one point there was a bug in which the following line was missing.
// This caused a traceback on "enclone 123085 RE". It is interesting because
// the traceback did not get back to the main program, even with
// "enclone 123085 RE NOPRETTY".
vs_notes.push(String::new());
vsnx = String::new();
}
Expand Down
40 changes: 0 additions & 40 deletions enclone/src/misc1.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,6 @@ use enclone_core::{
};
use equiv::EquivRel;
use itertools::Itertools;
#[cfg(not(target_os = "windows"))]
use pager::Pager;

use std::time::Instant;
use string_utils::stringme;
Expand All @@ -20,44 +18,6 @@ use vector_utils::{

// ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

// This section contains a function that supports paging. It does not work under Windows, and
// we describe here all the *known* problems with getting enclone to work under Windows.
// 1. It does not compile for us. When we tried, there was a problem with libhdf-5.
// 2. Paging is turned off, because the pager crate doesn't compile under Windows, and porting
// it to Windows appears nontrivial.
// 3. ANSI escape characters are not handled correctly, at least by default.
// In addition, we have some concerns about what it would mean to properly test enclone on Windows,
// given that some users might have older OS installs, and support for ANSI escape characters
// appears to have been changed in 2018. This is not made easier by the Windows Subsystem for
// Linux.

#[cfg(not(target_os = "windows"))]
pub fn setup_pager(pager: bool) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code was moved into enclone proper.

// If the output is going to a terminal, set up paging so that output is in effect piped to
// "less -R -F -X -K".
//
// ∙ The option -R is used to render ANSI escape characters correctly. We do not use
// -r instead because if you navigate backwards in less -r, stuff gets screwed up,
// which is consistent with the scary stuff in the man page for less at -r. However -R will
// not display all unicode characters correctly, so those have to be picked carefully,
// by empirically testing that e.g. "echo ◼ | less -R -F -X" renders correctly.
//
// ∙ The -F option makes less exit immediately if all the output can be seen in one screen.
//
// ∙ The -X option is needed because we found that in full screen mode on OSX Catalina, output
// was sent to the alternate screen, and hence it appeared that one got no output at all
// from enclone. This is really bad, so do not turn off this option!

if pager {
Pager::with_pager("less -R -F -X -K").setup();
}
}

#[cfg(target_os = "windows")]
pub fn setup_pager(_pager: bool) {}

// ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓

// Lookup for heavy chain reuse (special purpose experimental option).
// This is interesting but not likely to yield interesting examples of heavy chain reuse
// because biologically it doesn't make sense that one would have both H-L1 and H-L2 expanded.
Expand Down
37 changes: 0 additions & 37 deletions enclone_args/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
// Copyright (c) 2021 10x Genomics, Inc. All rights reserved.
#![allow(clippy::needless_range_loop)]

use io_utils::path_exists;

pub mod load_gex;
pub mod load_gex_core;
pub mod load_gex_util;
Expand All @@ -14,38 +12,3 @@ pub mod proc_args_post;
pub mod process_special_arg1;
pub mod process_special_arg2;
pub mod read_json;

// parse_csv_pure: same as parse_csv, but don't strip out quotes

pub fn parse_csv_pure(x: &str) -> Vec<&str> {
let w = x.char_indices().collect::<Vec<_>>();
let mut y = Vec::new();
let (mut quotes, mut i) = (0, 0);
while i < w.len() {
let mut j = i;
while j < w.len() {
if quotes % 2 == 0 && w[j].1 == ',' {
break;
}
if w[j].1 == '"' {
quotes += 1;
}
j += 1;
}
let (start, stop) = (w[i].0, w.get(j).map_or(x.len(), |(ind, _)| *ind));
y.push(&x[start..stop]);
i = j + 1;
}
if !w.is_empty() && w.last().unwrap().1 == ',' {
y.push("");
}
y
}

pub fn fnx(outs: &str, name: &str) -> String {
let mut file = format!("{outs}/../{name}");
if !path_exists(&file) {
file = format!("{outs}/{name}");
}
file
}
6 changes: 0 additions & 6 deletions enclone_args/src/load_gex.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ use vector_utils::{bin_position, unique_sort};
pub fn get_gex_info(ctl: &mut EncloneControl) -> Result<GexInfo, String> {
let mut gex_features = Vec::<Vec<String>>::new();
let mut gex_barcodes = Vec::<Vec<String>>::new();
let mut feature_refs = Vec::<String>::new();
let mut cluster = Vec::<HashMap<String, usize>>::new();
let mut cell_type = Vec::<HashMap<String, String>>::new();
let mut cell_type_specified = Vec::<bool>::new();
Expand All @@ -32,12 +31,10 @@ pub fn get_gex_info(ctl: &mut EncloneControl) -> Result<GexInfo, String> {
let mut h5_paths = Vec::<String>::new();
let mut feature_metrics = Vec::<HashMap<(String, String), String>>::new();
let mut json_metrics = Vec::<HashMap<String, f64>>::new();
let mut metrics = Vec::<String>::new();
load_gex(
ctl,
&mut gex_features,
&mut gex_barcodes,
&mut feature_refs,
&mut cluster,
&mut cell_type,
&mut cell_type_specified,
Expand All @@ -50,7 +47,6 @@ pub fn get_gex_info(ctl: &mut EncloneControl) -> Result<GexInfo, String> {
&mut h5_paths,
&mut feature_metrics,
&mut json_metrics,
&mut metrics,
)?;
if ctl.gen_opt.gene_scan.is_some() && !ctl.gen_opt.accept_inconsistent {
let mut allf = gex_features.clone();
Expand Down Expand Up @@ -137,7 +133,6 @@ pub fn get_gex_info(ctl: &mut EncloneControl) -> Result<GexInfo, String> {
Ok(GexInfo {
gex_features,
gex_barcodes,
feature_refs,
cluster,
cell_type,
cell_type_specified,
Expand All @@ -154,6 +149,5 @@ pub fn get_gex_info(ctl: &mut EncloneControl) -> Result<GexInfo, String> {
have_fb,
feature_metrics,
json_metrics,
metrics,
})
}
Loading
Loading