better readme generation

README.md

@@ -30,10 +30,48 @@ eval "$(_STEMIA_COMPLETE=bash_source stemia)"
 
 See [the click docs](https://click.palletsprojects.com/en/8.1.x/shell-completion/#enabling-completion) for how to do it for other shells.
 
-## Tools
+<!-- autogenerated content start here -->
 
-Everything is accessible through the main command line interface `stemia`. Try `stemia -h`.
+## Tools
 
+Everything is accessible through the main command line interface `stemia`.
+
+Try `stemia -h` for help, or `stemia -l` for the command tree:
+
+```
+.stemia
+├── aretomo:  A collection of AreTomo-related tools and scripts.
+│   ├── aln2xf:  Convert AreTomo `aln` file to imod `xf` format.
+│   └── batch:  Run AreTomo on a full directory.
+├── cryosparc:  A collection of Cryosparc-related tools and scripts.
+│   ├── csplot:  Read a cryosparc job directory and plot interactively any column.
+│   ├── fix_filament_ids:  Replace cryosparc filament ids with small unique integers.
+│   ├── generate_tilt_angles:  Generate angle priors for a tilted dataset.
+│   └── merge_defects_gainref:  Merge serialEM defects and gainref for cryosparc usage.
+├── image:  Simple image manipulation and processing.
+│   ├── center_filament:  Center an mrc image (stack) containing filament(s).
+│   ├── classify_densities:  Do hierarchical classification of particle stacks based on densities.
+│   ├── create_mask:  Create a mask for INPUT.
+│   ├── extract_z_snapshots:  Grab z slices at regular intervals from a tomogram as jpg images.
+│   ├── flip_z:  Flip the z axis for particles in a RELION star file.
+│   ├── fourier_crop:  Bin mrc images to the specified pixel size using fourier cropping.
+│   └── rescale:  Rescale an mrc image to the specified pixel size.
+├── imod:  A collection of IMOD-related tools and scripts.
+│   └── find_NAD_params:  Test a range of k and iteration values for nad_eed_3d.
+├── relion:  A collection of Relion-related tools and scripts.
+│   ├── align_filament_particles:  Fix filament PsiPriors so they are consistent within a filament.
+│   └── edit_star:  Simple search-replace utility for star files.
+└── warp:  A collection of Warp-related tools and scripts.
+    ├── fix_mdoc:  Fix mdoc files to point to the right data.
+    ├── merge_star:  Merge star files ignoring optic groups and ensuring columns for warp.
+    ├── offset_angle:  Offset tilt angles in warp xml files.
+    ├── parse_xml:  Parse a warp xml file and print its content.
+    ├── prepare_isonet:  Update an isonet starfile with preprocessing data from warp.
+    ├── spoof_mdoc:  Create dummy mdocs for warp.
+    ├── summarize:  Summarize the state of a Warp project.
+    └── preprocess_serialem:  Prepare and unpack data from sterialEM for Warp.
+```
+
 ### stemia aretomo aln2xf
 
 ```
@@ -46,53 +84,6 @@ Options:
   --help           Show this message and exit.
 ```
 
-### stemia aretomo batch_warp
-
-```
-Usage: stemia aretomo batch_warp [OPTIONS] WARP_DIR
-
-  Run aretomo in batch on data preprocessed in warp.
-
-  Needs to be ran after imod stacks were generated. Requires ccderaser and
-  AreTomo>=1.3.0. Assumes the default Warp directory structure with generated
-  imod stacks.
-
-Options:
-  -m, --mdoc-dir PATH
-  -o, --output-dir PATH           output directory for all the processing. If
-                                  None, defined as warp_dir/aretomo
-  -d, --dry-run                   only print some info, without running the
-                                  commands.
-  -v, --verbose                   print individual commands
-  -j, --just TEXT                 reconstruct just these tomograms
-  -e, --exclude TEXT              exclude these tomograms from the run
-  -t, --sample-thickness INTEGER  unbinned thickness of the SAMPLE (ice or
-                                  lamella) used for alignment
-  -z, --z-thickness INTEGER       unbinned thickness of the RECONSTRUCTION.
-  -b, --binning INTEGER           binning for aretomo reconstruction (relative
-                                  to warp binning)
-  -a, --tilt-axis FLOAT           starting tilt axis for AreTomo, if any
-  -p, --patches INTEGER           number of patches for local alignment in
-                                  aretomo (NxN), if any
-  -r, --roi-dir PATH              directory containing ROI files. Extension
-                                  does not matter, but names should be same as
-                                  TS.
-  -f, --overwrite                 overwrite any previous existing run
-  --train                         whether to train a new denosing model
-  --topaz-patch-size INTEGER      patch size for denoising in topaz.
-  --start-from [fix|align|tilt_mdocs|reconstruct|stack_halves|reconstruct_halves|denoise]
-                                  use outputs from a previous run, starting
-                                  processing at this step
-  --stop-at [fix|align|tilt_mdocs|reconstruct|stack_halves|reconstruct_halves|denoise]
-                                  terminate processing after this step
-  --ccderaser TEXT                command for ccderaser
-  --aretomo TEXT                  command for aretomo
-  --gpus TEXT                     Comma separated list of gpus to use for
-                                  aretomo. Default to all.
-  --tiltcorr / --no-tiltcorr      do not correct sample tilt
-  --help                          Show this message and exit.
-```
-
 ### stemia aretomo batch
 
 ```
@@ -310,7 +301,7 @@ Options:
 ```
 Usage: stemia imod find_NAD_params [OPTIONS] INPUT
 
-  Test a range of k and iteration values for nad_eed_3d
+  Test a range of k and iteration values for nad_eed_3d.
 
 Options:
   -k, --k-values TEXT

docs/generate_readme.py

@@ -1,8 +1,14 @@
 #!/usr/bin/env python3
 
+import contextlib
+import io
+from inspect import cleandoc
+from pathlib import Path
+
 import click
 
 from stemia import cli
+from stemia.utils.click import print_command_tree
 
 
 def get_help(name, cli):
@@ -22,4 +28,38 @@ def get_help(name, cli):
 
 help_msg = get_help("stemia", cli)
 
-print("\n\n".join(help_msg))
+readme_separator = "<!-- autogenerated content start here -->"
+
+readme = Path(__file__).parent.parent / "README.md"
+
+header = []
+with open(readme) as f:
+    for line in f.readlines():
+        if readme_separator in line:
+            break
+        header.append(line)
+
+output = io.StringIO()
+with contextlib.redirect_stdout(output):
+    print(f'{"".join(header)}{readme_separator}\n')
+    print(
+        cleandoc(
+            """
+    ## Tools
+
+    Everything is accessible through the main command line interface `stemia`.
+
+    Try `stemia -h` for help, or `stemia -l` for the command tree:
+
+    ```"""
+        )
+    )
+    print_command_tree(cli)
+    print(
+        """```
+    """
+    )
+    print("\n\n".join(help_msg))
+
+with open(readme, "w") as f:
+    f.write(output.getvalue())