Tracking issue for ark-analysis interoperability

[berombau]

Some interop with ark-analysis would be nice for running analyses and comparing and benchmarking. Seems like they are starting with AnnData interoperability themselves, mostly for the downstream analysis (angelolab/ark-analysis#1112).

• make sure the Harpy sdata.table work with anndata_dir (https://github.com/angelolab/ark-analysis/blob/3bb2f09b6eab5668ec74e27f2b25fed24af02f7f/templates/cell_neighbors_analysis.ipynb)
• most interesting are "2. Pixel clustering with Pixie" and "3. Cell clustering with Pixie", as they contain a Jupyter notebook GUI for manual cluster adjustment and annotation

[ArneDefauw]

This PR #21 implements features in the notebook 2. Pixel clustering with Pixie".

Resulting table is compatible with the ark gui to visualize clusters and meta clusters via a heatmap.
See notebook on how to use it.

[ArneDefauw]

This PR #22 implements much of the features of notebook 3. Cell clustering with pixie

See notebook and

harpy/src/sparrow/_tests/test_table/test_cell_clustering/test_cell_clustering.py

Line 12 in 34164eb

def test_cell_clustering(sdata_blobs):

for illustration on how to use it

[ArneDefauw]

This PR #23 implements the multiple fov scenario as in notebook cell clustering with pixie.

See notebook for an example on how to use it, and the unit test:

harpy/src/sparrow/_tests/test_table/test_cell_clustering/test_cell_clustering.py

Line 11 in 4957562

def test_cell_clustering(sdata_blobs):

I've compared our implementation of pixel and cell clustering with the implementation of ark:

• All preprocessing gives same results as ark,
• for flowsom clustering there is some randomness involved, so it is harder to compare, but qualitatively, we obtain similar results.
• our approach, working with dask arrays for pixel clustering instead of dataframes is more scalable.