Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: all_statistics file not generated if one image of the serie is missing the object #942

Open
7 tasks done
AdriSP2 opened this issue Oct 10, 2024 · 0 comments
Open
7 tasks done

[Bug]: all_statistics file not generated if one image of the serie is missing the object #942

AdriSP2 opened this issue Oct 10, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@AdriSP2
Copy link

AdriSP2 commented Oct 10, 2024 •
edited by ns-rse
Loading

Checklist

  • Re-run analysis with topostats process --core 1.
  • Describe the bug.
  • Include the configuration file.
  • Copy of the output.
  • The exact command that failed. This is what you typed at the command line, including any options.
  • TopoStats version, this is reported by topostats --version
  • Operating System and Python Version

Describe the bug

When processing several images at ones form a Highspeed AFM if one or more of does images is missing the object (because of drift or simply too blurry) and the mask does not get placed, after processing everything, the all_statistics file does not get created neither the height_profiles, this brings an unexpected issue since the rest of hundred images have been processed properly but we are unable to obtain any data from them since there are no files created.
It would be better if it was possible to force the creation of the file even if some images are 0

Copy of the output

[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Plotting tidied_border image
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Plotting removed_noise image
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Plotting removed_small_objects image
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Plotting labelled_regions_02 image
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Plotting coloured_regions image
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962_above_masked] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_962_above_masked.png | DPI: 100
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : *** Grain Statistics ***
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Processing grain: 0
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Height profiles extracted.
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] Calculation of DNA Tracing disabled, returning grainstats data frame.
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : *** Image Statistics ***
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] : Saving image to .topostats file
[Thu, 10 Oct 2024 18:55:03] [WARNING ] [topostats] Item 'grain_trace_data' is None. Skipping.
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] Cannot save key 'grain_trace_data' to HDF5. Item type: <class 'NoneType'>. Skipping. One of data, shape or dtype must be specified
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] Processing : 230523-01-0040_lactosa_50-968_963
Processing images from C:\Users\Adrià\Desktop\TopoStats, results are under output: 963it [16:24,  1.03it/s][Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] Image dimensions: (75, 75)
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_962] Processing completed.
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : *** Filtering ***
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Median flattening without mask
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plane tilt removal without mask
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : x-polyfit 1st order: [-0.10272048  3.06315789]
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : y-polyfit 1st order: [0. 0.]
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [{self.filename}] : y gradient is zero, skipping plane tilt y removal
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Remove quadratic bow without mask
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : x polyfit 2nd order: [-0.00519986  0.38478942 -1.61844668]
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Nonlinear polynomial removal optimal params: const: 7.368876529400875 xy: 0.00024135385823098598 x: 0.005041282970554284 y: 0.02701935314604087
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Zero averaging background : 2.812085909277408e-09 nm
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Masking (above) Threshold: 4.64036682296801
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Masking (below) Threshold: -38.66972352473342
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Median flattening with mask
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plane tilt removal with mask
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : x-polyfit 1st order: [-0.01215757  0.39504204]
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : y-polyfit 1st order: [-2.21097046e-19  1.11411854e-17]
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : removing y plane tilt
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Remove quadratic bow with mask
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : x polyfit 2nd order: [-0.00312699  0.2313974  -2.420293  ]
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Nonlinear polynomial removal optimal params: const: 2.65728140942684 xy: 7.442848387384515e-06 x: -0.003593117006874147 y: 0.011480821430595034
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Zero averaging background : 2.8363046608899937e-09 nm
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Applying Gaussian filter (mode : nearest; Gaussian blur (px) : 1.15).
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting Filtering Images
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_963.png | DPI: 100
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : *** Grain Finding ***
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Thresholding method (grains) : absolute
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Finding above grains, threshold: (5)
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Masking (above) Threshold: 5
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Labelling Regions
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Tidying borders
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Removing noise (above)
[Thu, 10 Oct 2024 18:55:03] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Area thresholding grains | Thresholds: L: 1.12,U: 5625.00 px^2, L: 0.50, U: 2500.00 nm^2.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Removing small / large grains (above)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Area thresholding grains | Thresholds: L: 2.25,U: 6750.00 px^2, L: 1.00, U: 3000.00 nm^2.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Labelling Regions
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Region properties calculated (above)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Coloured regions
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Extracted bounding boxes (above)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Grains found for direction above : 1
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting Grain Finding Images
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting above Grain Finding Images
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting mask_grains image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting labelled_regions_01 image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting tidied_border image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting removed_noise image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting removed_small_objects image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting labelled_regions_02 image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Plotting coloured_regions image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963_above_masked] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_963_above_masked.png | DPI: 100
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : *** Grain Statistics ***
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Processing grain: 0
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Height profiles extracted.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] Calculation of DNA Tracing disabled, returning grainstats data frame.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : *** Image Statistics ***
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] : Saving image to .topostats file
[Thu, 10 Oct 2024 18:55:04] [WARNING ] [topostats] Item 'grain_trace_data' is None. Skipping.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] Cannot save key 'grain_trace_data' to HDF5. Item type: <class 'NoneType'>. Skipping. One of data, shape or dtype must be specified
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] Processing : 230523-01-0040_lactosa_50-968_964
Processing images from C:\Users\Adrià\Desktop\TopoStats, results are under output: 964it [16:25,  1.05it/s][Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] Image dimensions: (75, 75)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_963] Processing completed.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : *** Filtering ***
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Median flattening without mask
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plane tilt removal without mask
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : x-polyfit 1st order: [-0.09797297  3.23333333]
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : y-polyfit 1st order: [0. 0.]
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [{self.filename}] : y gradient is zero, skipping plane tilt y removal
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Remove quadratic bow without mask
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : x polyfit 2nd order: [-0.00599438  0.44358444 -2.16361073]
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Nonlinear polynomial removal optimal params: const: 7.183007570802004 xy: 0.00012006711517242863 x: -0.004991134488481874 y: 0.020766214966296603
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Zero averaging background : 2.811675848432868e-09 nm
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Masking (above) Threshold: 4.651877143850615
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Masking (below) Threshold: -38.765642865421796
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Median flattening with mask
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plane tilt removal with mask
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : x-polyfit 1st order: [-0.01436898  0.52214216]
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : y-polyfit 1st order: [-2.21097046e-19  7.99555354e-18]
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : removing y plane tilt
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Remove quadratic bow with mask
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : x polyfit 2nd order: [-0.00317012  0.23458864 -2.33201959]
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Nonlinear polynomial removal optimal params: const: 2.6065707674810206 xy: -1.7912123706774537e-05 x: -0.007857293891788018 y: 0.00959619633408403
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Zero averaging background : 1.3389976481061481e-11 nm
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Applying Gaussian filter (mode : nearest; Gaussian blur (px) : 1.15).
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting Filtering Images
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_964.png | DPI: 100
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : *** Grain Finding ***
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Thresholding method (grains) : absolute
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Finding above grains, threshold: (5)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Masking (above) Threshold: 5
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Labelling Regions
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Tidying borders
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Removing noise (above)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Area thresholding grains | Thresholds: L: 1.12,U: 5625.00 px^2, L: 0.50, U: 2500.00 nm^2.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Removing small / large grains (above)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Area thresholding grains | Thresholds: L: 2.25,U: 6750.00 px^2, L: 1.00, U: 3000.00 nm^2.
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Labelling Regions
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Region properties calculated (above)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Coloured regions
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Extracted bounding boxes (above)
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Grains found for direction above : 1
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting Grain Finding Images
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting above Grain Finding Images
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting mask_grains image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting labelled_regions_01 image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting tidied_border image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting removed_noise image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting removed_small_objects image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting labelled_regions_02 image
[Thu, 10 Oct 2024 18:55:04] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Plotting coloured_regions image
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964_above_masked] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_964_above_masked.png | DPI: 100
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : *** Grain Statistics ***
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Processing grain: 0
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Height profiles extracted.
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] Calculation of DNA Tracing disabled, returning grainstats data frame.
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : *** Image Statistics ***
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] : Saving image to .topostats file
[Thu, 10 Oct 2024 18:55:05] [WARNING ] [topostats] Item 'grain_trace_data' is None. Skipping.
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] Cannot save key 'grain_trace_data' to HDF5. Item type: <class 'NoneType'>. Skipping. One of data, shape or dtype must be specified
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] Processing : 230523-01-0040_lactosa_50-968_965
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] Image dimensions: (75, 75)
Processing images from C:\Users\Adrià\Desktop\TopoStats, results are under output: 965it [16:26,  1.02it/s][Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : *** Filtering ***
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_964] Processing completed.
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Median flattening without mask
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plane tilt removal without mask
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : x-polyfit 1st order: [-0.09822191  3.15087719]
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : y-polyfit 1st order: [0. 0.]
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [{self.filename}] : y gradient is zero, skipping plane tilt y removal
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Remove quadratic bow without mask
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : x polyfit 2nd order: [-0.00587916  0.43505791 -2.14232741]
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Nonlinear polynomial removal optimal params: const: 7.386046168928121 xy: 0.00042217574186901593 x: 0.007587430884472521 y: 0.027849046852272342
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Zero averaging background : 1.4570966110719988e-10 nm
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Masking (above) Threshold: 4.58411307709485
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Masking (below) Threshold: -38.200942309123754
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Median flattening with mask
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plane tilt removal with mask
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : x-polyfit 1st order: [-0.01606563  0.61377642]
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : y-polyfit 1st order: [-2.05304400e-20 -4.79148884e-18]
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : removing y plane tilt
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Remove quadratic bow with mask
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : x polyfit 2nd order: [-0.0033181   0.24553938 -2.37361938]
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Nonlinear polynomial removal optimal params: const: 3.092552577750214 xy: 0.00020348861095320636 x: 0.0017574569848918253 y: 0.016413824731088433
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Zero averaging background : 2.8088153894460253e-09 nm
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Applying Gaussian filter (mode : nearest; Gaussian blur (px) : 1.15).
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting Filtering Images
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_965.png | DPI: 100
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : *** Grain Finding ***
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Thresholding method (grains) : absolute
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Finding above grains, threshold: (5)
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Masking (above) Threshold: 5
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Labelling Regions
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Tidying borders
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Removing noise (above)
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Area thresholding grains | Thresholds: L: 1.12,U: 5625.00 px^2, L: 0.50, U: 2500.00 nm^2.
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Removing small / large grains (above)
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Area thresholding grains | Thresholds: L: 2.25,U: 6750.00 px^2, L: 1.00, U: 3000.00 nm^2.
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Labelling Regions
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Region properties calculated (above)
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Coloured regions
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Extracted bounding boxes (above)
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Grains found for direction above : 1
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting Grain Finding Images
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting above Grain Finding Images
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting mask_grains image
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting labelled_regions_01 image
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting tidied_border image
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting removed_noise image
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting removed_small_objects image
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting labelled_regions_02 image
[Thu, 10 Oct 2024 18:55:05] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Plotting coloured_regions image
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965_above_masked] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_965_above_masked.png | DPI: 100
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : *** Grain Statistics ***
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Processing grain: 0
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Height profiles extracted.
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] Calculation of DNA Tracing disabled, returning grainstats data frame.
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : *** Image Statistics ***
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] : Saving image to .topostats file
[Thu, 10 Oct 2024 18:55:06] [WARNING ] [topostats] Item 'grain_trace_data' is None. Skipping.
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] Cannot save key 'grain_trace_data' to HDF5. Item type: <class 'NoneType'>. Skipping. One of data, shape or dtype must be specified
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] Processing : 230523-01-0040_lactosa_50-968_966
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] Image dimensions: (75, 75)
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : *** Filtering ***
Processing images from C:\Users\Adrià\Desktop\TopoStats, results are under output: 966it [16:27,  1.04it/s][Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Median flattening without mask
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_965] Processing completed.
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plane tilt removal without mask
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : x-polyfit 1st order: [-0.09927098  3.11469298]
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : y-polyfit 1st order: [0. 0.]
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [{self.filename}] : y gradient is zero, skipping plane tilt y removal
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Remove quadratic bow without mask
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : x polyfit 2nd order: [-0.00625041  0.46253008 -2.51275632]
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Nonlinear polynomial removal optimal params: const: 7.193627409518127 xy: 0.00021799734307403895 x: 0.004981383042935383 y: 0.017508528609587626
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Zero averaging background : 2.8097564912362333e-09 nm
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Masking (above) Threshold: 4.239536732772054
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Masking (below) Threshold: -35.32947277310045
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Median flattening with mask
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plane tilt removal with mask
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : x-polyfit 1st order: [-0.01466458  0.5651161 ]
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : y-polyfit 1st order: [-3.99553948e-19  2.62558007e-17]
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : removing y plane tilt
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Remove quadratic bow with mask
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : x polyfit 2nd order: [-0.00354379  0.26224064 -2.6254784 ]
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Nonlinear polynomial removal optimal params: const: 2.6976136084638473 xy: 8.146513867657301e-05 x: -0.0019259573572405486 y: 0.0066013545848899735
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Zero averaging background : 2.817570781227884e-09 nm
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Applying Gaussian filter (mode : nearest; Gaussian blur (px) : 1.15).
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting Filtering Images
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_966.png | DPI: 100
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : *** Grain Finding ***
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Thresholding method (grains) : absolute
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Finding above grains, threshold: (5)
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Masking (above) Threshold: 5
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Labelling Regions
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Tidying borders
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Removing noise (above)
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Area thresholding grains | Thresholds: L: 1.12,U: 5625.00 px^2, L: 0.50, U: 2500.00 nm^2.
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Removing small / large grains (above)
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Area thresholding grains | Thresholds: L: 2.25,U: 6750.00 px^2, L: 1.00, U: 3000.00 nm^2.
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Labelling Regions
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Region properties calculated (above)
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Coloured regions
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Extracted bounding boxes (above)
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Grains found for direction above : 1
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting Grain Finding Images
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting above Grain Finding Images
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting mask_grains image
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting labelled_regions_01 image
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting tidied_border image
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting removed_noise image
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting removed_small_objects image
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting labelled_regions_02 image
[Thu, 10 Oct 2024 18:55:06] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Plotting coloured_regions image
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966_above_masked] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_966_above_masked.png | DPI: 100
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : *** Grain Statistics ***
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Processing grain: 0
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Height profiles extracted.
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] Calculation of DNA Tracing disabled, returning grainstats data frame.
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : *** Image Statistics ***
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] : Saving image to .topostats file
[Thu, 10 Oct 2024 18:55:07] [WARNING ] [topostats] Item 'grain_trace_data' is None. Skipping.
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] Cannot save key 'grain_trace_data' to HDF5. Item type: <class 'NoneType'>. Skipping. One of data, shape or dtype must be specified
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] Processing : 230523-01-0040_lactosa_50-968_967
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] Image dimensions: (75, 75)
Processing images from C:\Users\Adrià\Desktop\TopoStats, results are under output: 967it [16:28,  1.06it/s][Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : *** Filtering ***
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_966] Processing completed.
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Median flattening without mask
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plane tilt removal without mask
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : x-polyfit 1st order: [-0.09423898  2.75350877]
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : y-polyfit 1st order: [0. 0.]
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [{self.filename}] : y gradient is zero, skipping plane tilt y removal
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Remove quadratic bow without mask
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : x polyfit 2nd order: [-0.00549387  0.40654641 -2.19280588]
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Nonlinear polynomial removal optimal params: const: 7.090453924460172 xy: 0.00032300377890587213 x: 0.007012306064288907 y: 0.026234212208592428
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Zero averaging background : 1.8189894035458565e-15 nm
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Masking (above) Threshold: 4.538985705063879
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Masking (below) Threshold: -37.82488087553232
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Median flattening with mask
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plane tilt removal with mask
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : x-polyfit 1st order: [-0.01604597  0.60368783]
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : y-polyfit 1st order: [-5.52742615e-21 -3.77378440e-18]
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Removing x plane tilt
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : removing y plane tilt
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Remove quadratic bow with mask
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : x polyfit 2nd order: [-0.00371576  0.27496606 -2.74173261]
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Nonlinear polynomial removal optimal params: const: 2.838815932914521 xy: 5.504652636423302e-05 x: -0.0020072754397857384 y: 0.006135085840952903
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Skipping scar removal as requested from config
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Zero averaging background : 2.795734489369427e-09 nm
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Applying Gaussian filter (mode : nearest; Gaussian blur (px) : 1.15).
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting Filtering Images
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_967.png | DPI: 100
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : *** Grain Finding ***
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Thresholding method (grains) : absolute
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Finding above grains, threshold: (5)
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Masking (above) Threshold: 5
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Labelling Regions
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Tidying borders
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Removing noise (above)
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Area thresholding grains | Thresholds: L: 1.12,U: 5625.00 px^2, L: 0.50, U: 2500.00 nm^2.
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Removing small / large grains (above)
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Area thresholding grains | Thresholds: L: 2.25,U: 6750.00 px^2, L: 1.00, U: 3000.00 nm^2.
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Labelling Regions
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Region properties calculated (above)
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Coloured regions
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Extracted bounding boxes (above)
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Grains found for direction above : 1
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting Grain Finding Images
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting above Grain Finding Images
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting mask_grains image
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting labelled_regions_01 image
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting tidied_border image
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting removed_noise image
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting removed_small_objects image
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting labelled_regions_02 image
[Thu, 10 Oct 2024 18:55:07] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Plotting coloured_regions image
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967_above_masked] : Image saved to : output\processed\230523-01-0040_lactosa_50-968_967_above_masked.png | DPI: 100
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : *** Grain Statistics ***
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Processing grain: 0
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Height profiles extracted.
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] Calculation of DNA Tracing disabled, returning grainstats data frame.
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : *** Image Statistics ***
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] : Saving image to .topostats file
[Thu, 10 Oct 2024 18:55:08] [WARNING ] [topostats] Item 'grain_trace_data' is None. Skipping.
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] Cannot save key 'grain_trace_data' to HDF5. Item type: <class 'NoneType'>. Skipping. One of data, shape or dtype must be specified
Processing images from C:\Users\Adrià\Desktop\TopoStats, results are under output: 968it [16:29,  1.00s/it][Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] [230523-01-0040_lactosa_50-968_967] Processing completed.
Processing images from C:\Users\Adrià\Desktop\TopoStats, results are under output: 968it [16:29,  1.02s/it]
[Thu, 10 Oct 2024 18:55:08] [INFO    ] [topostats] Saving image stats to : output/image_stats.csv.
Traceback (most recent call last):
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\Scripts\topostats.exe\__main__.py", line 7, in <module>
    sys.exit(entry_point())
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\topostats\entry_point.py", line 386, in entry_point
    args.func(args)
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\topostats\run_topostats.py", line 154, in run_topostats
    results = pd.concat(results.values())
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\reshape\concat.py", line 395, in concat
    return op.get_result()
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\reshape\concat.py", line 671, in get_result
    for ax, new_labels in enumerate(self.new_axes):
  File "properties.pyx", line 36, in pandas._libs.properties.CachedProperty.__get__
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\reshape\concat.py", line 702, in new_axes
    return [
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\reshape\concat.py", line 703, in <listcomp>
    self._get_concat_axis if i == self.bm_axis else self._get_comb_axis(i)
  File "properties.pyx", line 36, in pandas._libs.properties.CachedProperty.__get__
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\reshape\concat.py", line 760, in _get_concat_axis
    concat_axis = _concat_indexes(indexes)
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\reshape\concat.py", line 778, in _concat_indexes
    return indexes[0].append(indexes[1:])
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\indexes\base.py", line 5489, in append
    return self._concat(to_concat, name)
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\indexes\base.py", line 5497, in _concat
    result = concat_compat(to_concat_vals)
  File "C:\Pinokio\nou\bin\miniconda\envs\topostats\lib\site-packages\pandas\core\dtypes\concat.py", line 115, in concat_compat
    warnings.warn(
FutureWarning: The behavior of array concatenation with empty entries is deprecated. In a future version, this will no longer exclude empty items when determining the result dtype. To retain the old behavior, exclude the empty entries before the concat operation.

Include the configuration file

base_dir: ./ # Directory in which to search for data files
output_dir: ./output # Directory to output results to
log_level: info # Verbosity of output. Options: warning, error, info, debug
cores: 5 # Number of CPU cores to utilise for processing multiple files simultaneously.
file_ext: .asd # File extension of the data files.
loading:
  channel: TP # Channel to pull data from in the data files.
filter:
  run: true # Options : true, false
  row_alignment_quantile: 0.5 # lower values may improve flattening of larger features
  threshold_method: std_dev # Options : otsu, std_dev, absolute
  otsu_threshold_multiplier: 1.0
  threshold_std_dev:
    below: 10.0 # Threshold for data below the image background
    above: 1.2 # Threshold for data above the image background
  threshold_absolute:
    below: -1.0 # Threshold for data below the image background
    above: 1.0 # Threshold for data above the image background
  gaussian_size: 1.15 # Gaussian blur intensity in px
  gaussian_mode: nearest
  # Scar remvoal parameters. Be careful with editing these as making the algorithm too sensitive may
  # result in ruining legitimate data.
  remove_scars:
    run: false
    removal_iterations: 2 # Number of times to run scar removal.
    threshold_low: 0.250 # lower values make scar removal more sensitive
    threshold_high: 0.666 # lower values make scar removal more sensitive
    max_scar_width: 4 # Maximum thichness of scars in pixels.
    min_scar_length: 16 # Minimum length of scars in pixels.
grains:
  run: true # Options : true, false
  # Thresholding by height
  threshold_method: absolute # Options : std_dev, otsu, absolute
  otsu_threshold_multiplier: 1.0
  threshold_std_dev:
    below: 10.0 # Threshold for grains below the image background
    above: 1.0 # Threshold for grains above the image background
  threshold_absolute:
    below: 2 # Threshold for grains below the image background
    above: 5 # Threshold for grains above the image background
  direction: above # Options: above, below, both (defines whether to look for grains above or below thresholds or both)
  # Thresholding by area
  smallest_grain_size_nm2: 0.5 # Size in nm^2 of tiny grains/blobs (noise) to remove, must be > 0.0
  absolute_area_threshold:
    above: [1, 3000] # above surface [Low, High] in nm^2 (also takes null)
    below: [null, null] # below surface [Low, High] in nm^2 (also takes null)
  remove_edge_intersecting_grains: true # Whether or not to remove grains that touch the image border
grainstats:
  run: true # Options : true, false
  edge_detection_method: binary_erosion # Options: canny, binary erosion. Do not change this unless you are sure of what this will do.
  cropped_size: 50.0 # Length (in nm) of square cropped images (can take -1 for grain-sized box)
  extract_height_profile: true # Extract height profiles along maximum feret of molecules
dnatracing:
  run: false # Options : true, false
  min_skeleton_size: 10 # Minimum number of pixels in a skeleton for it to be retained.
  skeletonisation_method: topostats # Options : zhang | lee | thin | topostats
  spline_step_size: 7.0e-9 # The sampling rate of the spline in metres.
  spline_linear_smoothing: 5.0 # The amount of smoothing to apply to linear splines.
  spline_circular_smoothing: 0.0 # The amount of smoothing to apply to circular splines.
  pad_width: 1 # Cells to pad grains by when tracing
#  cores: 1 # Number of cores to use for parallel processing
plotting:
  run: true # Options : true, false
  style: topostats.mplstyle # Options : topostats.mplstyle or path to a matplotlibrc params file
  savefig_format: null # Options : null, png, svg or pdf. tif is also available although no metadata will be saved. (defaults to png) See https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html
  savefig_dpi: null # Options : null (defaults to the value in topostats/plotting_dictionary.yaml), see https://afm-spm.github.io/TopoStats/main/configuration.html#further-customisation and https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html
  pixel_interpolation: null # Options : https://matplotlib.org/stable/gallery/images_contours_and_fields/interpolation_methods.html
  image_set: core # Options : all, core
  zrange: [null, null] # low and high height range for core images (can take [null, null]). low <= high   
  colorbar: true # Options : true, false
  axes: true # Options : true, false (due to off being a bool when parsed)
  num_ticks: [null, null] # Number of ticks to have along the x and y axes. Options : null (auto) or integer > 1
  cmap: null # Colormap/colourmap to use (default is 'nanoscope' which is used if null, other options are 'afmhot', 'viridis' etc.)
  mask_cmap: blu # Options : blu, jet_r and any in matplotlib
  histogram_log_axis: false # Options : true, false
summary_stats:
  run: true # Whether to make summary plots for output data
  config: null

To Reproduce

https://drive.google.com/drive/folders/1Ko5IJeFi9jS1Dx2Ecx5PJuHfIsYXdZaj?usp=sharing

TopoStats Version

Git main branch

Python Version

3.1

Operating System

Windows

Python Packages

AFMReader==0.0.1
asttokens==2.4.1
cheap-repr==0.5.1
colorama==0.4.6
contourpy==1.2.1
cycler==0.12.1
decorator==5.1.1
exceptiongroup==1.2.2
executing==2.0.1
fonttools==4.53.1
h5py==3.11.0
igor2==0.5.8
imageio==2.34.2
ipython==8.26.0
jedi==0.19.1
joblib==1.4.2
kiwisolver==1.4.5
lazy_loader==0.4
loguru==0.7.2
matplotlib==3.9.1
matplotlib-inline==0.1.7
networkx==3.3
numpy==1.26.4
numpyencoder==0.3.0
packaging==24.1
pandas==2.2.2
parso==0.8.4
pillow==10.4.0
prompt_toolkit==3.0.47
psutil==5.9.8
pure-eval==0.2.2
pyfiglet==1.0.2
Pygments==2.18.0
pyparsing==3.1.2
pyspm==0.6.2
python-dateutil==2.9.0.post0
pytz==2024.1
PyYAML==6.0.1
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
schema==0.7.7
scikit-image==0.24.0
scikit-learn==1.5.1
scipy==1.14.0
seaborn==0.13.2
six==1.16.0
snoop==0.4.3
stack-data==0.6.3
threadpoolctl==3.5.0
tifffile==2024.7.2
topostats @ git+https://github.com/AFM-SPM/TopoStats.git@3cee358bd9b99585c6e5296b340d8d502c6b27d4
tqdm==4.66.4
traitlets==5.14.3
typing_extensions==4.12.2
tzdata==2024.1
wcwidth==0.2.13
win32-setctime==1.1.0
@AdriSP2 AdriSP2 added the bug Something isn't working label Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@AdriSP2