Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New flakefinding algorithm + improved code structure #1

Open
wants to merge 73 commits into
base: main
Choose a base branch
from
Open

New flakefinding algorithm + improved code structure #1

wants to merge 73 commits into from

Conversation

ky28059
Copy link

@ky28059 ky28059 commented Aug 7, 2023

@ky28059
Initial formatting, config
f7c7a6d
@ky28059
Use some with ... as, abstract some common utils
5557f12
@ky28059
More util separation
4104244
@ky28059
Move files to root folder
7f8bea3
@ky28059
Speed up pos_get and dim_get using regex
000cc4e
@ky28059
More small formatting
1ff2313
@ky28059
Fix util import name shadowing
aaa6b90
@ky28059
Plot utils
bed8625
@ky28059
Extract Box, imread
a16409d
@ky28059
Fix relative import error
4149a6e
@ky28059
Quick google drive unzip utility script
3ee44c5
@ky28059
Add logging, preliminary edgefind abstraction
7937e11
@ky28059
Replace prints with logging, use regex and list comp
c6e0a29 
TODO: fix multiprocessing logging
@ky28059
Abstract box merging
434b632
@ky28059
Comments, update edge finding algorithm, log to set file to prevent t…
de433a8 
…hreading issues
@ky28059
Remove unused edgefind variables, shuffle utils around
0229258
@ky28059
Abstract average RGB finding
3925b5c
@ky28059
Combine run_file and run_file_wrapped with starmap, default `--…
48c8502 
…q` to `Queue.txt`
@ky28059
Quick threshold test to experiment with cv2.inRange()
fe038b8
@ky28059
Use built-in argparse default instead of short-circuiting, upgrade …
44e5c87 
…threshold test for multiple images
@ky28059
Run open morph and close morph pass, show contours and clusters
3c758cc 
This operation is insanely fast (all operations < 1s as opposed to 16s+ for DBSCAN cluster-finder, plus separate edgefinding), but needs to be tuned to be more accurate using the following parameters:

- Threshold range (controls initial b&w image)
- Open morph size (larger size clears more islands, but could decay legitimate flakes)
- Close morph size (larger size fills more holes)
- Min cluster pixel count? (avoid if possible, because can then avoid call to contour area)
@ky28059
Document scripts in README
aa5bf93
@ky28059
Preliminary new algorithm usage in monolayer_finder
6148046 
Some errors with the manual text file logs that will be resolved later. Will need to check merge boxes as well.
@ky28059
Tune more forgiving min flake size, fix stage number bug
71ccb5a
@ky28059
Add benchmarking
5b3c61e
@ky28059
Fix loadtxt error from removed log file params
1cc4361
@ky28059
Remove plot prints, set console log level to INFO
725fd37
@ky28059
Delete unused script, utils
dd35aa6
@ky28059
Remove unused config params
57d811b
@ky28059
Abstract get_stage, sort files by max flake area and not last
99753e7
ky28059 and others added 30 commits July 20, 2023 15:20
@ky28059
Print when exiting image early, don't error when making plot with no …
468c8f5 
…points
@ky28059
Use with open() as ..., hopefully fix unpack error if no detections…
880378e 
… happen
@ky28059
Fix ambiguous array truthiness check
b11ef9f
@ky28059
Read input dir from queue instead of hard coding
fe09c09 
- Fix image mask in histogram test
@ky28059
Histogram abstraction, support swapping to np histograms
fab8578
@ky28059
Add tests for edge detection, equalization methods
3ee0949
@ky28059
Filter out "too dark" pixels and combine masks
7133f5b
@ky28059
Change constants to be in terms of microns, uppercase constants and a…
626a776 
…dd units
@ky28059
Write detected edges (and angles) to log file, use .csv
14f9f68
@ky28059
Temp equalize script and constants
9875b8f
@ky28059
Mask equalize by contrast instead of intensity, fix scripts, speed up…
4e6ed37 
… "on edge" check slightly

TODO: fix edge clarity on equalize
@ky28059
Fix contrast mask step in finder script
bb7b3f8
@ky28059
Fix open and close morph size / shape
1baf1f2
@ky28059
Add background detection back in instead of hard-coded alpha value
b5212df
@ky28059
Threshold via HSV range
7c6ccc6
@ky28059
Tune flake outer mask, remove background limits
0d7e551
@ky28059
Use HSV in monolayer script
cd8d3f7
@ky28059
Semi-hacky 3 range hue filter
4fa695a
@ky28059
Replace hacky hue ranges with perimeter filtering
2dde26e
@ky28059
Re-add slight hack to accommodate for 5_23_23
7dc4a03
@ky28059
Tighten saturation limit to exclude bilayers
af79eff
@ky28059
Use regex for queue loading
ecc99e1
@ky28059
Add corner detection to test, subpixel refinement
7904ad4
@ky28059
Don't require mask image to check box intersections
937a97f
@ky28059
Commit other tests, doc changes
acd7dba
@ky28059
Remove unnecessary to_mask method
3228320
@Ecourtney
Database Update
a0f739a 
Updated with data from n=1500 flakes to improve rgb+hsv masking
@Ecourtney
Edge + Bilayer finding
614e569 
Improved angle labeling and added rough functionality to find bilayers as an input option
@Ecourtney
Multilayer updates
56b934c 
Updated to sort detected monolayers attached to bi/tri/bulk/freestanding
@Ecourtney
Added 5x ID
bc74eac 
Added config values for 5x, added function to search .xlif file for objective magnification and choose values accordingly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ky28059 @Ecourtney