Add the GrabCut example #193
Conversation
|
@jlstevens if you don't mind having a look at this, I think that this time we're closer to what we wanted to have from the beginning! Too bad that the annotators seem to be broken in geoviews, I just realized it quite late in the process of updating the main grabcut class. |
That's alarming!! |
|
Thanks for the PR! I'll try this out tonight - this will be a nice test case for my deployment tool. :-)
That sounds like an issue to file on geoviews if you can submit a small, reproducible example... |
|
@jlstevens thanks! And there was already an issue opened on geoviews 🙃 |
Co-authored-by: James A. Bednar <jbednar@users.noreply.github.com>
jlstevens
force-pushed
the
add_grabcut_v2
branch
from
cb32fc0 to
996590f
Compare
|
I've rebased this PR and if I can get it to build I'll merge to master and update the public website. |
|
@maximlt just reminded me that there is one last change to be made to the grabcut project: we need to serve a panel dashboard with |
|
@jlstevens I've made Grabcut servable by creating a small dashboard with a pipeline that includes the first two stages introduced in the notebook. I've not added the 3rd stage (allow to manually edit the contour obtained after running grabcut) since the 4th stage which should allow to download the output (or do something with it, anything really) isn't included in the notebook, since it wasn't included in Earthsim. I don't even know if it's possible to export a Path or Polygon to a shapefile with geoviews? Last section of the notebook: First stage of the deployed app: |
|
Looks great! A two stage pipeline is fine for now: I'll see if I can deploy a running version of this project shortly and if that works well, we can think about whether it is worth introducing more stages. |
|
Cool! I believe shapely can export a shapefile. |
That would be with |
|
Oh man, I can definitely understand why Maxime had concerns about panel-chat-examples and future maintenance; for old, unmaintained repos, its a nightmare. Took me the whole day to get it barely functioning. It's cool when it works, but super, extremely finicky. Area.mp4This drained the life out of me, and I'd prefer to rebuild existing ideas thru slightly more modern tools like https://twitter.com/giswqs/status/1649416858645282822 instead next time. |
grabcut/grabcut.py
Outdated
| @@ -320,8 +320,8 @@ def view(self): | |||
| dmap = hv.DynamicMap(self.extract_foreground) | |||
| dmap = hv.util.Dynamic(dmap, operation=self._filter_contours) | |||
| dmap = hv.util.Dynamic(dmap, operation=self._simplify_contours) | |||
| return (regrid(self.image).options(**options) * self.bg_paths * self.fg_paths + | |||
There was a problem hiding this comment.
Why was the regrid add here? It kept making the shapes mismatched and took me forever to find what caused the shape mismatches :(
There was a problem hiding this comment.
Presumably to make it possible to display really large images without causing issues in the browser? That would be my initial guess at least.
Welcome to the gang :D Code that doesn't run dies quicker than one would think! |
|
Would like a review on this. |
grabcut/grabcut.ipynb
Outdated
| "\n", | ||
| "Written by Philipp Rudiger and Maxime Liquet<br>\n", | ||
| "Created: November 16, 2018<br>\n", | ||
| "Last updated: January 7, 2021" |
There was a problem hiding this comment.
Do the dates auto-update on merge?
|
Can we merge this? |
|
Please! |
|
Let me review please, I see some changes to make, to the project config for instance... |
|
@ahuang11 did you use anaconda-project to set up your environment to work on Grabcut? I see the project is not locked on |
|
It's been a while so I don't really remember. I believe I did, but I didn't lock the project. |
Sorry to come back to this old message @jbednar, I'm sure @ahuang11 would agree that "great" isn't an adjective that qualifies GrabCut 🙃 I don't know if you ran the notebook but I just did, after relocking since the last lock didn't support osx-arm64, and, oh boy, that's a painful example to run through. I wrote down a list of bugs to resolve and suggestions to improve the UX, which is pretty terrible at the moment. It's a non-trivial amount of work and merging GrabCut is just a nice-to-have on my current Examples todo list (#383), so I won't prioritize it, unless told so.
versus (from https://earthsim.holoviz.org/topics/GrabCut.html)
In my OP 3 years ago I wrote:
|
|
Your changes were successfully integrated in the dev site, make sure to review the pages of the projects you touched before merging this PR. |
What looked great to me (and still looks great) is that anaconda-project.yml no longer includes anything apparently crazy; those are all released, normal packages, and that's a great improvement. I don't know if I ran the code at the time, and I haven't run it today, but I agree that the behavior you outline above doesn't seem good. This example was one of the motivators for how we developed our ideas around editing and annotating shapes, so it's not surprising that it has a lot of barely ok ways of doing those things. I have very often wanted to demo the ideas covered by this example, but have been stymied by it being unable to build or having functionality that wasn't working. I'm perfectly happy with a different approach to showing the same thing, and don't need the algorithm involved to be grabcut specifically, though it's nice that it's simpler than e.g. Segment Anything. In any case the actual segmentation performance isn't the main point of the example, but instead it's about being able to draw shapes, then select regions by value based on the drawings, then annotate them, and then use that shape in subsequent computations. Those are all very important capabilities that we are currently almost entirely hiding from our users without having this example running. |
|
Oh yes, I agree the concept overall is okay and this example is a good candidate for the gallery. It's just that the example needs to be fixed (and maybe some upstream libraries too) before we publish it. I'm a bit sad we can't, as all the work I did on this repo was originally motivated by this example, merging new examples 3 years ago was pretty brittle. I don't have the time to work on these fixes at the moment, maybe early next year. Or someone else could pick this up. I don't think working on another non-GrabCut example using the same functionality will be less work so let's fix this one. Another suggestion to improve the example:
|
Ok, so we figured this out! Bokeh's behavior changed in version 3.4, to enter the edit mode with the PolyEditTool you no longer double-click but press tap (more than 300ms). This was changed in bokeh/bokeh#12831. I've opened an issue to track updating HoloViews' docs accordingly holoviz/holoviews#6474 |
This PR adds the GrabCut example taken from Earthsim's topics. While the original example relied on the
earthsimlibrary, this example relies on the moduleearthsim.pythat sits next to the notebook. This module has been extracted and adapted fromearthsim. Doing so has allowed to significantly reduce the number of dependencies required to run the grabcut example, and as a consequence made it possible to update the remaining ones.Compared to the original example:
questto download map tiles, instead the module relies on the functionget_tile_rgbavailable in theutil.pymodule of geoviews.PolyAnnotatorto modify the contour generated during the main/previous stage has been entirely removed since annotators are currently broken in geoviews (annotators wont work in geoviews holoviz/geoviews#526).