Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the "momentum" - reuse the information from the previous few images #8

Open
OndraZizka opened this issue Mar 3, 2021 · 2 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@OndraZizka
Copy link

OndraZizka commented Mar 3, 2021

From the results it seems that Deface detects faces in each frame individually.

This causes the blurred regions to jump wildly, sometimes a false positive just blinks in 1 frame, and what's worst:
Sometimes the face slips into 1 or 2 frames, if obscured by even something small. Then the whole point of blurring is gone.

Deface could work in two passes:

  1. Detect the faces in individual images, saving score, size and position.
  2. Distribute the values between images, using
    a) simple gausian distribution of a "detected face's existence (score + position)" to the surrounding images,
    b) even better - matching the "same face" by clustering their positions in surrounding images, computing a vector of its movement, and assume a face in the computed position where e.g. nothing is detected, but the surrounding images have a high score.

This would have benefits:

  1. The false negatives could be dramatically reduced.
  2. In combination with the fixed number of faces parameter, the false positives could be reduced.
  3. The computed vectors could go beyond the edges of the video, so a face which moves out or into the video, could be blurred when partly cropped. This solves another issue: If the face moves e.g. left to right, then the halves at the opposite video edges can be combined and whoever wants can have the whole face.

Sounds good?

@mdraw
Copy link
Member

mdraw commented Apr 6, 2021

This definitely sounds good :) I had actually planned including such a smoothing feature (2.a) from the start but dropped it because I couldn't get it to work properly and had other priorities when implementing the project as a whole. Now that most other things work, this is IMO the most important TODO again. #11 could partially solve this, but I'm not yet sure how effective it is and how it would perform against 2.a and 2.b.
Contributions in this domain are very welcome.

@OndraZizka
Copy link
Author

I might contribute but not sure if in Python. I could use some middle layer data and process them with a Kotlin utility, giving back more middle layer data. That can later be translated to Python if needed. That I could do after the summer holiday, unless you come up with something better in the meantime (which is likely :) )

@mdraw mdraw added enhancement New feature or request help wanted Extra attention is needed labels Jun 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants