fix features zero division causing nan value

[originlake]

Trying to fix this issue, wanted to get some comments but there seems no response.
mapillary#1014

[pierotofy]

Interesting find, can you share some test images to reproduce this?

[originlake]

I'm not able to share the datasets due to NDA.
In the issue, I shared a gray image that was dumped in the middle of processing, which was resized to 1/4 of its original size and then converted to gray, I just set a breakpoint before it passed to the below function and used cv2.imwrite("nan.png", image_gray) to save the data. It's in PNG format and I verified that it's not altered during uploading, so it should be identical to the image_gray input for the function. You can download it to reproduce the problem.

OpenSfM/opensfm/features.py

Line 620 in 2211a49

points, desc = extract_features_sift(image_gray, config, features_count)

https://user-images.githubusercontent.com/11423220/264133948-bee5c487-f534-4990-81f3-ce4cf732f6c7.png.

How I reproduce it with the latest docker build

# Download the image and save it as nan.png under <image_folder>
# Command line
docker run -ti --rm -v <image_folder>:/datasets --entrypoint python3 opendronemap/odm:latest
# Python console
import cv2
from opensfm import features
import numpy as np
image = cv2.imread("/datasets/nan.png", 0)
points, desc = features.extract_features_sift(image, {'sift_edge_threshold':10.0, 'sift_peak_threshold':0.066, 'feature_root':True}, 16000)
np.isnan(desc).any() # Will get True here

Related processing argument, feature-quality: medium, min-num-features: 16000. This should be a very rare case where the surrounding pixels of the keypoint are all zeros, I processed thousands of datasets, and this only happened for this one specific image.

[pierotofy]

Thanks for the repro steps; the issue makes sense.

We should probably check that s != 0 (and s_sub != 0) rather than searching for isnan, might be a bit faster (maybe)?

[originlake]

You are right, I'll update it sometime later.

[pierotofy]

Fantastic, thanks @originlake !