Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] pose_engine.py runtime exception #12

Open
ivelin opened this issue Feb 9, 2021 · 5 comments
Open

[BUG] pose_engine.py runtime exception #12

ivelin opened this issue Feb 9, 2021 · 5 comments
Assignees
Labels
bug Something isn't working

Comments

@ivelin
Copy link

ivelin commented Feb 9, 2021

Describe the bug
Pose engine throws intermittent runtime exceptions.

To Reproduce
Run pose engine through a variety of fall images.

Expected behavior
There should be no unexpected runtime exceptions caused by image processing during fall detections.

Host environment (please complete the following information):

  • Hardware: Raspberry Pi
  • OS: Raspberry OS
  • Container: docker
  • Version: ambianic edge 1.13.2

Additional context

2021-02-07 13:09:02 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1077.19 ms, 0.93 fps in pipeline area_watch
2021-02-07 13:09:02 ERROR /opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py.process_sample(101): Error "only size-1 arrays can be converted to Python scalars" while processing sample. Dropping sample: {'image': <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=1280x720 at 0xADB66570>}
Traceback (most recent call last):
  File "/opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py", line 85, in process_sample
    inference_result, thumbnail = self.fall_detect(image=image)
  File "/opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py", line 344, in fall_detect
    pose, thumbnail, pose_score, pose_dix = self.find_keypoints(image)
  File "/opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py", line 146, in find_keypoints
    poses, thumbnail = self._pose_engine.DetectPosesInImage(image)
  File "/opt/ambianic-edge/src/ambianic/pipeline/ai/pose_engine.py", line 146, in DetectPosesInImage
    kps = self.parse_output(template_heatmaps, template_offsets, 0.3)
  File "/opt/ambianic-edge/src/ambianic/pipeline/ai/pose_engine.py", line 89, in parse_output
    pose_kps[i, 0] = int(remap_pos[0] + offset_data[max_val_pos[0], max_val_pos[1], i])
TypeError: only size-1 arrays can be converted to Python scalars
2021-02-07 13:09:03 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1070.10 ms, 0.70 fps in pipeline area_watch
2021-02-07 13:09:04 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1003.87 ms, 0.99 fps in pipeline area_watch
2021-02-07 13:09:05 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1042.34 ms, 0.96 fps in pipeline area_watch

@ivelin ivelin changed the title [BUG] Pose engine runtime exception [BUG] pose_engine.py runtime exception Feb 9, 2021
@ivelin ivelin transferred this issue from ambianic/ambianic-edge Feb 9, 2021
@ivelin ivelin added the bug Something isn't working label Feb 9, 2021
@bhavikapanara
Copy link
Contributor

@ivelin

Can you please provide me with a sample test fall image so I can reproduce the error and fix it?

@ivelin
Copy link
Author

ivelin commented Feb 11, 2021

No, because the error is intermittent and since it is not handled in a way that provides much context, I cannot correlate the message to an image. Please think how we can better encapsulate this error with a contextual information in the log that allows us to track it down.

@bhavikapanara
Copy link
Contributor

ok...will check it.

@bhavikapanara
Copy link
Contributor

bhavikapanara commented Feb 12, 2021

@ivelin

I have tried to trace this issue but unable to locate the bug.
This function use output data(heatmap_data, offset_data) that are generated from TFLite posenet model.

pose_kps[i, 0] = int(remap_pos[0] + offset_data[max_val_pos[0], max_val_pos[1], i])

pose_kps[i, 0] = int(remap_pos[0] + offset_data[max_val_pos[0], max_val_pos[1], i])
TypeError: only size-1 arrays can be converted to Python scalars

And this error is raised when the function expects a single value but you pass an array instead.

I am trying to check at what circumstance model output is different. But not able to find it.
Can you please help me with your thoughts to deal with this bug?

thanks

@ivelin
Copy link
Author

ivelin commented Feb 12, 2021

It's not obvious to me what the issue is. Is it possible that
remap_pos[0] returns anything but a scalar?
Same question for offset_data[max_val_pos[0], max_val_pos[1], i].

One of these two returns a list or array, which confuses int().

I suggest surrounding these lines with an exception handling block that provides as much context as needed. That way the next time this happens, we would know exactly what caused it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants