Extracting point cloud on the contact surface

[tkelestemur]

Hello all -- I've been trying to generate the point cloud on the contact surface of the digit (at the points where it is in contact with the object) from the depth images coming from the sensors. If I understand the implementation correctly, the depth image is not the distance from the camera to the surface but it represents how much the gel is deformed. This is basically done by subtracting the depth during contacts from the nominal depth of the gel at this line. To avoid this, I tried commenting out this line but then I get the nominal shape of the gel when there is no contact. Finally, I ended up getting the "deformation" depth and subtracting it from the nominal depth where the values are larger than zero:

_, tactile_depth = self.digits.render()
depth_idx = tactile_depth[0] > 0
tactile_depth[0][depth_idx] = self.digits.renderer.depth0[0][depth_idx] - tactile_depth[0][depth_idx]

This works nicely when I disable the curvate of the gel here. But when I don't, I get noisy values in the pixels with no contact. I couldn't figure out the problem with curved gel and I'm also wondering if there is a better way to achieve this. I'm guessing that the points where there is no contact are somehow affected by the force applied on the other points. Please see the images below.

Without curvature:

With curvature