Experiencing issues with meshing results

[Beamsspam]

I've been attempting to utilise scAnt with a D7500 camera to create 3D models of a termite specimen in Meshroom following the recommended settings. However, meshroom seems to be failing to have generated the mesh and point cloud properly. Is there something I could be doing incorrectly or that needs to be changed? Does the point cloud look right?
Thus far I've been stacking the images in Zerene, batch masking them in photoshop using an action, converting them to JPG files, and then adding in the following Exif data via command prompt: exiftool -overwrite_original -Make="NIKON CORPORATION" -Model="NIKON D7500" -Lens="F/2.8" -FocalLength="60.0" -FocalLengthIn35mmFormat="90.0" "F:\pathtofiles"

[FabianPlum]

Hi @Beamsspam!
First of all, thanks so much for having built your own scAnt and using it in your work!

There are a few things that could be the culprit in your reconstruction issues. I suspect it's first and foremost an issue of image quality. While the pictures above resolve the termite somewhat reasonably, it does not occupy a lot of space in the image and thus any reconstruction software will have a hard time solving the camera alignment and point cloud steps (structure from motion) reasonably well.

I would recommend trying to get overall higher magnification in your setup by, e.g., moving the camera much closer to your subject by using extension tubes to decrease the minimal focal distance, or to use a higher magnification lens altogether. Ideally, the subject you wish to digitise should occupy at least ~50% of the image for a clean reconstruction and suffiently high levels of surface detail.

If you want specific recommendations regarding lenses or extension tubes (which would of course be much cheaper than a full lens) let me know and we can figure something out together that fits your system.

Two things on the processing side to try in the meantime:

1. Crop your images (uniformly, so all with the same crop factor) and adjust the image meta data accordingly. Then try to re-run the reconstruction in Meshroom.
2. Try alternative reconstruction software. We have had great success with Zephyr Lite, which is one of the cheapest commercial photogrammetry software solutions and produces excellent results, even with more challenging image inputs. I would recommend checking out their software by requesting a 30 day trial license and seeing if this already partially solves your issues. This way, you do not have to commit to spending any money to know whether any one solution is ideal for your purposes.

Regardless of which approach you try next: higher magnification for such a small specimen will go a long way!

Hope this provides a good starting point for you and please let us know if there is anything else you need help with.

All the best
Fabi

[Beamsspam]

Hello @FabianPlum, thank you very much for responding to my inquiry! I'll try some of these out and see how they go.

[Beamsspam]

Hello Fabian, I've tried moving the camera significantly closer and have also switched to a larger specimen (an adult musgraveia sulciventris) and have run the resulting images through meshroom. The size of the meshroom folder generated is now significantly larger (about 42gb) than what I had been generating previously, which was usually under 10gb, but is getting stuck at meshing with the resulting log shared below. Have you encountered a similar issue previously? Also, is this the expected size of a meshroom folder if things were otherwise going well?

16:33:16.936626][info] Found 1 image dimension(s):
[16:33:16.936718][info] - [5568x3712]
[16:33:17.501274][info] Overall maximum dimension: [5568x3712]
[16:33:17.501425][warning] repartitionMode: 1
[16:33:17.501436][warning] partitioningMode: 1
[16:33:17.501440][info] Meshing mode: multi-resolution, partitioning: single block.
[16:33:17.501466][info] Estimate space from depth maps.
[16:33:17.505972][info] Cannot find any depth map tile file (rc: 0).
[16:33:17.506004][warning] Can't find or invalid 'nbDepthValues' metadata in depth map (rc: 0). Recompute the number of valid values.
aliceVision_meshing: /opt/AliceVision_git/src/aliceVision/mvsUtils/mapIO.cpp:308: void aliceVision::mvsUtils::readMapFromFileOrTiles(int, const aliceVision::mvsUtils::MultiViewParams&, aliceVision::mvsUtils::EFileType, aliceVision::image::Image&, int, int, const string&) [with T = float; std::string = std::basic_string]: Assertion `scale > 0' failed.
[16:33:17.506189][info] Cannot find any depth map tile file (rc: 16).
[16:33:17.506279][info] Cannot find any depth map tile file (rc: 152).
[16:33:17.506591][info] Cannot find any depth map tile file (rc: 48).
[16:33:17.509087][info] Cannot find any depth map tile file (rc: 120).
[16:33:17.507831][info] Cannot find any depth map tile file (rc: 144).
[16:33:17.507912][info] Cannot find any depth map tile file (rc: 80).
[16:33:17.507643][info] Cannot find any depth map tile file (rc: 195).
[16:33:17.509259][warning] Can't find or invalid 'nbDepthValues' metadata in depth map (rc: 80). Recompute the number of valid values.
[16:33:17.509168][warning] Can't find or invalid 'nbDepthValues' metadata in depth map (rc: 16). Recompute the number of valid values.

[FabianPlum]

Hi @Beamsspam!
The stacked image looks incredible, quality-wise this is a great start! We need to check if anything else goes wrong in the reconstruction pipeline further upstream. If no depth maps are found that either means that they have failed to compute or perhaps there is an accidental disconnect in the node graph.

The reason for your Meshroom files to become so much bigger is that there is A LOT more information to work with. At the same sampling settings, Meshroom will now be able to find a lot more features to use for its reconstructions which takes more space and the depth maps (if they are computed) will also have a lot more detail.

Does Meshroom successfully generate a sparse point loud and do the cameras look well aligned?

Also - if Meshroom is the roadblock, perhaps try to use another reconstruction software as a sanity check. 3DF Zephyr Lite produces excellent results and has a trial version that you could test with your images.

All the best
Fabi

[Beamsspam]

HI Fabian, sorry for the delay. I haven't had success in Meshroom yet, and Zephyr hasn't gotten back to me about doing a trial yet, but I have found that KIRI Engine seems to be able to do the meshes pretty well, though it is a bit messy on the topside of the specimen due to the scan having ended prematurely while I wasn't around.

[FabianPlum]

Amazing! The scan looks really good, especially the textures are beautiful!
To help you further could you perhaps share your Meshroom project file with me?

[FabianPlum]

@Beamsspam Hi! Could you perhaps Update your progress so we can see where else we can help?

[Beamsspam]

Hello Fabian, my apologies for the delay. I've mostly moved away from Meshroom now, due to how easy I've been finding working with KIRI Engine to be. I've also adjusted a couple of things to be able to scan the top of the insect as I'd had a couple of issues beforehand which I think were due to the computer's limitations (I am working with a laptop and I have found that halving the y-axis step size to 40 can take up too much RAM memory for my device). We've also tried 3D printing it (not to scale)

We've also done a test scan of a cube for testing our ability to resize the .stl files for printing to scale and have had it be within the nozzle tolerances.

Outside of that, I've been trying to scan smaller specimens with the DSLR camera set-up we've been using by trying different mounting set-ups since we had some trouble getting the pin to stand-upright; since otherwise the specimen would move out of the field of view. This stage is still ongoing however.