Optimize command recording with VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT

[BeastLe9enD]

Hey, I just had an idea: If we record commands and VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT is enabled, couldn't we optimize the command recording by recording all commands to the MTLCommandEncoder directly instead of storing them as MVKCommands in the MVKCommandBuffer first?

I'm not sure if this is worth doing or even possible with the MoltenVK structure, I just had the idea and thought why not discuss it here :)

[billhollings]

The existing MVK_CONFIG_PREFILL_METAL_COMMAND_BUFFERS environment variable enables this capability. It's documented in vk_mvk_moltenvk.h.

For simplicity, and consistency in design and functionality with the more common usecase of deferred command playback upon queue submission, the design of this capability still uses MVKCommands and stores them in an MVKCommandBuffer first, , but it does allow the prefilling of multiple VkCommandBuffers/MTLCommandBuffers to take place on different threads.

If this design is demonstrated to add significant space-time overhead, then in the case where VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT is enabled, we could explore optimizing it, as you suggest, possibly by immediately recording each MVKCommand to the MTLCommandBuffer without first saving it to the MVKCommandBuffer.

Be aware that testing has shown that, in the case where you have many VkCommandBuffers in each queue submit, it is much more efficient to avoid prefilling individual MTLCommandBuffers, in favor of allowing the submit to use a single MTLCommandBuffer for all of the VkCommandBuffers it is processing.

[BeastLe9enD]

I agree that submitting multiple command buffers can be even slower, but I think when MVK_CONFIG_PREFILL_METAL_COMMAND_BUFFERS is already activated, directly submitting the MVKCommands without storing them should be fine.

I implemented it temporarily this evening. So far it seems to be working, but i haven't tested much yet, i will make a pull request when I'm done!
I`ll leave links here in case you are interested:
BeastLe9enD@0f167c4

[billhollings]

Thanks for taking this on.

I made some suggestions for design robustness in the commit you linked. But otherwise looks pretty good. Once it's cleaned up, feel free to submit it as a PR. Happy to discuss further in your commit link.