Integration of different Neural Network Optimizer

[jihwankimqd]

My understanding is that Autoware Universe supports a layered architecture where individual modules can be separated/extended. Currently, some of the Neural Network related functionalities in the Perception module have a CUDA dependency. I want to try to implement the functionality with a different optimizer such as OpenVINO (contrary to TensorRT). As of now, I can't seem to find any guidelines on the docs on developing and testing this functionality. I would appreciate any guidance or help!

[mitsudome-r]

We did have similar discussions in the past https://discourse.ros.org/t/unified-ml-inference-in-autoware-a-proposal/14058.

From the discussion, we decided to use TVM so we are thinking of converting TensorRT implementation into TVM implementation when we move Universe packages into Core as a stable release, but we don't have enough resources to do that so they are currently remaining as TensorRT.

[jihwankimqd]

Thanks for the prompt reply. After reading the thread you've provided and following the relevant links to a related TVM repo, it seems like the last commit was 2 years ago. I understand that TVM implementation is being considered, but I was wondering if there were documentations to help external/third-parties implement the function using their own preferred framework. For example, I have working experience with OpenVINO and just need a documentation on the formats/standards for designing the module and replacing it with the existing module in Universe.