Fix cuda graph support

[tanmayv25]

FindPackage(CUDA) has been deprecated and hence CUDA_VERSION is no longer set. Using CUDAToolkit_VERSION instead to determine whether or not to support CUDA graph in the TRT backend build.

To avoid silently failing to build with cuda graph support added a check on the CUDAToolkit_VERSION availability.

Related PR: triton-inference-server/backend@b5dab15

Before

# cmake -DTRT_VERSION=10.0.1.6 -DCMAKE_TOOLCHAIN_FILE= -DVCPKG_TARGET_TRIPLET= -DTRITON_ENABLE_NVTX:BOOL=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/tmp/tritonbuild/tensorrt/install -DTRITON_REPO_ORGANIZATION:STRING=https://github.com/triton-inference-server -DTRITON_COMMON_REPO_TAG:STRING=main -DTRITON_CORE_REPO_TAG:STRING=main -DTRITON_BACKEND_REPO_TAG:STRING=main -DTRITON_ENABLE_GPU:BOOL=ON -DTRITON_ENABLE_MALI_GPU:BOOL=OFF -DTRITON_ENABLE_STATS:BOOL=ON -DTRITON_ENABLE_METRICS:BOOL=ON -DTRITON_ENABLE_MEMORY_TRACKER:BOOL=ON ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- RapidJSON found. Headers: /usr/include
-- RapidJSON found. Headers: /usr/include
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
CMake Warning (dev) at /usr/share/cmake-3.27/Modules/FetchContent.cmake:1316 (message):
  The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
  not set.  The policy's OLD behavior will be used.  When using a URL
  download, the timestamps of extracted files should preferably be that of
  the time of extraction, otherwise code that depends on the extracted
  contents might not be rebuilt if the URL changes.  The OLD behavior
  preserves the timestamps from the archive instead, but this is usually not
  what you want.  Update your project to the NEW behavior or specify the
  DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
  robustness issue.
Call Stack (most recent call first):
  build/_deps/repo-common-src/src/test/CMakeLists.txt:34 (FetchContent_Declare)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found Python: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter 
-- Found CUDAToolkit: /usr/local/cuda/include (found version "12.4.131") 
-- Using CUDA 12.4.131
-- The CUDA compiler identification is NVIDIA 12.4.131
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Using CUDA 
CMake Warning at CMakeLists.txt:95 (message):
  CUDA does not support CUDA graphs.


-- TRT_VERSION envvar is 10.0.1.6
-- Configuring done (7.2s)
-- Generating done (0.0s)

-- Build files have been written to: /work/build

NOTE WARNING: CUDA does not support CUDA graphs.

After

# cmake -DTRT_VERSION=10.0.1.6 -DCMAKE_TOOLCHAIN_FILE= -DVCPKG_TARGET_TRIPLET= -DTRITON_ENABLE_NVTX:BOOL=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/tmp/tritonbuild/tensorrt/install -DTRITON_REPO_ORGANIZATION:STRING=https://github.com/triton-inference-server -DTRITON_COMMON_REPO_TAG:STRING=main -DTRITON_CORE_REPO_TAG:STRING=main -DTRITON_BACKEND_REPO_TAG:STRING=main -DTRITON_ENABLE_GPU:BOOL=ON -DTRITON_ENABLE_MALI_GPU:BOOL=OFF -DTRITON_ENABLE_STATS:BOOL=ON -DTRITON_ENABLE_METRICS:BOOL=ON -DTRITON_ENABLE_MEMORY_TRACKER:BOOL=ON ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- RapidJSON found. Headers: /usr/include
-- RapidJSON found. Headers: /usr/include
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
CMake Warning (dev) at /usr/share/cmake-3.27/Modules/FetchContent.cmake:1316 (message):
  The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
  not set.  The policy's OLD behavior will be used.  When using a URL
  download, the timestamps of extracted files should preferably be that of
  the time of extraction, otherwise code that depends on the extracted
  contents might not be rebuilt if the URL changes.  The OLD behavior
  preserves the timestamps from the archive instead, but this is usually not
  what you want.  Update your project to the NEW behavior or specify the
  DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
  robustness issue.
Call Stack (most recent call first):
  build/_deps/repo-common-src/src/test/CMakeLists.txt:34 (FetchContent_Declare)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found Python: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter 
-- Found CUDAToolkit: /usr/local/cuda/include (found version "12.4.131") 
-- Using CUDA 12.4.131
-- The CUDA compiler identification is NVIDIA 12.4.131
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Using CUDA 12.4.131
-- TRT_VERSION envvar is 10.0.1.6
-- Configuring done (7.3s)
-- Generating done (0.0s)

-- Build files have been written to: /work/build

Background

L0_cuda_graph was failing as no cuda graph was being captured. RCA The backend was built with no cuda graph support as With this change the test is passes successfully.

[rmccorm4]

Nice find!