Skip to content

Commit

Permalink
Add environment variables for ilab container and increase shm size fo…
Browse files Browse the repository at this point in the history
…r vllm to 10GB

Include ILAB_GLOBAL_CONFIG, VLLM_LOGGING_LEVEL, and NCCL_DEBUG as environment variables when starting the ilab container. Also add shared memory size of 10G to enable vllm execution. Resolves: #721

Signed-off-by: Tyler Lisowski <lisowski@us.ibm.com>
  • Loading branch information
relyt0925 committed Aug 4, 2024
1 parent 7eae618 commit ea64b86
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions training/ilab-wrapper/ilab
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,15 @@ PODMAN_COMMAND=("sudo" "podman" "run" "--rm" "-it"
"${IMPERSONATE_CURRENT_USER_PODMAN_FLAGS[@]}"
"--device" "${CONTAINER_DEVICE}"
"--security-opt" "label=disable" "--net" "host"
"--shm-size" "10G"
"-v" "$HOME:$HOME"
"${ADDITIONAL_MOUNT_OPTIONS[@]}"
# This is intentionally NOT using "--env" "HOME" because we want the HOME
# of the current shell and not the HOME set by sudo
"--env" "HOME=$HOME"
"--env" "ILAB_GLOBAL_CONFIG=$ILAB_GLOBAL_CONFIG"
"--env" "VLLM_LOGGING_LEVEL=$VLLM_LOGGING_LEVEL"
"--env" "NCCL_DEBUG=$NCCL_DEBUG"
"--entrypoint" "$ENTRYPOINT"
"--env" "HF_TOKEN"
"${IMAGE_NAME}")
Expand Down

0 comments on commit ea64b86

Please sign in to comment.