[FEATURE]: Enable Istio in my garden Environment

[msharma38]

About

In my current garden environment setup i am using Kaniko to build my images in different namespace i.e garden-kaniko. Now when i am enabling istio in my garden environment i am getting the below error.

Error :-

[verbose] Starting Pod kaniko-simple-golang-988f12 with command '/bin/sh -c '/kaniko/executor' '--context' 'dir:///.garden/context' '--dockerfile' './docker/simple-golang.dockerfile' '--destination' 'us-docker.pkg.dev/wf-gcp-us-plat-gar-prod/docker-dev/garden-webhook/simple-golang:v-1fc17eb72a' '--cache=true' '--build-arg' 'GARDEN_MODULE_VERSION=v-1fc17eb72a' '--build-arg' 'GARDEN_ACTION_VERSION=v-1fc17eb72a' '--build-arg' 'ENVIRONMENT=dev' '--build-arg' 'GID=80' '--build-arg' 'GROUP=www' '--build-arg' 'UID=1001' '--build-arg' 'USER=javamambauser';
export exitcode=$?;
'touch' '/.garden/done';
exit $exitcode;'
ℹ build.simple-golang → [verbose] [kaniko] Configuring supplied registries....
2024-10-18T14:50:32.789687062Z Adding config for registries: us-docker.pkg.dev
2024-10-18T14:50:32.789874592Z /kaniko/.docker/config.json configured to use this credential helper for GCR registries
2024-10-18T14:50:32.815029734Z Error: error resolving dockerfile path: please provide a valid path to a Dockerfile within the build context with --dockerfile
ℹ build.simple-golang → [verbose] [kaniko] Usage:
2024-10-18T14:50:32.815760534Z executor [flags]
2024-10-18T14:50:32.815784665Z executor [command]
2024-10-18T14:50:32.815787945Z
ℹ build.simple-golang → [verbose] [kaniko] Available Commands:
2024-10-18T14:50:32.815795694Z completion Generate the autocompletion script for the specified shell
2024-10-18T14:50:32.815800165Z help Help about any command
2024-10-18T14:50:32.815803514Z version Print the version number of kaniko
2024-10-18T14:50:32.815807494Z
2024-10-18T14:50:32.815812485Z Flags:
2024-10-18T14:50:32.815818794Z --build-arg multi-arg type This flag allows you to pass in ARG values at build time. Set it repeatedly for multiple values. (default )
2024-10-18T14:50:32.815825914Z --cache Use cache when building image
2024-10-18T14:50:32.815830905Z --cache-copy-layers Caches copy layers
2024-10-18T14:50:32.815836334Z --cache-dir string Specify a local directory to use as a cache. (default "/cache")
2024-10-18T14:50:32.815841694Z --cache-repo string Specify a repository to use as a cache, otherwise one will be inferred from the destination provided; when prefixed with 'oci:' the repository will be written in OCI image layout format at the path provided
2024-10-18T14:50:32.815847074Z --cache-run-layers Caches run layers (default true)
ℹ build.simple-golang → [verbose] [kaniko] --cache-ttl duration Cache timeout, requires value and unit of duration -> ex: 6h. Defaults to two weeks. (default 336h0m0s)
2024-10-18T14:50:32.815857674Z --cleanup Clean the filesystem at the end
2024-10-18T14:50:32.815862104Z --compressed-caching Compress the cached layers. Decreases build time, but increases memory usage. (default true)
2024-10-18T14:50:32.815865374Z --compression compression Compression algorithm (gzip, zstd)
2024-10-18T14:50:32.815868664Z --compression-level int Compression level (default -1)
2024-10-18T14:50:32.815872594Z -c, --context string Path to the dockerfile build context. (default "/workspace/")
2024-10-18T14:50:32.815876354Z --context-sub-path string Sub path within the given context.
2024-10-18T14:50:32.815879914Z --custom-platform string Specify the build platform if different from the current host
ℹ build.simple-golang → [verbose] [kaniko] --customPlatform string This flag is deprecated. Please use '--custom-platform'.
2024-10-18T14:50:32.815886504Z -d, --destination multi-arg type Registry the final image should be pushed to. Set it repeatedly for multiple destinations. (default )
2024-10-18T14:50:32.815889734Z --digest-file string Specify a file to save the digest of the built image to.
2024-10-18T14:50:32.815893094Z -f, --dockerfile string Path to the dockerfile to be built. (default "Dockerfile")
2024-10-18T14:50:32.815896364Z --force Force building outside of a container
2024-10-18T14:50:32.815899554Z --force-build-metadata Force add metadata layers to build image
2024-10-18T14:50:32.815927824Z --git gitoptions Branch to clone if build context is a git repository (default branch=,single-branch=false,recurse-submodules=false)
2024-10-18T14:50:32.815934074Z -h, --help help for executor
2024-10-18T14:50:32.815940034Z --ignore-path multi-arg type Ignore these paths when taking a snapshot. Set it repeatedly for multiple paths.
2024-10-18T14:50:32.815945984Z --ignore-var-run Ignore /var/run directory when taking image snapshot. Set it to false to preserve /var/run/ in destination image. (default true)
2024-10-18T14:50:32.815950934Z --image-download-retry int Number of retries for downloading the remote image
2024-10-18T14:50:32.815955884Z --image-fs-extract-retry int Number of retries for image FS extraction
2024-10-18T14:50:32.815960864Z --image-name-tag-with-digest-file string Specify a file to save the image name w/ image tag w/ digest of the built image to.
2024-10-18T14:50:32.815966114Z --image-name-with-digest-file string Specify a file to save the image name w/ digest of the built image to.
2024-10-18T14:50:32.815970854Z --insecure Push to insecure registry using plain HTTP
2024-10-18T14:50:32.815974024Z --insecure-pull Pull from insecure registry using plain HTTP
2024-10-18T14:50:32.815977374Z --insecure-registry multi-arg type Insecure registry using plain HTTP to push and pull. Set it repeatedly for multiple registries.
2024-10-18T14:50:32.815980574Z --kaniko-dir string Path to the kaniko directory, this takes precedence over the KANIKO_DIR environment variable. (default "/kaniko")
2024-10-18T14:50:32.815998614Z --label multi-arg type Set metadata for an image. Set it repeatedly for multiple labels.
2024-10-18T14:50:32.816002554Z --log-format string Log format (text, color, json) (default "color")
ℹ build.simple-golang → [verbose] [kaniko] --log-timestamp Timestamp in log output
2024-10-18T14:50:32.816013484Z --no-push Do not push the image to the registry
2024-10-18T14:50:32.816018564Z --no-push-cache Do not push the cache layers to the registry
2024-10-18T14:50:32.816023924Z --oci-layout-path string Path to save the OCI image layout of the built image.
2024-10-18T14:50:32.816028674Z --push-ignore-immutable-tag-errors If true, known tag immutability errors are ignored and the push finishes with success.
2024-10-18T14:50:32.816033624Z --push-retry int Number of retries for the push operation
2024-10-18T14:50:32.816039204Z --registry-certificate key-value-arg type Use the provided certificate for TLS communication with the given registry. Expected format is 'my.registry.url=/path/to/the/server/certificate'.
2024-10-18T14:50:32.816048674Z --registry-client-cert key-value-arg type Use the provided client certificate for mutual TLS (mTLS) communication with the given registry. Expected format is 'my.registry.url=/path/to/client/cert,/path/to/client/key'.
2024-10-18T14:50:32.816054744Z --registry-map key-multi-value-arg type Registry map of mirror to use as pull-through cache instead. Expected format is 'orignal.registry=new.registry;other-original.registry=other-remap.registry' (default )
2024-10-18T14:50:32.816059764Z --registry-mirror multi-arg type Registry mirror to use as pull-through cache instead of docker.io. Set it repeatedly for multiple mirrors. (default )
2024-10-18T14:50:32.816073884Z --reproducible Strip timestamps out of the image to make it reproducible
2024-10-18T14:50:32.816078975Z --single-snapshot Take a single snapshot at the end of the build.
2024-10-18T14:50:32.816083935Z --skip-default-registry-fallback If an image is not found on any mirrors (defined with registry-mirror) do not fallback to the default registry. If registry-mirror is not defined, this flag is ignored.
2024-10-18T14:50:32.816094555Z --skip-push-permission-check Skip check of the push permission
2024-10-18T14:50:32.816098024Z --skip-tls-verify Push to insecure registry ignoring TLS verify
2024-10-18T14:50:32.816101155Z --skip-tls-verify-pull Pull from insecure registry ignoring TLS verify
2024-10-18T14:50:32.816104435Z --skip-tls-verify-registry multi-arg type Insecure registry ignoring TLS verify to push and pull. Set it repeatedly for multiple registries.
2024-10-18T14:50:32.816109215Z --skip-unused-stages Build only used stages if defined to true. Otherwise it builds by default all stages, even the unnecessaries ones until it reaches the target stage / end of Dockerfile
2024-10-18T14:50:32.816115404Z --snapshot-mode string Change the file attributes inspected during snapshotting (default "full")
2024-10-18T14:50:32.816121095Z --snapshotMode string This flag is deprecated. Please use '--snapshot-mode'.
2024-10-18T14:50:32.816126244Z --tar-path string Path to save the image in as a tarball instead of pushing
2024-10-18T14:50:32.816130904Z --tarPath string This flag is deprecated. Please use '--tar-path'.
2024-10-18T14:50:32.816135675Z --target string Set the target build stage to build
2024-10-18T14:50:32.816140604Z --use-new-run Use the experimental run implementation for detecting changes without requiring file system snapshots.
2024-10-18T14:50:32.816155835Z -v, --verbosity string Log level (trace, debug, info, warn, error, fatal, panic) (default "info")
2024-10-18T14:50:32.816160795Z
2024-10-18T14:5
✖ build.simple-golang → Failed (took 151.9 sec)
ℹ deploy.simple-golang → Aborting because upstream dependency failed.
ℹ deploy.simple-golang-istio → Aborting because upstream dependency failed.

Note:-
If we are using ingress instead of istio everything is working fine.

❓ A question

How important is this for you/your team?

As this feature is very important for our team as we are moving towards using istio instead of ingress.

🥀 Crucial, Garden is unusable for us without it

[twelvemo]

Hi @msharma38 , the error causing kaniko to fail is that it can't find the Dockerfile. The Dockerfile as part of the build context is being synced first to the util pod and then from the util pod to the kaniko instance on startup. Can you make sure that as part of your istio deployment, there are no network rules forbidding pods in the same namespace to communicate with each other?