[ADXT-199] Early start long e2e tests environment deployment on the CI (priority on EKS) #26891

KevinFairise2 · 2024-06-19T12:14:47Z

What does this PR do?

Add options and configure CI to run EKS test with a previous job that early start the creation of EKS cluster.
It makes things way more complex but it allows us to reduce by 30 minutes the duration of the containers EKS test which shorten by 30 minutes the duration of the whole CI.
Per se, it does not reduce the overall duration of the EKS jobs, but it allows to start the setup earlier in the CI which should reduce the overall duration of the CI by 30 minutes since EKS jobs are in the critical path

Needs this change on test-infra to work because containers tests does not use the Provider defined on datadog-agent repository

The overall idea is to have two jobs that share the same Pulumi stack, the first job will apply the stack and stop when we reach a point defined in the run function. The second call that is made when all the dependencies are ready will run the complete run fonction and execute the tests. All the resources created in the first call are reused.

With that we're still able to rerun the container test manually, recreating the stack from scratch

Motivation

Speed up the CI by creating earlier the resources that can be created

Additional Notes

In the future we plan to remove the cleanup part from the test itself and instead mark the stack for deletion. And let an external workflow to handle the deletion. Which would allow us to remove the cleanup job as well

Possible Drawbacks / Trade-offs

The setup is quite manual and would require us to create the two jobs everytime we want to speed EKS tests. However we do not expect a lot of EKS tests, as we rather recommend to use Kind. So it should be fine

Describe how to test/QA your changes

pr-commenter · 2024-06-19T15:28:28Z

Test changes on VM

Use this command from test-infra-definitions to manually test this PR changes on a VM:

inv create-vm --pipeline-id=46300838 --os-family=ubuntu

Note: This applies to commit 4f173fc

…/test-early-create-eks

pr-commenter · 2024-07-05T10:31:37Z

Regression Detector

Regression Detector Results

Run ID: b41ed032-d8c5-43fd-baf7-e50fa8ef8fbd Metrics dashboard Target profiles

Baseline: 304623b
Comparison: 4f173fc

Performance changes are noted in the perf column of each table:

✅ = significantly better comparison variant performance
❌ = significantly worse comparison variant performance
➖ = no significant change in performance

No significant changes in experiment optimization goals

Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%

There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.

Fine details of change detection per experiment

perf	experiment	goal	Δ mean %	Δ mean % CI	trials	links
➖	uds_dogstatsd_to_api_cpu	% cpu utilization	+1.67	[+0.95, +2.40]	1	Logs
➖	basic_py_check	% cpu utilization	+0.64	[-3.28, +4.57]	1	Logs
➖	file_to_blackhole_1000ms_latency	egress throughput	+0.25	[-0.25, +0.75]	1	Logs
➖	file_to_blackhole_300ms_latency	egress throughput	+0.15	[-0.04, +0.33]	1	Logs
➖	file_to_blackhole_500ms_latency	egress throughput	+0.04	[-0.20, +0.28]	1	Logs
➖	uds_dogstatsd_to_api	ingress throughput	+0.01	[-0.09, +0.11]	1	Logs
➖	file_to_blackhole_100ms_latency	egress throughput	+0.01	[-0.22, +0.23]	1	Logs
➖	tcp_dd_logs_filter_exclude	ingress throughput	-0.00	[-0.02, +0.01]	1	Logs
➖	otel_to_otel_logs	ingress throughput	-0.01	[-0.82, +0.80]	1	Logs
➖	file_to_blackhole_0ms_latency	egress throughput	-0.03	[-0.35, +0.30]	1	Logs
➖	pycheck_lots_of_tags	% cpu utilization	-0.04	[-3.49, +3.41]	1	Logs
➖	idle	memory utilization	-0.15	[-0.21, -0.09]	1	Logs
➖	file_tree	memory utilization	-1.26	[-1.41, -1.12]	1	Logs
➖	tcp_syslog_to_blackhole	ingress throughput	-1.70	[-1.77, -1.63]	1	Logs
➖	idle_all_features	memory utilization	-2.41	[-2.56, -2.27]	1	Logs

Bounds Checks

perf	experiment	bounds_check_name	replicates_passed
✅	file_to_blackhole_0ms_latency	memory_usage	10/10
✅	file_to_blackhole_1000ms_latency	memory_usage	10/10
✅	file_to_blackhole_100ms_latency	memory_usage	10/10
✅	file_to_blackhole_300ms_latency	memory_usage	10/10
✅	file_to_blackhole_500ms_latency	memory_usage	10/10
✅	idle	memory_usage	10/10

Explanation

A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".

For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:

Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
Its configuration does not mark it "erratic".

pducolin · 2024-07-05T14:59:31Z

@vboulineau would you consider migrating to new-e2e/test Provisioners, to reduce iterations in updates to your tests and to have a more standard pattern?

test/new-e2e/pkg/e2e/suite.go

test/new-e2e/pkg/runner/ci_profile.go

…/test-early-create-eks

pr-commenter · 2024-08-02T11:38:02Z

Gitlab CI Configuration Changes

Modified Jobs

stages (configuration)

  stages:
  - .pre
  - setup
  - maintenance_jobs
  - deps_build
  - deps_fetch
  - lint
  - source_test
  - source_test_stats
  - software_composition_analysis
  - binary_build
  - package_deps_build
  - kernel_matrix_testing_prepare
  - kernel_matrix_testing_system_probe
  - kernel_matrix_testing_security_agent
  - kernel_matrix_testing_cleanup
  - integration_test
  - benchmarks
  - package_build
  - packaging
  - pkg_metrics
  - kitchen_deploy
  - kitchen_testing
  - container_build
  - container_scan
  - check_deploy
  - dev_container_deploy
  - deploy_containers
  - deploy_packages
  - deploy_cws_instrumentation
  - deploy_dca
  - choco_and_install_script_build
  - trigger_release
  - choco_and_install_script_deploy
  - internal_image_deploy
  - install_script_testing
  - e2e_pre_test
+ - e2e_init
  - e2e
+ - e2e_cleanup
  - e2e_k8s
  - e2e_install_packages
  - kitchen_cleanup
  - functional_test
  - functional_test_cleanup
  - junit_upload
  - internal_kubernetes_deploy
  - post_rc_build
  - check_merge
  - notify
  - .post

variables (configuration)

  variables:
    AGENT_BINARIES_DIR: bin/agent
    AGENT_GITHUB_APP: agent-github-app
    AGENT_GITHUB_APP_ID: ci.datadog-agent.platform-github-app-id
    AGENT_GITHUB_INSTALLATION_ID: ci.datadog-agent.platform-github-app-installation-id
    AGENT_GITHUB_KEY: ci.datadog-agent.platform-github-app-key
    AGENT_QA_PROFILE: ci.datadog-agent.agent-qa-profile
    API_KEY_DDDEV: ci.datadog-agent.datadog_api_key
    API_KEY_ORG2: ci.datadog-agent.datadog_api_key_org2
    APP_KEY_ORG2: ci.datadog-agent.datadog_app_key_org2
    ARTIFACT_DOWNLOAD_ATTEMPTS: 2
    ATLASSIAN_WRITE: atlassian-write
    BTFHUB_ARCHIVE_BRANCH: main
    BUCKET_BRANCH: dev
    CHANGELOG_COMMIT_SHA: ci.datadog-agent.gitlab_changelog_commit_sha
    CHOCOLATEY_API_KEY: ci.datadog-agent.chocolatey_api_key
    CLANG_LLVM_VER: 12.0.1
    CLUSTER_AGENT_BINARIES_DIR: bin/datadog-cluster-agent
    CLUSTER_AGENT_CLOUDFOUNDRY_BINARIES_DIR: bin/datadog-cluster-agent-cloudfoundry
    CODECOV_TOKEN: ci.datadog-agent.codecov_token
    CWS_INSTRUMENTATION_BINARIES_DIR: bin/cws-instrumentation
    DATADOG_AGENT_ARMBUILDIMAGES: v45979046-89a73a40
    DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX: ''
    DATADOG_AGENT_BTF_GEN_BUILDIMAGES: v45979046-89a73a40
    DATADOG_AGENT_BTF_GEN_BUILDIMAGES_SUFFIX: ''
    DATADOG_AGENT_BUILDIMAGES: v45979046-89a73a40
    DATADOG_AGENT_BUILDIMAGES_SUFFIX: ''
    DATADOG_AGENT_EMBEDDED_PATH: /opt/datadog-agent/embedded
    DATADOG_AGENT_SYSPROBE_BUILDIMAGES: v45979046-89a73a40
    DATADOG_AGENT_SYSPROBE_BUILDIMAGES_SUFFIX: ''
    DATADOG_AGENT_WINBUILDIMAGES: v45979046-89a73a40
    DATADOG_AGENT_WINBUILDIMAGES_SUFFIX: ''
    DD_AGENT_TESTING_DIR: $CI_PROJECT_DIR/test/kitchen
    DD_PKG_VERSION: latest
    DEB_GPG_KEY: ci.datadog-agent.deb_signing_private_key_${DEB_GPG_KEY_ID}
    DEB_GPG_KEY_ID: c0962c7d
    DEB_GPG_KEY_NAME: Datadog, Inc. APT key
    DEB_RPM_TESTING_BUCKET_BRANCH: testing
    DEB_S3_BUCKET: apt.datad0g.com
    DEB_SIGNING_PASSPHRASE: ci.datadog-agent.deb_signing_key_passphrase_${DEB_GPG_KEY_ID}
    DEB_TESTING_S3_BUCKET: apttesting.datad0g.com
    DOCKER_REGISTRY_LOGIN: ci.datadog-agent.docker_hub_login
    DOCKER_REGISTRY_PWD: ci.datadog-agent.docker_hub_pwd
    DOCKER_REGISTRY_RO: dockerhub-readonly
    DOCKER_REGISTRY_URL: docker.io
    DOGSTATSD_BINARIES_DIR: bin/dogstatsd
+   E2E_PULUMI_CONFIG_PASSPHRASE: ci.datadog-agent.pulumi_password
    E2E_TESTS_API_KEY: ci.datadog-agent.e2e_tests_api_key
    E2E_TESTS_APP_KEY: ci.datadog-agent.e2e_tests_app_key
    E2E_TESTS_AZURE_CLIENT_ID: ci.datadog-agent.e2e_tests_azure_client_id
    E2E_TESTS_AZURE_CLIENT_SECRET: ci.datadog-agent.e2e_tests_azure_client_secret
    E2E_TESTS_AZURE_SUBSCRIPTION_ID: ci.datadog-agent.e2e_tests_azure_subscription_id
    E2E_TESTS_AZURE_TENANT_ID: ci.datadog-agent.e2e_tests_azure_tenant_id
    E2E_TESTS_GCP_CREDENTIALS: ci.datadog-agent.e2e_tests_gcp_credentials
    E2E_TESTS_RC_KEY: ci.datadog-agent.e2e_tests_rc_key
    EXECUTOR_JOB_SECTION_ATTEMPTS: 2
    FF_KUBERNETES_HONOR_ENTRYPOINT: true
    FF_SCRIPT_SECTIONS: 1
    GENERAL_ARTIFACTS_CACHE_BUCKET_URL: https://dd-agent-omnibus.s3.amazonaws.com
    GET_SOURCES_ATTEMPTS: 2
    GITHUB_PR_COMMENTER_APP_KEY: pr-commenter.github_app_key
    GITHUB_PR_COMMENTER_INSTALLATION_ID: pr-commenter.github_installation_id
    GITHUB_PR_COMMENTER_INTEGRATION_ID: pr-commenter.github_integration_id
    GITLAB_FULL_API_TOKEN: ci.datadog-agent.gitlab_full_api_token
    GITLAB_READ_API_TOKEN: ci.datadog-agent.gitlab_read_api_token
    GITLAB_SCHEDULER_TOKEN: ci.datadog-agent.gitlab_pipelines_scheduler_token
    GO_TEST_SKIP_FLAKE: 'true'
    INSTALL_SCRIPT_API_KEY: ci.agent-linux-install-script.datadog_api_key_2
    INTEGRATION_WHEELS_CACHE_BUCKET: dd-agent-omnibus
    JIRA_READ_API_TOKEN: ci.datadog-agent.jira_read_api_token
    KERNEL_MATRIX_TESTING_ARM_AMI_ID: ami-02fbd245475d6a63d
    KERNEL_MATRIX_TESTING_X86_AMI_ID: ami-013ea43cbe85107c0
    KITCHEN_AZURE_CLIENT_ID: ci.datadog-agent.azure_kitchen_client_id
    KITCHEN_AZURE_CLIENT_SECRET: ci.datadog-agent.azure_kitchen_client_secret
    KITCHEN_AZURE_SUBSCRIPTION_ID: ci.datadog-agent.azure_kitchen_subscription_id
    KITCHEN_AZURE_TENANT_ID: ci.datadog-agent.azure_kitchen_tenant_id
    KITCHEN_EC2_SSH_KEY: ci.datadog-agent.aws_ec2_kitchen_ssh_key
    KITCHEN_INFRASTRUCTURE_FLAKES_RETRY: 2
    MACOS_GITHUB_APP_1: macos-github-app-one
    MACOS_GITHUB_APP_2: macos-github-app-two
    MACOS_GITHUB_APP_ID: ci.datadog-agent.macos_github_app_id
    MACOS_GITHUB_APP_ID_2: ci.datadog-agent.macos_github_app_id_2
    MACOS_GITHUB_INSTALLATION_ID: ci.datadog-agent.macos_github_installation_id
    MACOS_GITHUB_INSTALLATION_ID_2: ci.datadog-agent.macos_github_installation_id_2
    MACOS_GITHUB_KEY: ci.datadog-agent.macos_github_key_b64
    MACOS_GITHUB_KEY_2: ci.datadog-agent.macos_github_key_b64_2
    MACOS_S3_BUCKET: dd-agent-macostesting
    OMNIBUS_BASE_DIR: /omnibus
    OMNIBUS_GIT_CACHE_DIR: /tmp/omnibus-git-cache
    OMNIBUS_PACKAGE_DIR: $CI_PROJECT_DIR/omnibus/pkg/
    OMNIBUS_PACKAGE_DIR_SUSE: $CI_PROJECT_DIR/omnibus/suse/pkg
    PROCESS_S3_BUCKET: datad0g-process-agent
    RELEASE_VERSION_6: nightly
    RELEASE_VERSION_7: nightly-a7
    RESTORE_CACHE_ATTEMPTS: 2
    RPM_GPG_KEY: ci.datadog-agent.rpm_signing_private_key_${RPM_GPG_KEY_ID}
    RPM_GPG_KEY_ID: b01082d3
    RPM_GPG_KEY_NAME: Datadog, Inc. RPM key
    RPM_S3_BUCKET: yum.datad0g.com
    RPM_SIGNING_PASSPHRASE: ci.datadog-agent.rpm_signing_key_passphrase_${RPM_GPG_KEY_ID}
    RPM_TESTING_S3_BUCKET: yumtesting.datad0g.com
    RUN_E2E_TESTS: auto
    RUN_KMT_TESTS: auto
    RUN_UNIT_TESTS: auto
    S3_ARTIFACTS_URI: s3://dd-ci-artefacts-build-stable/$CI_PROJECT_NAME/$CI_PIPELINE_ID
    S3_CP_CMD: aws s3 cp $S3_CP_OPTIONS
    S3_CP_OPTIONS: --no-progress --region us-east-1 --sse AES256
    S3_DD_AGENT_OMNIBUS_BTFS_URI: s3://dd-agent-omnibus/btfs
    S3_DD_AGENT_OMNIBUS_LLVM_URI: s3://dd-agent-omnibus/llvm
    S3_DSD6_URI: s3://dsd6-staging
    S3_OMNIBUS_CACHE_BUCKET: dd-ci-datadog-agent-omnibus-cache-build-stable
    S3_PERMANENT_ARTIFACTS_URI: s3://dd-ci-persistent-artefacts-build-stable/$CI_PROJECT_NAME
    S3_PROJECT_ARTIFACTS_URI: s3://dd-ci-artefacts-build-stable/$CI_PROJECT_NAME
    S3_RELEASE_ARTIFACTS_URI: s3://dd-release-artifacts/$CI_PROJECT_NAME/$CI_PIPELINE_ID
    S3_RELEASE_INSTALLER_ARTIFACTS_URI: s3://dd-release-artifacts/datadog-installer/$CI_PIPELINE_ID
    S3_SBOM_STORAGE_URI: s3://sbom-root-us1-ddbuild-io/$CI_PROJECT_NAME/$CI_PIPELINE_ID
    SLACK_AGENT_CI_TOKEN: ci.datadog-agent.slack_agent_ci_token
    SMP_ACCOUNT_ID: ci.datadog-agent.single-machine-performance-account-id
    SMP_AGENT_TEAM_ID: ci.datadog-agent.single-machine-performance-agent-team-id
    SMP_API: ci.datadog-agent.single-machine-performance-api
    SMP_BOT_ACCESS_KEY: ci.datadog-agent.single-machine-performance-bot-access-key
    SMP_BOT_ACCESS_KEY_ID: ci.datadog-agent.single-machine-performance-bot-access-key-id
    SSH_KEY: ci.datadog-agent.ssh_key
    SSH_KEY_RSA: ci.datadog-agent.ssh_key_rsa
    SSH_PUBLIC_KEY_RSA: ci.datadog-agent.ssh_public_key_rsa
    STATIC_BINARIES_DIR: bin/static
    SYSTEM_PROBE_BINARIES_DIR: bin/system-probe
    USE_S3_CACHING: --omnibus-s3-cache
    VCPKG_BLOB_SAS_URL: ci.datadog-agent-buildimages.vcpkg_blob_sas_url
    WINDOWS_BUILDS_S3_BUCKET: $WIN_S3_BUCKET/builds
    WINDOWS_POWERSHELL_DIR: $CI_PROJECT_DIR/signed_scripts
    WINDOWS_TESTING_S3_BUCKET_A6: pipelines/A6/$CI_PIPELINE_ID
    WINDOWS_TESTING_S3_BUCKET_A7: pipelines/A7/$CI_PIPELINE_ID
    WINGET_PAT: ci.datadog-agent.winget_pat
    WIN_S3_BUCKET: dd-agent-mstesting

.on_npm_or_e2e_changes

  .on_npm_or_e2e_changes:
  - if: $RUN_E2E_TESTS == "off"
    when: never
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - if: $RUN_E2E_TESTS == "on"
    when: on_success
  - if: $CI_COMMIT_BRANCH == "main"
    when: on_success
  - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
    when: on_success
  - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
    when: on_success
  - changes:
      compare_to: main
      paths:
      - test/new-e2e/pkg/**/*
      - test/new-e2e/go.mod
      - flakes.yaml
  - changes:
      compare_to: main
      paths:
+     - pkg/network/**/*
      - test/new-e2e/tests/npm/**/*

new-e2e-containers

  new-e2e-containers:
    after_script:
    - $CI_PROJECT_DIR/tools/ci/junit_upload.sh
    artifacts:
      expire_in: 2 weeks
      paths:
      - $E2E_OUTPUT_DIR
      - junit-*.tgz
      reports:
        annotations:
        - $EXTERNAL_LINKS_PATH
      when: always
    before_script:
    - mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
    - rm -f modcache_e2e.tar.xz
    - mkdir -p ~/.aws
    - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
      exit $?
    - export AWS_PROFILE=agent-qa-ci
    - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
      || exit $?
    - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
      $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
    - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
    - ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
      || exit $?; export ARM_CLIENT_ID
    - ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
      || exit $?; export ARM_CLIENT_SECRET
    - ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
      || exit $?; export ARM_TENANT_ID
    - ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
      || exit $?; export ARM_SUBSCRIPTION_ID
    - GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
      || exit $?; export GOOGLE_CREDENTIALS
    - inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
    image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
    needs:
    - go_e2e_deps
    - qa_agent
    - qa_dca
    - qa_dogstatsd
    parallel:
      matrix:
      - EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.19
      - EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.22
      - EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.27
      - EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.29
      - EXTRA_PARAMS: --run TestKindSuite -c ddinfra:osDescriptor=ubuntu:20.04
      - EXTRA_PARAMS: --run TestKindSuite -c ddinfra:osDescriptor=ubuntu:22.04
-     - EXTRA_PARAMS: --run TestEKSSuite
      - EXTRA_PARAMS: --run TestECSSuite
      - EXTRA_PARAMS: --run TestDockerSuite
      - EXTRA_PARAMS: --skip "Test(Kind|EKS|ECS|Docker)Suite"
    rules:
    - if: $RUN_E2E_TESTS == "off"
      when: never
    - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
      when: never
    - if: $RUN_E2E_TESTS == "on"
      when: on_success
    - if: $CI_COMMIT_BRANCH == "main"
      when: on_success
    - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
      when: on_success
    - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
      when: on_success
    - changes:
        compare_to: main
        paths:
        - test/new-e2e/pkg/**/*
        - test/new-e2e/go.mod
        - flakes.yaml
    - changes:
        compare_to: main
        paths:
        - comp/core/tagger/**/*
        - comp/core/workloadmeta/**/*
        - comp/core/autodiscovery/listeners/**/*
        - comp/core/autodiscovery/providers/**/*
        - comp/languagedetection/**/*
        - pkg/clusteragent/admission/mutate/**/*
        - pkg/collector/corechecks/cluster/**/*
        - pkg/collector/corechecks/containers/**/*
        - pkg/collector/corechecks/containerimage/**/*
        - pkg/collector/corechecks/containerlifecycle/**/*
        - pkg/collector/corechecks/sbom/**/*
        - pkg/sbom/**/*
        - pkg/util/clusteragent/**/*
        - pkg/util/containerd/**/*
        - pkg/util/containers/**/*
        - pkg/util/docker/**/*
        - pkg/util/ecs/**/*
        - pkg/util/kubernetes/**/*
        - pkg/util/cgroups/**/*
        - pkg/util/trivy/**/*
        - test/new-e2e/tests/containers/**/*
        - test/new-e2e/go.mod
      when: on_success
    - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
      when: never
    - allow_failure: true
      when: manual
    script:
    - inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
      -c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
      --junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
    stage: e2e
    tags:
    - arch:amd64
    variables:
      E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
      E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
      E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
      E2E_PIPELINE_ID: $CI_PIPELINE_ID
      E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
      E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
      EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
      KUBERNETES_CPU_REQUEST: 6
      KUBERNETES_MEMORY_LIMIT: 16Gi
      KUBERNETES_MEMORY_REQUEST: 12Gi
      SHOULD_RUN_IN_FLAKES_FINDER: 'true'
      TARGETS: ./tests/containers
      TEAM: container-integrations

new-e2e-npm-docker

  new-e2e-npm-docker:
    after_script:
    - $CI_PROJECT_DIR/tools/ci/junit_upload.sh
    artifacts:
      expire_in: 2 weeks
      paths:
      - $E2E_OUTPUT_DIR
      - junit-*.tgz
      reports:
        annotations:
        - $EXTERNAL_LINKS_PATH
      when: always
    before_script:
    - mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
    - rm -f modcache_e2e.tar.xz
    - mkdir -p ~/.aws
    - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
      exit $?
    - export AWS_PROFILE=agent-qa-ci
    - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
      || exit $?
    - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
      $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
    - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
    - ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
      || exit $?; export ARM_CLIENT_ID
    - ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
      || exit $?; export ARM_CLIENT_SECRET
    - ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
      || exit $?; export ARM_TENANT_ID
    - ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
      || exit $?; export ARM_SUBSCRIPTION_ID
    - GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
      || exit $?; export GOOGLE_CREDENTIALS
    - inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
    image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
    needs:
    - go_e2e_deps
    - qa_dca
    - qa_agent
    rules:
    - if: $RUN_E2E_TESTS == "off"
      when: never
    - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
      when: never
    - if: $RUN_E2E_TESTS == "on"
      when: on_success
    - if: $CI_COMMIT_BRANCH == "main"
      when: on_success
    - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
      when: on_success
    - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
      when: on_success
    - changes:
        compare_to: main
        paths:
        - test/new-e2e/pkg/**/*
        - test/new-e2e/go.mod
        - flakes.yaml
    - changes:
        compare_to: main
        paths:
+       - pkg/network/**/*
        - test/new-e2e/tests/npm/**/*
    - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
      when: never
    - allow_failure: true
      when: manual
    script:
    - inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
      -c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
      --junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
    stage: e2e
    tags:
    - arch:amd64
    variables:
      E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
      E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
      E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
      E2E_PIPELINE_ID: $CI_PIPELINE_ID
      E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
      E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
      EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
-     EXTRA_PARAMS: --run "Test(ECSVM|EC2VMContainerized|EKSVM)Suite"
?                                                       ------
+     EXTRA_PARAMS: --run "Test(ECSVM|EC2VMContainerized)Suite"
      KUBERNETES_CPU_REQUEST: 6
      KUBERNETES_MEMORY_LIMIT: 16Gi
      KUBERNETES_MEMORY_REQUEST: 12Gi
      SHOULD_RUN_IN_FLAKES_FINDER: 'true'
      TARGETS: ./tests/npm
      TEAM: network-performance-monitoring

new-e2e-npm-packages

  new-e2e-npm-packages:
    after_script:
    - $CI_PROJECT_DIR/tools/ci/junit_upload.sh
    artifacts:
      expire_in: 2 weeks
      paths:
      - $E2E_OUTPUT_DIR
      - junit-*.tgz
      reports:
        annotations:
        - $EXTERNAL_LINKS_PATH
      when: always
    before_script:
    - mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
    - rm -f modcache_e2e.tar.xz
    - mkdir -p ~/.aws
    - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
      exit $?
    - export AWS_PROFILE=agent-qa-ci
    - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
      || exit $?
    - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
      $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
    - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
    - ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
      || exit $?; export ARM_CLIENT_ID
    - ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
      || exit $?; export ARM_CLIENT_SECRET
    - ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
      || exit $?; export ARM_TENANT_ID
    - ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
      || exit $?; export ARM_SUBSCRIPTION_ID
    - GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
      || exit $?; export GOOGLE_CREDENTIALS
    - inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
    image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
    needs:
    - go_e2e_deps
    - deploy_deb_testing-a7_x64
    - deploy_rpm_testing-a7_x64
    - deploy_windows_testing-a7
    rules:
    - if: $RUN_E2E_TESTS == "off"
      when: never
    - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
      when: never
    - if: $RUN_E2E_TESTS == "on"
      when: on_success
    - if: $CI_COMMIT_BRANCH == "main"
      when: on_success
    - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
      when: on_success
    - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
      when: on_success
    - changes:
        compare_to: main
        paths:
        - test/new-e2e/pkg/**/*
        - test/new-e2e/go.mod
        - flakes.yaml
    - changes:
        compare_to: main
        paths:
+       - pkg/network/**/*
        - test/new-e2e/tests/npm/**/*
    - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
      when: never
    - allow_failure: true
      when: manual
    script:
    - inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
      -c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
      --junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
    stage: e2e
    tags:
    - arch:amd64
    variables:
      E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
      E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
      E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
      E2E_PIPELINE_ID: $CI_PIPELINE_ID
      E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
      E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
      EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
      EXTRA_PARAMS: --run "TestEC2(VM|VMSELinux|VMWKit)Suite"
      KUBERNETES_CPU_REQUEST: 6
      KUBERNETES_MEMORY_LIMIT: 16Gi
      KUBERNETES_MEMORY_REQUEST: 12Gi
      SHOULD_RUN_IN_FLAKES_FINDER: 'true'
      TARGETS: ./tests/npm
      TEAM: network-performance-monitoring

Added Jobs

new-e2e-containers-eks

new-e2e-containers-eks:
  after_script:
  - $CI_PROJECT_DIR/tools/ci/junit_upload.sh
  artifacts:
    expire_in: 2 weeks
    paths:
    - $E2E_OUTPUT_DIR
    - junit-*.tgz
    reports:
      annotations:
      - $EXTERNAL_LINKS_PATH
    when: always
  before_script:
  - mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
  - rm -f modcache_e2e.tar.xz
  - mkdir -p ~/.aws
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
    exit $?
  - export AWS_PROFILE=agent-qa-ci
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
    || exit $?
  - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
    $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
  - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
  - ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
    || exit $?; export ARM_CLIENT_ID
  - ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
    || exit $?; export ARM_CLIENT_SECRET
  - ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
    || exit $?; export ARM_TENANT_ID
  - ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
    || exit $?; export ARM_SUBSCRIPTION_ID
  - GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
    || exit $?; export GOOGLE_CREDENTIALS
  - inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
  image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
  needs:
  - go_e2e_deps
  - new-e2e-containers-eks-init
  - qa_agent
  - qa_dca
  rules:
  - if: $RUN_E2E_TESTS == "off"
    when: never
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - if: $RUN_E2E_TESTS == "on"
    when: on_success
  - if: $CI_COMMIT_BRANCH == "main"
    when: on_success
  - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
    when: on_success
  - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
    when: on_success
  - changes:
      compare_to: main
      paths:
      - test/new-e2e/pkg/**/*
      - test/new-e2e/go.mod
      - flakes.yaml
  - changes:
      compare_to: main
      paths:
      - comp/core/tagger/**/*
      - comp/core/workloadmeta/**/*
      - comp/core/autodiscovery/listeners/**/*
      - comp/core/autodiscovery/providers/**/*
      - comp/languagedetection/**/*
      - pkg/clusteragent/admission/mutate/**/*
      - pkg/collector/corechecks/cluster/**/*
      - pkg/collector/corechecks/containers/**/*
      - pkg/collector/corechecks/containerimage/**/*
      - pkg/collector/corechecks/containerlifecycle/**/*
      - pkg/collector/corechecks/sbom/**/*
      - pkg/sbom/**/*
      - pkg/util/clusteragent/**/*
      - pkg/util/containerd/**/*
      - pkg/util/containers/**/*
      - pkg/util/docker/**/*
      - pkg/util/ecs/**/*
      - pkg/util/kubernetes/**/*
      - pkg/util/cgroups/**/*
      - pkg/util/trivy/**/*
      - test/new-e2e/tests/containers/**/*
      - test/new-e2e/go.mod
    when: on_success
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - allow_failure: true
    when: manual
  script:
  - inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
    -c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
    --junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
  stage: e2e
  tags:
  - arch:amd64
  variables:
    E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
    E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
    E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
    E2E_PIPELINE_ID: $CI_PIPELINE_ID
    E2E_PRE_INITIALIZED: 'true'
    E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
    E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
    EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
    EXTRA_PARAMS: --run TestEKSSuite
    KUBERNETES_CPU_REQUEST: 6
    KUBERNETES_MEMORY_LIMIT: 16Gi
    KUBERNETES_MEMORY_REQUEST: 12Gi
    SHOULD_RUN_IN_FLAKES_FINDER: 'true'
    TARGETS: ./tests/containers
    TEAM: container-integrations

new-e2e-containers-eks-init

new-e2e-containers-eks-init:
  after_script:
  - $CI_PROJECT_DIR/tools/ci/junit_upload.sh
  allow_failure: true
  artifacts:
    expire_in: 2 weeks
    paths:
    - $E2E_OUTPUT_DIR
    - junit-*.tgz
    reports:
      annotations:
      - $EXTERNAL_LINKS_PATH
    when: always
  before_script:
  - mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
  - rm -f modcache_e2e.tar.xz
  - mkdir -p ~/.aws
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
    exit $?
  - export AWS_PROFILE=agent-qa-ci
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
    || exit $?
  - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
    $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
  - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
  - ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
    || exit $?; export ARM_CLIENT_ID
  - ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
    || exit $?; export ARM_CLIENT_SECRET
  - ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
    || exit $?; export ARM_TENANT_ID
  - ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
    || exit $?; export ARM_SUBSCRIPTION_ID
  - GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
    || exit $?; export GOOGLE_CREDENTIALS
  - inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
  image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
  needs:
  - go_e2e_deps
  rules:
  - if: $RUN_E2E_TESTS == "off"
    when: never
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - if: $RUN_E2E_TESTS == "on"
    when: on_success
  - if: $CI_COMMIT_BRANCH == "main"
    when: on_success
  - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
    when: on_success
  - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
    when: on_success
  - changes:
      compare_to: main
      paths:
      - test/new-e2e/pkg/**/*
      - test/new-e2e/go.mod
      - flakes.yaml
  - changes:
      compare_to: main
      paths:
      - comp/core/tagger/**/*
      - comp/core/workloadmeta/**/*
      - comp/core/autodiscovery/listeners/**/*
      - comp/core/autodiscovery/providers/**/*
      - comp/languagedetection/**/*
      - pkg/clusteragent/admission/mutate/**/*
      - pkg/collector/corechecks/cluster/**/*
      - pkg/collector/corechecks/containers/**/*
      - pkg/collector/corechecks/containerimage/**/*
      - pkg/collector/corechecks/containerlifecycle/**/*
      - pkg/collector/corechecks/sbom/**/*
      - pkg/sbom/**/*
      - pkg/util/clusteragent/**/*
      - pkg/util/containerd/**/*
      - pkg/util/containers/**/*
      - pkg/util/docker/**/*
      - pkg/util/ecs/**/*
      - pkg/util/kubernetes/**/*
      - pkg/util/cgroups/**/*
      - pkg/util/trivy/**/*
      - test/new-e2e/tests/containers/**/*
      - test/new-e2e/go.mod
    when: on_success
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - allow_failure: true
    when: manual
  script:
  - inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
    -c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
    --junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
  stage: e2e_init
  tags:
  - arch:amd64
  variables:
    E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
    E2E_INIT_ONLY: 'true'
    E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
    E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
    E2E_PIPELINE_ID: $CI_PIPELINE_ID
    E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
    E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
    EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
    EXTRA_PARAMS: --run TestEKSSuite
    KUBERNETES_CPU_REQUEST: 6
    KUBERNETES_MEMORY_LIMIT: 16Gi
    KUBERNETES_MEMORY_REQUEST: 12Gi
    SHOULD_RUN_IN_FLAKES_FINDER: 'true'
    TARGETS: ./tests/containers
    TEAM: container-integrations

new-e2e-eks-cleanup-on-failure

new-e2e-eks-cleanup-on-failure:
  allow_failure: true
  dependencies:
  - go_e2e_deps
  image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
  script:
  - mkdir -p ~/.aws
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
    exit $?
  - export AWS_PROFILE=agent-qa-ci
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
    || exit $?
  - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
    $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
  - PULUMI_CONFIG_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_PULUMI_CONFIG_PASSPHRASE)
    || exit $?; export PULUMI_CONFIG_PASSPHRASE
  - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
  - inv -e new-e2e-tests.cleanup-remote-stacks --stack-regex "ci-$CI_PIPELINE_ID.*eks.*"
    --pulumi-backend=dd-pulumi-state
  stage: e2e_cleanup
  tags:
  - arch:amd64
  variables:
    E2E_PIPELINE_ID: $CI_PIPELINE_ID
    E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
    E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
  when: always

new-e2e-npm-eks

new-e2e-npm-eks:
  after_script:
  - $CI_PROJECT_DIR/tools/ci/junit_upload.sh
  artifacts:
    expire_in: 2 weeks
    paths:
    - $E2E_OUTPUT_DIR
    - junit-*.tgz
    reports:
      annotations:
      - $EXTERNAL_LINKS_PATH
    when: always
  before_script:
  - mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
  - rm -f modcache_e2e.tar.xz
  - mkdir -p ~/.aws
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
    exit $?
  - export AWS_PROFILE=agent-qa-ci
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
    || exit $?
  - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
    $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
  - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
  - ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
    || exit $?; export ARM_CLIENT_ID
  - ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
    || exit $?; export ARM_CLIENT_SECRET
  - ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
    || exit $?; export ARM_TENANT_ID
  - ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
    || exit $?; export ARM_SUBSCRIPTION_ID
  - GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
    || exit $?; export GOOGLE_CREDENTIALS
  - inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
  image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
  needs:
  - go_e2e_deps
  - new-e2e-npm-eks-init
  - qa_agent
  - qa_dca
  rules:
  - if: $RUN_E2E_TESTS == "off"
    when: never
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - if: $RUN_E2E_TESTS == "on"
    when: on_success
  - if: $CI_COMMIT_BRANCH == "main"
    when: on_success
  - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
    when: on_success
  - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
    when: on_success
  - changes:
      compare_to: main
      paths:
      - test/new-e2e/pkg/**/*
      - test/new-e2e/go.mod
      - flakes.yaml
  - changes:
      compare_to: main
      paths:
      - pkg/network/**/*
      - test/new-e2e/tests/npm/**/*
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - allow_failure: true
    when: manual
  script:
  - inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
    -c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
    --junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
  stage: e2e
  tags:
  - arch:amd64
  variables:
    E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
    E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
    E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
    E2E_PIPELINE_ID: $CI_PIPELINE_ID
    E2E_PRE_INITIALIZED: 'true'
    E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
    E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
    EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
    EXTRA_PARAMS: --run "TestEKSVMSuite"
    KUBERNETES_CPU_REQUEST: 6
    KUBERNETES_MEMORY_LIMIT: 16Gi
    KUBERNETES_MEMORY_REQUEST: 12Gi
    SHOULD_RUN_IN_FLAKES_FINDER: 'true'
    TARGETS: ./tests/npm
    TEAM: network-performance-monitoring

new-e2e-npm-eks-init

new-e2e-npm-eks-init:
  after_script:
  - $CI_PROJECT_DIR/tools/ci/junit_upload.sh
  allow_failure: true
  artifacts:
    expire_in: 2 weeks
    paths:
    - $E2E_OUTPUT_DIR
    - junit-*.tgz
    reports:
      annotations:
      - $EXTERNAL_LINKS_PATH
    when: always
  before_script:
  - mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
  - rm -f modcache_e2e.tar.xz
  - mkdir -p ~/.aws
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
    exit $?
  - export AWS_PROFILE=agent-qa-ci
  - $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
    || exit $?
  - touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
    $SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
  - pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
  - ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
    || exit $?; export ARM_CLIENT_ID
  - ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
    || exit $?; export ARM_CLIENT_SECRET
  - ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
    || exit $?; export ARM_TENANT_ID
  - ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
    || exit $?; export ARM_SUBSCRIPTION_ID
  - GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
    || exit $?; export GOOGLE_CREDENTIALS
  - inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
  image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
  needs:
  - go_e2e_deps
  rules:
  - if: $RUN_E2E_TESTS == "off"
    when: never
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - if: $RUN_E2E_TESTS == "on"
    when: on_success
  - if: $CI_COMMIT_BRANCH == "main"
    when: on_success
  - if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
    when: on_success
  - if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
    when: on_success
  - changes:
      compare_to: main
      paths:
      - test/new-e2e/pkg/**/*
      - test/new-e2e/go.mod
      - flakes.yaml
  - changes:
      compare_to: main
      paths:
      - pkg/network/**/*
      - test/new-e2e/tests/npm/**/*
  - if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
    when: never
  - allow_failure: true
    when: manual
  script:
  - inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
    -c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
    --junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
  stage: e2e_init
  tags:
  - arch:amd64
  variables:
    E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
    E2E_INIT_ONLY: 'true'
    E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
    E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
    E2E_PIPELINE_ID: $CI_PIPELINE_ID
    E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
    E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
    EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
    EXTRA_PARAMS: --run "TestEKSVMSuite"
    KUBERNETES_CPU_REQUEST: 6
    KUBERNETES_MEMORY_LIMIT: 16Gi
    KUBERNETES_MEMORY_REQUEST: 12Gi
    SHOULD_RUN_IN_FLAKES_FINDER: 'true'
    TARGETS: ./tests/npm
    TEAM: network-performance-monitoring

Changes Summary

Removed	Modified	Added	Renamed
0	6	5	0

ℹ️ Diff available in the job log.

Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>

…atadog-agent into kfairise/test-early-create-eks

test/new-e2e/pkg/e2e/suite.go

test/new-e2e/pkg/environments/aws/kubernetes/eks.go

test/new-e2e/pkg/e2e/suite.go

test/new-e2e/pkg/runner/ci_profile.go

chouetz

LGTM, thanks for the updates

tasks/new_e2e_tests.py

test/new-e2e/tests/npm/eks_1host_test.go

f4usto

lgtm

davidor

👍 for container-platform files

…/test-early-create-eks

…atadog-agent into kfairise/test-early-create-eks

KevinFairise2 added 3 commits June 18, 2024 14:54

WIP

20586da

First test in CI to use pre initialized envs

67c952a

First test in CI to use pre initialized envs

5f643fc

github-actions bot added the team/agent-developer-tools label Jun 19, 2024

Add init only mode for containers and npm eks tests

8a66aba

KevinFairise2 added 4 commits June 19, 2024 17:39

Fix initOnly param

bfe3f20

Fix initOnly param

b0e0f34

Compare with and without init

e10f67c

Try to work with containers test [skip cancel]

15e289c

KevinFairise2 changed the title ~~WIP~~ [ADXT-199] Early start long e2e tests environment deployment on the CI (priority on EKS) Jun 20, 2024

Merge branch 'main' of github.com:DataDog/datadog-agent into kfairise…

acc4b06

…/test-early-create-eks

KevinFairise2 added changelog/no-changelog qa/no-code-change Skip QA week as there is no code change in Agent code labels Jul 5, 2024

Clean duplicated jobs [skip cancel]

496a632

KevinFairise2 marked this pull request as ready for review July 5, 2024 10:59

KevinFairise2 requested review from a team as code owners July 5, 2024 10:59

hmahmood approved these changes Jul 5, 2024

View reviewed changes

pducolin approved these changes Jul 5, 2024

View reviewed changes

test/new-e2e/pkg/e2e/suite.go Show resolved Hide resolved

test/new-e2e/pkg/runner/ci_profile.go Outdated Show resolved Hide resolved

KevinFairise2 added 3 commits August 2, 2024 13:34

Merge branch 'main' of github.com:DataDog/datadog-agent into kfairise…

c2d84df

…/test-early-create-eks

Test infra init only

9212a87

Remove leftover

8b6c0c9

Get go mod deps

2510a13

KevinFairise2 and others added 8 commits September 25, 2024 09:20

Print error

ecc96a3

Update .gitlab/e2e/e2e.yml

661f531

Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>

Update .gitlab/e2e/e2e.yml

4706f26

Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>

Update .gitlab/e2e/e2e.yml

d77ed79

Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>

Update tasks/new_e2e_tests.py

412bda8

Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>

Fix suggestion

565746b

Apply suggestion

aa43015

Update .gitlab/e2e/e2e.yml

74be619

Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>

KevinFairise2 mentioned this pull request Sep 25, 2024

Fix gitlab configuration check to not crash on job without rules #29558

Merged

KevinFairise2 added 2 commits September 25, 2024 14:36

Remove print and use -e

69bb994

Merge branch 'kfairise/test-early-create-eks' of github.com:DataDog/d…

6f12bd1

…atadog-agent into kfairise/test-early-create-eks

KevinFairise2 requested review from f4usto and chouetz September 25, 2024 13:17

vboulineau reviewed Sep 27, 2024

View reviewed changes

chouetz approved these changes Sep 30, 2024

View reviewed changes

KevinFairise2 added 2 commits September 30, 2024 13:01

Address comments

efb6955

Fix test and remove no longer existing params

658fac5

KevinFairise2 requested a review from vboulineau September 30, 2024 14:58

vboulineau reviewed Oct 1, 2024

View reviewed changes

tasks/new_e2e_tests.py Outdated Show resolved Hide resolved

test/new-e2e/tests/npm/eks_1host_test.go Outdated Show resolved Hide resolved

KevinFairise2 added 3 commits October 1, 2024 13:05

Remove unneeded line

cb50ebd

Make cleanup_remote_stack more generic

0e9807b

Use pulumi stack ls --all

7114f3b

KevinFairise2 requested a review from vboulineau October 7, 2024 07:47

Merge branch 'main' into kfairise/test-early-create-eks

d99c2ef

f4usto approved these changes Oct 8, 2024

View reviewed changes

davidor approved these changes Oct 10, 2024

View reviewed changes

KevinFairise2 added 4 commits October 10, 2024 13:11

Merge branch 'main' of github.com:DataDog/datadog-agent into kfairise…

75bdc8d

…/test-early-create-eks

Merge branch 'kfairise/test-early-create-eks' of github.com:DataDog/d…

fee3d34

…atadog-agent into kfairise/test-early-create-eks

Authorize my cleanup job

9243dd4

Fix linter

4f173fc

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ADXT-199] Early start long e2e tests environment deployment on the CI (priority on EKS) #26891

[ADXT-199] Early start long e2e tests environment deployment on the CI (priority on EKS) #26891

KevinFairise2 commented Jun 19, 2024 •

edited

Loading

pr-commenter bot commented Jun 19, 2024 •

edited by agent-platform-auto-pr bot

Loading

pr-commenter bot commented Jul 5, 2024 •

edited by cit-pr-commenter bot

Loading

Fine details of change detection per experiment

Explanation

pducolin commented Jul 5, 2024

pr-commenter bot commented Aug 2, 2024 •

edited by agent-platform-auto-pr bot

Loading

chouetz left a comment

f4usto left a comment

davidor left a comment

[ADXT-199] Early start long e2e tests environment deployment on the CI (priority on EKS) #26891

Are you sure you want to change the base?

[ADXT-199] Early start long e2e tests environment deployment on the CI (priority on EKS) #26891

Conversation

KevinFairise2 commented Jun 19, 2024 • edited Loading

What does this PR do?

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

pr-commenter bot commented Jun 19, 2024 • edited by agent-platform-auto-pr bot Loading

Test changes on VM

pr-commenter bot commented Jul 5, 2024 • edited by cit-pr-commenter bot Loading

Regression Detector

Regression Detector Results

No significant changes in experiment optimization goals

Fine details of change detection per experiment

Bounds Checks

Explanation

pducolin commented Jul 5, 2024

pr-commenter bot commented Aug 2, 2024 • edited by agent-platform-auto-pr bot Loading

Gitlab CI Configuration Changes

Modified Jobs

Added Jobs

Changes Summary

chouetz left a comment

Choose a reason for hiding this comment

f4usto left a comment

Choose a reason for hiding this comment

davidor left a comment

Choose a reason for hiding this comment

KevinFairise2 commented Jun 19, 2024 •

edited

Loading

pr-commenter bot commented Jun 19, 2024 •

edited by agent-platform-auto-pr bot

Loading

pr-commenter bot commented Jul 5, 2024 •

edited by cit-pr-commenter bot

Loading

pr-commenter bot commented Aug 2, 2024 •

edited by agent-platform-auto-pr bot

Loading