Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve aws tests #3035

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open

improve aws tests #3035

wants to merge 32 commits into from

Conversation

wconti27
Copy link
Contributor

@wconti27 wconti27 commented Sep 13, 2024

Motivation

Fixes

  • Fix invalid parameter for PHP aws requests since weblog name contains a period: ., an illegal queue name character
  • Explicitly set AWS_REGION to us-east-1.
  • Change all CI AWS environment variables to be prefixed with SYSTEM_TESTS_[AWS Var name]
  • Add check for AWS env variables being set when running INTEGRATIONS and CROSSED_TRACING_LIBRARIES suites, and throw an error if the variables are not set. The message provides info on how to authenticate with AWS to run the test suite.

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@wconti27 wconti27 requested a review from a team as a code owner September 13, 2024 21:09
@wconti27 wconti27 self-assigned this Sep 13, 2024
@cbeauchesne cbeauchesne marked this pull request as draft September 16, 2024 11:55
@cbeauchesne
Copy link
Collaborator

i guess it's still in progress? So I converted to draft. If no, please revert 😃

@wconti27 wconti27 marked this pull request as ready for review September 16, 2024 12:38
@wconti27
Copy link
Contributor Author

i guess it's still in progress? So I converted to draft. If no, please revert 😃

This is the full PR. Simply ensuring no periods and other special characters make it into the queue name

@wconti27 wconti27 changed the title fix php aws error fix aws test errors Sep 16, 2024
@wconti27 wconti27 changed the title fix aws test errors improve aws tests Sep 16, 2024
.github/workflows/run-end-to-end.yml Outdated Show resolved Hide resolved
@cbeauchesne cbeauchesne marked this pull request as draft September 26, 2024 08:57
@wconti27 wconti27 marked this pull request as ready for review September 30, 2024 19:19
@wconti27
Copy link
Contributor Author

All system tests ci runs within each tracer's CI has been updated to include the new AWS env variables, can we get this merged?

.github/workflows/run-end-to-end.yml Outdated Show resolved Hide resolved
utils/_context/containers.py Outdated Show resolved Hide resolved
.github/workflows/run-end-to-end.yml Outdated Show resolved Hide resolved
utils/_context/containers.py Outdated Show resolved Hide resolved
utils/_context/containers.py Outdated Show resolved Hide resolved
utils/_context/containers.py Outdated Show resolved Hide resolved
@wconti27 wconti27 marked this pull request as draft October 3, 2024 13:25
@wconti27 wconti27 marked this pull request as ready for review October 3, 2024 13:43
@wconti27 wconti27 requested review from a team as code owners October 3, 2024 15:08
@wconti27 wconti27 requested review from a team as code owners October 3, 2024 21:37
@@ -154,6 +155,22 @@ def get_span_from_agent(weblog_request):
raise ValueError(f"Span is not found for {weblog_request.request.url}")


# set AWS credentials for runner
os.environ["AWS_ACCESS_KEY_ID"] = os.environ.get("SYSTEM_TESTS_AWS_ACCESS_KEY_ID", "")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to pass those value to the sdk directly ? If it's possible to not hack to much os.environ, it's better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this code, the code now just instantiates an AWS client session, and uses the appropriate variables without modifying the user's env.

@wconti27
Copy link
Contributor Author

wconti27 commented Oct 7, 2024

Ive added info to the AWS authentication error message on how to setup authentication (it uses aws-vault, which is the DD preferred way to deal with AWS auth), should we also add a CLI argument to skip AWS tests? That way if users don't want to deal with AWS auth, and are running tests other than those using AWS within the two suites of INTEGRATIONS / CROSSED TRACING, they can easily skip it? I was thinking --skip-aws

tests/integrations/utils.py Outdated Show resolved Hide resolved
tests/integrations/utils.py Outdated Show resolved Hide resolved
tests/integrations/utils.py Outdated Show resolved Hide resolved
tests/integrations/utils.py Outdated Show resolved Hide resolved
@@ -1120,3 +1115,23 @@ def post_start(self):

logger.stdout(f"Library: {self.library}")
logger.stdout(f"Image: {self.image.name}")


def set_aws_auth_environment(image):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it private to prevent external usage :

Suggested change
def set_aws_auth_environment(image):
def _set_aws_auth_environment(image):

utils/_context/containers.py Show resolved Hide resolved
@@ -23,6 +26,18 @@ def _get_unique_id(replay: bool, host_log_folder: str) -> str:
return unique_id


def _check_aws_variables(scenario: EndToEndScenario):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be more clean to make this a method of EndToEndScenario ?

class EndToEndScenario:
   ...
   def _check_aws_variables(self):  
      ...

utils/_context/_scenarios/integrations.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants