Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

fix: 320 Subnet validation in Network Configuration fails when multiple subnets exist in the VPC #321

Conversation

chotalia
Copy link
Contributor

@chotalia chotalia commented Oct 6, 2023

Closes #320
Subnet validation in Network Configuration fails when multiple subnets exist in the VPC.

  • Resolved an issue where defining subnets in network configuration would erroneously report them as missing in the VPC.
  • Adjusted the boto3 describe_subnets call to specifically compare against SubnetId, as it retrieves full subnet details.

Checklist

  • References any related issue by including "Closes #" or "Closes ".
    • If no issue exists and your change is not a small fix, please create an issue first.
  • Includes tests or only affects documentation.
  • Passes pre-commit checks.
    • Run pre-commit install && pre-commit run --all locally for formatting and linting.
  • Includes screenshots of documentation updates.
    • Run mkdocs serve view documentation locally.
  • Summarizes PR's changes in CHANGELOG.md

@desertaxle
Copy link
Member

Thanks for the PR @chotalia! Based on a review of the code and the tests passing without any changes, this change appears to be functionally equivalent to the previous implementation. Have you tested this change with your setup to confirm it fixes the error you see?

…le subnets exist in the VPC.

fixes PrefectHQ#320

Subnet validation in Network Configuration fails when multiple subnets exist in the VPC.

- Resolved an issue where defining subnets in network configuration would erroneously report them as missing in the VPC.
- Adjusted the boto3 describe_subnets call to specifically compare against SubnetId, as it retrieves full subnet details.
@chotalia chotalia force-pushed the 360-network-configuration-subnet-verification branch from 9aab406 to b31d9ed Compare October 8, 2023 10:02
@chotalia
Copy link
Contributor Author

chotalia commented Oct 8, 2023

Thanks for the PR @chotalia! Based on a review of the code and the tests passing without any changes, this change appears to be functionally equivalent to the previous implementation. Have you tested this change with your setup to confirm it fixes the error you see?

Thanks @desertaxle for reviewing the PR. Please find more details.

The code was specifically failing when more than 1 subnets exist in VPC in AWS. Existing tests were testing only when VPC has single subnet hence they were passing. I have added 3 new tests. I can confirm these 3 new tests failed without the change of the code.

There was two main issues in this line:

[conf_sn in sn.values() for conf_sn in config_subnets for sn in subnets]

List of Boolean Values:
This comprehension generates a list of boolean values. The list can have duplicate True or False values for a subnet depending on how many subnets are in the VPC. This is especially problematic if there's more than one subnet in the VPC.

For example, let's say config_subnets contains a single valid subnet ID and the VPC has 3 subnets. The resulting boolean list might look like [True, False, False] if only the first subnet in subnets matches. The use of all() on this list will then return False, even though the subnet ID from config_subnets was valid for one of the subnets in the VPC.

False Positives: Boto3 describe_subnets retrieves not only subnet ids but all the subnet specific attributes. The conf_sn in sn.values() checks if the subnet ID from config_subnets exists anywhere in the values of the subnet dictionary. This is not precise, as we only want to match against the actual subnet IDs, not other values like availability zones, states, etc.

@chotalia chotalia changed the title fix: 320 subnet validation in Network Configuration fix: 320 Subnet validation in Network Configuration fails when multiple subnets exist in the VPC Oct 8, 2023
@chotalia
Copy link
Contributor Author

@desertaxle, @urimandujano this has been a show stopper for us to migrate to Prefect 2

Appreciate if you could look sooner if possible?

Thank you in advance.

Copy link
Member

@desertaxle desertaxle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@desertaxle desertaxle merged commit 8650dbf into PrefectHQ:main Oct 10, 2023
6 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
2 participants