Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PingOne testing strategy #41

Closed
patrickcping opened this issue Feb 7, 2023 · 1 comment · Fixed by #250
Closed

PingOne testing strategy #41

patrickcping opened this issue Feb 7, 2023 · 1 comment · Fixed by #250
Assignees
Milestone

Comments

@patrickcping
Copy link
Contributor

patrickcping commented Feb 7, 2023

Multi-region acceptance testing
The PingOne provider runs a nightly acceptance test, and also runs acceptance tests on merge into the main branch. The acceptance tests run every single test case in every region (EU, NA, APAC, CA) to ensure that region-specific nuances are accounted for

Test cases
The following test cases are expected to be defined:

  1. New environment test case - Where multiple instances of a resource can be created in an environment (1 to many relationship), this test case is designed to test creation of the first object in a new environment. The purpose of this test is to identify potential race conditions that need additional retry logic. Example
  2. Existing environment test cases - Where multiple instances of a resource can be created in an environment (1 to many relationship), this test case is designed to test the bulk of the attribute/use case tests, to avoid many simultaneous environment creations that slow the acceptance test routine. Where a single instance of a resource is the limit, then all tests will use a new environment strategy as above. Example
  3. Use case testing with "full" and "minimal" data models - This is a test that creates resources (and validates) the "full" data model (defining custom Required, Optional and validating Computed data in the schema; to validate all schema items have been mapped to the API successfully), and the "minimal" data model (defining custom Required, leaving default Optional and validating Computed attributes; to validate the TF schema has the correct defaults). The test should have steps to create both from fresh, and test modifications from one to the other. Example
  4. Invalid data / custom errors - To test whether custom errors are successfully overridden in response to bad data input. Example
  5. Bootstrapping specific - To test whether creates/updates behave correctly with new resources alongside bootstrapped resources. Example

Check for destroy confirmation
The PingOne provider will be enforcing the CheckDestroy implementation, to ensure that the delete routine works successfully. Example

@patrickcping
Copy link
Contributor Author

#85 introduces regular multi-region acceptance test CI. That PR resolves part of this issue, but not all.

@patrickcping patrickcping self-assigned this Dec 29, 2023
@patrickcping patrickcping linked a pull request Jan 8, 2024 that will close this issue
@patrickcping patrickcping added this to the v0.3.0 milestone Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant