More robust tools for E2E integration tests

[MaxwellGarceau]

Is your enhancement related to a problem? Please describe.

We are currently blocked from finishing E2E automated testing because of the inability to control environment conditions inside the test user's Mailchimp account. In particular, we are blocked on validation testing and form submission testing.

A detailed list of blockers on a test by test and expectation by expectation basis can be seen in ❗this checklist❗ that I duplicated from the original Testing instructions for critical flows Mailchimp.

Some of the functionality we lack inside the Mailchimp account is the ability to:

• Delete contacts
• Verify contacts
• Adjust merge field visibility
• Adjust merge field required params
• Create and delete account (for account creation tests)

Designs

• ✅ (already done) Add .env.test file for easier local testing
• Update documentation to use credentials locally for the same test account that runs in CI/CD - Set required Mailchimp conditions in global before function to ensure the Mailchimp account is set up to properly test scenarios
• ✅ (already done) Use the Mailchimp API to fetch and modify contacts and merge fields as necessary to set up tests
  • Alternatively, we could use Cypress to log into the test user's account, but I believe this is harder than it needs to be
• (On hold, open to ideas) Use an email server where we can retrieve subscriber verifications that Mailchimp sends. Current best option seems to be Mailtrap. Even with single opt in, Mailchimp will send verification emails if it detects the possibility of fake addresses or spam subscriptions.
• Define the order the tests must be run in and clarify that the tests can not be run individually
  • OR ensure that the before/after methods in each testing file are sufficient to run each file individually

Describe alternatives you've considered

• Document E2E tests that can't be automated and use QA to test those areas
  • Cons: Not automated testing
  • Pro: Easy, low lift, and already well documented testing instructions
• Create a separate test suite for BE functionality that can verify successful integration between the Mailchimp WP plugin BE and the user's Mailchimp account
  • Cons: Not a true E2E test because a change to FE form submission could be made without also failing the BE tests causing a drift in integration between the FE and BE over time
• Use Cypress to log into a Mailchimp user's account to set up testing conditions as needed
  • Con: Brittle and will break over time based on unpredictable external environmental conditions (cookie pop up, change in Mailchimp UI, etc).
  • Con: Mailchimp UI is not built for jQuery selectors
  • Con: 3rd party app security issues to circumvent

Code of Conduct

• I agree to follow this project's Code of Conduct

[MaxwellGarceau]

@jeffpaul and @dkotter, I created a new enhancement issue to address some of the blockers I came across in #61. This issue would also partially address #34 for testing purposes.

Let me know what you think. I'm going to finish up the E2E tests that can be made with the current limitations and move onto other backlog items in order to give myself more space to think up better solutions for the testing blockers I've documented here.