Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Josh Black <raskchanky@gmail.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: miagilepner <mia.epner@hashicorp.com>

* Backport initial release notes drafting

* Updates for GA

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: Rowan Smith <86935689+rowansmithhc@users.noreply.github.com>

Co-authored-by: Meggie <meggie@hashicorp.com>

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>

Co-authored-by: Doug Knight <doug.knight@karmix.org>

Co-authored-by: Shubham P <shubham.prajapat@sourcefuse.com>

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>

Co-authored-by: Ryan Cragun <me@ryan.ec>

Co-authored-by: Ryan Cragun <me@ryan.ec>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

Add an updated `target_ec2_instances` module that is capable of
dynamically splitting target instances over subnet/az's that are
compatible with the AMI architecture and the associated instance type
for the architecture. Use the `target_ec2_instances` module where
necessary. Ensure that `raft` storage scenarios don't provision
unnecessary infrastructure with a new `target_ec2_shim` module.

After a lot of trial, the state of Ec2 spot instance capacity, their
associated APIs, and current support for different fleet types in AWS
Terraform provider, have proven to make using spot instances for
scenario targets too unreliable.

The current state of each method:
* `target_ec2_fleet`: unusable due to the fact that the `instant` type
  does not guarantee fulfillment of either `spot` or `on-demand`
  instance request types. The module does support both `on-demand` and
  `spot` request types and is capable of bidding across a maximum of
  four availability zones, which makes it an attractive choice if the
  `instant` type would always fulfill requests. Perhaps a `request` type
  with `wait_for_fulfillment` option like `aws_spot_fleet_request` would
  make it more viable for future consideration.
* `target_ec2_spot_fleet`: more reliable if bidding for target instances
  that have capacity in the chosen zone. Issues in the AWS provider
  prevent us from bidding across multiple zones succesfully. Over the
  last 2-3 months target capacity for the instance types we'd prefer to
  use has dropped dramatically and the price is near-or-at on-demand.
  The volatility for nearly no cost savings means we should put this
  option on the shelf for now.
* `target_ec2_instances`: the most reliable method we've got. It is now
  capable of automatically determing which subnets and availability
  zones to provision targets in and has been updated to be usable for
  both Vault and Consul targets. By default we use the cheapest medium
  instance types that we've found are reliable to test vault.

* Update .gitignore
* enos/modules/create_vpc: create a subnet for every availability zone
* enos/modules/target_ec2_fleet: bid across the maximum of four
  availability zones for targets
* enos/modules/target_ec2_spot_fleet: attempt to make the spot fleet bid
  across more availability zones for targets
* enos/modules/target_ec2_instances: create module to use
  ec2:RunInstances for scenario targets
* enos/modules/target_ec2_shim: create shim module to satisfy the
  target module interface
* enos/scenarios: use target_ec2_shim for backend targets on raft
  storage scenarios
* enos/modules/az_finder: remove unsed module

Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

Co-authored-by: Ryan Cragun <me@ryan.ec>

Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com>

Co-authored-by: Yoko Hyakuna <yoko@hashicorp.com>

Co-authored-by: Christophe Deliens <chris@deliens.be>

Co-authored-by: Rebecca Willett <47540675+rebwill@users.noreply.github.com>

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

…ashicorp#21537)

Co-authored-by: Kianna <30884335+kiannaquach@users.noreply.github.com>

Co-authored-by: Max Bowsher <maxbowsher@gmail.com>

Co-authored-by: miagilepner <mia.epner@hashicorp.com>

Co-authored-by: Peter Wilson <peter.wilson@hashicorp.com>

Co-authored-by: mickael-hc <86245626+mickael-hc@users.noreply.github.com>

Co-authored-by: Jordan Reimer <zofskeez@gmail.com>

Co-authored-by: Jordan Reimer <zofskeez@gmail.com>

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

Co-authored-by: Max Bowsher <maxbowsher@gmail.com>

Improve our build workflow execution time by using custom runners,
improved caching and conditional Web UI builds.

Runners
-------
We improve our build times[0] by using larger custom runners[1] when
building the UI and Vault.

Caching
-------
We improve Vault caching by keeping a cache for each build job. This
strategy has the following properties which should result in faster
build times when `go.sum` hasn't been changed from prior builds, or
when a pull request is retried or updated after a prior successful
build:

* Builds will restore cached Go modules and Go build cache according to
  the Go version, platform, architecture, go tags, and hash of `go.sum`
  that relates to each individual build workflow. This reduces the
  amount of time it will take to download the cache on hits and upload
  the cache on misses.
* Parallel build workflows won't clobber each others build cache. This
  results in much faster compile times after cache hits because the Go
  compiler can reuse the platform, architecture, and tag specific build
  cache that it created on prior runs.
* Older modules and build cache will not be uploaded when creating a new
  cache. This should result in lean cache sizes on an ongoing basis.
* On cache misses we will have to upload our compressed module and build
  cache. This will slightly extend the build time for pull requests that
  modify `go.sum`.

Web UI
------
We no longer build the web UI in every build workflow. Instead we separate
the UI building into its own workflow and cache the resulting assets.
The same UI assets are restored from cache during build worklows. This
strategy has the following properties:

* If the `ui` directory has not changed from prior builds we'll restore
  `http/web_ui` from cache and skip building the UI for no reason.
* We continue to use the built-in `yarn` caching functionality in
  `action/setup-node`. The default mode saves the `yarn` global cache.
  to improve UI build times if the cache has not been modified.

Changes
-------
* Add per platform/archicture Go module and build caching
* Move UI building into a separate job and cache the result
* Restore UI cache during build
* Pin workflows

Notes
-----
[0] https://hashicorp.atlassian.net/browse/QT-578
[1] https://github.com/hashicorp/vault/actions/runs/5415830307/jobs/9844829929

Signed-off-by: Ryan Cragun <me@ryan.ec>

Co-authored-by: Bianca Moreira <48203644+biazmoreira@users.noreply.github.com>

Co-authored-by: Jordan Reimer <zofskeez@gmail.com>

Co-authored-by: Peter Wilson <peter.wilson@hashicorp.com>

Co-authored-by: Max Bowsher <maxbowsher@gmail.com>

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>

Co-authored-by: Chelsea Shaw <82459713+hashishaw@users.noreply.github.com>

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>

…21625) (hashicorp#21672)

Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>

Co-authored-by: Angel Garbarino <Monkeychip@users.noreply.github.com>

Co-authored-by: Jordan Reimer <zofskeez@gmail.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

Co-authored-by: miagilepner <mia.epner@hashicorp.com>

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* combine into one checker

* combine and simplify ci checks

* add to test package list

* remove testing test

* only run deprecations check

* only run deprecations check

* remove unneeded repo check

* fix bash options

Co-authored-by: miagilepner <mia.epner@hashicorp.com>

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>

Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>

Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

Co-authored-by: Max Winslow <43095669+maxiscoding28@users.noreply.github.com>

Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>

Co-authored-by: Rebecca Willett <47540675+rebwill@users.noreply.github.com>

Co-authored-by: miagilepner <mia.epner@hashicorp.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

… (hashicorp#21790)

In order to reliably store Go test times in the Github Actions cache we
need to reduce our cache thrashing by not using more than 10gb over all
of our caches. This change reduces our cache usage significantly by
sharing Go module cache between our Go CI workflows and our build
workflows. We lose our per-builder cache which will result in a bit of
performance hit, but we'll enable better automatic rebalancing of our CI
workflows. Overall we should see a per branch reduction in cache sizes
from ~17gb to ~850mb.

Some preliminary investigation into this new strategy:

Prior build workflow strategy on a cache miss:
  Download modules: ~20s
  Build Vault: ~40s
  Upload cache: ~30s
  Total: ~1m30s

Prior build workflow strategy on a cache hit:
  Download and decompress modules and build cache: ~12s
  Build Vault: ~15s
  Total: ~28s

New build workflow strategy on a cache miss:
  Download modules: ~20
  Build Vault: ~40s
  Upload cache: ~6s
  Total: ~1m6s

New build workflow strategy on a cache hit:
  Download and decompress modules: ~3s
  Build Vault: ~40s
  Total: ~43s

Expected time if we used no Go caching:
  Download modules: ~20
  Build Vault: ~40s
  Total: ~1m

Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>

Co-authored-by: Ryan Cragun <me@ryan.ec>

* use shas instead of versions and fix milestones

* remove trailing space

Co-authored-by: miagilepner <mia.epner@hashicorp.com>

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>

Co-authored-by: Rebecca Willett <47540675+rebwill@users.noreply.github.com>

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

…corp#21736) (hashicorp#21825)

Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>

Co-authored-by: akshya96 <87045294+akshya96@users.noreply.github.com>