Skip to content

Commit

Permalink
bug-1909870: Rename old system tests to smoke tests.
Browse files Browse the repository at this point in the history
  • Loading branch information
smarnach committed Jul 29, 2024
1 parent 19466b0 commit 6e3dc91
Show file tree
Hide file tree
Showing 18 changed files with 43 additions and 44 deletions.
2 changes: 1 addition & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ frontend/node_modules/
# Ephemeral data for local development. These will get volume mounted.
archive/
myanalysis/
systemtests/data/
smoketests/data/
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,5 @@ version.json
yarn-debug.log*
yarn-error.log*

# Ephemeral systemtest data
systemtests/data/
# Ephemeral smoke test data
smoketests/data/
2 changes: 1 addition & 1 deletion bin/run_lint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

set -e

FILES="bin tecken systemtests"
FILES="bin tecken smoketests"
PYTHON_VERSION=$(python --version)


Expand Down
10 changes: 5 additions & 5 deletions docs/dev.rst
Original file line number Diff line number Diff line change
Expand Up @@ -634,19 +634,19 @@ this code and it has to be tested manually. You can do something like this:
11. click on "Sign out"


System tests
------------
Smoke tests
-----------

System tests are located in the repository in ``systemtests/``. See the
Smoke tests are located in the repository in ``smoketests/``. See the
``README.rst`` there for usage.

System tests can be run against any running environment:
Smoke tests can be run against any running environment:

* local: local dev environment
* stage: the stage server environment
* prod: the prod server environment--will not run destructive tests

System tests can help verify that upload API and download API work. They
Smoke tests can help verify that upload API and download API work. They
periodically need to be updated as symbols files expire out of the systems.


Expand Down
3 changes: 1 addition & 2 deletions docs/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,11 @@ Here's a bunch of top-level directories and what's in them::
docs/ -- documentation
favicons/ -- favicons used by Symbols webapp
frontend/ -- Symbols webapp JavaScript frontend
schemas/ -- API schemas
tecken/ -- Symbols service unit tests and code

These directories have test stuff in them::

systemtests/ -- systemtests for Symbols Server
smoketests/ -- smoke tests for Symbols Server


.. Note::
Expand Down
28 changes: 14 additions & 14 deletions systemtests/README.rst → smoketests/README.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
=================
Systemtest README
=================
==================
Smoke tests README
==================

These test Tecken stage and prod environments.

Expand All @@ -14,30 +14,30 @@ Contents of this directory::
Setting up tests
================

Before running the systemtests, you need to `build a local dev environment
Before running the smoke tests, you need to `build a local dev environment
<https://tecken.readthedocs.io/en/latest/dev.html>`__.

Then set up the tests this way::

$ make shell
root@e62fb7ae586f:/app# cd systemtests
root@e62fb7ae586f:/app/systemtests# ./setup_tests.py
root@e62fb7ae586f:/app# cd smoketests
root@e62fb7ae586f:/app/smoketests# ./setup_tests.py

That creates files in directories under ``data/``.

You only need to set up the tests once to run the system tests against all
You only need to set up the tests once to run the smoke tests against all
environments.


Running tests
=============

The system tests are run using the ``test_env.py`` Python script. You can get
The smoke tests are run using the ``test_env.py`` Python script. You can get
help about the command-line invocation of that script using::

$ make shell
root@e62fb7ae586f:/app# cd systemtests
root@e62fb7ae586f:/app/systemtests# ./test_env.py --help
root@e62fb7ae586f:/app# cd smoketests
root@e62fb7ae586f:/app/smoketests# ./test_env.py --help

The help includes a list of available environments.

Expand All @@ -49,13 +49,13 @@ variable names in the help output, e.g. ``STAGE_AUTH_TOKEN`` and
For destructive tests run in local and stage, you need separate auth tokens for
try uploads with "Upload Try Symbols Files" permissions. See Bug 1759740.

To run the systemtests, do::
To run the smoke tests, do::

root@e62fb7ae586f:/app/systemtests# ./test_env.py <ENV_NAME>
root@e62fb7ae586f:/app/smoketests# ./test_env.py <ENV_NAME>


Rules of systemtest
===================
Rules of smoke tests
====================

1. Don't run destructive tests against the prod server environment.

Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ def download_sym_files(base_url, test_headers, csv_file):
click.echo(click.style(f"Working on {url} ...", fg="yellow"))

headers = {
"User-Agent": "tecken-systemtests",
"User-Agent": "tecken-smoketests",
# We know storage backends will honor "Accept-Encoding": "gzip",
# so we test the unusual case only to ensure the response is
# still gzipped.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
QUEUE = "https://firefox-ci-tc.services.mozilla.com/api/queue/v1/"
NAMESPACE = "gecko.v2.mozilla-central.revision.REV.firefox"

HTTP_HEADERS = {"User-Agent": "tecken-systemtests"}
HTTP_HEADERS = {"User-Agent": "tecken-smoketests"}


def index_namespaces(namespace, limit=1000):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import markus
from markus.backends import BackendBase

from systemtestslib.utils import build_zip_file, download_sym_file, get_sym_files
from smoketestslib.utils import build_zip_file, download_sym_file, get_sym_files


# Number of seconds to wait for a response from server
Expand Down Expand Up @@ -219,7 +219,7 @@ def setup_download_tests(start_page, auth_token, csv_output_path, zip_output_dir
"""
Generates a list of sym files, writes them to a CSV, and downloads
them to two separate zip folders: one for try symbols files, and one for
regular symbols files. This is used for the download system tests.
regular symbols files. This is used for the download smoke tests.
Runtime is dependent on what files have been uploaded recently.
Expand Down Expand Up @@ -258,7 +258,7 @@ def setup_download_tests(start_page, auth_token, csv_output_path, zip_output_dir
# Figure out the ZIP file names and final path
# Try files go into a separate zip from regular files, so they
# can be uploaded to the correct bucket later as part of the
# upload system tests.
# upload smoke tests.
zip_filename_try = datetime.datetime.now().strftime(
"symbols_%Y%m%d_%H%M%S__try.zip"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import markus
from markus.backends import BackendBase

from systemtestslib.utils import build_zip_file, download_sym_file, get_sym_files
from smoketestslib.utils import build_zip_file, download_sym_file, get_sym_files


# Number of seconds to wait for a response from server
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ def upload_symbols(ctx, expect_code, auth_token, base_url, symbolsfile):
)

api_url = urljoin(base_url, "/upload/")
headers = {"auth-token": auth_token, "User-Agent": "tecken-systemtests"}
headers = {"auth-token": auth_token, "User-Agent": "tecken-smoketests"}
basename = os.path.basename(symbolsfile)

# This is an upload and it's success is partially dependent on the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ def upload_symbols_by_download(ctx, base_url, auth_token, url, expect_code):
"""Upload SYM file to a host using a download url."""

api_url = urljoin(base_url, "/upload/")
headers = {"auth-token": auth_token, "User-Agent": "tecken-systemtests"}
headers = {"auth-token": auth_token, "User-Agent": "tecken-smoketests"}

# It's possible this can fail, so we put it in a retry loop.
for i in range(MAX_ATTEMPTS):
Expand Down
6 changes: 3 additions & 3 deletions systemtests/setup_tests.py → smoketests/setup_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@

import click

from systemtests.bin.setup_download_tests import setup_download_tests
from systemtests.bin.setup_upload_tests import setup_upload_tests
from smoketests.bin.setup_download_tests import setup_download_tests
from smoketests.bin.setup_upload_tests import setup_upload_tests

# Usage: ./setup-tests.py
# This setup is only for non-prod envs (local and stage), as it uploads
Expand All @@ -26,7 +26,7 @@ def setup_tests(ctx):
# Create the zip output directory if it doesn't exist
os.makedirs(ZIPS_DIR)

click.echo("Generating systemtest data files ...")
click.echo("Generating smoketest data files ...")
zips_count = len(
[name for name in os.listdir(ZIPS_DIR) if os.path.isfile(f"{ZIPS_DIR}/{name}")]
)
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at https://mozilla.org/MPL/2.0/.

# Utilities used to setup upload and download system tests.
# Utilities used to setup upload and download smoke tests.

import os
from urllib.parse import urljoin
Expand All @@ -25,7 +25,7 @@ def get_sym_files(baseurl, auth_token, params):
"""
url = urljoin(baseurl.rstrip("/"), "/api/uploads/files")
headers = {"auth-token": auth_token, "User-Agent": "tecken-systemtests"}
headers = {"auth-token": auth_token, "User-Agent": "tecken-smoketests"}
params = params or {}

sym_files = []
Expand Down Expand Up @@ -89,7 +89,7 @@ def build_zip_file(zip_filename, sym_dir):

def download_sym_file(url, sym_file_path):
"""Download SYM file at url into sym_file_path."""
headers = {"User-Agent": "tecken-systemtests"}
headers = {"User-Agent": "tecken-smoketests"}
resp = requests.get(url, headers=headers, timeout=CONNECTION_TIMEOUT)
resp.raise_for_status()

Expand Down
10 changes: 5 additions & 5 deletions systemtests/test_env.py → smoketests/test_env.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@

import click

from systemtests.bin.download_sym_files import download_sym_files
from systemtests.bin.list_firefox_symbols_zips import list_firefox_symbols_zips
from systemtests.bin.upload_symbols import upload_symbols
from systemtests.bin.upload_symbols_by_download import upload_symbols_by_download
from smoketests.bin.download_sym_files import download_sym_files
from smoketests.bin.list_firefox_symbols_zips import list_firefox_symbols_zips
from smoketests.bin.upload_symbols import upload_symbols
from smoketests.bin.upload_symbols_by_download import upload_symbols_by_download

ZIP_FILES = list(Path("./data/zip-files").glob("*.zip"))

Expand Down Expand Up @@ -109,7 +109,7 @@ def auth_token(self, try_storage: bool) -> str:
\b
Usage:
1. run "make shell" to get a shell in the container
2. then do "cd systemtests"
2. then do "cd smoketests"
3. run "./setup_tests.py" if you are running the tests for the first time.
4. run "./test_env.py [{"|".join(ENVIRONMENTS)}]"
Expand Down

0 comments on commit 6e3dc91

Please sign in to comment.