Fix node warning calling rmdirSync() when running Playwright setup (F…

…ixes mozilla#15058) (mozilla#15061)

* Fix node warning calling rmdirSync() when running Playwright setup (Fixes mozilla#15058)

* Add linnks to Axe rules in testing docs

* Add link to a11y test job and fix heading levels in docs

docs/cms.rst

@@ -318,7 +318,7 @@ When you add a new page to the CMS, it will be available to add as a new child p
 So if you ship a page that needs to be used immediately in Production (which will generally be most cases), you must remember to add it to ``CMS_ALLOWED_PAGE_MODELS`` in Bedrock's settings. If you do not, it will not be selectable as a new Child Page in the CMS.
 
 Why do we have this behaviour?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+------------------------------
 
 Two reasons:
 
@@ -472,12 +472,12 @@ Localization process
 ====================
 
 Manual updates
-~~~~~~~~~~~~~~
+--------------
 
 At its most basic, there's nothing stopping us using copy-and-paste to enter translations into lang-specific pages, which might work well if we have a page in just one non-en-US lang and an in-house colleague doing the translation.
 
 Automated via Smartling
-~~~~~~~~~~~~~~~~~~~~~~~
+-----------------------
 
 However, we also have automation available to send source strings to translation vendor Smartling. This uses the ``wagtail-localize-smartling`` package.
 
@@ -498,7 +498,7 @@ Here's the workflow:
 
 
 Automated via Pontoon
-~~~~~~~~~~~~~~~~~~~~~
+---------------------
 
 It should also be possible to use `Pontoon`_ with `wagtail-localize`. (There are notes on the `Pontoon integration`_ here, but we have not yet tried to enable this alongside `wagtail-localize-smartling`).
 

docs/testing.rst

@@ -265,7 +265,7 @@ load a different override instead of using ``openPage``:
 Accessibility testing (Axe)
 ===========================
 
-Axe tests are run as a subset of Playwright tests using the ``@a11y`` tag. These
+`Axe`_ tests are run as a subset of Playwright tests using the ``@a11y`` tag. These
 tests are run against the dev environment once per day via a GitHub action. The
 axe spec files can be found in the ``./tests/playwright/specs/a11y/`` directory.
 
@@ -283,9 +283,12 @@ desktop and mobile viewport sizes.
 
 Test results are output to the console, and any issues found will be created as
 HTML report files in the ``./tests/playwright/test-results-a11y/`` directory. When
-run via the GitHub action, the reports are also output to the annotation logs for
+run via the `GitHub action`_, the reports are also output to the annotation logs for
 each test job.
 
+A list of all the Axe rules that are checked by the tests can be viewed in the
+`axe-core repo`_.
+
 Running Selenium tests
 ======================
 
@@ -377,7 +380,7 @@ dev:
 For more information on command line options, see the `pytest documentation`_.
 
 Running tests in Sauce Labs
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+---------------------------
 
 You can also run tests in Sauce Labs directly from the command line. This can be useful
 if you want to run tests against Internet Explorer when you're on Mac OSX, for instance.
@@ -540,3 +543,5 @@ via product details are well formed and return valid 200 responses.
 .. _how to write tests: https://playwright.dev/docs/writing-tests
 .. _locator strategy: https://playwright.dev/docs/locators#locate-by-test-id
 .. _Axe: https://github.com/dequelabs/axe-core-npm/blob/develop/packages/playwright/README.md
+.. _axe-core repo: https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md
+.. _GitHub action: https://github.com/mozilla/bedrock/actions/workflows/a11y_tests.yml

tests/playwright/global-setup.js

@@ -6,26 +6,29 @@
 
 'use strict';
 
-const fs = require('fs');
+const fs = require('fs').promises;
 
 /**
  * Clean up /test-results-a11y/ directory before running tests.
  */
-function cleanA11yReportDir() {
+async function cleanA11yReportDir() {
     const dir = 'test-results-a11y/';
-    if (fs.existsSync(dir)) {
-        fs.rmdirSync(dir, {
-            recursive: true
-        });
+    try {
+        const stats = await fs.stat(dir);
+        if (stats.isDirectory()) {
+            await fs.rm(dir, { recursive: true, force: true });
+        }
+    } catch (err) {
+        if (err.code !== 'ENOENT') {
+            throw err;
+        }
     }
 
-    if (!fs.existsSync(dir)) {
-        fs.mkdirSync(dir);
-    }
+    await fs.mkdir(dir);
 }
 
-function globalSetup() {
-    cleanA11yReportDir();
+async function globalSetup() {
+    await cleanA11yReportDir();
 }
 
 module.exports = globalSetup;