testing_vscode_extensions.org

Testing VSCode extensions

whoami

Dan Čermák

Agenda

• Open Build Service Connector
• Testing Extensions
  • Unit testing
  • Manual testing
  • Integration testing
• Q&A

Visual Studio Code

OBS Connector

Roadblocks

1. need to connect to OBS
2. have to store passwords
3. unit tests are brittle

1. use upstream’s development environment
2. use a dirty LD_PRELOAD trick
3. test the workflows of the extension

Testing Extensions

Why is it hard?

• UIs cary a lot of state
• testing workflows is uncommon
• what you get is not what you see

Unit testing

• test one functionality of one unit/function
• anything touching the UI needs extensive setup & teardown
• external services need to be mocked

Getting started

• documentation has an example setup
• code coverage setup more involved

Extension Settings

• can be read & modified in tests

Events

quickPick.onDidChangeValue(async (val: string) => {
  if (verifyInput(val)) {
    await launchBackgroundTask();
  }
});

→ use fake events when possible

Disposables

• “destructors” in VSCode
• use after() or afterEach()

UI Elements

• only check the interesting parts
• preferably keep UI part as small as possible

Manual testing

Integration testing

vscode-extension-tester

const editor = new TextEditor();
const pkgJsonEditor = await new EditorView().openEditor('package.json');

await pkgJsonEditor.setText('{"foo": [1, 2, 3], "bar": "baz"}');
await pkgJsonEditor.formatDocument();

What to test?

• check your main workflow(s)
• don’t test corner cases & minor regressions

How to test?

• upstream using mocha
• use root hooks for setup
• run steps as individual it()

Catches

• integration tests tend to be very slow and resource demanding
• avoid explicit sleeps
• certain elements invisible by default
• no test coverage

Legal

• Visual Studio Codo Logo © Microsoft
• openSUSE Logo CC-BY-SA 3.0
• Selenium Logo CC-BY-SA 4.0
• reveal.js MIT
• Font Awesome CC-BY-4.0 and SIL OFL 1.1 and MIT
• EOS Icons MIT
• everything else is my work under CC-BY-4.0

Questions?

Thank you for your time!