Skip to content

Latest commit

 

History

History
56 lines (45 loc) · 3.8 KB

testpack.md

File metadata and controls

56 lines (45 loc) · 3.8 KB

Test case packer

Tests for Whitespace assemblers are often tiny input files with short Whitespace output or an error message. Since I am testing external assemblers which expect file input, that's a lot of small files. Sometimes I've automated the generation of these inputs with shell scripts, but this leads to duplication.

I want a human-readable structured data format for snapshot testing of many small files, which can encode strings that can have arbitrary bytes. The strings are usually valid UTF-8, but they also need to be able to encode invalid UTF-8 sequences. JSON does not have this property.

Prior art

jq .test files are a set test cases, each with three or more lines: the program, an input, and the expected outputs. Comments are ignored and tests are separated by blank lines.

Rust UI tests are snapshot integration tests in tests/ui/. There is ongoing work to migrate to an out-of-tree test runner shared with Miri and Clippy.

The TypeScript compiler has unit tests, baseline tests, and fourslash tests.

  • Baseline tests are snapshot integration tests with a .ts/.tsx TypeScript source file in tests/cases/ and corresponding files in tests/baselines/reference/: .js with the TypeScript source delimited with //// [filename.ts] and JavaScript output delimited with //// [filename.js], .symbols with symbol information, .types with inline type information, and, if applicable, .errors.txt with type errors.

    Two examples:

  • Fourslash tests are integration tests for the user-facing parts of the API, such as for IDE clients. Lines starting with //// represent the text of an input file and lines starting with // are TSConfig settings or filename declarations. The syntax is described in fourslash.ts.

    An example: addAllMissingImportsNoCrash.ts

insta is a snapshot testing library for Rust. As an example, it is used by if-to-let-chain.