Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include test code in results.json #8

Open
ErikSchierboom opened this issue Apr 14, 2021 · 0 comments
Open

Include test code in results.json #8

ErikSchierboom opened this issue Apr 14, 2021 · 0 comments
Labels
help wanted Extra attention is needed

Comments

@ErikSchierboom
Copy link
Member

We've recently updated the test runner interface specification to allow for two types of test runners, identified via a new version property in the results.json file (see the spec for the version property):

  1. version: 1 test runners are quite basic, and only detect if either all tests passed or not. These test runners capture the console output of the test runner and output that. They do not include details on individual tests passing/failing
  2. version: 2 test runners contain detailed information on individual tests passing/failing. We've recently added a required field named test_code, which contains the code the test ran to verify the behavior (see the spec for the test_code property).

The test runner's output is currently what I refer to as version 1.5: it contains the individual test information, but does not yet contain the test_code key. If possible, the test runner should be updated to also include the test code.

There is an existing TODO in the code where the test code is written: https://github.com/exercism/scala-test-runner/blob/main/src/main/scala/Application.scala#L63 You are completely free on how you want to extract the test code, either via the AST or via finding the test code in the source code text via string manipulation. Whatever is most convenient.

Let me know if there are any questions.

@ErikSchierboom ErikSchierboom added the help wanted Extra attention is needed label Apr 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant