Adding Test Coverage to Console Tests

[st0012]

Why?

We're looking to shed light on parts of the codebase that could use more tests.

Assumptions

• The coverage report will be viewed locally in HTML format. We haven't planned to sync it with Ruby CI's coverage report yet.
  • Is there a way to sync this even if debug isn't a default gem?
• Our focus will be on console tests, as they cover most of the debugger's core functionalities.

How?

• We're using simplecov as our coverage library because it creates an easy-to-read HTML report.
• The coverage is only enabled when ENV["COVERAGE"] is present.
• When running tests with coverage enabled, the Ruby script for local console tests will execute test/support/simplecov_rdbg.rb instead of exe/rdbg. This script will:
  1. Require and activate simplecov.
  2. Configure the debugger to ignore simplecov related calls.
  3. Kick off the program.

(We need to clear previous results before each run with rm -rf coverage to ensure accuracy for now, but it can easily be turned into a Rake task.)

Issues

We've run into a few issues during the prototyping phase:

• simplecov relies on the at_exit hook to generate results, which means that test cases can't use kill! to terminate processes.
  • This implies that we'll have to update a number of test cases. However, if we can replace kill! with quit!, this may not be a big issue.
• Some tests break when simplecov runs alongside them. We can mitigate this to some extent by setting the skip_path config.
• However, the biggest concern is that some tests take significantly longer to run with simplecov, leading to timeouts. In addition to that, the entire test suite runs slower, which isn't ideal for developers.