Add DSL for compilation failing tests

[pyrsmk]

When developing my Origin shard (https://github.com/pyrsmk/origin), I needed to test that the compilation is failing properly when using the shard wrongly. This is especially useful when coding a macro.

Here's what I came with in a helper:

class String
  def trail(str : String) : String
    return self + str if !(self =~ Regex.new("#{Regex.escape(str)}$"))
    self
  end
end

class SuccessfulCompileError < Exception; end

module CompileHelper
  def compile_fails(path : String) : String
    buffer = IO::Memory.new
    result = Process.run(
      "crystal",
      ["run", "--no-color", "--no-codegen", "spec/" + path.trail(".cr")],
      error: buffer,
    )
    raise SuccessfulCompileError.new if result.success?
    output = buffer.to_s
    buffer.close
    output
  end
end

Here, we're defining a wire/wrong_typing.cr with a wrong implementation of the macro. We can then test it in an example:

it "does not compile when mismatching type definition" do
  expect(compile_fails("wire/wrong_typing")).to match(
    /method must return Int32 but it is returning Float64/i
  )
end

What are your thoughts?

[icy-arctic-fox]

I've done something similar within Spectator itself to test syntax. I can see how this would be useful in other scenarios.

https://gitlab.com/arctic-fox/spectator/-/blob/specs/spec/runtime_example_spec.cr
https://gitlab.com/arctic-fox/spectator/-/blob/specs/spec/helpers/example.cr

Maybe a new type of matcher that compiles crystal code? Perhaps something like:

let(:code) { "puts \"some crystal code here\"" }
let(:malformed_code) { "this won't work" }

specify { expect(code).to compile }
specify { expect(code).to_not compile.with_error(/syntax/) }

[pyrsmk]

Nice matcher! Wouldn't be easier to have a way to specify a file instead of a string? Because it's simpler to write clean code in a file instead of a string, and most of the time it seems the code to compile would have many lines.

edit: I mean an easier way instead of having to do a File.read each time; but another solution could be to have this documented in the wiki

[icy-arctic-fox]

The code could be cleaned up with some multi-line strings and macros. Something like:

let(:code) do
  <<-EOS
    puts "do something"
  EOS
end

macro source_code(name, &block)
  let({{name.id}}) do
      {{block.body.stringify}}
  end
end

source_code(:sample) do
  puts "another example"
end

The matcher could also accept a File (or even IO) type.

let(:code_file) { File.new("foo.cr") }
specify { expect(code_file).to compile }

or even a utility:

macro expect_file(file)
  expect(File.new({{file}})).to compile
end

specify { expect_file("foo.cr").to compile }

Templates and string substitution can be done with the first approach, but for passing a file, something like ECR could be used.

[pyrsmk]

All your examples are really interesting. I didn't think about defining a macro to handle the source code in place and having the syntax highlighting at the same time., this is nice.

But maybe the simpler (to implement, and for the user) would be to just have an expect_file utility.