WIP: Opentelemetry

[TheAngryByrd]

I used the Aspire dashboard. You can run it via docker:

docker run --rm -it -p 18888:18888 -p 4317:18889 -d --name aspire-dashboard mcr.microsoft.com/dotnet/nightly/aspire-dashboard:8.0.0-preview.4

Then you can run the Expecto.Tests project. It should export OpenTelemetry traces to Aspire.

Example of traces for tests showing up:

Clicking View on one of them with a red dot (red dot indicates there were exceptions/errors):

Clicking on the top span, you can see details about its run.

Clicking on one of the child spans you can see details such as Exception data.

I didn't show it here but you can also export logs that are associated with a test trace.

[TheAngryByrd]

One option is to have Expecto have a dedicate ActivitySource, the other is to add it the the Expecto config and allow a user to pass it along.

[TheAngryByrd]

As mentioned, we can have an activity source in the config but there's not really a function that lets people update the config.

It might be more tricky to do this option if people are using the dotnet test integration so maybe a dedicated source is appropriate.

[TheAngryByrd]

Lots of helper functions. Don't have to live here, just here for the moment.

[TheAngryByrd]

This is the starting point for creating an span. Later we add additional information to it based on whether we pass/fail/ignore.

[TheAngryByrd]

This is adding the OpenTelemetry Exporter to the test project, mostly for demo purposes. Could create a separate project if desired.

[TheAngryByrd]

Here a consumer would pass in the ActivitySource. Otherwise they'd have to add the dedicated Expecto one to the .AddSource call on line 29.

[farlee2121]

I'm still thinking over generalization approaches. I haven't found a clear nice answer yet.

Have you considered how the current span approach would behave with stress testing?

[TheAngryByrd]

Oh one of the things missing too is how to integrate with dotnet test since we can't call addOpenTelemetry_SpanPerTest, we'd probably need some attribute that would do the call on our behalf when calling into testFromAssembly

[farlee2121]

Good point. That's a problem for shuffle and filter too.
I'm not sure why no discovery methods (like testFromThisAssembly) are officially exposed. They are available, if unofficially, under the Expecto.Impl module.

Seems like exposing such a method would be desirable for library extension.

[farlee2121]

Upon review, I think I got wrap up in my own line of thought and didn't interpret your statement correctly.

I'm not sure I see the incompatibility with dotnet test. Doesn't YoloDev.Expecto.TestSdk respect whatever is configured in main?

[TheAngryByrd]

As far as I know, dotnet test doesn't care whats in your main/entrypoint. Yolodev uses Expecto.Impl.testFromAssembly to find tests.

I suppose you can call addOpenTelemetry_SpanPerTest for every testlist that uses [<Tests>]. There might be some tricks required to get ActivitySource to not be null, I'll have to look later.

Specifically the tricks that @baronfel has in https://github.com/baronfel/otel-startup-hook

[farlee2121]

Interesting. I hadn't thought of such a limitation.

Tests names can be listed via dotnet run, but that doesn't provide fully-hydrated Test records.
I don't see a clear solution to this either, since I'm not sure anonymous functions could be effectively transferred out-of-process.

[TheAngryByrd]

So creating the TraceProvider statically:

do
    let provider = traceProvider()
    AppDomain.CurrentDomain.ProcessExit.Add(fun _ -> provider.Dispose())

And addOpenTelemetry_SpanPerTest has to be added per [<Tests>]. Which isn't too bad but might get annoying for people that have lots of [<Tests>] instead of a few.

After that, this does work for dotnet test