Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve NUnit result plotting #421

Open
alelom opened this issue Feb 22, 2023 · 0 comments
Open

Improve NUnit result plotting #421

alelom opened this issue Feb 22, 2023 · 0 comments
Assignees
Labels
type:feature New capability or enhancement

Comments

@alelom
Copy link
Member

alelom commented Feb 22, 2023

Description:

Currently, the NUnit results when there is a failure are plotted like this (example taken from here):

image

The test failing here is only one (BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest), but there are other 2 tests that were run. Additionally, those two test may have some RecordMessage log that could be useful to show (see #420).

After discussing with @FraserGreenroyd , we agreed on this example structure for the results plotting:

Inner results:
    BH.Tests.Engine.Base.Query.UnpackTests.ErrorReportingTest (Passed):
        Messages: 
            Some error logged via BH.Engine.Base.Compute.RecordError
            Some error logged via BH.Engine.Base.Compute.RecordWarning
            Some error logged via BH.Engine.Base.Compute.RecordNote

    BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest (Failed):
        Shouldly.ShouldAssertException : Comparing object equivalence, at path:
        result.OfType<BHoMObject>().FirstOrDefault() [BH.oM.Base.BHoMObject]
            BHoM_Guid [System.Guid]
            Expected value to be
        a1380fbb-b004-4bac-b28c-0d45780a5e9c
            but was
        4fa87007-d24a-d44b-9d3e-aa9d743b7059
        at BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest() in C:\ProgramData\BHoMGitHub\BHoM_Engine\.ci\unit-tests\Base_Engine_Tests\Query\Unpack.cs:line 143

        Error:
            Some warning logged via BH.Engine.Base.Compute.RecordError

        Warnings:
            Some warning logged via BH.Engine.Base.Compute.RecordWarning
    
    BH.Tests.Engine.Base.Query.UnpackTests.WarningReportingTest (Passed):
        Warnings:
            Some warning logged via BH.Engine.Base.Compute.RecordWarning
    
    BH.Tests.Engine.Base.Query.UnpackTests.NoteReportingTest (Passed):
        Messages:
            Some note logged via BH.Engine.Base.Compute.RecordNote

The information from the RecordMessage log is shown differently depending on whether the test has passed or failed:

  • when the test has passed, the recorded messages are clustered under a "Messages" category (no differentiation btw Errors/Warnings/Note). This was suggested by @FraserGreenroyd to avoid confusing readers.
  • when the test has failed, the recorded messages are grouped as normal (Errors/Warnings/Notes).

For each test, the recorded messages should appear after any message from NUnit.

@alelom alelom added the type:feature New capability or enhancement label Feb 22, 2023
@alelom alelom self-assigned this Feb 22, 2023
alelom added a commit that referenced this issue Feb 24, 2023
FraserGreenroyd pushed a commit that referenced this issue Mar 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:feature New capability or enhancement
Projects
None yet
Development

No branches or pull requests

1 participant