Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds failing Subset assertion #92944

Open
ericstj opened this issue Oct 3, 2023 · 12 comments
Labels
arch-x64 area-System.Diagnostics.Process Known Build Error Use this to report build issues in the .NET Helix tab os-mac-os-x macOS aka OSX runtime-mono specific to the Mono runtime
Milestone

Comments

@ericstj
Copy link
Member

ericstj commented Oct 3, 2023

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=425348
Build error leg or test failing: System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds
Pull request: #92730

Error Message

Similar issue reported in past as #28922

Child exception:
  Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 571
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs:line 22
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 178

Child process:
  System.Diagnostics.Process.Tests, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)

Child arguments:
  504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds.String userId",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=426977
Error message validated: at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds.String userId
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 10/4/2023 7:30:00 PM UTC

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0
@ericstj ericstj added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab labels Oct 3, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Oct 3, 2023
@ghost
Copy link

ghost commented Oct 3, 2023

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process
See info in area-owners.md if you want to be subscribed.

Issue Details

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=425348
Build error leg or test failing: System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds
Pull request: #92730

Error Message

Similar issue reported in past as #28922

Child exception:
  Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 571
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.Mono.cs:line 22
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 178

Child process:
  System.Diagnostics.Process.Tests, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)

Child arguments:
  504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}
Author: ericstj
Assignees: -
Labels:

area-System.Diagnostics.Process, blocking-clean-ci, untriaged, Known Build Error

Milestone: -

@carlossanlop
Copy link
Member

Hit this in 8.0. Not sure why KnownBuildError isn't linking the PRs:

Expand
===========================================================================================================
/private/tmp/helix/working/B1790977/w/A4B20994/e /private/tmp/helix/working/B1790977/w/A4B20994/e
  Discovering: System.Diagnostics.Process.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Diagnostics.Process.Tests (found 250 of 322 test cases)
  Starting:    System.Diagnostics.Process.Tests (parallel test collections = on, max threads = 12)
    System.Diagnostics.Tests.ProcessStartInfoTests.ShellExecute_Nano_Fails_Start [SKIP]
      Condition(s) not met: "IsWindowsNanoServer"
Darwin
    System.Diagnostics.Tests.ProcessTests.TestBasePriorityOnUnix [SKIP]
      Condition(s) not met: "IsPrivilegedProcess"
Unhandled exception. Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 703, ...]
   at Xunit.Assert.Subset[T](ISet`1 expectedSuperset, ISet`1 actual) in /_/src/xunit.assert/Asserts/SetAsserts.cs:line 71
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 571
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs:line 36
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 178
--- End of stack trace from previous location ---
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args) in /_/src/Microsoft.DotNet.RemoteExecutor/src/Program.cs:line 97
    System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds [FAIL]
      Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Remote process failed with an unhandled exception.
      Stack Trace:
        
        Child exception:
          Xunit.Sdk.SubsetException: Assert.Subset() Failure
        Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
        Actual:   HashSet<UInt32> [80, 12, 20, 61, 703, ...]
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(571,0): at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs(36,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(178,0): at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
        
        Child process:
          System.Diagnostics.Process.Tests, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)
        
        Child arguments:
          504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False
        
    System.Diagnostics.Tests.ProcessTests.TestPriorityClassUnix [SKIP]
      Condition(s) not met: "IsPrivilegedProcess"
    System.Diagnostics.Tests.ProcessTests.TestProcessRecycledPid [SKIP]
      Condition(s) not met: "IsStressModeEnabledAndRemoteExecutorSupported"
    System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIdsElevated [SKIP]
      Condition(s) not met: "IsRemoteExecutorSupportedAndPrivilegedProcess"
  Finished:    System.Diagnostics.Process.Tests
=== TEST EXECUTION SUMMARY ===
   System.Diagnostics.Process.Tests  Total: 306, Errors: 0, Failed: 1, Skipped: 5, Time: 24.023s
/private/tmp/helix/working/B1790977/w/A4B20994/e
----- end Mon Oct 2 19:19:37 EDT 2023 ----- exit code 1 ----------------------------------------------------------

@buyaa-n
Copy link
Contributor

buyaa-n commented Oct 3, 2023

Same failure found in #92846, Build Analysis might link this issue automatically if we remove the ProcessTests.Unix.cs(571,0): part from the "ErrorMessage" as that part is not visible in the Build Analysis tab:

UPDATE: looks it was originally: "ErrorMessage": "at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)", and apparently the automation still did not work

@carlossanlop carlossanlop added Known Build Error Use this to report build issues in the .NET Helix tab and removed Known Build Error Use this to report build issues in the .NET Helix tab labels Oct 4, 2023
@carlossanlop
Copy link
Member

Hit in 7.0:

Unhandled Exception:
Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at Xunit.Assert.Subset[UInt32](ISet`1 expectedSuperset, ISet`1 actual) in /_/src/xunit.assert/Asserts/SetAsserts.cs:line 72
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 568
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args) in /_/src/Microsoft.DotNet.RemoteExecutor/src/Program.cs:line 97
[ERROR] FATAL UNHANDLED EXCEPTION: Xunit.Sdk.SubsetException: Assert.Subset() Failure
Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
   at Xunit.Assert.Subset[UInt32](ISet`1 expectedSuperset, ISet`1 actual) in /_/src/xunit.assert/Asserts/SetAsserts.cs:line 72
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 568
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args) in /_/src/Microsoft.DotNet.RemoteExecutor/src/Program.cs:line 97
    System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds [FAIL]
      Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Remote process failed with an unhandled exception.
      Stack Trace:
        
        Child exception:
          Xunit.Sdk.SubsetException: Assert.Subset() Failure
        Expected: HashSet<UInt32> [12, 33, 61, 80, 98, ...]
        Actual:   HashSet<UInt32> [80, 12, 20, 61, 701, ...]
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(568,0): at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact)
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodInvoker.Mono.cs(33,0): at System.Reflection.MethodInvoker.InterpretedInvoke(Object obj, Span`1 args, BindingFlags invokeAttr)
        
        Child process:
          System.Diagnostics.Process.Tests, Version=7.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)
        
        Child arguments:
          504, 80, 12,33,61,80,98,100,204,250,395,398,399,400,701,702,703, False

@radical
Copy link
Member

radical commented Oct 4, 2023

Updated the error pattern to correctly match the failure.

@carlossanlop
Copy link
Member

Updated the error pattern to correctly match the failure.

It seems you missed a (.

@radical
Copy link
Member

radical commented Oct 4, 2023

Updated the error pattern to correctly match the failure.

It seems you missed a (.

That was deliberate, because it is a regex pattern. So, used . instead to match any character.

@ericstj
Copy link
Member Author

ericstj commented Oct 4, 2023

Updated the error pattern to correctly match the failure.

previously this did match, which is why the tool wrote this
image

Still with your changes we aren't getting the issue updated.

@radical
Copy link
Member

radical commented Oct 4, 2023

It was showing it as unmatched, which prompted me to change the pattern. This is what I see right now:

Screenshot 2023-10-04 at 16 57 24

@ericstj
Copy link
Member Author

ericstj commented Oct 4, 2023

I see - seems like that happened after the search string was changed earlier today:
image

This was working before - seems like the issue table insertion and PR linking is not though.

@radical
Copy link
Member

radical commented Oct 4, 2023

This was working before - seems like the issue table insertion and PR linking is not though.

Yeah, I'm seeing the same issue in #93023 . It should have had 2 chances by now to do that (8, and 9pm UTC).
@missymessa any clues?

@adamsitnik adamsitnik added this to the 9.0.0 milestone Oct 20, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Oct 20, 2023
@build-analysis build-analysis bot removed this from the 9.0.0 milestone Nov 15, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Nov 15, 2023
@akoeplinger akoeplinger added this to the 9.0.0 milestone Nov 24, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Nov 24, 2023
@jeffschwMSFT jeffschwMSFT removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Feb 16, 2024
@jeffschwMSFT
Copy link
Member

removing blocking-clean-ci as it has not been hit in 30 days

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0

@adamsitnik adamsitnik modified the milestones: 9.0.0, Future Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-x64 area-System.Diagnostics.Process Known Build Error Use this to report build issues in the .NET Helix tab os-mac-os-x macOS aka OSX runtime-mono specific to the Mono runtime
Projects
None yet
Development

No branches or pull requests

7 participants