Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

system.text.regularexpressions.tests work item #72834

Open
runfoapp bot opened this issue Jul 26, 2022 · 5 comments
Open

system.text.regularexpressions.tests work item #72834

runfoapp bot opened this issue Jul 26, 2022 · 5 comments
Labels
area-System.Text.RegularExpressions disabled-test The test is disabled in source code against the issue os-tvos Apple tvOS
Milestone
Future

Comments

@runfoapp
Copy link

runfoapp bot commented Jul 26, 2022 •
edited
Loading

Build terminating in CI with little indication why on TvOS:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-03666e5d9e604c64bd/System.Text.RegularExpressions.Tests/1/console.adbd082e.log?%3Fhelixlogtype%3Dresult

  [211/217] xunit.execution.dotnet.dll -> xunit.execution.dotnet.dll.s, xunit.execution.dotnet.dll-llvm.o, xunit.execution.dotnet.aotdata
  [212/217] xunit.runner.utility.netcoreapp10.dll -> xunit.runner.utility.netcoreapp10.dll.s, xunit.runner.utility.netcoreapp10.dll-llvm.o, xunit.runner.utility.netcoreapp10.aotdata
  [213/217] System.Text.RegularExpressions.Tests.dll -> System.Text.RegularExpressions.Tests.dll.s, System.Text.RegularExpressions.Tests.dll-llvm.o, System.Text.RegularExpressions.Tests.aotdata
./xharness-runner.apple.sh: line 173: 16241 Terminated: 15          source command.sh
Removing empty log files:
ls: /tmp/helix/working/AADA097D/w/AF48095E/uploads/xunit-*.xml: No such file or directory
+ exit_code=143

Runfo Tracking Issue: system.text.regularexpressions.tests work item

Build Definition Kind Run Name

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
0 0 0
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jul 26, 2022
@ghost
Copy link

ghost commented Jul 26, 2022

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

Runfo Creating Tracking Issue (data being generated)

Author: runfoapp[bot]
Assignees: -
Labels:

area-System.Text.RegularExpressions
Milestone: -

@noahfalk noahfalk added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' os-tvos Apple tvOS labels Jul 26, 2022
@ghost
Copy link

ghost commented Jul 26, 2022

Tagging subscribers to 'os-tvos': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

Build terminating in CI with little indication why on TvOS:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-03666e5d9e604c64bd/System.Text.RegularExpressions.Tests/1/console.adbd082e.log?%3Fhelixlogtype%3Dresult

  [211/217] xunit.execution.dotnet.dll -> xunit.execution.dotnet.dll.s, xunit.execution.dotnet.dll-llvm.o, xunit.execution.dotnet.aotdata
  [212/217] xunit.runner.utility.netcoreapp10.dll -> xunit.runner.utility.netcoreapp10.dll.s, xunit.runner.utility.netcoreapp10.dll-llvm.o, xunit.runner.utility.netcoreapp10.aotdata
  [213/217] System.Text.RegularExpressions.Tests.dll -> System.Text.RegularExpressions.Tests.dll.s, System.Text.RegularExpressions.Tests.dll-llvm.o, System.Text.RegularExpressions.Tests.aotdata
./xharness-runner.apple.sh: line 173: 16241 Terminated: 15          source command.sh
Removing empty log files:
ls: /tmp/helix/working/AADA097D/w/AF48095E/uploads/xunit-*.xml: No such file or directory
+ exit_code=143

Runfo Tracking Issue: system.text.regularexpressions.tests work item

Build Definition Kind Run Name Console Core Dump Test Results Run Client
1902387 runtime-extra-platforms Rolling net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1902034 runtime-extra-platforms Rolling net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20220523150939-4f64125 console.log runclient.py
1902000 runtime-extra-platforms PR 72706 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1902000 runtime-extra-platforms PR 72706 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1901961 runtime-extra-platforms PR 72802 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1901750 runtime-extra-platforms PR 72545 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1901717 runtime-extra-platforms PR 72554 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1901444 runtime-extra-platforms PR 72707 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1901068 runtime-extra-platforms Rolling net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1900887 runtime-extra-platforms Rolling net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20220523150939-4f64125 console.log runclient.py
1900783 runtime-extra-platforms PR 67861 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1900774 runtime-extra-platforms PR 72545 net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1900774 runtime-extra-platforms PR 72545 net7.0-MacCatalyst-Release-arm64-Mono_Release-OSX.1200.Arm64.Open console.log runclient.py
1900571 runtime-extra-platforms Rolling net7.0-tvOS-Release-arm64-Mono_Release-OSX.1100.Amd64.AppleTV.Open console.log runclient.py
1900480 runtime-extra-platforms Rolling net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20220523150939-4f64125 console.log runclient.py
1900064 runtime-extra-platforms Rolling net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20220523150939-4f64125 console.log runclient.py
1899653 runtime-extra-platforms Rolling net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20220523150939-4f64125 console.log runclient.py
1899117 runtime-extra-platforms Rolling net7.0-Linux-Release-x64-CoreCLR_release-(Fedora.34.Amd64.Open)ubuntu.1604.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20220523150939-4f64125 console.log runclient.py

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
8 16 16
Author: runfoapp[bot]
Assignees: -
Labels:

area-System.Text.RegularExpressions, blocking-clean-ci, untriaged, os-tvos
Milestone: -

@noahfalk
Copy link
Member

This may be the same build issue as: #72836

@steveisok steveisok mentioned this issue Jul 26, 2022
Merged
@steveisok
Copy link
Member

The issue is the test suite untrimmed will generate an app size around 1GB. For whatever reason, the helix machines churn when trying to build it and end up timing out. The solution I think will work is to bring over the aggressive trimming test targets that exist in

runtime/eng/testing/tests.mobile.targets

Lines 179 to 217 in 73374e8

<Target Name="ConfigureTrimming" Condition="('$(EnableAggressiveTrimming)' == 'true' or '$(EnableSoftTrimming)' == 'true') And '$(SkipConfigureTrimming)' != 'true'" AfterTargets="AddTestRunnersToPublishedFiles">
<PropertyGroup>
<TrimMode Condition="'$(EnableAggressiveTrimming)' == 'true'">link</TrimMode>
<TrimMode Condition="'$(EnableSoftTrimming)' == 'true'">copyused</TrimMode>
</PropertyGroup>
<ItemGroup>
<!-- Mark all the assemblies for link. We will explicitly mark the non-trimmable ones -->
<ResolvedFileToPublish TrimMode="$(TrimMode)" />
<!-- Don't trim the main assembly.
TrimMode="" is needed so the root assemblies are correctly identified -->
<ResolvedFileToPublish TrimMode="" Condition="'%(FileName)' == '$(AssemblyName)'" />
<!-- TODO: find out why these assemblies aren't copied by copyused even though they're referenced -->
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.ComponentModel.EventBasedAsync'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Diagnostics.FileVersionInfo'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Drawing.Primitives'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.IO.Pipelines'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Memory'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Net.WebHeaderCollection'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Runtime.Serialization.Formatters'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Security.AccessControl'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Security.Claims'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Security.Permissions'" />
<ResolvedFileToPublish TrimMode="Copy" Condition="'$(EnableSoftTrimming)' == 'true' and '%(FileName)' == 'System.Transactions.Local'" />
<!-- Even though we are trimming the test runner assembly, we want it to be treated
as a root -->
<TrimmerRootAssembly
Condition="$([System.String]::Copy('%(ResolvedFileToPublish.FileName)%(ResolvedFileToPublish.Extension)').EndsWith('TestRunner.dll'))"
Include="%(ResolvedFileToPublish.FullPath)" />
</ItemGroup>
<ItemGroup>
<TrimmerRootDescriptor Include="$(MSBuildThisFileDirectory)ILLink.Descriptor.xunit.xml" Condition="'$(SkipXunitTrimmerDescriptor)' != 'true'" />
<TrimmerRootDescriptor Include="$(MSBuildThisFileDirectory)ILLink.Descriptor.TestUtilities.xml" Condition="'$(SkipTestUtilitiesReference)' != 'true'" />
</ItemGroup>
</Target>

That will bring the app size down to about 300MB, which the helix machines should be able to handle without much trouble. Until the targets and xml descriptor files are sent over, the tests should be skipped.

steveisok pushed a commit to steveisok/runtime that referenced this issue Jul 26, 2022
[iOS] Skip test suites that take too long to build on helix
30855e8 
System.Text.RegularExpressions, Microsoft.Extensions.Logging.Generators.Roslyn4.0.Tests, and STJ source generator tests take too long to build on helix and time out before any tests are run. This is due in part to the amount of aot'd code, which ends up ballooning the app size to around 1GB.  The solution is to enable aggressive trimming targets that are in tests.mobile.targets as this will shrink the app size considerably.

Skipping these tests for now.

Addresses dotnet#72834

Addresses dotnet#72836
@runfoapp runfoapp bot mentioned this issue Jul 26, 2022
Closed
steveisok added a commit that referenced this issue Jul 27, 2022
@steveisok
[iOS] Skip test suites that take too long to build on helix (#72849)
c421a7c 
System.Text.RegularExpressions, Microsoft.Extensions.Logging.Generators.Roslyn4.0.Tests, and STJ source generator tests take too long to build on helix and time out before any tests are run. This is due in part to the amount of aot'd code, which ends up ballooning the app size to around 1GB.  The solution is to enable aggressive trimming targets that are in tests.mobile.targets as this will shrink the app size considerably.

Skipping these tests for now.

Addresses #72834

Addresses #72836
@steveisok steveisok removed blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' untriaged New issue has not been triaged by the area owner labels Aug 1, 2022
@steveisok
Copy link
Member

Suite was disabled and will look to bring it back in 8.0

@steveisok steveisok added this to the 8.0.0 milestone Aug 1, 2022
@steveisok steveisok added the disabled-test The test is disabled in source code against the issue label Jul 13, 2023
@steveisok steveisok modified the milestones: 8.0.0, Future Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-System.Text.RegularExpressions disabled-test The test is disabled in source code against the issue os-tvos Apple tvOS
Projects
None yet
Milestone
Future
Development

No branches or pull requests
2 participants
@steveisok @noahfalk