Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tests] Stand up iOS arm runtime tests and enable tracing/eventpipe tests #79169

Merged
merged 170 commits into from
Apr 18, 2023

Conversation

mdh1418
Copy link
Member

@mdh1418 mdh1418 commented Dec 2, 2022

Fixes #74891

On iOS, runtime tests were previously only stood up on x64 (Simulator) in #43954.
This PR looks to extend the runtime testing coverage to iOS devices and arm simulators by doing the following:

  • Provide AOT'd assemblies to the AppleAppBuilder in the runtime build flow following the conventions of library tests (previously no assemblies were provided to the AppleAppBuilder)
  • Default RunAOTCompilation and MonoForceInterpreter properties in the runtime build flow for iOS
  • Enable diagnostics_tracing for iOS runtime tests
  • Reenable runtime tracing/eventpipe tests on iOS not involving subprocesses
  • Tweak ProcessInfo and ProcessInfo2 tests to pass on iOS
  • Skip IpcTraceTest.EnsureCleanEnvironment as ProcessManager.GetProcessInfos is not supported on iOS
  • Align AppleAppBuilder parameter order with the order of definition in the task

Fixes: #81233

@ghost
Copy link

ghost commented Dec 2, 2022

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #74891

On iOS, runtime tests were previously only stood up on x64 (Simulator) in #43954.
This PR looks to extend the runtime testing coverage to iOS devices and arm simulators by doing the following:

  • Provide AOT'd assemblies to the AppleAppBuilder in the runtime build flow following the conventions of library tests (previously no assemblies were provided to the AppleAppBuilder)
  • Default RunAOTCompilation and MonoForceInterpreter properties in the runtime build flow for iOS
  • Enable diagnostics_tracing for iOS runtime tests
  • Reenable runtime tracing/eventpipe tests on iOS not involving subprocesses
  • Tweak ProcessInfo and ProcessInfo2 tests to pass on iOS
  • Skip IpcTraceTest.EnsureCleanEnvironment as ProcessManager.GetProcessInfos is not supported on iOS
  • Align AppleAppBuilder parameter order with the order of definition in the task
Author: mdh1418
Assignees: -
Labels:

area-Infrastructure-mono, runtime-mono

Milestone: -

@ghost ghost assigned mdh1418 Dec 2, 2022
@@ -274,6 +274,9 @@
<BuildDir>$(IntermediateOutputPath)\iOSApps\$(Category)</BuildDir>
<FinalPath>$(XUnitTestBinBase)$(CategoryWithSlash)\$(Category).app</FinalPath>
<RuntimeComponents>diagnostics_tracing;marshal-ilgen</RuntimeComponents>
<DiagnosticPorts>127.0.0.1:9000,nosuspend,listen</DiagnosticPorts>
<RunAOTCompilation>True</RunAOTCompilation>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused here; what happens if we are running iOSSimulator and both RunAOTCompilation and MonoForceInterpreter are true? Does this change how iOSSimulator tests are run?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For iOSlike devices and arm based simulators, AOT is needed for the tests to build. (At least that was the case for library tests, which motivated this PR #76551) When setting RunAOTCompilation and MonoForceInterpreter to true, what will happen is AOT will be ran, and if anything goes wrong, interp will be the fallback instead of JIT (which is important because JIT does not work on iOSlike platforms).
I believe the RunAOTCompilation is necessary here for arm architectures, but we can exclude x64 if thats preferred.

@mdh1418
Copy link
Member Author

mdh1418 commented Dec 16, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@mdh1418
Copy link
Member Author

mdh1418 commented Dec 19, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@mdh1418
Copy link
Member Author

mdh1418 commented Dec 19, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@naricc
Copy link
Contributor

naricc commented Dec 19, 2022

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@naricc
Copy link
Contributor

naricc commented Dec 20, 2022

/azp run runtime-ioslike

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@naricc
Copy link
Contributor

naricc commented Jan 6, 2023

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines failed to run 2 pipeline(s).

@naricc naricc requested a review from vargaz as a code owner January 6, 2023 20:53
@kotlarmilos
Copy link
Member

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member

The failure shouldn't be related. It has been tracked in dotnet/arcade#11683.

Copy link
Member Author

@mdh1418 mdh1418 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a bunch of nits and Jeremy's suggestion for the test grouping, but looks good overall to me!

Would approve, but the approve button is greyed out for me because I was the original PR author

src/tests/issues.targets Outdated Show resolved Hide resolved
eng/pipelines/libraries/helix-queues-setup.yml Outdated Show resolved Hide resolved
src/tests/Common/Coreclr.TestWrapper/MobileAppHandler.cs Outdated Show resolved Hide resolved
src/tests/Common/helixpublishwitharcade.proj Outdated Show resolved Hide resolved
src/tests/Common/helixpublishwitharcade.proj Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
src/tests/issues.targets Outdated Show resolved Hide resolved
Copy link
Member

@steveisok steveisok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@kotlarmilos
Copy link
Member

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member

/azp run runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mdh1418
Copy link
Member Author

mdh1418 commented Apr 17, 2023

/azp run runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@kotlarmilos
Copy link
Member

/azp run runtime-ioslike,runtime-ioslikesimulator

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member

Failures shouldn't be related. The TCP failure is known issue dotnet/arcade#11683.

@kotlarmilos
Copy link
Member

The failures are known #84979 and #84995.

@kotlarmilos kotlarmilos merged commit 790b14b into dotnet:main Apr 18, 2023
@ghost ghost locked as resolved and limited conversation to collaborators May 18, 2023
@mdh1418 mdh1418 deleted the ios_investigate_eventpipe_failures branch June 10, 2024 20:34
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-Infrastructure-mono runtime-mono specific to the Mono runtime
Projects
None yet
7 participants