how can I run runtime tests with monovm on ios?

[srxqds]

No description provided.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

null

Author:	srxqds
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[steveisok]

/cc @SamMonoRT

[ghost]

Tagging subscribers to 'os-ios': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

null

Author:	srxqds
Assignees:	-
Labels:	untriaged, area-Infrastructure-mono, os-ios
Milestone:	-

[SamMonoRT]

@srxqds - please follow instructions in https://github.com/dotnet/runtime/blob/main/docs/workflow/README.md and let us know if you have any questions.
@kotlarmilos - if there are limitation based on feedback, please update the documentation as needed

[JamboWu2]

The instruction what I follow is https://github.com/dotnet/runtime/blob/main/docs/workflow/testing/mono/testing.md
but the instruction do not have any information about iOS device or iOSSimulation testing, just have Android testing

I try to build the runtime tests for iOSSimulation use command in src/tests

./build.sh -mono os iOSSimulation arm64 release

It will pass but the test will crash while running

build the runtime tests for iOS device use command

./build.sh -mono os iOS arm64 release

It will throw many exceptions: Need list of AOT files for device builds.

[kotlarmilos]

The instruction what I follow is https://github.com/dotnet/runtime/blob/main/docs/workflow/testing/mono/testing.md but the instruction do not have any information about iOS device or iOSSimulation testing, just have Android testing

Check out the following instructions https://github.com/dotnet/runtime/blob/main/docs/workflow/testing/libraries/testing-apple.md. They are for the library tests, but could be useful.

I try to build the runtime tests for iOSSimulation use command in src/tests

./build.sh -mono os iOSSimulation arm64 release

First, run the following command from the root directory to build the runtime:

./build.sh -arch arm64 -os ios -s mono+libs -c Release

Second, run the following command from src/tests to AOT the runtime tests:

./build.sh os ios arm64 Release -mono tree tracing/eventpipe /p:LibrariesConfiguration=Release

Please note that a subset of runtime tests is supported currently. We will introduce more tests in the future.

It will pass but the test will crash while running

build the runtime tests for iOS device use command

./build.sh -mono os iOS arm64 release

It will throw many exceptions: Need list of AOT files for device builds.

Such error is usually happening when AOT compilation was disabled and AppleAppBuilderTask didn't get any AOT-ed assemblies. If you are running a HelloiOS sample app, make sure to set ForceAOT parameter to TRUE. Additionally, if you have an example, feel free to provide it here, so we can help you with the compilation and identify what is missing in the build process.

[JamboWu2]

./build.sh -mono os iOS arm64 release /p:DevTeamProvisioning=ZF3Z3HS5W2 /p:ForceAOT=True /p:LibrariesConfiguration=Release

The problam still exists
Need list of AOT files for device builds.

[kotlarmilos]

Can you share the error logs?

Parameter /p:ForceAOT=True is only required while running a HelloiOS sample app. If you want to run runtime tests, it is not necessary.

[JamboWu2]

There is a error:

xcrun simctl install 15ED0483-DDCB-4C06-B4E8-5230AB119F52 ./artifacts/tests/coreclr/iOSSimulator.arm64.Release/JIT/Math/JIT_Math.app
xcrun simctl launch --console booted net.dot.JIT_Math===========Apple builder build:==bindir:================/Users/game-netease/dotnet/runtime/artifacts/tests/coreclr/obj/iOS.arm64.Release/Managed/./build/iOSApps/Interop_ICustomMarshaler/bin-Interop_ICustomMarshaler-arm64 False
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: “AppleAppBuilderTask”任务意外失败。
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: System.InvalidOperationException: Need list of AOT files for device builds.
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: at AppleAppBuilderTask.Execute() in /Users/game-netease/dotnet/runtime/src/tasks/AppleAppBuilder/AppleAppBuilder.cs:line 233
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)

the following output is the code I add in AppleAppBuilder.cs execute function
===========Apple builder build:==bindir:================/Users/game-netease/dotnet/runtime/artifacts/tests/coreclr/obj/iOS.arm64.Release/Managed/./build/iOSApps/Interop_ICustomMarshaler/bin-Interop_ICustomMarshaler-arm64 False

[kotlarmilos]

Thank you. According to the logs, there are artifacts built for iOSSimulator.arm64.Release configuration. I suggest removing ./artifacts directory and building the runtime for ios-arm64 only. Additionally, please note that only tracing/eventpipe subset of runtime tests is enabled on iOS, so make sure to have -mono tree tracing/eventpipe parameter included.

The steps should be:

rm -rf ./artifacts
./build.sh -arch arm64 -os ios -s mono+libs -c Release
./src/tests/build.sh os ios arm64 Release -mono tree tracing/eventpipe /p:LibrariesConfiguration=Release

Also, I would recommend trying HelloiOS sample app to get familiar with the process, and to verify if the runtime is built correctly. To run the HelloiOS sample app, configure parameters in the Makefile and run make all.

[JamboWu2]

So follow your instructions :
(in Runtime directory)
1: ./build.sh mono+libs -os ios -a arm64 -c Release
2: cd src/tests
3: ./build.sh os iOS arm64 Release -mono tree:tracing/eventpipe /p:DevTeamProvisioning=ZF3Z3HS5W2 /p:LibrariesConfiguration=Release

then still the same error:

===========Apple builder build:==bindir:================/Users/game-netease/dotnet/runtime/artifacts/tests/coreclr/obj/iOS.arm64.Release/Managed/./build/iOSApps/tracing_eventpipe/bin-tracing_eventpipe-arm64 False
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: “AppleAppBuilderTask”任务意外失败。
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: System.InvalidOperationException: Need list of AOT files for device builds.
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: at AppleAppBuilderTask.Execute() in /Users/game-netease/dotnet/runtime/src/tasks/AppleAppBuilder/AppleAppBuilder.cs:line 233
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
/Users/game-netease/dotnet/runtime/src/tests/build.proj(330,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)

==================
HelloiOS sample app is OK, It can make succeed and run normally on the device

[kotlarmilos]

The provided command appears to have an invalid tree parameter. To fix this, try using tree tracing/eventpipe instead of tree:tracing/eventpipe. The current invalid parameter is causing the script to build all runtime tests on iOS, instead of only the supported subset.

[JamboWu2]

./build.sh os iOS arm64 Release -mono tree tracing/eventpipe /p:DevTeamProvisioning=ZF3Z3HS5W2 /p:LibrariesConfiguration=Release

The command is invalid,the errors below:

The switch that responds to file appends:
switch:tracing/eventpipe

[kotlarmilos]

Could you check if you are using the latest version from the main branch?

If you are using the latest version and the issues still persists, I suggest you zip and share your local version of the runtime repository here, so I can try to reproduce the issue locally.

Thank you.

[JamboWu2]

The branch what I used is release/7.0 instead of main, is that OK?

[kotlarmilos]

The iOS runtime tests have been introduced recently, in #79169. Please try with the main branch.

[JamboWu2]

Ok, I will check out main branch and try it, thank you

anyway,why we must focus on tracing/eventpipe? what about the other tests? Like JIT/Math, GC... and so on?

[kotlarmilos]

We plan to include other testing groups that could be executed on iOS/iOSSimulator in the future. It may be possible to filter other testing groups for execution, but they have not been tested yet.

If you find that certain groups can be run on a device without any modifications, we encourage you to contribute by opening a PR, or providing details here.

[JamboWu2]

Ok, thank you for help!

[kotlarmilos]

Thanks! I will close this issue, but feel free to reopen it if you have further questions.