Beginnings of native Android build #110471

grendello · 2024-12-06T12:56:15Z

No description provided.

dotnet-policy-service · 2024-12-06T20:23:17Z

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

…ndk' into dev/grendel/android-build-with-ndk

…ING_PAL_TCP

…stem.Security.Cryptography.Native.Android. It will incorrectly try to install in the coreclr build

…em.native library for the singlefilehost

janvorli · 2024-12-12T23:55:12Z

eng/native/build-commons.sh

@@ -80,7 +80,7 @@ build_native()

    if [[ "$targetOS" == android || "$targetOS" == linux-bionic ]]; then
        # Keep in sync with $(AndroidApiLevelMin) in Directory.Build.props in the repository rooot
-        local ANDROID_API_LEVEL=28
+        local ANDROID_API_LEVEL=29


@grendello has mentioned in #110470 that even API level 28 is unfortunate and that we will want to support older API level, e.g. the originally used level 21.
So I wonder if moving to the level 29 is a good thing.

It's not good in the long term and is something we should fix. Moving it to 29 for now allows us to not have to deal with emulated TLS usage.

Unless we can fix it now, I don't think this should block progress. We can do it sometime down the road.

In perspective, this would drop 12.8% of the devices (with API level from 21 to 29) according to https://apilevels.com/

Then again:
Level 21 is Android 5 and 6 years EOL
Level 29 is Android 10 and 2 years EOL (https://endoflife.date/android)
Level 34 or higher is necessary to publish apps to the Play Store (https://developer.android.com/google/play/requirements/target-sdk)

I think I can make the emulated TLS work without too much hassle. The question though is if we are worried about the perf implications of the emulated TLS vs the standard TLS and thus we would like to end up having support for both.

Perf is important, so I think the long-term solution should be that we have two builds of the runtime (alas): one with emulated TLS for the older devices (we mustn't ignore the older Android versions, 12.8% is still a lot of devices out there), and another targeting API 29 and onwards. It will increase nuget size and the runtime build times on CI, but I think it would be the best way forward.

I'd like to avoid runtime explosion if we can help it.

… R2R that acts like linux

MichalStrehovsky · 2024-12-18T05:50:17Z

src/coreclr/runtime.proj

-      <_CoreClrBuildArg Include="-DANDROID_STL=none"/>
-      <_CoreClrBuildArg Include="-DANDROID_CPP_FEATURES=&quot;no-rtti no-exceptions&quot;"/>
+      <_CoreClrBuildArg Include="-DANDROID_STL=c++_static"/>
+      <_CoreClrBuildArg Include="-DANDROID_CPP_FEATURES=&quot;no-rtti exceptions&quot;"/>


Does this affect size of native AOT runtime packs? (Build with ./build.sh -ci -arch x64 -os linux-bionic -c Release /p:DotNetBuildRuntimeNativeAOTRuntimePack=true and check the size of libRuntime.WorkstationGC.a). We'll likely have to move this part of configuration someplace else, native AOT is the most size sensitive workload we have. Native AOT doesn't need C++ runtime library and doesn't need exceptions.

Yes, it will affect the size, but less than if we included the shared libc++ (which is something we cannot do anyway) and I think we just need to pay the price. NativeAOT will be but one of the supported ways to build the application, we need to keep that in mind. I don't like the fact that CoreCLR needs exceptions (Android runtime doesn't use exceptions nor RTTI), but that's what we have and we need to support it (at least for the foreseeable future).

Ideally, for NativeAOT we should be able to link the application .so at the app build time (from a set of .o or .a files), in which case this setting here won't matter at all.

My question is whether we could set this in the CMake files instead of as argument to CMake. That way pieces of this can still compile with exceptions turned off instead of globally forcing the VM policy.

I'm not sure we can? The variables are used by the Android toolchain definition file - I think they need to be set on the command line for this to work.

Yeah this needs to be set at the CMake build level sadly. There's not a supported way to do it per-target. We could likely manually set flags to do it if we figure out how the toolchain file sets flags.

src/coreclr/tools/Common/TypeSystem/Common/TargetDetails.cs

vitek-karas · 2024-12-18T08:37:04Z

eng/native/naming.props

@@ -2,9 +2,36 @@
  <PropertyGroup>
    <StaticLibPrefix>lib</StaticLibPrefix>
    <ExeSuffix Condition="'$(HostOS)' == 'windows'">.exe</ExeSuffix>
+    <Blah>$(HostOS)</Blah>


Has this been removed?

… msbuild places instead.

…-build-with-ndk

* main: JIT: Model GT_RETURN kills with contained operand (dotnet#111230) Update dependencies from https://github.com/dotnet/runtime-assets build 20250110.2 (dotnet#111290) [NativeAOT/ARM64] Generate frames compatible with Apple compact unwinding (dotnet#107766) Cleanup unused JIT stubs in vm (dotnet#111237) Ensure that Shuffle is marked as HW_Flag_CanBenefitFromConstantProp (dotnet#111303) Fix CMP0173 policy warning with cmake 3.31 (dotnet#110522) [RISC-V] Fix HostActivation.Tests unknown-rid (dotnet#110687) Fix accidentally duplicated global-build-step.yml in runtime-official.yml (dotnet#111302) JIT: run extra SPMI queries for arrays (dotnet#111293) Split the Runtime Shared Framework project and combine legs in the official build (dotnet#111136) Do not ignore `MemoryMarshal.TryWrite` result (dotnet#108661) Update dependencies from https://github.com/dotnet/emsdk build 20250109.1 (dotnet#111263) Clean up in Number.Formatting.cs (dotnet#110955)

src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj

…arget

src/coreclr/tools/aot/crossgen2/crossgen2.props

ivanpovazan · 2025-01-15T14:49:34Z

eng/native/tryrun.cmake

-   EXISTS ${CROSS_ROOTFS}/usr/lib/gcc/i586-alpine-linux-musl OR
-   EXISTS ${CROSS_ROOTFS}/usr/lib/gcc/x86_64-alpine-linux-musl OR
-   EXISTS ${CROSS_ROOTFS}/usr/lib/gcc/riscv64-alpine-linux-musl)
+if(NOT DEFINED ANDROID_PLATFORM)


@janvorli I changed the workaround so that we don't set DARWIN when cross-building. We still rely on setting ANDROID_PLATFORM externally though, what we should revisit in the follow-up PRs.

… host lib prefix/suffix on cross builds

…ndk' into dev/grendel/android-build-with-ndk

AaronRobinsonMSFT · 2025-01-15T16:26:00Z

eng/native/naming.props

  </PropertyGroup>

  <!-- Add path globs specific to native binaries to exclude unnecessary files from packages. -->
+  <Choose>


Doesn't this logic already exist?

@jkoritzinsky ?

I don't believe it exists elsewhere. I put this here because I found host / target confusion when crossgen2 was cross building android from a mac.

The logic we have is for the "target" platform, not the host (and it's right below this).

src/coreclr/crossgen-corelib.proj

AaronRobinsonMSFT · 2025-01-15T16:30:00Z

src/coreclr/hosts/inc/coreclrhost.h

+#ifdef __cplusplus
+#define CORECLR_HOSTING_API_LINKAGE extern "C"
+#else
+#define CORECLR_HOSTING_API_LINKAGE
+#endif


FYI @elinor-fung

AaronRobinsonMSFT · 2025-01-15T16:31:52Z

src/coreclr/pal/src/eventprovider/CMakeLists.txt

@@ -1,6 +1,7 @@
 set(EVENT_MANIFEST ${VM_DIR}/ClrEtwAll.man)

-if(CLR_CMAKE_HOST_LINUX)
+# TODO: LTTNG is not supported on Android. We need a solution.


In lieu of a TODO, please file a github issue and provide a link in the comment.

AaronRobinsonMSFT · 2025-01-15T16:34:12Z

src/coreclr/runtime.proj

-      <!-- No LTTNG on Android -->
-      <_CoreClrBuildArg Include="-cmakeargs -DFEATURE_EVENT_TRACE=0"/>
+      <!-- 
+        TODO: No LTTNG on Android, it's disabled at the eventprovider level. Will need an answer. 


Same as above, please file an issue and add a link to the issue here.

AaronRobinsonMSFT · 2025-01-15T16:38:08Z

src/native/eventpipe/CMakeLists.txt

@@ -12,7 +12,9 @@ check_symbol_exists(
    HAVE_ACCEPT4)

 # Use TCP for EventPipe on mobile platforms
-if (CLR_CMAKE_HOST_IOS OR CLR_CMAKE_HOST_TVOS OR CLR_CMAKE_HOST_ANDROID)
+#
+# TODO: Resolve TCP/IP EventPipe support on CoreCLR Android


Please file a github issue and record the link here.

AaronRobinsonMSFT · 2025-01-15T16:38:16Z

src/native/external/libunwind_extras/CMakeLists.txt

+
+

Suggested change

AaronRobinsonMSFT · 2025-01-15T16:40:01Z

src/native/external/libunwind_extras/CMakeLists.txt

+    # Instead of patching libunwind, don't treat the handful of warnings it has as errors
+    add_compile_options(-Wno-error)


This should be an issue. We regularly upstream changes to libunwind. See libunwind/libunwind#734.

/cc @am11 @janvorli

Could you show the warnings? We should fix them here instead and upstream the patch simultaneously, and add a line in ../libunwind-version.txt.

AaronRobinsonMSFT · 2025-01-15T16:40:19Z

src/native/external/libunwind_extras/CMakeLists.txt

+
+

Suggested change

akoeplinger · 2025-01-15T16:52:30Z

Directory.Build.props

@@ -68,7 +68,7 @@
        - src/tasks/MobileBuildTasks/Apple/AppleProject.cs
        - https://github.com/dotnet/sdk repo > src/Installer/redist-installer/targets/GeneratePKG.targets
     -->
-    <AndroidApiLevelMin>21</AndroidApiLevelMin>
+    <AndroidApiLevelMin>29</AndroidApiLevelMin>


this needs to be conditioned on CoreCLR so we don't change the Mono build. same in eng/native/build-commons.sh

Should this be 28? @janvorli tested it recently with 28 https://github.com/dotnet/arcade/blob/88a10ba00ed798ee289d786db9f6339acd682a99/eng/common/cross/build-android-rootfs.sh#L24. We can provide a fallback for that glob API to further lower the requirement.

I think 29 is necessary because that's the first that does not use emulated TLS.

We can add a run-time check for emulated TLS selection and still lower the requirement.

Did we add a workaround for that? When I ran things previously we only had that for NativeAOT. Assuming that's true, the build will fail.

Emulated TLS works on linux-bionic and has been for a while, so in theory it should work on older Androids, newer ones can use the real TLS. I have recently dealt with Emulated TLS on riscv64 (and later switched to real TLS but still having some issues #111317 (comment)). We can tweak things in asm code to select which path to take GETTHREAD_ETLS_1 vs. INLINE_GETTHREAD. Right now it's under the build condition FEATURE_EMULATED_TLS.

It appears that there is going to be some amount of work to get this right < API level 29. My preference would be to do any of that work in a follow up.

akoeplinger · 2025-01-15T16:57:28Z

src/native/corehost/apphost/static/CMakeLists.txt

-add_compile_definitions(NATIVE_LIBS_EMBEDDED)
+# TODO: Android uses a separate System.Security.Cryptography lib that does not have 
+# the exported functions the singlefilehost expects. 
+# Figure out how that's going to work and turn this back on.


please file an issue for this, though I'm kinda surprised that the host depends on specific S.C.Cryptography functions?

jkoritzinsky

Can we run a test official build to make sure the official build will build the CoreCLR assets in the android legs and that we won't have conflicts/double-writes with any package names?

jkoritzinsky · 2025-01-15T19:22:17Z

src/coreclr/.nuget/coreclr-packages.proj

@@ -9,11 +9,11 @@
    <ProjectReference Include="ILCompiler.Reflection.ReadyToRun.Experimental\ILCompiler.Reflection.ReadyToRun.Experimental.pkgproj" />
  </ItemGroup>

-  <ItemGroup Condition="'$(RuntimeFlavor)' == 'CoreCLR'">
+  <ItemGroup Condition="'$(RuntimeFlavor)' == 'CoreCLR' and '$(TargetsAndroid)' != 'true' ">


Suggested change

<ItemGroup Condition="'$(RuntimeFlavor)' == 'CoreCLR' and '$(TargetsAndroid)' != 'true' ">

<ItemGroup Condition="'$(RuntimeFlavor)' == 'CoreCLR' and '$(TargetsMobile)' != 'true' ">

jkoritzinsky · 2025-01-15T19:22:26Z

src/coreclr/.nuget/coreclr-packages.proj

    <ProjectReference Include="Microsoft.NETCore.TestHost\Microsoft.NETCore.TestHost.proj" />
  </ItemGroup>

-  <ItemGroup>
+  <ItemGroup Condition=" '$(TargetsAndroid)' != 'true' ">


Suggested change

<ItemGroup Condition=" '$(TargetsAndroid)' != 'true' ">

<ItemGroup Condition=" '$(TargetsMobile)' != 'true' ">

jkoritzinsky · 2025-01-15T19:24:12Z

src/coreclr/CMakeLists.txt

@@ -250,8 +250,10 @@ if(NOT CLR_CMAKE_HOST_MACCATALYST AND NOT CLR_CMAKE_HOST_IOS AND NOT CLR_CMAKE_H
  add_subdirectory(utilcode)
  add_subdirectory(inc)

-  add_subdirectory(ilasm)
-  add_subdirectory(ildasm)
+  if (NOT CLR_CMAKE_TARGET_ANDROID)


Could we instead define a property for "build executables" instead of per-platform?
And also define one for the case above for "NativeAOT-only"?

jkoritzinsky · 2025-01-15T19:26:29Z

src/coreclr/pal/src/eventprovider/CMakeLists.txt

@@ -1,6 +1,7 @@
 set(EVENT_MANIFEST ${VM_DIR}/ClrEtwAll.man)

-if(CLR_CMAKE_HOST_LINUX)
+# TODO: LTTNG is not supported on Android. We need a solution.
+if(CLR_CMAKE_HOST_LINUX AND NOT CLR_CMAKE_TARGET_ANDROID)


This should also use the host defines.

Suggested change

if(CLR_CMAKE_HOST_LINUX AND NOT CLR_CMAKE_TARGET_ANDROID)

if(CLR_CMAKE_HOST_LINUX AND NOT CLR_CMAKE_HOST_ANDROID)

jkoritzinsky · 2025-01-15T19:28:15Z

src/coreclr/runtime.proj

-      <_CoreClrBuildArg Include="-DANDROID_STL=none"/>
-      <_CoreClrBuildArg Include="-DANDROID_CPP_FEATURES=&quot;no-rtti no-exceptions&quot;"/>
+      <_CoreClrBuildArg Include="-DANDROID_STL=c++_static"/>
+      <_CoreClrBuildArg Include="-DANDROID_CPP_FEATURES=&quot;no-rtti exceptions&quot;"/>


Yeah this needs to be set at the CMake build level sadly. There's not a supported way to do it per-target. We could likely manually set flags to do it if we figure out how the toolchain file sets flags.

jkoritzinsky · 2025-01-15T19:32:45Z

src/coreclr/runtime.proj

+      <!--
+        Unsure if something like this is actually defined. We need this to differentiate between
+        features that are enabled in mono android, but not in coreclr.
+      -->
+      <_CoreClrBuildArg Include="-cmakeargs -DRUNTIME_CORECLR=1" />


Is this used anywhere?

Yeah, one spot. Is there a better existing way to do this?

runtime/src/native/eventpipe/CMakeLists.txt

Line 17 in 9779aeb

if (CLR_CMAKE_HOST_IOS OR CLR_CMAKE_HOST_TVOS OR (CLR_CMAKE_HOST_ANDROID AND NOT RUNTIME_CORECLR))

Could we add a general feature flag that the Mono vs CoreCLR builds would set?

Is this expected to work for the NativeAOT build?

src/coreclr/tools/aot/crossgen2/crossgen2.props

src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Runtime.CoreCLR.sfxproj

Beginnings of native Android build

90c22cb

dotnet-issue-labeler bot added the area-Infrastructure-coreclr label Dec 6, 2024

dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Dec 6, 2024

grendello added 2 commits December 6, 2024 15:46

Don't package what's not needed on Android

6f1b917

TARGET_LINUX == TARGET_ANDROID for crosscomp.h

c25fbd9

steveisok added os-android and removed community-contribution Indicates that the PR has been added by a community member labels Dec 6, 2024

grendello added 3 commits December 6, 2024 21:43

Beginnings of native Android build

bfcbb3e

Don't package what's not needed on Android

97f2555

TARGET_LINUX == TARGET_ANDROID for crosscomp.h

3d9da10

grendello force-pushed the dev/grendel/android-build-with-ndk branch from c25fbd9 to 3d9da10 Compare December 6, 2024 20:43

steveisok added 2 commits December 6, 2024 15:50

Merge remote-tracking branch 'grendel/dev/grendel/android-build-with-…

9e173df

…ndk' into dev/grendel/android-build-with-ndk

Fix cmake endif / make sure RuntimeFlavor is CoreCLR

bb9f323

build-analysis bot mentioned this pull request Dec 6, 2024

restarted. Azure DevOps can't recover from restarts. dotnet/dnceng#3879

Open

3 tasks

steveisok added 3 commits December 9, 2024 10:41

Use dummyprovider instead of lttngprovider / disable FEATURE_PERFTRAC…

b0f3d55

…ING_PAL_TCP

Put a GEN_SHARED_LIB check around install_with_stripped_symbols in Sy…

f9eb950

…stem.Security.Cryptography.Native.Android. It will incorrectly try to install in the coreclr build

Bump min API level to 29 (proper TLS support) and link the right syst…

a69fcfd

…em.native library for the singlefilehost

janvorli reviewed Dec 12, 2024

View reviewed changes

Fix host/target confusion in R2R tool publish / Add Android target to…

54fbb50

… R2R that acts like linux

build-analysis bot mentioned this pull request Dec 13, 2024

[mono] Runtime tests LargeStructSize_Mono and xprecise1_cs_do failing on Apple mobile #110149

Closed

MichalStrehovsky reviewed Dec 18, 2024

View reviewed changes

src/coreclr/tools/Common/TypeSystem/Common/TargetDetails.cs Outdated Show resolved Hide resolved

vitek-karas reviewed Dec 18, 2024

View reviewed changes

build-analysis bot mentioned this pull request Dec 18, 2024

Compiler crashes when failing to release Mutex #53420

Open

steveisok added 2 commits January 10, 2025 13:52

Revert changes to R2R that alias Android to linux and put it in a few…

2e4c737

… msbuild places instead.

Merge remote-tracking branch 'upstream/main' into dev/grendel/android…

2516884

…-build-with-ndk

build-analysis bot mentioned this pull request Jan 11, 2025

"We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection" dotnet/dnceng#1886

Open

3 tasks

grendello added 2 commits January 13, 2025 10:00

Oops, too much

240ad01

This was referenced Jan 13, 2025

The hosted runner encountered an error while running your job. (Error Type: Disconnect). dotnet/dnceng#1919

Open

[wasm] Timeout in System.Text.Json.Tests #101204

Open

WASM MT: memory access out of bounds in System.Private.Xml.Tests #110805

Open

ivanpovazan reviewed Jan 14, 2025

View reviewed changes

src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj Outdated Show resolved Hide resolved

Undo host awareness in crossgen_publish as building it is about the t…

34cdc56

…arget

ivanpovazan reviewed Jan 15, 2025

View reviewed changes

src/coreclr/tools/aot/crossgen2/crossgen2.props Outdated Show resolved Hide resolved

Fix setting target platform in tryrun.cmake

c171ce3

ivanpovazan reviewed Jan 15, 2025

View reviewed changes

steveisok added 2 commits January 15, 2025 09:55

Make sure android runtime pack is generated and crossgen_inbuild uses…

a515ed8

… host lib prefix/suffix on cross builds

Merge remote-tracking branch 'grendel/dev/grendel/android-build-with-…

3831805

…ndk' into dev/grendel/android-build-with-ndk

grendello marked this pull request as ready for review January 15, 2025 16:22

grendello requested review from AaronRobinsonMSFT and a team as code owners January 15, 2025 16:22

AaronRobinsonMSFT reviewed Jan 15, 2025

View reviewed changes

src/coreclr/crossgen-corelib.proj Show resolved Hide resolved

AaronRobinsonMSFT reviewed Jan 15, 2025

View reviewed changes

src/native/external/libunwind_extras/CMakeLists.txt Outdated

Comment on lines 14 to 15

Copy link

Member

AaronRobinsonMSFT Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

AaronRobinsonMSFT reviewed Jan 15, 2025

View reviewed changes

src/native/external/libunwind_extras/CMakeLists.txt Outdated

Comment on lines 181 to 182

Copy link

Member

AaronRobinsonMSFT Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

akoeplinger reviewed Jan 15, 2025

View reviewed changes

jkoritzinsky reviewed Jan 15, 2025

View reviewed changes

Condition min api level for coreclr and not.

9779aeb

		# Instead of patching libunwind, don't treat the handful of warnings it has as errors
		add_compile_options(-Wno-error)

	<ItemGroup Condition="'$(RuntimeFlavor)' == 'CoreCLR' and '$(TargetsAndroid)' != 'true' ">
	<ItemGroup Condition="'$(RuntimeFlavor)' == 'CoreCLR' and '$(TargetsMobile)' != 'true' ">

	<ItemGroup Condition=" '$(TargetsAndroid)' != 'true' ">
	<ItemGroup Condition=" '$(TargetsMobile)' != 'true' ">

	if(CLR_CMAKE_HOST_LINUX AND NOT CLR_CMAKE_TARGET_ANDROID)
	if(CLR_CMAKE_HOST_LINUX AND NOT CLR_CMAKE_HOST_ANDROID)

Beginnings of native Android build #110471

Are you sure you want to change the base?

Beginnings of native Android build #110471

Conversation

grendello commented Dec 6, 2024

dotnet-policy-service bot commented Dec 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkoritzinsky Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkoritzinsky left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkoritzinsky Jan 15, 2025 •

edited

Loading