How does Briefcase handle dynamic libraries on iOS?

[marcpabst]

Hi, from reading https://peps.python.org/pep-0730/, I know that iOS is quite particular when it comes to loading dynamic libraries at runtime. I can't really find any documentation on how Briefcase handles these kinds of files. I can see that the prebuilt wheels (like numpy) come with *.dylib files, but I'm very curious about what happens within the Briefcase machinery.

Particularly, I would like to bundle in my own Python wheel that contains a *.dylib but I can't find a way to hook into Briefcase

[freakboy3742]

There's a high level answer to this; and then specific implementations of that answer.

The PEP essentially covers the high level answer. iOS will not allow any executable binary artefacts in the app bundle; all library content must be distributed as Frameworks, one framework per binary. How you access those frameworks is then up to your app - you can explicitly load the filename with dlopen, you can link against it - whatever you need to do.

How this works in practice depends on exactly which version of Python you're using.

As of Python 3.13.0b1 (which will be released any day now), CPython includes support for dynamic framework modules, by installing a custom importlib.machinery.ExtensionFileLoader subclass that knows how to accomodate iOS's dynamic library requirement. The expectation is that the Xcode project using the embedded Python interpreter will include build steps to do any library post-processing. The CPython docs contain a section on running CPython on iOS that detail how this works, and how to adapt your Xcode project to do the necessary post-processing.

Briefcase doesn't currently support Python 3.13. The code that has landed in Python 3.13 is deeply inspired by what Briefcase uses, but it has gone through some revisions as a result of being landed upstream. Fundamentally, the concepts are the same, but the Briefcase versions of Python 3.8-3.12 use a Metapath loader (which is a more complex interaction, and has some other complications and edge cases), and doesn't use the .fwork and .origin files. The post-processing build steps are subtly different as a result. The dylibs on 3.8-3.12 are also not linked against libPython whereas the official 3.13 solution is linked against libPython; and there's been a couple of subtle changes to the tag used to identify iOS (e.g., 3.13 uses arm64_iphoneos, whereas Briefcase uses iphoneos_arm64).

Before 3.13 is finalised, we're planning to back port the 3.13 accepted code back to the 3.9-3.12 branches (3.8 will be EOL when 3.13 is released), so that there is complete parity between all the implementations. This will be a backwards incompatible change - but it's a change that will ultimately result in an ecosystem that is easier to maintain.

In practical terms:

• Briefcase isn't doing anything when it comes to these binary files - it's unpacking wheels into an app_packages directory, and leaving it at that.
• The Xcode template that Briefcase uses includes the post-processing build step to move all the binaries from their original location into the Frameworks folder
• The Python support library for iOS knows how to load CPython binary modules from a framework, instead of their "in tree" location (although technically, it will load them from the in-tree location - but any app loading in that way will be rejected by App Store pre-validation).

Beyond that, dynamic loading is a straightforward dlopen of a file from its final location in the Framework folder. The 3.13-style loader makes this process a little easier, as you can modify whatever code is invoking dlopen to read the .fwork file and redirect the load to the final binary location. In the current state of 3.8-3.12, you need to do some sort of hand waving to derive the final location from the name of the file you're loading.

[marcpabst]

Thanks, this is helpful! Would probbaly be good to cover some of this in the docs, at least a small paragraph on how to provide your own binary wheels - as far as I can see, they just need to be named correctly and copied into the appropriate folder so that the xcode template can move them automatically. I'm now geting EXC_BAD_ACCESS when I try to load my package, but that seems to be a problem with compilation as Python is at least picking up on the the dylib.

[freakboy3742]

It's mentioned in passing here - but I agree that the definition of requires could be explicit that the value can be any valid PEP 508 version specifier (which includes both a local directory reference an a wheel reference). I've opened #1786 to track this idea.

[marcpabst]

I admit that that line was basically the start of my confusion. It mentions that you can provide your own wheels for iOS, but it doesn't explain how. I attempted to install through pip, but this failed due to incompatible architecture. In the end, manually copying files seemed to work. It's great to hear that Briefcase can handle this automatically, though!