Build python wheels and upload to pypi

[ermakov-oleg]

Hi @chdsbd,

I have started working on issue #261 and have successfully built Python packages for macOS, Windows, and Linux (x86_64).

However, I’m encountering issues building libpg_query-sys for Linux aarch64. Could you please assist me with this?

Thanks!

[netlify]

👷 Deploy request for squawkhq pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	169331f

[ermakov-oleg]

Hi @ods,

I came across your repository where you are already building Python packages: squawk-cli-python-package. Thank you for your work on this!

What do you think about moving the build process to this repository, publishing it to your PyPI squawk-cli, and transferring the rights to @sbdchd and @chdsbd?

[ods]

Hi @ermakov-oleg,

What do you think about moving the build process to this repository, publishing it to your PyPI squawk-cli, and transferring the rights to @sbdchd and @chdsbd?

It would be the best option!

[ermakov-oleg]

Hi @ermakov-oleg,

What do you think about moving the build process to this repository, publishing it to your PyPI squawk-cli, and transferring the rights to @sbdchd and @chdsbd?

It would be the best option!

Great!

Now we just need to figure out the build process for the Linux aarch64 package.

[ermakov-oleg]

@ermakov-oleg How do you handle libpg_query FFI dependency when cross-compiling?

I’m not very experienced in this, but as far as I understand, when selecting a target different from the current system, the entire necessary toolchain for the specific platform is downloaded, and bindgen sets the sysroot path to the header files of that platform from this toolchain

[ermakov-oleg]

A cold build under QEMU takes about 40 minutes, and with caching (sccache), around 20 minutes.

I had to update the Rust version to start using the sparse index in Cargo, otherwise, the crates.io index update would hang indefinitely.

[ods]

I’m not very experienced in this, but as far as I understand, when selecting a target different from the current system, the entire necessary toolchain for the specific platform is downloaded, and bindgen sets the sysroot path to the header files of that platform from this toolchain

I'm neither experienced. What I saw so far, this works perfectly for pure rust code, but not for FFI. Also, I saw maturin docs suggest to use zig for cross-compilation in such cases.

[ermakov-oleg]

I’m not very experienced in this, but as far as I understand, when selecting a target different from the current system, the entire necessary toolchain for the specific platform is downloaded, and bindgen sets the sysroot path to the header files of that platform from this toolchain

I'm neither experienced. What I saw so far, this works perfectly for pure rust code, but not for FFI. Also, I saw maturin docs suggest to use zig for cross-compilation in such cases.

Thanks for the idea!
The libpg_query-sys library cross-compiles using Zig, but only with glibc version >= 2.38.
I’m currently trying to figure out how to provide Maturin with the appropriate glibc version.

[ermakov-oleg]

@sbdchd I’ve finished building the Python wheels for all major platforms. Could you please review this PR?
Thank you in advance!

[ermakov-oleg]

Hi @sbdchd and @chdsbd,

I hope you’re doing well! I just wanted to kindly check in on the status of this pull request. I’ve finished the work on building the Python wheels and would love to hear your thoughts on the feature. Do you think there might be some time in the near future for a code review?

Thank you in advance!

[sbdchd]

Is the PR description still accurate, specifically:

However, I’m encountering issues building libpg_query-sys for Linux aarch64. Could you please assist me with this?

otherwise looks good, thank you for doing this, lot of hard work!!

[ermakov-oleg]

Yes, I was able to resolve the issue with the aarch64 build. The package now builds correctly for all major platforms, including Linux aarch64. The description is outdated — everything is working fine now.

[ermakov-oleg]

Hi! When is the release planned?
Do you need my help with uploading the package to PyPI or will you and @ods manage it yourselves?

[chdsbd]

I'm pushed a new version but I think we need to figure out authentication

[chdsbd]

@ods I think since you own squawk-cli, you'd need to either transfer the project, add @sbdchd and myself as collaborators, or configure https://docs.pypi.org/trusted-publishers/adding-a-publisher/ so we can publish via OpenID Connect

[ods]

Hi @chdsbd, is chdsbd the correct user?

[ods]

@ermakov-oleg, is ermakov the correct user? I'm planning to also add you as a maintainer, so that you can help with setting it up.

[ermakov-oleg]

@ermakov-oleg, is ermakov the correct user? I'm planning to also add you as a maintainer, so that you can help with setting it up.

Yeah, it's me.

[chdsbd]

@ods I use cdignam on PyPi

[ods]

@ods I use cdignam on PyPi

Added. Feel free removing me after successful publishing.

[chdsbd]

I finally got publishing working, so 1.2.0 is on pypi

[ods]

Excellent! Is there a change this way of installation being mentioned in the docs?

[chdsbd]

Hey @ods, I updated the readme to mention you can install squawk via pip:

squawk/README.md

Lines 19 to 21 in 013a414

	# or via PYPI
	pip install squawk-cli