feat: Download duckdb library from GitHub

[krlmlr]

To avoid the 500000 bytes boundary on CRAN.

[hannes]

Should we not just get the tarball from duckdb/duckdb releases?

[krlmlr]

We could at some point, yes. I wouldn't change it right now, for a variety of reasons.

[david-cortes]

There was some discussion about dealing with package sizes in r-polars some time ago.

Turns out some packages are able to get exceptions - example.

Perhaps you could ask CRAN maintainers for a size exception for DuckDB? Would be a lot more convenient for users if the package could keep the full source in the same style as most CRAN packages.

[krlmlr]

Thanks, I'd like to know more about that. It's not that we can't release the package, but the release no longer is automatic. What does the polars exception look like exactly?

[eitsupi]

I don't think CRAN allows downloading files from the internet.
How about attaching a compressed version of the DuckDB source code?
This is what all Rust-based packages submitted to CRAN do.
https://yutani.rbind.io/post/rust-and-cran-repository-policy/#downloads-the-rust-sources

[krlmlr]

Thanks. I remember the rule about binary files, but what about a source tarball?

xz-compressing the sources seems like a neat idea, but it will complicate the release process.

[eitsupi]

but what about a source tarball?

It is listed as a last option and I am not sure if CRAN will allow downloads from GitHub.
The general limit for packages is 10MB, so I don't think there is any compelling reason for duckdb, which is below 10MB and in the normal package category, to choose it.

xz-compressing the sources seems like a neat idea, but it will complicate the release process.

I think we can add the tarball to gitignore and branch the build process by the presence or absence of the tarball and only attach the tarball if we are submitting to CRAN.
prqlr has been doing that for over a year.
https://github.com/PRQL/prqlc-r/blob/2b447001d14e4ed301a9110ec8ffa3dc42e26524/src/Makevars.in#L24-L29

[krlmlr]

It looks like I could create a ./cleanup file that is run as part of R CMD build . and xz-compresses the duckdb sources. The configure script would then extract the sources if they are compressed. That file could be added to .Rbuildignore too.

Curious if this works on Windows.

[krlmlr]

#530