Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osbuild: Add Live ISO/PXE build support #3976

Merged
merged 7 commits into from
Nov 27, 2024

Conversation

dustymabe
Copy link
Member

This backports and takes advantage of the new
org.osbuild.coreos.live-artifacts.mono stage.

Copy link

openshift-ci bot commented Nov 26, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Since we are going to modify it let's ensure it's writable.

We are changing up some of the code that builds the Live artifacts
and it exposed the fact that we are relying here on the files in
the builddir to have the writable bit set. Let's not rely on that.
In 230139d we started copying in `src/config/live` into an overlay to be
stored under `/usr/share/coreos-assembler/live` in our resulting images.
The way this is currently implemented yields a nested `live/live` folder
and duplicate contents getting created becuase we don't clean up between
runs:

```
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/README-devel.md
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/zipl.prm
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/isolinux
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/isolinux/isolinux.cfg
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/isolinux/boot.msg
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/EFI
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/EFI/fedora
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/EFI/fedora/grub.cfg
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/README-devel.md
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/zipl.prm
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/isolinux
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/isolinux/isolinux.cfg
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/isolinux/boot.msg
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/EFI
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/EFI/fedora
/srv/tmp/override/usr-share-cosa/usr/share/coreos-assembler/live/live/EFI/fedora/grub.cfg
```

This results in the same being duplicated in our images. Let's clean up
between runs so we can not have the duplicate folder/files.
@dustymabe dustymabe force-pushed the dusty-osbuild-live-iso-pxe branch from 7a95e07 to 91b470a Compare November 26, 2024 05:16
@dustymabe
Copy link
Member Author

currently this is still WIP but it's getting closer.

@dustymabe dustymabe force-pushed the dusty-osbuild-live-iso-pxe branch 3 times, most recently from 7c2affe to 2291091 Compare November 26, 2024 20:44
@jlebon jlebon force-pushed the dusty-osbuild-live-iso-pxe branch from 2291091 to d8e6a15 Compare November 27, 2024 15:30
We run osbuild inside of supermin, so for building the live ISO we'll
need all the tools it needs in there.

We don't currently name `squashfs-tools` as a toplevel dependency. It's
currently pulled in by libguestfs, but we need it in supermin too, so
explicitly list it there.
@jlebon jlebon force-pushed the dusty-osbuild-live-iso-pxe branch from d8e6a15 to 4be456d Compare November 27, 2024 16:22
@jlebon
Copy link
Member

jlebon commented Nov 27, 2024

Pushed a few minor fixes.

cosa diff is clean. kola testiso -S is clean. On all arches. LGTM.

I'm pretty comfortable with getting this in and turning it on in rawhide at this point.

@jlebon jlebon marked this pull request as ready for review November 27, 2024 17:45
jlebon
jlebon previously approved these changes Nov 27, 2024
@jlebon
Copy link
Member

jlebon commented Nov 27, 2024

Also, I think I'll turn it on in the 9.6 streams in RHCOS since those are not being consumed yet by a wider audience and I didn't test RHCOS with this change so far (though I don't really expect any issues). It'll also make it easier for the AI/ABI folks to poke at it, and bundling it with 9.6 is a nice switchover point so that it gets validated in OCP together with 9.6 validation.

ravanelli and others added 4 commits November 27, 2024 16:35
This backports and takes advantage of the new
org.osbuild.coreos.live-artifacts.mono stage.

Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
 - Add checkpoints for metal and metal4k pipelines due LiveISO builds;
 - Bump cache size from 14G to 20G.

Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
There is a difference between the build ID and the OSTree version.
They're almost always the same, except if you're rebuilding disk
images/ISOs on top of the same OSTree commit.

Here, we want the OSTree version, not the build ID.
Might as well make it a meaningful CI run.
@jlebon jlebon force-pushed the dusty-osbuild-live-iso-pxe branch from 23b4076 to 54197e5 Compare November 27, 2024 21:35
@jlebon jlebon enabled auto-merge (rebase) November 27, 2024 21:45
@jlebon
Copy link
Member

jlebon commented Nov 27, 2024

Test fixture option seems to be working fine:

$ ls -l builds/latest/x86_64/*.iso
-r--r--r--. 1 jlebon jlebon 1048576 Nov 27 16:47 builds/latest/x86_64/fedora-coreos-41.20241127.dev.0-live-iso.x86_64.iso

which is in line with sizes in https://github.com/coreos/coreos-installer/tree/main/fixtures/iso once compressed. Using e.g. coreos-installer iso kargs modify on it works as expected.

@dustymabe dustymabe disabled auto-merge November 27, 2024 22:16
@dustymabe dustymabe enabled auto-merge (rebase) November 27, 2024 22:19
@dustymabe dustymabe merged commit 9d38a32 into coreos:main Nov 27, 2024
5 checks passed
@dustymabe dustymabe deleted the dusty-osbuild-live-iso-pxe branch November 27, 2024 23:51
@gursewak1997
Copy link
Member

I see that we added a few more patches to patch_osbuild. Should I keep them when I bump to Fedora 41 since we were initially deleting the call to this function. Or should I just remove the old patches and keep the new patches that were added in this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants