Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[openSUSE] Add packaging for documentation in info format #65

Open
wants to merge 24 commits into
base: factory
Choose a base branch
from

Conversation

Thaodan
Copy link

@Thaodan Thaodan commented Apr 2, 2024

Add packaging for documentation in the info format.

The original source of the patch adding the info target is Guix and was proposed to upstream
in:
https://lists.gnu.org/archive/html/qemu-devel/2024-03/msg04257.html

@dfaggioli
Copy link
Member

Hello, and thanks for your contribution! Thanks also for providing the link to the upstream discussion.

It's an interesting topic and while I don't use info docs much myself, I see how it could be useful. My problem with this is that upstream does not seem keen on taking it for now, which means it might become a downstream only patch which we'll then have to carry ourselves. And however simple it looks, there's always the chance of clashes with new versions or fixes backported from upstream, which is something we really much would like to avoid.

I'll leave this open for now, (and I'll point it to my fellow QEMU package maintainers) to see what others think, but my inclination is to not deviate from upstream for something like this

@Thaodan
Copy link
Author

Thaodan commented Apr 16, 2024 via email

dfaggioli and others added 17 commits September 5, 2024 18:10
Update to latest upstream major release, 9.1.0:

  https://lore.kernel.org/qemu-devel/172549088090.3334224.10887376086844748499@amd.com/

Full changelog available here:

  https://wiki.qemu.org/ChangeLog/9.1

Some of the most notable features/fixes:
 * migration: compression offload support via Intel In-Memory Analytics
   Accelerator (IAA) or User Space Accelerator Development Kit (UADK),
   along with enhanced support for postcopy failure recovery
 * virtio: support for VIRTIO_F_NOTIFICATION_DATA, allowing guest
   drivers to provide additional data as part of sending device notifications
   for performance/debug purposes
 * guest-agent: support for guest-network-get-route command on linux,
   guest-ssh-* commands on Windows, and enhanced CLI support for
   configuring allowed/blocked commands
 * block: security fixes for QEMU NBD server and NBD TLS encryption
 * ARM: emulation support for FEAT_NMI, FEAT_CSV2_3, FEAT_ETS2,
   FEAT_Spec_FPACC, FEAT_WFxT, FEAT_Debugv8p8 architecture features
 * ARM: nested/two-stage page table support for emulated SMMUv3
 * ARM: xilinx_zynq board support for cache controller and multiple
   CPUs, and B-L475E-IOT01A board support for a DM163 display
 * LoongArch: support for directly booting an ELF kernel and for running
   up to 256 vCPUs via extioi virt extension
 * LoongArch: enhanced debug/GDB support
 * RISC-V: support for version 1.13 of privileged architecture specification
 * RISC-V: support for Zve32x, Zve64x, Zimop, Zcmop, Zama16b, Zabha,
   Zawrs, and Smcntrpmf extensions
 * RISC-V: enhanced debug/GDB support and general fixes
 * SPARC: emulation support for FMAF, IMA, VIS3, and VIS4 architecture
   features
 * x86: KVM support for running AMD SEV-SNP guests
 * x86: CPU emulation support for Icelake-Server-v7, SapphireRapids-v3,
   and SierraForest

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
The avx512f, live-block-migration and pvrdma options no longer exist
in upstream configure because those features were removed. Make the
corresponding changes in the spec files.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Upstream commit 7c08eef (tests/data/acpi: Move x86 ACPI tables
under x86/${machine} path, 2024-06-25) has moved some files under
tests/data. Update the spec file to match.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
The nios2 emulation target has been removed upstream by commit
6c30148 (target/nios2: Remove the deprecated Nios II target,
2024-03-27).

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
These files are not provided by the QEMU packages.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
There was a typo on the subshell invocation: s/%/$

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…in qapi.h (bsc#1211000)

The following patches will add co_wrapper annotations to functions
declared in qapi.h. Add that header to the set of files used by
block-coroutine-wrapper.py.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
… mixed (bsc#1211000)

Some callers of this function are about to be converted to run in
coroutines, so allow it to be executed both inside and outside a
coroutine while we convert all the callers.

This will be reverted once all callers of bdrv_do_query_node_info run
in a coroutine.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…000)

This function has up until now always ran in the main loop, outside of
a coroutine. We're about to make it run inside a coroutine so start
actually taking the graph lock.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…sc#1221812)

There is a small window at the end of block device migration when
devices are being re-activated. This includes a resetting of some
fields of BDRVQcow2State at qcow2_co_invalidate_cache(). A concurrent
QMP query-block command can call qcow2_get_specific_info() during this
window and see the cleared values, which leads to an assert:

  qcow2_get_specific_info: Assertion `false' failed

This is the same issue as Gitlab #1933, which has already been
resolved[1], but there the fix applied only to non-coroutine
commands. Once we move query-block to a coroutine the problem will
manifest again.

Add an operation blocker to the invalidation function to block the
query info path during this window.

Instead of failing query-block, which would be disruptive to users,
use the blocker to know when to reschedule the coroutine back into the
iohandler so it doesn't run while the BDRVQcow2State is inconsistent.

To avoid failing query-block when all block operations are blocked,
unblock the INFO operation at various places. This preserves the prior
situations where query-block used to work.

1 - https://gitlab.com/qemu-project/qemu/-/issues/1933

Link: https://lore.kernel.org/all/87bk6trl9i.fsf@suse.de/
Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
References: bsc#1221812
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…1000)

Move this function into a coroutine so we can convert the whole
qmp_query_block command into a coroutine in the next patches.

Placing the entire command in a coroutine allow us to yield all the
way back to the main loop, releasing the BQL and unblocking the main
loop.

When the whole conversion is completed, we'll be able to avoid a
priority inversion that happens when a QMP command calls a slow
(buggy) system call and blocks the vcpu thread from doing mmio due to
contention on the BQL.

About coroutine safety:

Most callees have coroutine versions themselves and thus are safe to
call in a coroutine. The remaining ones:

- bdrv_refresh_filename, bdrv_get_full_backing_filename: String
  manipulation, nothing that would be unsafe for use in coroutines;

- bdrv_get_format_name: Just accesses a field;

- bdrv_get_specific_info, bdrv_query_snapshot_info_list: No locks or
  anything that would poll or block.

(using a mixed wrapper for now, but after all callers are converted,
this can become a coroutine exclusively)

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…sc#1211000)

We're converting callers of bdrv_co_get_allocated_file_size() to run
in coroutines because that function will be made asynchronous when
called (indirectly) from the QMP dispatcher.

This function is a candidate because it calls bdrv_do_query_node_info(),
which in turn calls bdrv_co_get_allocated_file_size().

All the functions called from bdrv_do_query_node_info() onwards are
coroutine-safe, either have a coroutine version themselves[1] or are
mostly simple code/string manipulation[2].

1) bdrv_co_getlength(), bdrv_co_get_allocated_file_size(),
   bdrv_co_get_info();

2) bdrv_refresh_filename(), bdrv_get_format_name(),
   bdrv_get_full_backing_filename(), bdrv_query_snapshot_info_list(),
   bdrv_get_specific_info();

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…1000)

This function is a caller of bdrv_do_query_node_info(), which have
been converted to a coroutine. Convert this function as well so we're
closer from having the whole qmp_query_block as a single coroutine.

Also remove the wrapper for bdrv_co_do_query_node_info() now that all
its callers are converted.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…#1211000)

We're converting callers of bdrv_co_get_allocated_file_size() to run
in coroutines because that function will be made asynchronous when
called (indirectly) from the QMP dispatcher.

This function is a candidate because it calls bdrv_query_image_info()
-> bdrv_co_do_query_node_info() -> bdrv_co_get_allocated_file_size().

It is safe to turn this is a coroutine because the code it calls is
made up of either simple accessors and string manipulation functions
[1] or it has already been determined to be safe [2].

1) bdrv_refresh_filename(), bdrv_is_read_only(),
   blk_enable_write_cache(), bdrv_cow_bs(), blk_get_public(),
   throttle_group_get_name(), bdrv_write_threshold_get(),
   bdrv_query_dirty_bitmaps(), throttle_group_get_config(),
   bdrv_filter_or_cow_bs(), bdrv_skip_implicit_filters()

2) bdrv_co_do_query_node_info() (see previous commits);

This was the only caller of bdrv_query_image_info(), so we can remove
the wrapper for that function now.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…rt (bsc#1211000)

We're currently doing a full query-block just to enumerate the devices
for qmp_nbd_server_add and then discarding the BlockInfoList
afterwards. Alter hmp_nbd_server_start to instead iterate explicitly
over the block_backends list.

This allows the removal of the dependency on qmp_query_block from
hmp_nbd_server_start. This is desirable because we're about to move
qmp_query_block into a coroutine and don't need to change the NBD code
at the same time.

Add the GRAPH_RDLOCK_GUARD_MAINLOOP macro because
bdrv_skip_implicit_filters() needs the graph lock.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
…odes to coroutine (bsc#1211000)

Convert the remaining functions to make the QMP commands query-block
and query-named-block-nodes run in their entirety in a coroutine. With
this, any yield from those commands will return all the way back to
the main loop. This releases the BQL and the main loop and avoids
having the QMP command block another more important task from running.

Both commands need to be converted at once because hmp_info_block
calls both and it needs to be moved to a coroutine as well.

Now the wrapper for bdrv_co_get_allocated_file_size() can be made not
mixed and the wrapper for bdrv_co_block_device_info() can be removed.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
Signed-off-by: Lin Ma <lma@suse.com>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
The fstat call can take a long time to finish when running over
NFS. Add a version of it that runs in the thread pool.

Adapt one of its users, raw_co_get_allocated_file size to use the new
version. That function is called via QMP under the qemu_global_mutex
so it has a large chance of blocking VCPU threads in case it takes too
long to finish.

Link: https://lore.kernel.org/r/20240409145917.6780-1-farosas@suse.de
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: João Silva <jsilva@suse.de>
References: bsc#1211000
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Package qemu-vmsr-helper for letting VMs access the RAPL MSR.

I'll live in its own package and only makes sense on x86_64.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Upstream provides services for qemu-pr-helper. So far, we've not needed
them, so let's continue not to ship them for now.

However, in case at some point we want to start offering them, stash the
commented out runes for that in the spec file.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Convert conditional build of features to the %bcond_without, so they
can actually be disabled, e.g., at the project level.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
… in a macro

When running configure, first of all we disable everything, and then we
enable only the feature that we know we want (and, of course, system
and user emulation use different sets of such features).

Consolidate the first part in a macro, that can be share between the two
spec files, making everything simpler and prettier.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
dfaggioli and others added 3 commits September 11, 2024 18:47
Was disabled by mistake, when updating the QEMU version.
Reinstate it.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Take advantage of the Sphinx texinfo backend to generate a QEMU info
manual.  The Texinfo format allows for more structure and info readers
provide more advanced navigation capabilities compared to manpages
readers.

The original source of the patch is Guix and was proposed to upstream
in:
https://lists.gnu.org/archive/html/qemu-devel/2024-03/msg04257.html

* docs/meson.build (texi, info): New targets.

Signed-off-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Signed-off-by: Björn Bidar <bjorn.bidar@jolla.com>
Signed-off-by: Björn Bidar <bjorn.bidar@jolla.com>
@Thaodan Thaodan force-pushed the Thaodan/openSUSE/factory/info_package branch from 92f5efc to 271a3f2 Compare September 19, 2024 07:15
@dfaggioli dfaggioli force-pushed the factory branch 2 times, most recently from 036b767 to 080ae7a Compare October 15, 2024 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants