Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ubuntu/noble pre-upload review against released version ubuntu/24.2-0ubuntu1_24.04.1 #26

Closed
wants to merge 195 commits into from

Conversation

blackboxsw
Copy link
Owner

To aid initial SRU upload review, we are providing this pull request to represent all unreleased changes to Ubuntu Noble in cloud-init.

Review of this PR gates landing canonical#5669.

leavelet and others added 30 commits July 3, 2024 14:10
The /etc/hostname.* files should have the mtu on
a separate line otherwise it gives error:

  ifconfig: mtu: bad value

The lines are executed in order by ifconfig and
mtu should be on it's own line.

Fixes: canonicalGH-5413
)

When there is no IPv6 addr given in the customization configuration,
we shall set IPv6 type to dhcp6, then customized Linux network will be
set to dhcp IPv6 explicitly.
* Remove `unittest` constructs and remove base classes.
* Replace tests that don't test things with tests that do
* Add fstab and mounts combinations test
There are no call sites requesting not decoding the environment vars.
This change decodes then always, simplifying typing and logic.
Instead of a broad try/except, do properly check for conditions that
invalidate a mount location.
Support for jumbo frames requires that the underlying physical interfaces
and the parent bond interface all have the larger MTU configured, not just
the physical interfaces.
…canonical#5501)

At least one of (or both) 'baseurl' or 'metalink' should be provided for yum
repository specification. Add schema changes to enforce it. Without this,
with just 'metalink' property set, one would get the schema validator error

\---
Error: Cloud config schema errors: yum_repos.epel-release: 'baseurl' is a required property
\---

Signed-off-by: Ani Sinha <anisinha@redhat.com>
On systemd, services are started by PID 1. When this doesn't happen, cloud-init
is in an unknown run state and should warn the user.

Reorder pid log to be able to reuse Distro information.

Add docstring deprecating util.is_Linux().
…5209)

Implement verify_clean_boot() to ignore certain expected logs
in a platform-specific way.
…anonical#5209)

Ensure ignore_warnings=True or ignore_errors=True is honored and
not overridden by supplemental warning texts appended.
…l#5209)

Avoid using warning level messages as there may be some
use-cases in the wild that need to invoke cloud-init boot
stages after boot for some reason unknown to upstream.

Provide a detailed warning message informing admins to file
issues against cloud-init to better represent those feature
needs before dropping this feature altogether.
…onical#5504)

Commit 604d80e introduced assertions expecting exit 2 from the
CLI when calling cloud-init init --local. Revert this test assertion
as only cloud-init status command exits (2) on deprecations/warnings.

Invoking cloud-init's boot stages on the commmand line will only exit
1 if critical errors are encountered to avoid degrading overall
systemd health as seen from cloud-init systemd units. When cloud-init
boot stages encounter recoverable_errors of any type, there is no
need to exit non-zero as those deprecation logs are not-critical to
the health of the system as a whole.
It is pretty consistently failing due to canonical#5373 with no fix in
sight.
Ensure DNS server addresses are parsed from the proper location
of network_data.json

Fixes canonical#5386

Co-authored-by: Alberto Contreras <alberto.contreras@canonical.com>
If DNS information is added to a NetworkManager managed interface where
the given protocol family is disabled, NetworkManager will be unable to
activate the interface.

canonical#5387
'mirrorlist' config can be specified instead or along with 'baseurl' in the yum
repository config. Add support for specifying mirrorlist instead of 'baseurl'.

Fixes canonicalGH-5520
Signed-off-by: Ani Sinha <anisinha@redhat.com>
9929a00 added the ability to used a cached datasource when none is
found. This was supposed to be per-datasource, but the lack of cache
cleaning got applied universally. This commit makes it so cache will be
cleaned as it was before if fallback isn't implemented in datasource.

Fixes canonicalGH-5486
This is useful for logs we want hidden by default but can be turned
on via configuration.
Switch to pathlib where appropriate and call consistently
…5515)

With this change, the following config in cloud.cfg.d/ will select NoCloud in
network stage.

```
datasource_list: [ GCE, NoCloud, None ]
datasource:
  NoCloud:
    seedfrom: http://0.0.0.0:8000/
```

Previously a two or less datasources in the datasource_list were required to
get this behavior, which was undocumented and not intuitive.

The ds-identify already allowed inline user-data and meta-data to
trigger detection.

Add ds-identify unittests for seedfrom and inline user-data.
Add DataSourceNoCloud.ds_detect() unittests for seedfrom and inline
user-data.
The nocloud datasource logs messages that are sometimes confused by users
for errors. Clarify them.

Also, remove redundant information from the logs:

- simplify log wording
- only include seed and dsmode information in nocloud string when
  non-default values are used
KsenijaS and others added 14 commits August 27, 2024 07:36
Add PPS support for azure-proxy agent and improve error logging.
…nical#5633)

Do not treat the emptiness of .cloud-init/ as an error in the logs
if agent.yaml is present.

Fixes canonicalGH-5632
…ply (canonical#5622)

Perform the same steps that cloud-init daily recipe builds performs
to assert any packaging branch updates will not break daily builds
due to quilt patch apply issues.

Steps of daily build recipe reflected in this workflow:
- checkout main
- merge packaging branch topmost commit
- quilt push -a
- run unittests (via tox -e py3)
- quilt pop -a
…cal#5641)

Reintroduce strict assert that cloud-init's cert in userdata is the
only root cert defined on the platform. Google guest agent was
installed a secondary root cert in ca_certifications.crt for a period of
time and this was determined to be less than ideal practice.

Allow cloud-init's integration tests to remain strict validation of
cert checksum to provide a signal if other platforms or agents
attempt to extend or alter the system-wide CA.
…d field (canonical#5355)

Currently cc_user_groups assumes that "useradd" never locks the password
field of newly created users. This is an incorrect assumption.

Change add_user (in both __init__.py and alpine.py) to
explicitly call either lock_passwd or unlock_passwd at all times to
achieve the desired final result.

For existing users with empty or empty locked passwords, no
password unlock will be performed and warnings will be issued.
To support empty password validation, provide functionality to
parse /etc/shadow and /var/lib/extrausers/shadow to assert
existing users do not have empty passwords before unlocking.

Additionally in this commit: 
- add NetworkBSD.ifs property to avoid subp side-effect in ___init__
  which calls ifconfig -a at every instance initialization

Useradd background:

From the useradd manpage:

'-p, --password PASSWORD
The encrypted password, as returned by crypt(3). The default is to
disable the password.'

That is, if cloud-init runs 'useradd' but does not pass it the "-p"
option (with an encrypted password) then the new user's password field
will be locked by "useradd".

cloud-init only passes the "-p" option when calling "useradd" when
user-data specifies the "passwd" option for a new user. For user-data
that specifies either the "hashed_passwd" or "plain_text_passwd"
options instead then cloud-init calls "useradd" without the "-p" option
and so the password field of such a user will be locked by "useradd".

For user-data that specifies "hashed_passwd" for a new user then
"useradd" is called with no "-p" option, so causing "useradd" to lock the
password field, however then cloud-init calls "chpasswd -e" to set the
encrypted password which also results in the password field being
unlocked.

For user-data that specifies either "plain_text_passwd" for a new user
then "useradd" is called with no "-p" option, so causing "useradd" to
lock the password. cloud-init then calls "chpasswd" to set the password
which also results in the password field being unlocked.

For user-data that specifies no password at all for a new user then
"useradd" is called with no "-p" option, so causing "useradd" to lock
the password. The password field is left locked.

In all the above scenarios "passwd -l" may be called later by
cloud-init to enforce "lock_passwd: true").

Conversely where "lock_passwd: false" applies the above "usermod"
situation (for "hash_passwd", "plain_text_passwd" or no
password) means that newly created users may have password
fields locked when they should be unlocked.

For Alpine, "adduser" does not support any form of password being
passed and it always locks the password field (the same point
applies about password field being unlocked when/if "chpasswd" is
called). Therefore in some situations (i.e. no password specified
in user-data) the password needs to be unlocked if
"lock_passwd: false".
Bump the version in cloudinit/version.py to 24.3 and
update ChangeLog.
Bump the version in cloudinit/version.py to 24.3.1 and
update ChangeLog.
patches:
debian/patches/no-nocloud-network.patch
Comment on lines +14 to +18
cloud-init (24.2-0ubuntu1~24.04.2) noble; urgency=medium

* Declare breaks on the python3-minimal version that is affected by the
py3clean failure when using alternate character set (LP: #2075337)

-- Benjamin Drung <bdrung@ubuntu.com> Wed, 07 Aug 2024 20:54:01 +0200

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, this is sync'd from direct release of 24.2-0ubuntu1~24.04.2 to the distribution packaging branches for ubuntu/noble-proposed/updates

@@ -43,6 +43,7 @@ Depends: cloud-guest-utils | cloud-utils,
${python3:Depends}
Recommends: eatmydata, gdisk, gnupg, python3-apt, software-properties-common
Suggests: openssh-server, ssh-import-id
Breaks: python3-minimal (<< 3.12.3-0ubuntu2~)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, this is sync'd from direct release of 24.2-0ubuntu1~24.04.2 to the distribution packaging branches for ubuntu/noble-proposed/updates

@panlinux
Copy link

panlinux commented Sep 6, 2024

This works, I can verify that ubuntu/noble-released matches pkg/ubuntu/noble-devel with the exception of the (easy to deal with) Breaks change.
Starting review.

debian/changelog Outdated
* d/p/no-nocloud-network.patch: Remove nocloud network feature
* Upstream snapshot based on upstream/main at acf04d61.
* refresh patches:
- d/p/no-nocloud-network.patch
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, but we are adding a patch, and refreshing it? :)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @panlinux we shouldn't document both, our tooling automatically does this as we perform a new_upstream_snapshot and mentions any refreshes applied since last unreleased snapshot sync. We typically have to manually redact this duplicated from d/changelog so I'll do that now

Wants=cloud-init-local.service cloud-init.service
After=cloud-init-local.service cloud-init.service
Wants=cloud-init-local.service cloud-init-network.service
After=cloud-init-local.service cloud-init-network.service
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a debian patch in debian/patches/no-single-process.patch which disables the single-process for the SRU. That patch also removes the cloud-init-network.service.tmpl file, so I'm thinking this rename here should be reverted in that patch too?

@panlinux
Copy link

panlinux commented Sep 9, 2024

Like PR canonical#5669, +1 from pre-SRU review.

Copy link

@panlinux panlinux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 from pre-SRU review.

@blackboxsw
Copy link
Owner Author

Thank you @panlinux. I'll close this branch as approved and merge the upstream PR canonical#5678 once our upload gets accepted into -proposed

@blackboxsw blackboxsw closed this Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.