Ubuntu/devel #4391
Merged
Ubuntu/devel #4391
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…#4167) Added a DataSource for the Akamai cloud platform, enabling metadata and user data retrieval and instance provisioning via cloud-init.
full re-runs will no longer write duplicate rules to sudoers file Rerunning cloud-init would always append sudoer rules to the sudoers file without checking if the rules already existed first. This is no longer the case - only unique rules not already found in the sudoers file will be added to it. Fixes canonicalGH-4048 LP: #1998539
Bump the version in cloudinit/version.py to 23.2.2 and update ChangeLog.
…4277) Add Alpine to distros supported by this module. If sudo is not available then try and use doas as an alternative if it is available.
…64 (canonical#4291) By default, NetworkManager renderer in cloud-init does not set any specific method for IPV6 addr-gen-mode in the keyfiles it writes. Hence, implicitly the mode is set to `eui64` in the absence of any global addr-gen-mode option in NetworkManager configuration. Later when other interfaces get added via D-Bus API or by using nmcli commands without explictly setting an addr-gen-mode, NM auto generates new profiles for those interfaces with addr-gen-mode set to `stable-privacy`. This introduces inconsistency of configurations between interfaces based on how they were added. This can cause problems for the customers. In this change, cloud-init overrides NetworkManager's preferred default of `stable-privacy` to use EUI64 using a drop in NetworkManager configuration file. This setting can be overriden by using global-connection-defaults setting in /etc/NetworkManager/NetworkManager.conf file. RHBZ: 2188388 Signed-off-by: Ani Sinha <anisinha@redhat.com>
Ruff is much faster and replicates everything flake8 can do. Additionally, I added a few more checks that seemed relevant to our project and would not require many changes. Long term, we should be able to replace isort with ruff as well, and maybe eventually pylint, though ruff is not yet able to do everything that pylint can.
…al#4317) Reference: https://gitee.com/src-openeuler/cloud-init/issues/I3SQF9?from=project-issue https://gitee.com/src-openeuler/cloud-init/issues/I4KMYA?from=project-issue Signed-off-by: shixuantong <shixuantong1@huawei.com> Signed-off-by: chengquan <chengquan3@huawei.com> Signed-off-by: yang_zhuang_zhuang <yangzhuangzhuang1@huawei.com>
…ical#4336) get_master() was checking against system's sysfs which may cause failures when the mocked interface conflicted with real one. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
Reflect this in the test, by mocking the kernel_version every time. Sponsored by: The FreeBSD Foundation
ec2: Support double encoded userdata The Amazon APIs expect userdata to be base64 encoded when passed as input to e.g. RunInstances. A number of tools, including the AWS CLI, perform this base64 encoding implicitly, but it's common for users to base64 encode the data prior to passing it to them. This results in two layers of base64 encoding and effectively results in a failed EC2 launch. This change adds the ability to decode the redundant layer of encoding. Fixes amazonlinux/amazon-linux-2023#401 Signed-off-by: Noah Meyerhans <nmeyerha@amazon.com>
…cal#4281) Currently, ds-identify determines if CloudStack is supported by checking dmi product name. It works fine on KVM, as the VM has the following system information System Information Manufacturer: Apache Software Foundation Product Name: CloudStack KVM Hypervisor However, it does not work on VMware or XenServer/XCP-ng. For example, on VMware, the system information is System Information Manufacturer: VMware, Inc. Product Name: VMware Virtual Platform Version: None
To make a swapfile work on btrfs, we need to create an empty file and add the "no copy-on-write" attribute before making it a swapfile. See https://btrfs.readthedocs.io/en/latest/Swapfile.html Fixes canonicalGH-3713 LP: #1884127
Address complaints by newer versions of flake8, these are not picked up by ruff
Add support for Alpine Linux to cc_keyboard.
When required setuptools functions are not available, we skip running the module. Doing this requires specifying `allow_module_level=True`, which we were not doing before. Also, the setuptools import will fail if tools tests are run after the standard unit tests.
…rce (canonical#4328) Signed-off-by: Ani Sinha <anisinha@redhat.com>
These tests relied on the SUT to provide sensible return values for util.kernel_version(). On FreeBSD, those values are 14.0-CURRENT for my development machine, which doesn't work well with when split at '.' and compared against a pair of integers (14, '0-CURRENT') < (5, 4) doesn't make much sense. This patch adds `@mock`s for util.kernel_version wherever it is used. That way, test_cc_wireguard.py tests passes on a FreeBSD system. Sponsored by: The FreeBSD Foundation
`/proc/$pid/environ` and `/proc/$pid/stat` don't exist on BSD. procfs on BSDs is highly limited, and its use very discourged. While FreeBSD has procstat(1), which could reasonably read a process environment, that use-case in the code is Linux specific and doesn't map on FreeBSD. Guard get_proc_env() tests with is_Linux(). The code already mostly uses this as a fallback, when all else fails. implement get_proc_ppid() with `ps(1)`. This should work on most platforms that have `ps(1)`. Sponsored by: The FreeBSD Foundation Fixes canonicalGH-4332
change clean_script to use `#!/bin/sh` which is guaranteed to exist on pretty much any Unix (like) system, unlike `/bin/bash`, which could be anywhere in the PATH. Sponsored by: The FreeBSD Foundation
Fix a test failure that only seemed to occur when running the full test
suite. Probably due to something not being cleaned up properly.
This adds an env={"LANG": "C"} to simplify the test execution, and
makes the failure go away.
Sponsored by: The FreeBSD Foundation
Fixes canonicalGH-4354
Because /etc/rc.d/growfs exists on FreeBSD, we need to be selective about when to allow tests access to it, so that they will pass. Sponsored by: The FreeBSD Foundation
In case of very unexpected timestamps, we pass them on to date. Unfortunately, only GNU date is able to perform this level of deduction (guess work). Rework the code and tests to look for GNU date on non-Linux platforms. Only fail, and loudly! if GNU date cannot be found. Sponsored by: The FreeBSD Foundation Fixes canonicalGH-4333
Simplify the cloud.cfg.tmpl file. There is a lot of duplication (e.g. the same sudo rule specified multiple times). This also addresses canonical#4267 and aligns Debian-specific template configuration with Debian's own packaged cloud.cfg, located here: https://salsa.debian.org/cloud-team/cloud-init/-/blob/master/debian/cloud.cfg Use uniform YAML indentation. Modify jinja template to avoid duplicate blank lines.
1!5.0.0 removed 'wait' on launch along with a cloud 'clean' method, so update our integration testing code appropriately.
This enables running cloud-init integration tests from distros other than Ubuntu.
…l#4364) When user-data defines apt: config to setup repos with GPG keyids, GPG spawns a two background daemons gpg-agent and dirmngr for the root user. In systemd 253 and later, systemd will report the systemd unit as SubState=running instead of SubState=exited. This results in cloud-init status --wait blocking indefinitely despite the fact that the cloud-config.service StatusErrno=0 and Result=success. While the SubState=running doesn't block systemd proceeding to any units/services declared `After=cloud-config.service` it does block cloud-init status --wait which will affect external tools/utilities which may be waiting for cloud-init to complete initial system configuration. Rather than altering cloud-init status --wait to observe the conditions StatusErrno=0 and Result=success, it makes sense for cc_apt_configure to teardown the gpg-agent and dirmngr daemon cleanly with gpgconf --kill all as cloud-init shouldn't be leaving around the daemons which can immediately be spawned automatically on demand by any subsequent gpg invocation. Fixes: canonicalGH-4344
…cal#4366) Annotated git tags are not present in shallow git clones. In shallow clones, git describe may fail with exit 128 with one of the following errors: - No tags can describe <hash> - No names found, cannot describe anything If either message is present, fallback to cloudinit.version.version_string. This fixes readthedocs CI builds which may use git clone --depth 1 in Annotated git tags are not present in shallow git clones. In shallow clones, git describe may fail with exit 128 with one of the following errors: - No tags can describe <hash> - No names found, cannot describe anything If either message is present, fallback to cloudinit.version.version_string. Also pin sphinx=7.1.2 as the release 7.2.0 on August 17 dropped the extension setup_js_tag_helper used by our sphinx builds. This fixes readthedocs CI builds which may use git clone --depth 1 in the doc build process.
In f146fe cloud-init developed capability to automatically determine nocloud mode. Remove documentation related to nocloud-net since this is no longer a necessary argument and may confuse users during debugging.
Swapfile works fine with fallocate on btrfs. Btrfs official document also use fallocate instead of dd. See https://btrfs.readthedocs.io/en/latest/Swapfile.html
collect-logs now streams certain outputs to avoid large memory usage _stream_command_output_to_file() was added to mimic functionality of _write_command_output_to_file() except instead of reading the output of subprocess calls into memory, it streams the outputs directly to the target file. This new function is used when the output of a subprocess call does not need to be saved into a variable. As far as usage goes, the main difference between the two functions is that the stream function does not return result of the subprocess call, while the write function does. Fixes canonicalGH-3994 LP: #1980150
When fetching metadata or reprovisiondata, allow up to 30 seconds for a read timeout. For cases where a read request is delayed, this would eliminate extra follow-up request(s). Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
status: treat SubState=running and MainPID=0 as service exited Avoid cloud-init status --wait blocking indefinitely on systemd status SubState=running and MainPID=0 condition for cloud-init systemd services. When daemons are spawned by a systemd unit in the unified hierarchy, long-lived processes or daemons launched by the systemd unit will be tracked in the cgroup/slice. As long as spawned daemons of processes live, the systemd tooling will report the parent systemd unit as SubState=running instead of SubState=exited. This affected Ubuntu across the 23.04 -> 23.10 release boundary because a long-term patch to ignore cgroup status was dropped at systemd 252.5 which last existed in Ubuntu 23.04 (Lunar). Cloud-init status --wait checks MainPID == 0 when SubState=running as a secondary indicator the the executing process of the boot stage has completed despite any forked daemons. Additionally fix typo in puppet integration test name. Fixes canonicalGH-4373
PEP 668 disallows installing packages via pip outside of virtual environments, to avoid breaking due to conflicting system packages. Ansible is unlikely to be installed in the base system, and if it is and the user still chooses to use `install_method: pip`, that would be an invalid configuration. This flag exists to allow users to bootstrap ansible to run as a controller. The alternative solution would be to require virtual environments, however this is often packaged as a separate dependency, which further complicates bootstrap. Allow installing ansible outside of virtual environments via pip's --break-system-packages. Fixes canonicalGH-4244
Add doas (an alternative to sudo) support for Alpine, FreeBSD, and OpenBSD to cc_users_groups module for user creation/update.
…l#4387) canonical#4276 uncovered an issue with the initialization of the return value for get_instance_userdata(). The return value was initialized with user_data = "" which is a str class. It then calls url_helper.read_file_or_url(), which attempts to retrieve user-data content from IMDS. read_file_or_url() returns its results as a bytes object, which is then passed directly up to the caller. In the event that read_file_or_url() does not successfully retrieve content (e.g. because it was given a file:// path to a nonexistent file or an http:// path that generates a 404 code), an exception is raised an get_instance_userdata returns the string object initially stored in user_data. Rather than make the caller cope with return data potentially encoded as either bytes or str, this commit changes the initialization of user_data to an empty bytes object, ensuring type consistency in get_instance_userdata()'s return value. Fixes canonicalGH-4386 Signed-off-by: Noah Meyerhans <nmeyerha@amazon.com>
localectl set-x11-keymap is now a disabled sub-command for new debian-based installs. Given we can no longer use this command, write /etc/default/keyboard directly as directed in man keyboard. LP: #2030788
Bump the version in cloudinit/version.py to 23.3 and update ChangeLog.
blackboxsw
requested review from
TheRealFalcon and
holmanb
and removed request for
TheRealFalcon
TheRealFalcon
approved these changes
Aug 28, 2023
There was a problem hiding this comment.
LGTM. No changes (other than author) on my end.
I get a lintian warning though:
W: cloud-init source: newer-debconf-templates [debian/po/templates.pot]
debconf-updatepo does provide me a small diff:
diff --git a/debian/po/templates.pot b/debian/po/templates.pot
index 3171457cb..eada8db11 100644
--- a/debian/po/templates.pot
+++ b/debian/po/templates.pot
@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: cloud-init\n"
"Report-Msgid-Bugs-To: cloud-init@packages.debian.org\n"
-"POT-Creation-Date: 2021-05-12 12:22-0400\n"
+"POT-Creation-Date: 2023-08-28 14:08-0500\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
@@ -29,12 +29,6 @@ msgstr ""
msgid "ConfigDrive: Reads data from Openstack Config Drive"
msgstr ""
-#. Type: multiselect
-#. Choices
-#: ../cloud-init.templates:1001
-msgid "LXD: Reads /dev/lxd/sock representation of instance data"
-msgstr ""
-
#. Type: multiselect
#. Choices
#: ../cloud-init.templates:1001
@@ -173,12 +167,24 @@ msgstr ""
msgid "Vultr: Vultr Cloud"
msgstr ""
+#. Type: multiselect
+#. Choices
+#: ../cloud-init.templates:1001
+msgid "LXD: Reads /dev/lxd/sock representation of instance data"
+msgstr ""
+
#. Type: multiselect
#. Choices
#: ../cloud-init.templates:1001
msgid "NWCS: NWCS"
msgstr ""
+#. Type: multiselect
+#. Choices
+#: ../cloud-init.templates:1001
+msgid "Akamai: Akamai and Linode platforms"Given that we have no translations I'm not sure it's something we need to update here, but we may want to fix it after upload.
blackboxsw
force-pushed
the
ubuntu/devel
branch
from
e768042
to
397c108
Compare
Will pull that change in and git rebase reorder it into 23.3-0ubuntu1 |
TheRealFalcon
approved these changes
Aug 28, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Perform
new_upstream_snapshot.py -c 23.3 --first-devel-uploadto create an upload for mantic from verified annotated tag 23.3.Do not squash merge
Additional Context
Test Steps
Process followed from https://github.com/orgs/canonical/projects/29/views/1?pane=issue&itemId=21362354
Publish ubuntu/develproject cardChecklist: