v3.51.0

Known Issues

⚠️ There is a known issue with buildkite-agent step get. This is fixed in v3.52.0

v3.51.0 (2023-08-15)

Full Changelog

Added

• Add --strict-single-hooks #2268 (@DrJosh9000)
• Add missing 'an' in annotation help #2285 (@mdb)
• [Experimental] Verify step signatures #2210 (@moskyb)
• [Experimental] Pipeline Signing/Verification with JWS #2252 (@moskyb)
• [Experimental] Include plugins in command step signatures #2292 (@DrJosh9000)

Changed

• Make the agent send a SIGTERM (configurable) before a SIGKILL to subprocesses #2250 (@triarius)
• Limit job log length #2192 (@DrJosh9000)
• Refactor redactor into streaming replacer and use it to redact secrets #2277 (@DrJosh9000)
• Dependency upgrades #2278 #2274 #2271 #2272 #2275 #2266 (@dependabot[bot])

Fixed

• Fix fatal: bad object not detected from git fetch #2286 (@triarius)
• Fix scalar plugin parsing #2264 (@DrJosh9000)

Internal

• Reorganise step types among files #2267 (@DrJosh9000)
• Upload test coverage #2270 (@DrJosh9000)
• Remove unwrapping in error Is methods #2269 (@triarius)
• Use capacity hint in concat #2288 (@DrJosh9000)
• Add ordered.Unmarshal, and use it in pipeline parsing #2279 (@DrJosh9000)
• Create a setup method for config and logger to reduce boilerplate #2281 (@triarius)
• Add retry for publishing RPMs #2280 (@triarius)
• Fix data race in testAgentEndpoint #2265 (@DrJosh9000)
• Fix missing "fmt" import #2287 (@DrJosh9000)

v3.50.4

v3.50.4 (2023-07-31)

Full Changelog

This point release fixes more parser cases in previous point releases of 3.50. We recommend anyone using v3.50.0 through v3.50.3 upgrade, especially when experiencing problems with previously-working pipeline YAML.

• Legacy step forms (where the step type is specified with type) should parse correctly
• Scalar (i.e. non-map) block steps should parse correctly
• Explicit env: null should parse correctly
• Parser errors should no longer cause the pipeline to be missing steps; either errors should halt processing or unknown step types remain in the output (though these might still be rejected by buildkite.com).

⚠️ There is a remaining known issue concerning environment variable substitution in pipelines. When environment variables that are defined as null at the top level of a pipeline upload are substituted, they will no longer fall back to values defined in the environment of the buildkite-agent. For example, if you had a pipeline

env:
  FOO: null
steps:
- command: echo "FOO=$FOO"

and upload the pipeline as

FOO=fromJob buildkite-agent pipeline upload

the uploaded pipeline was previously equivalent to:

env:
  FOO: null
steps:
- command: echo "FOO=fromJob"

but now, the null value will result in an empty string being substituted instead.

To work around this, we recommend you defined the fallback manually like so:

env:
  FOO: ${FOO:-}
steps:
- command: echo "FOO=$FOO"

Fixed

• Even More Pipeline Parsing Fixes #2253 (@moskyb)
• Fix missing return statements when unmarshalling fails #2245 (@moskyb), #2257 (@DrJosh9000)
• Add future-proofing UnknownStep type #2254 (@DrJosh9000)
• Nil handling fixes, particularly parsing env: null #2260 (@DrJosh9000)

Changed

• Remove docker-compose v1 from ubuntu 22.04 and replace with compatibility script #2248 (@triarius)
• Authentication failure errors when using S3 now mention BUILDKITE_S3_PROFILE and AWS_PROFILE #2247 (@DrJosh9000)

Internal

• Remove a double check for the existence of a local hook and log when it is missing in debug #2249 (@triarius)
• Refactor some code in process.go #2251 (@triarius)
• Store GOCACHE outside container #2256 (@DrJosh9000)
• Get mime types from github, rather than Apache's SVN Server #2255 (@moskyb)
• Check that go.mod is tidy in CI #2246 (@moskyb) and fix flakiness of this check #2261 (@triarius)

v3.50.3

v3.50.3 (2023-07-24)

Full Changelog

This release contains a known issue:

Severity	Description	Fixed in
Medium	Several forms of pipelines fail to parse and can no longer be uploaded	✅ Fixed in v3.50.4

Changed

• Two-phase pipeline parsing #2238 (@DrJosh9000)
• Remove installing qemu-binfmt from agent pipeline #2236 (@triarius)

v3.50.2

v3.50.2 (2023-07-21)

Full Changelog

This release contains a known issue:

Severity	Description	Fixed in
Medium	When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround	✅ Fixed in v3.50.3

Workaround for yaml merge key issue

For example, this pipeline would fail to parse:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Fixed

• Fix an issue introduced in #2207 where jobs wouldn't check if they'd been cancelled #2231 (@triarius)
• Fix avoid-recursive-trap experiment not recognised #2235 (@triarius)
• Further refactor to agent.JobRunner #2222 #2230 (@moskyb)

v3.50.1

v3.50.1 (2023-07-20)

Full Changelog

This release contains multiple issues:

Severity	Description	Fixed in
⚠️ Very High	Jobs running on this version of the agent are not cancellable from the UI/API	✅ Fixed in v3.50.2
Medium	When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround	✅ Fixed in v3.50.3

Workaround for yaml merge key issue

For example, this pipeline would fail to parse:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Fixed

• Empty or zero-length steps is no longer a parser error, and is normalised to [] instead #2225, #2229 (@DrJosh9000)
• Group steps now correctly include the group key #2226 (@DrJosh9000)
• Increases to test coverage for the new parser #2227 (@DrJosh9000)

v3.50.0

v3.50.0 (2023-07-18)

Full Changelog

This release contains multiple issues:

Severity	Description	Fixed in
Medium	When uploading pipelines, some group steps are not correctly parsed, and were ignored.	✅ Fixed in v3.50.1
Low	Uploading pipelines with empty or zero-length steps failed, where they should've been a no-op.	✅ Fixed in v3.50.1
⚠️ Very High	Jobs running on this version of the agent are not cancellable from the UI/API	✅ Fixed in v3.50.2
Medium	When uploading pipelines, if any object in the pipeline YAML contained multiple merge keys, the pipeline would fail to parse. See below for a workaround	✅ Fixed in v3.50.3

Workaround for yaml merge key issue

For example, this pipeline would fail to parse:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: *default_plugins
    <<: *default_retry
    command: "echo 'hello, world!'"

As a workaround for this, you can use yaml array merge syntax instead:

default_plugins: &default_plugins
  plugins:
    - docker#4.0.0:
        image: alpine:3.14

default_retry: &default_retry
  retry:
    automatic:
      - exit_status: 42

steps:
  - <<: [*default_plugins, *default_retry]
    command: "echo 'hello, world!'"

Added

• We're working on making pipeline signing a feature of the agent! But it's definitely not ready for primetime yet... #2216, #2200, #2191, #2186, #2190, #2181, #2184, #2173, #2180 (@moskyb, @DrJosh9000)
• Add option to configure location of Job Log tmp file #2174 (@yhartanto)
• Add avoid-recursive-trap experiment to avoid a recursive trap #2209 (@triarius)
• Load the AWS Shared Credentials for s3 operations #1730 (@lox)

Fixed

• Add workaround for fatal: bad object errors when fetching from a git mirror #2218 (@DrJosh9000)
• Fix missing fetch when updating git mirrors of submodules (#2203) (@DrJosh9000)
• Use a unique name for each agent started using the systemd template unit file #2205 (@DavidGregory084)
• Polyglot hooks wasn't documented in EXPERIMENTS.md, so we fixed that #2169 (@moskyb)
• De-experimentify wording on the status page #2172 (@DrJosh9000)
• The secrets redactor now properly redacts multi-line secrets and overlapping secrets #2154 (@DrJosh9000)

Changed

• Print agent version and build in debug logs #2211 (@triarius)
• Include the version each experiment was promoted #2199 (@DrJosh9000)

Various code cleanups and meta-fixes

• Fix docker builds for Ubuntu 22.04 #2217 (@moskyb)
• JobRunner cleanup #2207 (@moskyb)
• Simplify command phase #2206 (@triarius)
• Rename Bootstrap struct (and friends) to Executor #2188 (@moskyb)
• Upgrade docker compose plugin to v4.14, use docker compose v2 #2189 (@moskyb)
• Rename package bootstrap -> job #2187 (@moskyb)
• Clarify code around creating a process group #2185 (@triarius)
• Fix docker builds for Ubuntu 22.04 #2217 (@moskyb)

And the usual amount of @dependabot[bot] updates!

v3.49.0

Fixed

• CreateArtifacts & UpdateArtifacts: remove sometimes-too-short timeout after 4 attempts #2159 (@pda)
• Fix submodule mirror repository remote using main repo URL #1998 (@francoiscampbell)
• Update job log file to include line transforms #2157 (@chasestarr)
• Clearer HTTP error logging from API client #2156 (@moskyb)

Changed

• Buildkite-Timeout-Milliseconds API request header #2160 (@pda)
• Extract pipeline parser to package internal/pipeline #2158 (@DrJosh9000)
• Minor dependency updates #2165 #2164 #2162 #2161 #2153 #2152 #2151 (@dependabot[bot])
• Lock library #2145 (@DrJosh9000)

v3.48.0

v3.48.0 (2023-06-06)

Full Changelog

The de-experimentification release!

• The ansi-timestamps experiment is now enabled by default. To remove the
  timestamps from your logs, you can pass the --no-ansi-timestamps flag.
• The flock-file-locks experiment is now enabled by default. Because the old
  and new file lock systems don't interact, we strongly recommend not running
  multiple agents of different versions on the same host.
• The inbuilt-status-page experiment is now enabled by default. For those
  running the agent with --health-check-addr, go to /status to see a
  human-friendly status page.

And whatever happened to git-mirrors? It graduated from experiment-hood in
v3.47.0!

Changed

• De-experimentify ansi-timestamps #2133 (@DrJosh9000)
• Preserve plugin config env var names with consecutive underscores #2116 (@triarius)
• De-experimentify flock-file-locks #2131 (@DrJosh9000)
• Report more AWS metadata #2118 (@david-poirier)
• De-experimentify inbuilt-status-page #2126 (@DrJosh9000)

Fixed

• Fix origin for mirrored submodules #2144 (@DrJosh9000)
• Wipe checkout directory on git checkout and git fetch failure and retry #2137 (@triarius)

v3.47.0

v3.47.0 (2023-05-25)

Full Changelog

Two new and very noteworthy experiments!

1. Have you ever wanted to write hooks in a compiled language? Or in Python or Ruby? Well now you can! With --experiment=polyglot-hooks the agent can run all sorts of hooks and plugins directly. Combined with --experiment=job-api, your hooks-of-a-different-language can alter environment variables through the local Job API!
2. Concurrency groups are great, but have you ever wanted to manage multiple agents running on the same host concurrently accessing a shared resource? Well now you can! With --experiment=agent-api, the agent now has an inbuilt locking service, accessible through new lock subcommands and also via a Unix socket (like the job-api).

Added

• Experiment: Polyglot hooks #2040 (@moskyb)
• Experiment: Local Agent API, with locking service #2042 (@DrJosh9000)
• New flag --upload-skip-symlinks (on artifact upload) allows skipping symlinks when uploading files. --follow-symlinks has been deprecated and renamed to --glob-resolve-follow-symlinks #2072 (@triarius)

Fixed

• The normalised-upload-paths experiment was unintentionally left out of the available experiments list #2076 (@MatthewDolan)

Changed

• The git-mirrors experiment is promoted to full functionality #2032 (@moskyb)
• Errors in the git checkout process are now easier to diagnose #2074 (@moskyb)
• Alpine images updated to Alpine 3.18 #2098 (@moskyb)

v3.46.1

3.46.1 (2023-05-08)

Full Changelog

Fixed

• Avoid long --no-patch arg added to git show in v1.8.4, to e.g. support CentOS 7 #2075 (@pda)