[CI] Package elastic agent job #38011

pazone · 2024-02-14T14:28:09Z

Proposed commit message

Externally triggered job for elastic-agent packaging and publishing

Our release team is going to trigger this job from their pipeline

Relates #2644

elasticmachine · 2024-02-14T14:42:08Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-03-14T18:57:13.970+0000
Duration: 130 min 27 sec

Test stats 🧪

Test	Results
Failed	0
Passed	20613
Skipped	1421
Total	22034

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-03-11T16:50:11Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

cmacknz · 2024-03-11T20:38:14Z

x-pack/elastic-agent/magefile.go

+					os.Setenv(snapshotEnv, "true")
+					mage.Snapshot = true
+				}
+				os.Setenv("BEAT_VERSION", parsedVersion.CoreVersion())


Why does the 7.17 branch need to set the version from a manifest file? Can't it always use the version in the branch?

Do you mean to hardcode 7.17?

No I don't understand what this manifest file block is doing. Is this something that applies to all DRA built projects?

Mostly I want to double check that this isn't something from the elastic-agent repository that was done support early agent releases that isn't actually needed here. The early agent releases work flow can't work for 7.17 as it is missing the changes the agent itself needed to support it.

It was introduced before the faster agent release project was started. If I understand correctly, it get the version from a provided manifest(specified in build params) and just sets it to BEAT_VERSION

Got it, it came from here and is used to know which dependency versions to download. elastic/elastic-agent#2727

I still don't really understand why it sets BEAT_VERSION since I doubt that ever doesn't match the version in the branch.

.buildkite/x-pack/elastic-agent/pipeline.xpack.elastic-agent.package.yml

dev-tools/mage/artifacts/artifacts_api.go

dev-tools/mage/crossbuild.go

dev-tools/mage/manifest/manifest.go

dev-tools/mage/manifest/manifestspecs.go

dev-tools/mage/settings.go

belimawr · 2024-03-12T15:09:56Z

dev-tools/mage/version/version_parser.go

+	"strings"
+)
+
+// regexp taken from https://semver.org/ (see the FAQ section/Is there a suggested regular expression (RegEx) to check a SemVer string?)


Would github.com/Masterminds/semver work here? Could it be used instead of this regexp to parse the semver?

I think it would be better to use the semver. I just brought this code from the elastic-agent repo just for consistency

We can move this code from elastic-agent to elastic-agent-libs so you can just import it instead of copying it.

dev-tools/mage/version/version_parser.go

belimawr · 2024-03-12T15:21:23Z

x-pack/elastic-agent/magefile.go

+	if devtools.PackagingFromManifest {
+		fmt.Println(">>>> Using manifest to package Agent")
+		if manifestResponse, err := manifest.DownloadManifest(devtools.ManifestURL); err != nil {
+			log.Panicf("failed to download remote manifest file %s", err)


Why panic here instead of returning an error?

I see we often use panic in our magefiles instead of returning an error. Honestly, I don't know why we do that. Perhaps for simplification

elasticmachine · 2024-03-14T18:59:45Z

💚 Build Succeeded

Buildkite Build
Commit: 14dcba0

History

💚 Build #902 succeeded 966a763
💚 Build #900 succeeded 6dc5e06

cc @pazone

elasticmachine · 2024-03-14T18:59:57Z

💔 Build Failed

Buildkite Build
Commit: 14dcba0

Failed CI Steps

:linux: Load dynamic x-pack metricbeat pipeline

History

💔 Build #915 failed 966a763
💔 Build #913 failed 6dc5e06

cc @pazone

elasticmachine · 2024-03-14T19:00:04Z

💚 Build Succeeded

Buildkite Build
Commit: 14dcba0

History

💚 Build #905 succeeded 966a763
💚 Build #903 succeeded 6dc5e06

cc @pazone

elasticmachine · 2024-03-14T19:03:56Z

💔 Build Failed

Buildkite Build
Commit: 14dcba0

Failed CI Steps

History

💔 Build #2474 failed 966a763
💔 Build #2472 failed 6dc5e06
💔 Build #2260 failed a65cb8d
💔 Build #2253 failed 8374cd1

cc @pazone

elasticmachine · 2024-03-14T19:04:51Z

💔 Build Failed

Buildkite Build
Commit: 14dcba0

Failed CI Steps

History

💔 Build #3820 failed 966a763
💔 Build #3818 failed 6dc5e06
💔 Build #3601 failed a65cb8d
💔 Build #3594 failed 8374cd1

cc @pazone

elasticmachine · 2024-03-14T19:12:50Z

💔 Build Failed

Buildkite Build
Commit: 14dcba0

Failed CI Steps

History

💔 Build #945 failed 6dc5e06

cc @pazone

elasticmachine · 2024-03-14T19:13:43Z

💔 Build Failed

Buildkite Build
Commit: 14dcba0

Failed CI Steps

History

💔 Build #2182 failed 6dc5e06
💔 Build #1970 failed a65cb8d
💔 Build #1963 failed 8374cd1
💔 Build #1900 failed c577cb9

cc @pazone

elasticmachine · 2024-03-14T19:16:54Z

💔 Build Failed

Buildkite Build
Commit: 14dcba0

Failed CI Steps

:go: Go Integration Tests

History

💚 Build #2175 succeeded 6dc5e06
💔 Build #1963 failed a65cb8d
💚 Build #1956 succeeded 8374cd1
💚 Build #1893 succeeded c577cb9
💚 Build #1884 succeeded a5dd15a

cc @pazone

elasticmachine · 2024-03-14T19:27:19Z

💚 Build Succeeded

Buildkite Build
Commit: 14dcba0

History

💚 Build #1338 succeeded 6dc5e06
💚 Build #1126 succeeded a65cb8d
💚 Build #1119 succeeded 8374cd1
💚 Build #1056 succeeded c577cb9
💚 Build #1047 succeeded a5dd15a
💚 Build #1023 succeeded 3a07c72

cc @pazone

elasticmachine · 2024-03-14T19:38:20Z

💚 Build Succeeded

Buildkite Build
Commit: 14dcba0

History

💚 Build #3393 succeeded 6dc5e06
💔 Build #3177 failed a65cb8d
💔 Build #3170 failed 8374cd1
💔 Build #3107 failed c577cb9
💔 Build #3098 failed a5dd15a
💔 Build #3074 failed 3a07c72

cc @pazone

pazone · 2024-03-18T15:30:46Z

@belimawr @cmacknz most of the code is just copied here from 8.x. I fixed as many lint issues as I could

DaveSys911 · 2024-03-19T09:35:45Z

@cmacknz Apologies for intruding, this PR is a blocker for the unified release 7.x migration.
Any chance we can expedite the review on this and fix noncritical issues in subsequent prs?

cmacknz · 2024-03-19T19:41:53Z

This was merged with the check step failing, several of the files use the wrong license type for Beats and are showing up in other PRs. https://github.com/elastic/beats/pull/38432/files

[CI] Package elastic agent job

4c47fcc

pazone added the enhancement label Feb 14, 2024

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 14, 2024

mergify bot assigned pazone Feb 14, 2024

pazone added the Team:Elastic-Agent Label for the Agent team label Feb 14, 2024

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 14, 2024

pazone added 23 commits February 22, 2024 15:40

Packaging

714155d

Use own images

e95c73c

Set mage verison

6261bdc

Set mage verison

eccc68f

Added wait step

7766e8c

Added wait step

d423661

Added wait step

7014d57

Added package group

750460c

Added package group

1091f85

Added package group

64c72fd

Added package group

b15e604

Added package group

127c0ee

Added package group

1110f2a

Added dependencies report

dec06a6

publish step

aadc408

publish step

6907a88

publish step

a6a31ac

publish step

41aa643

Fixed artifacts paths

f626acb

Fixed artifacts paths

8637c8a

Fixed artifacts paths

717342f

test: use artifacts from previous build

87cd526

test: use artifacts from previous build

8fedace

pazone requested a review from a team as a code owner March 11, 2024 16:50

pazone requested review from AndersonQ and faec and removed request for a team March 11, 2024 16:50

pazone requested review from belimawr and DaveSys911 and removed request for a team March 11, 2024 16:50

cmacknz reviewed Mar 11, 2024

View reviewed changes

belimawr reviewed Mar 12, 2024

View reviewed changes

pazone added 3 commits March 14, 2024 18:58

Fixes

6dc5e06

Fixes: lint

966a763

Fixes: lint

14dcba0

AndersonQ removed their request for review March 18, 2024 13:28

pazone merged commit 2bb2fcb into 7.17 Mar 19, 2024
81 of 92 checks passed

pazone deleted the packaging_elastic_agent branch March 19, 2024 10:15

cmacknz mentioned this pull request Mar 19, 2024

[7.17] Fix license headers in dev-tools #38443

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] Package elastic agent job #38011

[CI] Package elastic agent job #38011

pazone commented Feb 14, 2024 •

edited

Loading

elasticmachine commented Feb 14, 2024 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Mar 11, 2024

cmacknz Mar 11, 2024

pazone Mar 12, 2024

cmacknz Mar 12, 2024

pazone Mar 14, 2024

cmacknz Mar 18, 2024

belimawr Mar 12, 2024

pazone Mar 14, 2024

cmacknz Mar 18, 2024

belimawr Mar 12, 2024

pazone Mar 14, 2024

elasticmachine commented Mar 14, 2024

elasticmachine commented Mar 14, 2024

elasticmachine commented Mar 14, 2024

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024

elasticmachine commented Mar 14, 2024

pazone commented Mar 18, 2024

DaveSys911 commented Mar 19, 2024

cmacknz commented Mar 19, 2024

[CI] Package elastic agent job #38011

[CI] Package elastic agent job #38011

Conversation

pazone commented Feb 14, 2024 • edited Loading

Proposed commit message

elasticmachine commented Feb 14, 2024 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Mar 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Mar 14, 2024

💚 Build Succeeded

History

elasticmachine commented Mar 14, 2024

💔 Build Failed

Failed CI Steps

History

elasticmachine commented Mar 14, 2024

💚 Build Succeeded

History

elasticmachine commented Mar 14, 2024 • edited Loading

💔 Build Failed

Failed CI Steps

History

elasticmachine commented Mar 14, 2024 • edited Loading

💔 Build Failed

Failed CI Steps

History

elasticmachine commented Mar 14, 2024 • edited Loading

💔 Build Failed

Failed CI Steps

History

elasticmachine commented Mar 14, 2024 • edited Loading

💔 Build Failed

Failed CI Steps

History

elasticmachine commented Mar 14, 2024 • edited Loading

💔 Build Failed

Failed CI Steps

History

elasticmachine commented Mar 14, 2024

💚 Build Succeeded

History

elasticmachine commented Mar 14, 2024

💚 Build Succeeded

History

pazone commented Mar 18, 2024

DaveSys911 commented Mar 19, 2024

cmacknz commented Mar 19, 2024

pazone commented Feb 14, 2024 •

edited

Loading

elasticmachine commented Feb 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading

elasticmachine commented Mar 14, 2024 •

edited

Loading