fix packetbeat packaging - exclude darwin/arm64 package

[sharbuz]

What is the problem this PR solves?

Jenkins->Buildkite pipelines migration

Packetbeat packaging fix

How does this PR solve the problem?

Exclude darwin/arm64 package from the build because this platform is unsupported

Example of the error:
https://buildkite.com/elastic/beats-packetbeat/builds/1197#018d8e8a-c9cc-4f5f-8e2a-f26fc94ccc7c/421-511

>> Building using: cmd='build/mage-linux-amd64 buildGoDaemon', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, PLATFORM_ID=linux-amd64]
--
  | /tmp/ccgSaqPI.o: In function `main':
  | adb73258c8ab: Pull complete
  | Digest: sha256:b28467475b52ee1a5ea108c4bf7041c562b0cea9c065813dac587988a5f36437
  | Status: Downloaded newer image for docker.elastic.co/beats-dev/golang-crossbuild:1.21.7-darwin-debian10
  | >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
  | package ran for 1m44.93844247s
  | Error: unsupported cross build platform darwin/arm64
  | 🚨 Error: The command exited with status 1
  | user command error: exit status 1

Related issues

https://github.com/elastic/ingest-dev/issues/1693

[botelastic]

This pull request doesn't have a Team:<team> label.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @sharbuz? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 5 min 4 sec

Pipeline error

This error is likely related to the pipeline itself. Click here
and then you will see the error (either incorrect syntax or an invalid configuration).

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[v1v]

As far as I see in Jenkins the packaging pipeline for amd uses the below platforms:

• beats/.ci/packaging.groovy

  Lines 323 to 325 in e5000b6

  	'windows/amd64',
  	'darwin/amd64',
  	'darwin/arm64'

While for arm uses

beats/.ci/packaging.groovy

Line 259 in e5000b6

withEnv(["HOME=${env.WORKSPACE}", 'PLATFORMS=linux/arm64','PACKAGES=docker', "BEATS_FOLDER=${beat}"]) {

For instance, this build ran mage package snapshot packetbeat with these platforms: +all linux/amd64 linux/arm64 windows/amd64 darwin/amd64 darwin/arm64

[2024-02-11T09:11:29.919Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2024-02-11T09:12:40.781Z] >> golangCrossBuild: Building for darwin/arm64
[2024-02-11T09:12:40.781Z] Unable to find image 'docker.elastic.co/beats-dev/golang-crossbuild:1.21.7-darwin-arm64-debian10' locally
[2024-02-11T09:12:41.283Z] 1.21.7-darwin-arm64-debian10: Pulling from beats-dev/golang-crossbuild

What's the reason it works in Jenkins but it fails in Buildkite?

[sharbuz]

As far as I see in Jenkins the packaging pipeline for amd uses the below platforms:

• beats/.ci/packaging.groovy

  Lines 323 to 325 in e5000b6

  	'windows/amd64',
  	'darwin/amd64',
  	'darwin/arm64'

While for arm uses

beats/.ci/packaging.groovy

Line 259 in e5000b6

withEnv(["HOME=${env.WORKSPACE}", 'PLATFORMS=linux/arm64','PACKAGES=docker', "BEATS_FOLDER=${beat}"]) {

For instance, this build ran mage package snapshot packetbeat with these platforms: +all linux/amd64 linux/arm64 windows/amd64 darwin/amd64 darwin/arm64

[2024-02-11T09:11:29.919Z] >> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=o64-clang, CXX=o64-clang++, GOARCH=amd64, GOARM=, GOOS=darwin, PLATFORM_ID=darwin-amd64]
[2024-02-11T09:12:40.781Z] >> golangCrossBuild: Building for darwin/arm64
[2024-02-11T09:12:40.781Z] Unable to find image 'docker.elastic.co/beats-dev/golang-crossbuild:1.21.7-darwin-arm64-debian10' locally
[2024-02-11T09:12:41.283Z] 1.21.7-darwin-arm64-debian10: Pulling from beats-dev/golang-crossbuild

What's the reason it works in Jenkins but it fails in Buildkite?

I'm not sure we mean the same packaging.
In this case, means the mage package, and you mentioned the packaging.groovy pipeline
Also, I found that the mage package returns the mentioned error only in the 7.17 branch.

[v1v]

A bit more context about what package means will help with understanding how to review these PRs 🙏

In this case, means the mage package, and you mentioned the packaging.groovy pipeline

I don't understand those changes and can not see any errors in any Jenkins pipeline related to the package itself.

Please point out where this particular variable is used in the Jenkins pipeline I cannot find any references about cross-compile for packetbeat in https://github.com/elastic/beats/blob/main/packetbeat/Jenkinsfile.yml.

In addition, mage package should behave similarly in Jenkinsfile or <beat>/Jenkinsfile.yml and .ci/packaging.groovy, this platform configuration and this other one are equally the same and use when running mage package

Also, I found that the mage package returns the mentioned error only in the 7.17 branch.

If the error is only happening in 7.17, I wonder whether the fix should be done in that branch, as far as I see in

beats/Jenkinsfile

Lines 413 to 431 in 97682f1

	def packagingLinux(Map args = [:]) {
	def PLATFORMS = [ '+all',
	'linux/amd64',
	'linux/386',
	'linux/arm64',
	// armv7 packaging isn't working, and we don't currently
	// need it for release. Do not re-enable it without
	// confirming it is fixed, you will break the packaging
	// pipeline!
	//'linux/armv7',
	// The platforms above are disabled temporarly as crossbuild images are
	// not available. See: https://github.com/elastic/golang-crossbuild/issues/71
	//'linux/ppc64le',
	//'linux/mips64',
	//'linux/s390x',
	'windows/amd64',
	'windows/386',
	(params.macos ? '' : 'darwin/amd64'),
	].join(' ')

is not used , that's likely there are no failures in Jenkins for the 7.17.

[sharbuz]

opened another PR to 7.17, @v1v , let's discuss the changes in that PR.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6510c09

History

• 💚 Build #1230 succeeded b38a54c

cc @sharbuz

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6510c09

History

• 💚 Build #1555 succeeded b38a54c

cc @sharbuz

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6510c09

History

• 💚 Build #1225 succeeded b38a54c

cc @sharbuz

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6510c09

History

• 💚 Build #395 succeeded b38a54c

cc @sharbuz

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6510c09

History

• 💚 Build #2894 succeeded b38a54c

cc @sharbuz

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6510c09

History

• 💚 Build #2443 succeeded b38a54c

cc @sharbuz