ARM support is a must for cLab in 2023

[ankudinov]

According to the installation docs ARM is currently not supported by Containerlab.
The reasons are explained in #927
Nevertheless it's not valid in 2023 any more. There is quite some work around this as more users are getting ARM based MAC laptops and this can hit at scale very soon.
I'd say an assumption that there will be only x86 in the field is incorrect now and every container build must be adjusted to multi-platform.

Normally it's as easy as:

# setup qemu
  - name: setup QEMU
    uses: docker/setup-qemu-action@v2
    with:
      platforms: "linux/arm64/v8,linux/amd64"
  # setup buildx
  - name: setup buildX
    uses: docker/setup-buildx-action@v2

I assume, that would be very similar for a bit more advanced clab build process.
Please do not close this issue as there is a number of Containerlab users that will be desperately waiting for ARM compatible release.

[hellt]

Hi
I am on ARM64 system myself and I am curious to know which network OSes currently have ARM64 images?

[ankudinov]

None yet. But it's time to start preparing cLab for that. There are some options to compile ARM images internally and it's only a question of time when it will become more critical. ARM-based MACs are becoming more popular and we have already started bringing our community containers, like Ansible AVD up to date. Containerlab has to lead here, as it will be quite sad to have an ARM compatible container and not being able to use cLab with it.
And some community members will be happy to help with testing once ARM-compatible cLab will be released.
Is there any cLab internals that can be impacted and have to be tested or updating the build workflow will be sufficient?

[marcosfsch]

At least cRPD has an ARM64 image and it’s possible to build SONiC and RARE/freeRtr ARM64 images too.

[hellt]

We are open for pull requests. Last time I checked building Mac OS compatible arm images required developer account to be purchased.

Containerlab uses goreleaser build system, contributors are welcome trying building any Go project with it for Mac OS Arm system and see if it works (it wasn't in the past due to apple's signing process)

[ankudinov]

Fair. Let me test a custom build first. I'll certainly come back with questions.

[hellt]

Also I think the way it has a chance to work is by creating a container image with linux/arm64.
Building a binary for darwin/arm64 is not possible, since, for example, netlink is not available on darwin which we use to talk to kernel for interface provisioning.

So at best, we can only build a container that will run nicely on M1 using docker desktop or rancher or similar virtualization framework

[ankudinov]

So, after some initial testing I have to admit, that I was too fast in opening an issue for this.

1. From my side there is no expectation that Containerlab will work outside of the container on MacOS. In fact, I personally prefer to run cLab as D-in-D or Docker-outside-Docker on any machine, including Linux.
2. I've created a simple test with 2 alpine containers and do not see any issues when running that on ARM. There can be many caveats not exposed by this test though, but at least cLab itself runs. I guess the only real issue here is that cLab container is not multi-platform. That can be changed, but I personally do not use it and prefer VSCode devcontainers. So, it's not hard to craft a custom container with cLab.
3. It is significantly harder to do any advanced testing currently. That can only be done when there will be images available for public use as stated by @hellt

My conclusions:

• Containerlab can run on M1/M2 Mac laptops. It may not be supported, but there is technical possibility. There is not need to do any cLab changes, except may be mentioning custom container approach in the docs.
• Any other issues have to be specific, based on some failed tests.

[ankudinov]

Also while it runs well on x86 platforms, the performance impact on ARM isn't clear without better testing.

INFO[0000] Adding containerlab host entries to /etc/hosts file 
+---+-------------------------------+--------------+---------------+-------+---------+----------------+--------------+
| # |             Name              | Container ID |     Image     | Kind  |  State  |  IPv4 Address  | IPv6 Address |
+---+-------------------------------+--------------+---------------+-------+---------+----------------+--------------+
| 1 | clab-simple-clab-on-arm-node1 | 3190d43671c3 | alpine:latest | linux | running | 192.168.0.2/24 | N/A          |
| 2 | clab-simple-clab-on-arm-node2 | db8e0b44a7c3 | alpine:latest | linux | running | 192.168.0.3/24 | N/A          |
+---+-------------------------------+--------------+---------------+-------+---------+----------------+--------------+
vscode ➜ /workspaces/clab-arm-test (master) $ sysctl kernel.version
kernel.version = #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023
vscode ➜ /workspaces/clab-arm-test (master) $ uname -a
Linux 50812784f215 5.15.49-linuxkit-pr #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023 aarch64 GNU/Linux