Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark CI is down #3245

Closed
anonrig opened this issue Mar 21, 2023 · 9 comments
Closed

Benchmark CI is down #3245

anonrig opened this issue Mar 21, 2023 · 9 comments
Labels

Comments

@anonrig
Copy link
Member

anonrig commented Mar 21, 2023

It seems the benchmark machines are down or the job is wrong. I'm seeing the following log on the build history of https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/

(pending—All nodes of label ‘[benchmark-ubuntu1804-intel-64](https://ci.nodejs.org/label/benchmark-ubuntu1804-intel-64)’ are offline)
@targos
Copy link
Member

targos commented Mar 21, 2023

The benchmark machines are unreachable.

/cc @efrisby

@targos
Copy link
Member

targos commented Mar 22, 2023

We should setup these machines so that they restart automatically after a power outage, like the macs. It may require physical access to tweak the BIOS settings.

@targos
Copy link
Member

targos commented Mar 28, 2023

Ping @efrisby, @RafaelGSS

@efrisby
Copy link

efrisby commented Mar 29, 2023

Hi,

I will get those machines started tomorrow morning when I'm on site in HQ. I was on annual leave and just back a few days.

Thanks,
Eamonn

@efrisby
Copy link

efrisby commented Mar 30, 2023

Hi, those servers are now back online, we are putting a better alert notification in place, to make sure we don't miss this again. We will also investigate the power settings to see if they are in place, as we would have odd power failures from time to time and they would restart, it seems hit and miss, not sure if it is related to the age of the machines themselves and any hardware issues.

Can someone confirm who is the provider of those machines, can we look to replace perhaps with more modern hardware?

@richardlau
Copy link
Member

Can someone confirm who is the provider of those machines, can we look to replace perhaps with more modern hardware?

According to #791 those machines are Intel's. I wasn't in Build nor the TSC at the time. Maybe @mcollina or @mhdawson have more information.

@mcollina
Copy link
Member

mcollina commented Apr 3, 2023

Intel shipped those two machines to NearForm back in the day. Since then, those machines have been operated by NearForm, and there was no direct involvement by Intel. @efrisby manages them for NearForm (unless things changed).

Can someone confirm who is the provider of those machines, can we look to replace perhaps with more modern hardware?

This would require a donor to set things up. Given the availability of cheapish dedicated HW online (Hetzner sells for 50-100€/month, other providers have similar prices), foundation resources could be asked for.

Ultimately, I don't think having more modern HW matters in this case, as long as those machines are up. We do not want to measure maximum absolute speed, but rather do a comparison on the same HW.

Copy link

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

@github-actions github-actions bot added the stale label Jan 29, 2024
@richardlau
Copy link
Member

Let's move/keep discussions about whether replacements are needed to #3615.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants