-
Notifications
You must be signed in to change notification settings - Fork 299
Run Leela Chess Zero clients on Google Cloud (10 minute guide)
This guide will allow you to have Leela Chess Zero clients running in the cloud in 10 minutes or less. These clients will run self-play training games and help make Leela stronger. This guide is aimed at everyone and assumes no technical understanding.
If you've never had Google Cloud account before then this is something you can do for free. (For more information about the free trial click here)!
Note, that if you're on the free trial you will NOT be charged if you run out of credits and forget to stop the trial (see 'Do I pay anything for the free trial?' here).
Create a free account on Google Cloud Platform.
Once you've created an account, navigate to Google Cloud Compute. In the interface, this is also listed as "Compute Engine."
Here you should have the option to create a VM instance. Click Create. On the Create an instance page you have many options. You can ignore most of these for now. Here are the ones you need to modify:
- Machine type: Here you can select your memory and number of CPUs. Click on the dropdown and select 8 vCPUs / 7.2GB memory / n1-highcpu-8. This will change the number of cores (CPUs) to 8 (the free trial maximum), and the allotted memory to 7.2 GB (the minimum).
Note that you get $300 in credit free for the period of the free trial. With these settings, your virtual machine will cost about $150 per month, meaning the free trial will help you train Leela for about two whole months!
- Boot disk: Select 'Ubuntu 16.04 LTS'
Now click on the 'Management, disks, networking, SSH keys' drop-down. Under 'Startup script' copy and paste the following:
#! /bin/bash
mkdir /lczero
wget https://github.com/glinscott/leela-chess/releases/download/v0.7/client_linux -P /lczero/
chmod +x /lczero/client_linux
git clone https://github.com/glinscott/leela-chess.git /lczero/leela-chess
apt-get -qq update
apt-get install -y cmake g++ libboost-all-dev libopenblas-dev opencl-headers ocl-icd-libopencl1 ocl-icd-opencl-dev zlib1g-dev
sed -i '0,/\*\//s//\*\/ #define FEATURE_USE_CPU_ONLY/' /lczero/leela-chess/src/config.h
make -j -C /lczero/leela-chess/src/
cp /lczero/leela-chess/src/lczero /lczero/lczero
And that's most of the hard work done. Now grab yourself a drink while the VM boots up and the startup script runs (which should take around 3-5 minutes).
Once your instance has built itself, you can now SSH into the VM. To do this, click on the 'SSH' button on the 'VM Instances' page. This should open up a terminal window for you where you're able to communicate with your VM. If you're logging into the instance for the first time then there is one thing you need to do.
- Run the command
sudo cp /lczero/* .(with the full stop included).
If you get the error: lczero: no such file or directory, then the script probably hasn't finished running. If this happens then just wait a couple of minutes and run the commands like you're logging in for the first time again.
First, to check that everything is working, run `./client_linux'. You will be prompted to create a username and password. This username will allow you to track the number of games you've contributed on the main page.
Wait a few seconds, and you'll see some startup text appear, and then after a few more seconds you'll see a line that begins "info depth..." and contains a list of chess moves. If you see this, then everyone is working properly. Success!
You'll now want to start the client persistently, so you can exit the SSH window and the the client will play training games without any input from you. To do this, first exit the current client by pressing control-C.
To start the client running persistently, run nohup ./client_linux &. This will run the client in the background so you'll be able to close the terminal session and the client will continue to run. Each time you run this command a new client will start running and on your VM. With the above settings you will be able to run 7 clients simultaneously, so enter the command 6 more times.
To see how many clients you have running you can use the command jobs -l. This should show you the clients along with their ID.
To stop any of these clients use kill JOB_ID where JOB_ID is the 4-5 digit ID found when using jobs -l. (If you have reconnected after logging out use ps ax | grep client_linux instead.)
You can monitor your VMs by double clicking on your instance name on the 'VM instances' page. This will direct you to the 'VM Settings' page where you can see how much of your instances CPU is being used up (along with many other things). With 7 clients, you should be at about 90% CPU.
After about 20-30 minutes, games from your clients will complete. After this happens, you'll be able to see the number of games you've contributed on the main page, lczero.org. To do this, navigate to the main page and ctrl-f for the username you entered earlier.
That's it! You can close all the windows and go on with your life. You will continue contributing training games until your free trial runs out.
Occasionally, bugs or client version updates will require the client to be restarted. You can check to see if this is needed by monitoring the main page. At the time of writing, the client is on version 6. If a new version is released, simply delete the current instance on Google Cloud and repeat the steps above.
Advanced users who want to maximize their Google Cloud credits will want to consider using "preemptible" instances, which Google offers for an ~80% discount. The downside is that Google can turn off these instances when they please, and each instance will only run for a maximum 24 hours before being turned off. However, restarting the preemptible instance after a shutdown is a 1-click operation (or one command using Google Cloud Shell), and a cron job to automatically restart the Leela client on reboot should be simple to set up.
The scripts above can also be used with Azure, Microsoft's version of Google Cloud. Azure has $200 in free trial credits for new users, but be aware that they expire in 1 month.
Warning: It appears Microsoft will disable Azure accounts for TOS violation for constant 100% CPU usage. This has not been reported with Google Cloud. If you want to try Azure even with this warning, try limiting overall CPU usage to <=25% per server (one client on a 4-core server) to avoid detection.
The following is a simple guide to Azure's interface:
- Go to the Azure dashboard after signing up.
- Click "Create a Resource". Search for "Ubuntu 16.04 LTS," select it, and click Create.
- Put in a random name, user/password of your choice, resource group name of your choice, and region.
- Choose F- or F_v2-series server (these have the fastest CPU).
- Click OK for optional features tab, and click Create to deploy.
- Once deployed, click the VPS tile, copy IP address, and use an SSH client like PuTTY to login to your server:
ssh <user>@<ip>. Once logged in, typesudo -sto become the root user. Open the nano text editor and create a file called "script" withnano script. Using right-click menu (not keyboard shortcut), Paste the Google Cloud "Startup script" in the first section above into nano,CTRL-Xto exit andYto save. Type./scriptto run the script and install everything. Then, everything proceeds as above in the "Starting Leela Chess Zero" section.