githubrunner

What it does?

Runs a specific Nomad job matching the labels requested by pending GitHub actions jobs. Controls over the number of Nomad jobs depending on the number of pending or in progress GithHub actions jobs.

How it works?

• Every CONFIG.pool seconds
  • Get Github state:
    • For each configured CONFIG.repos
      • If the repository has no /
        • Get all repositories under this organization or user
      • else use the repository
    • For each such repository
      • Get all repository actions runs
      • For each action run
        • Get all action runs jobs
        • For each job that is queued or in_progress
          • add it to the list for later
  • Get Nomad state:
    • Get all Nomad jobs that start with CONFIG.nomad.jobprefix
    • The Nomad job info is extracted from automatically set metadata.
  • Apply scheduling decisions to scale the number of Nomad jobs
    • If there are not enough jobs associated with specific repo specific label
      • run a new job for this repo for this label
    • If there are too many jobs associated with specific repo specific label
      • stop these jobs
    • If CONFIG.nomad.purge and there are any stopped successful jobs
      • purge them

How to configure?

Use a YAML file, pass it to the -c command line option.

The default configuration can be listed with nomadtools githubrunner dumpconfig. You can use it also to test.

How is Nomad job template configured?

• The project uses Jinja2 templates and matches a list of labels joined with a comma with configured regex.
• When the regex matches, the string under the key is rendered as a template.
• All strings in the form a=v present in the labels split by using shlex.split are added variables available to the template.
• After rendering the template, it is loaded with a JSON or HCL.
• Then the name of the job is replaced with automatically generated one.
• Some predefined needed metadata are added to the job for identification porpouses.
• And the job is run.

The following variables are available in the template:

variable	what	example
run	A "global" variable with provided context.	{"JOB_NAME": "...", ....
run.JOB_NAME	The job name for the job. Max 64 character, no special chars. Use this for the runner name to easily identify which job runs which runner.	GITHUBRUNNER-1-user-repo-description-that-fix
run.ACCESS_TOKEN	Equal to CONFIG.github.token.
run.REPO_URL	The repository URL the runner will run for.
run.JOB_URL	The best efforted action run job URL that cause to run this job. May be empty and non-unique.
run.LABELS	Comma separated list of labels of the runner.
arg	A dictionary of arguments passed with labels

The command line tool rendertemplate can be used to check the resulting template value.

How to configure it?

Create a file config.yml with the following content:

---
github:
  token: the_access_token_or_set_GH_TOKEN_env_variable
repos:
  # for single repos
  - user/repo1
  - user/repo2
  # for all repositories of this user
  - user

Run the command line:

nomadtools githubrunner -c ./config.yml run

And it should start running.

How to use it from Github workflow?

The default runner detects all labels with regex nomadtools.*. Following text is split on spaces and parsed as variable assignments.

The default template takes following arguments:

argument	what	possible values
imge	myoung34/github-runner	anything
tag	The tag to use with myoung34/github-runner	see https://hub.docker.com/r/myoung34/github-runner/tags
docker	=dind runs docker in docker, =host mounts /var/run/docker.sock	dind host
cpu
mem
maxmem
debug

Examples:

   runs_on: nomadtools tag=latest mem=160000 docker=host
   runs_on: nomadtools tag=debian-sid mem=1000 maxmem=30000 docker=dind