Skip to content

Commit

Permalink
rework readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@julianjandeleit
julianjandeleit committed Mar 31, 2024
1 parent 035210e commit 9f38adf
Showing 1 changed file with 38 additions and 26 deletions.
64 changes: 38 additions & 26 deletions ReadMe.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,20 @@
[![LinkedIn][linkedin-shield]][linkedin-url]

## About
`ai-cli` is a command line script, developed to simplify the execution of [MLflow Projects](https://www.mlflow.org/docs/latest/projects.html).
Whether you intend to run from a local folder structure or want to execute a remote project from a GitLab repository, both ways are supported in `ai-cli`.
Docker containers are used to manage the dependencies and isolate different projects from the host system.
`ai-cli` helps to manages these containers and provides additional functionality, like:

- Run MLflow projects with one command
- Manage project dependecies in differnet docker images
- CUDA support
- [Jupyter Lab](https://github.com/jupyterlab/jupyterlab) integration
- easy management of different [CML runners](https://github.com/iterative/cml) for differnet git repositories.
`ai-cli` is a command line script, developed to simplify the execution of machine learning projects.
Docker containers are used to manage dependencies and isolate different projects from the host system.

`ai-cli` helps to manages these containers and provides functionality, like:

- Manage project dependecies in differnet docker images
- Separate users
- [Jupyter Lab](https://github.com/jupyterlab/jupyterlab) integration
- Run MLflow projects with one command
- Management of [CML runners](https://github.com/iterative/cml) for differnet git repositories
- Conda support using Mamba
- GPU support with CUDA
- Service based domain names using nginx reverse proxy

## Setup

Expand All @@ -38,13 +42,13 @@ Note:

Open `/etc/ai-cli/config` and edit
- DVC_DATA
- MLFLOW_DATA (can contain ${USER} for user destinction)
- MLFLOW_DATA (should contain ${USER} for user destinction)
- CERTS_PATH (should contain .pem, .crt .key files for ssl)
- URLNAME (can be left as localhost if desired)
- DEFAULT_HOST (can be left as error.localhost if desired)
to reflect respective values of your system

Configure a first user like below and start reverse proxy: `ai-cli start-proxy`. _This needs to be repeated after reboot and must be invoked to reach exposed web services like jupyter._
Configure a first user like below and start reverse proxy: `ai-cli start-proxy`. _This needs to be invoked in order to reach exposed web services like jupyter._

**User Configuration**

Expand All @@ -61,42 +65,50 @@ The following needs to be done for _every_ new user in order to use `ai-cli`.
ai-cli [OPTIONS] COMMAND [ARGS...]

### Options

-v VOLUME Specify additional docker VOLUMEs
-e FILE Specify environment FILE
-g GPUS Enable gpu support. Set specific GPUS, e.g. 0,1 for gpu 0 and 1.
-r RUNTIME Specify specific RUNTIME. Default is read from docker info.
-n NAME Specify experiment NAME
-c CUDA Build with cuda support. Specify CUDA version to use. Tags image with 'c<VERSION>' by default.
-i IMAGE Specify docker IMAGE to be used.
-t TAG Specify image TAG to be used.
-h Show help

### Commands

init initialize this script for your user
start-server start mlflow server
stop-server stop mlflow server
restart-server restart mlflow server
start-mlflow start mlflow server
stop-mlflow stop mlflow server
restart-mlflow restart mlflow server
start-proxy start nginx reverse proxy
stop-proxy stop nginx reverse proxy
status show status of your environment
build build image from dockerfile
bash start bash shell inside container
notebook start jupyter notebook server
lab start jupyter lab server
runner start cml runner for repo
exec PROGRAMM execute PROGRAMM inside container
run PATH run MLProject from PATH inside container
run-from-git URI run MLProject from git repo at URI inside container
info show basic paths and env variables
version show git commit hash this script is installed with
reset-password Reset Password for mlflow you will setup a new one at next server start

### Workflow

After a fresh install, each user needs to execute `ai-cli init` before starting to work.
To use the mlflow dashboard, the mlflow server needs to be started using `ai-cli start-server`.
### Overview

`ai-cli` is based on docker images/containers. The list of images associated to a user can be seen with `ai-cli status`.
The image ai-cli/<USER>:default is used by default.
The image ai-cli/<USER>:workspace is used by default.
A different associated image can selected by specifying the -t flag.
All below commands need to be executed as the user that should use the respective feature.

After a fresh install, each user needs to execute `ai-cli init` before starting to work.
To use the mlflow dashboard, the mlflow server needs to be started using `ai-cli start-mlflow`.
To use the jupyter lab environmen start the jupyter server using `ai-cli lab`.
For debugging and local execution, a bash shell can be entered using `ai-cli bash`.
Hooks can be injected as .sh-files for and after starting a container at `/etc/ai-cli/hooks`. An example is provided there.
Conda environments is supported via Mamba by default and are mounted on startup.

At the start, no image exists. A basic image get created using `ai-cli build`.
Now `ai-cli` is ready to be used, examples are provided below.

### Example commands
Expand All @@ -106,7 +118,7 @@ Assume a mlflow project at current working directory:
| comand | comment |
|-------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
| `ai-cli lab` | opens jupyter lab at https://[USER]-lab.[HOST]. Use token from command output to login. |
| `ai-cli run .` | runs local mlflow project |
| `ai-cli start-mlflow` | opens mlflow server at https://[USER]-mlflow.[HOST]. Can be secured with password |
| `ai-cli bash` | open bash for local project in docker image. |
| --- legacy --- | --- --- |
| `ai-cli -e .env run .` | with .env as defautl environment file |
Expand All @@ -120,9 +132,9 @@ To use cuda, a cuda-enabled image is necessary.
To create a cuda image, use the build command and specify the desired cuda version using the `-c` switch.
`ai-cli -c 11.2 build`.

For the cuda image to work, add the `-g` option when using any command to actually include gpus that cuda can use.
For the cuda container to use the gpu, add the `-g` option when using any command to actually include gpus that cuda can use.
This makes the specified gpu(s) available and switches the runtime to "nvidia".
Note that, the docker-nvidia-runtime needs to be set up correctly beforehand.
Note, that docker-nvidia-runtime needs to be set up correctly beforehand.

For example `ai-cli -g 0 -c 11.2 bash` opens a bash command line with inside the container, exposing gpu "0" with support for cuda 11.2. Note, that the images for cuda 11.2 need to be built beforhand as stated above.
It can be verified that the gpu is used, using `nvidia-smi` inside the container. Note, that the cuda version shown by nvidia-smi does not necessarily represent the correct version because of differences between driver API and runtime API version.
Expand Down

0 comments on commit 9f38adf

Please sign in to comment.