Installation

1. Optional: Install MuJoCo

   • If you are using Ubuntu you also need to install some requirements:

     sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3 patchelf

   • Keep in mind that the the most recent version of MuJoCo is not compatible with the most recent version of the OpenAI Gym. Therefore we use mjpro150 instead of mujoco200.

2. Setup the virtual environment

   • Clone the repository for the training results as well as the repository for the code

     git clone git@github.com:neuroevolution-ai/CTRNN_Simulation_Results.git
     git clone git@github.com:neuroevolution-ai/NeuroEvolution-CTRNN.git
     cd NeuroEvolution-CTRNN

   • Install dependencies and convenience tools. If you are running a Linux with a different package manager google the names of the packages for your system.

     sudo apt install cmake python3 python3-dev python3-pip swig python3-tk tmux

   • Create the virtual environment, activate it and install the required Python packages

     sudo pip3 install virtualenv --system
     virtualenv $HOME/.venv/neuro --python=python3
     . $HOME/.venv/neuro/bin/activate
     pip install --upgrade -r requirements.txt

   • Optional: If you are using MuJoCo the following steps are required

     export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mjpro150/bin
     pip install mujoco-py 'gym[mujoco]'

   • Optional: install custom envs

     sudo apt install qt5-default cmake
     python -m pip install git+https://github.com/neuroevolution-ai/procgen.git@3a5d86c3b89d362983c1a90df970b9173c0bd79b

Quick Start

1. Optional: Change the configuration nano configurations/default.json

2. Start the Training

   • Keep in mind that this will take some time, depending on your hardware and the configuration

   • Run the following commands

     tmux
     . $HOME/.venv/neuro/bin/activate`
     python neuro_evolution_ctrnn/train.py configurations/default.json

3. Visualize the results

   python neuro_evolution_ctrnn/render_hof.py results/data/2021-02-01_11-28-24

4. Optional: Publish the results

   • All results are stored in a git submodule which is located in a different folder

     cd ../CTRNN_Simulation_Results

   • After changing directories, pull recent changes and add the new results

     git pull --rebase && git add .

   • Now commit the results with a corresponding commit message and push them

     git commit -m "New simulation results" && git push

usage hints

CLI parameter

you can call all our scripts with the CLI parameter -h to show all available CLI parameters. For example:

$ python neuro_evolution_ctrnn/render_hof.py -h
pygame 2.0.0 (SDL 2.0.12, python 3.8.5)
Hello from the pygame community. https://www.pygame.org/contribute.html
usage: render_hof.py [--no-render] [--render-raw-ob] [--record] [--record-force] [--neuron-vis] [--hof HOF]
                     [--rounds ROUNDS] [--slow-down SLOW_DOWN] [--neuron-vis-width NEURON_VIS_WIDTH]
                     [--neuron-vis-height NEURON_VIS_HEIGHT] [-h]
                     dir

Visualize Experiments

positional arguments:
  dir                   (str, default=None) Directory path to the simulation result

optional arguments:
  --no-render           (bool, default=False) disable rendering to screen?
  --render-raw-ob       (bool, default=False) should the env be rendered from the perspective of the agent?
  --record              (bool, default=False) record rendering to store it to file?
  --record-force        (bool, default=False) force rendering even if file exists
  --neuron-vis          (bool, default=False) show neuron visualizer?
  --hof HOF             (int, default=1) how many members of hall-of-fame should be shown?
  --rounds ROUNDS       (int, default=1) how many episodes should be shown per HOF-member?
  --slow-down SLOW_DOWN
                        (int, default=0) how many milliseconds should be pause between time steps?
  --neuron-vis-width NEURON_VIS_WIDTH
                        (int, default=1600) how wide should the neuron_vis window be?
  --neuron-vis-height NEURON_VIS_HEIGHT
                        (int, default=900) how high should the neuron_vis window be?
  -h, --help            show this help message and exit

configuration file

For the configuration json-file, all available options are listed in neuro_evolution_ctrnn/tools/configurations.py. The contents of the json-file are directly mapped to objects in the configurations.py. The json-file as a whole especially is represented by the class ExperimentCfg.

Development

Tests and checks

. $HOME/.venv/neuro/bin/activate
python -m mypy .
PYTHONPATH=neuro_evolution_ctrnn pytest -q --disable-pytest-warnings tests

Tips and tricks

PyCharm

• For running mypy simply add a new Python configuration, choose Module name instead of Script path and use mypy as the module. Then add the absolute path to the directory containing the code as a parameter, like so:

  $HOME/PycharmProjects/NeuroEvolution-CTRNN_new/neuro_evolution_ctrnn

  (Assuming that you cloned the repository to the PycharmProjects folder.)

  Mypy can now be started by running the configuration.

• Another way can be to use PyCharm plugins

  • https://plugins.jetbrains.com/plugin/11086-mypy
    • Does not work for us, see: leinardi/mypy-pycharm#60
  • https://plugins.jetbrains.com/plugin/13348-mypy-official-
    • Config parameter Path suffix has to be set to the bin folder of the virtual environment
    • Run command: dmypy run . -- --follow-imports skip

• For running pytest add a Python tests -> pytest configuration. Select Custom as the target.

  Write tests in the Additional Arguments field. Then simply select the configuration and run it.

Generate statistics

python neuro_evolution_ctrnn/batch_generate_plot.py
cd ../CTRNN_Simulation_Results
python resuts_to_csv.py

output.csv can now be opened in LibreOffice Calc. The hyperlinks in the plot column do not get formatted when loading. LibreOffice will recognize the hyperlinks when you enter the cell, add a space to the end and leave the cell again. When you then click on the hyperlinks, a window with the plot should open.

Note: Make sure that the numbers are imported correctly. A comma can be a decimal separator, a thousands separator (e.g., 100,000,000) or a column separator. A dot can be a decimal separator and a thousands separator. Sometimes LibreOffice uses the dot as a thousands separator in some columns and as a decimal separator in other columns during the import of the same CSV.

Scrolling in tmux:

echo "set -g mouse on" >> $HOME/.tmux.conf
tmux kill-server && tmux

Parameters for the scripts

Every script in this repository that can be executed directly has a help function which can be executed by providing the --help parameter.

JSON formatting

All JSON files are formatted with the following style. Applying different formatting, or none at all, will result in a lot of clutter in the git commits.

For PyCharm one can change the formatting of JSON by going to:

File --> Settings --> Editor --> Code Style --> JSON --> Wrapping and Braces

Hardwrap at	120
Wrap on typing	yes
Visual guides	80, 120
Keep when reformatting
Line breaks	☑
Trailing comma	☐
Ensure right margin is not exceeded	☑
Arrays	Wrap if long
Objects	Wrap if long
Align	Do not align

visualizer

If it is not possible to install pygame via pip, try with apt-manager: sudo apt-get install python-pygame

Interaktion

You can individualize the visualizer via keyboard and mouse.

Troubleshooting

Visualization fails with "ERROR: GLEW initialization error: Missing GL version"

Solution: export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGLEW.so

Source: openai/mujoco-py#268 (comment)