vcmi-gym is a project which aims to create a gym-compatible reinforcement
learning environment for VCMI (the open-source recreation of
Heroes of Might & Magic III game) along with implementations of RL algorithms
and other supplementary code (orchestration, hyperparameter tuning, observability)
needed to produce VCMI combat AI models.
Models trained by vcmi-gym can be loaded in VCMI through the changes proposed in this pull request: vcmi/vcmi#4788. When the VCMI team accepts the contribution, vcmi-gym's models will become readily available in VCMI through the VCMI mod ecosystem.
Training new models is an ongoing effort and does not block players from using the already trained MMAI models once released, as they enhance gameplay experience by adding engaging and unpredictable behaviour of the enemy troops during battle.
A high-level overview of the vcmi-gym project is given in the below diagram:
The W&B external component is optional. VCMI (the game itself) is required --
it is a fork of VCMI with some modifications for RL training purposes and is
managed in a separate repo, tracked as a
git submodule located
at ./vcmi.
A step-by-step setup guide can be found below:
Please refer to this document for more information about the RL environment.
Please refer to this document for information about the RL training setup and tools used in this project.
Please refer to this document for information about the Connector component.
Fellow HOMM3 AI enthusiasts are more than welcome to help with this project. There is a lot of headroom for improvement, be it in the form of NN architectures, RL algorithm implementations, hyperparameter search, reward shaping, etc. The preferred approach is to submit a Pull request, but if you have stumbled upon a bug which you can't fix yourself, submitting an issue can help me (and others) fix it. You can also help with the ongoing AI training process by "plugging in" your own piece of hardware into the mix.
Please check for existing issues and verify that your issue is not already submitted. If it is, it's highly recommended to add to that issue with your reports.
When submitting a new issue, please be as detailed as possible - OS and Python versions, what did you do, what did you expect to happen, and what actually happened.
- Find an existing issue to work on or follow "Submitting an issue" to first create one that you're also going to fix. Make sure to notify that you're working on a fix for the issue you picked.
- Branch out from latest
mainand organize your code changes there. - Commit, push to your branch and submit a PR to
main.
If you have spare hardware and would like to help with this project, please reach out to me - together, we can ponder on putting it to good use: e.g. for training new models (GPU-bound task), evaluating existing models (CPU-bound task), or creating and rebalancing new training maps (CPU and HDD-bound task).