Skip to content

2019.02
Compare
Choose a tag to compare
@ryanjulian ryanjulian released this 02 Mar 01:41
· 812 commits to master since this release
v2019.02.0
7fefd49

The Reinforcement Learning Working Group is proud to announce the 2019.02 release of garage.

We are actively seeking new contributors. If you use garage, please consider submitting a PR with your algorithm or improvements to the framework.

Summary

Please see the CHANGELOG for detailed information on the changes in this release.

Splitting garage into packages

Most changes in this released are focused on moving garage towards a modular future. We are moving the framework from a single monolithic repository to a family of independent Python packages, where each package serves a well-defined single purpose.

This will help garage have the widest impact by:

  • Allowing users to pick-and-choose which parts of the software fit well for their project, making using garage not an all-or-nothing decision
  • Making the the framework more stable, because smaller codebases are easier to test and maintain
  • Making it easier to introduce new frameworks (e.g. PyTorch) and features more easily, by forcing API separation between different parts of the software
  • Separating parts of the software at different maturity levels into different packages, making it easier for users to know which parts are stable and well-tested, and which parts are experimental and quickly-changing

In service of that goal, in this release we moved 3 packages to independent repositories with their own packages on PyPI (e.g. you can pip install <package>).

  • akro: Spaces types for reinforcement learning (from garage.spaces)
  • viskit: Hyperparamter-tuning dashboard for reinforcement learning experiments (from garage.viskit)
  • metaworlds: Environments for benchmarking meta-learning and multi-task learning (from garage.envs.mujoco and garage.envs.box2d)
  • gym-sawyer: Simulations and ROS bindings for the Sawyer robot, based on the openai/gym interface (from garage.envs.mujoco.sawyer and garage.envs.ros)

Deleting redundant or unused code

We've also started aggressively deleting unused code, or code where a better implementation already exists in the community. The largest example of this is MuJoCo and Box2D environments, many of which we removed because they have well-tested equivalents in openai/gym. Expect to find many other smaller examples in this and future releases.

Deleting Theano

We completed feature-parity between the Theano and TensorFlow trees, and deleted the Theano tree because we have not found any future interest in maintaining it. We made sure to port over all algorithms available in Theano to TensorFlow before making this change.

Preparing garage for PyTorch and other frameworks

We have started a full rewrite of the experiment definition, experiment deployment, snapshotting, and logging functionality in garage. This will allow new algorithm libraries or research projects to easily use garage tooling (e.g. logging, snapshotting, environment wrappers), irrespective of what numerical framework they use.

conda is now optional

While we still use conda in the CI environment for garage, we've moved all Python dependency information into a canonical setup.py file. While we are not releasing garage on PyPI yet, this means you can use any Python environment manager you'd like (e.g. pipenv, virtualenv, etc.) for your garage projects. In the future, we will add CI checks to make sure that the environment installs successfully in the most popular Python environment managers.

Primitives for pixel-based policies

We added CNN and wrapper primitives useful for pixel-based algorithms. Our implementation of DQN is forthcoming, since we are still benchmarking to make we can guarantee state-of-the-art performance.

Updated Docker support

We completely rewrote the garage Dockerfiles, added docker-compose examples for using them in your projects, and added a Makefile to help you easily execute your experiments using Docker (for both CPU and GPU machines). We use these Dockerfiles to run out own CI environment, so you can be sure that they are always updated.

Who should use this release, and how

Users who want to base a project on a semi-stable version of this software, and are not interested in bleeding-edge features should use the release branch and tags.

As always, we recommend existing rllab users migrate their code to a garage release ASAP.

Platform support

This release has been tested extensively on Ubuntu 16.04 and 18.04. We have also used it successfully on macOS 10.12, 10.13, and 10.14.

Maintenance Plan

We plan on supporting this branch until at least October 2019. Our support will come mostly in the form of attempting to reproduce and fix critical user-reported bugs, conducting quality control on user-contributed PRs to the release branch, and releasing new versions when fixes are committed.

We haven no intention of performing proactive maintenance such as dependency upgrades, nor new features, tests, platform support, or documentation. However, we welcome PRs to the maintenance branch (release-2019.02) from contributors wishing see these enhancements to this version of the software.

Hotfixes

We will post backwards-compatible hotfixes for this release to the branch release-2019.02. New hotfixes will also trigger a new release tag which complies with semantic versioning, i.e. the first hotfix release would be tagged v2019.02.1, the second would be tagged v2019.02.2, etc.

We will not add new features, nor remove existing features from the branch release-2019.02 unless it is absolutely necessary for the integrity of the software.

Next release

We hope to release 2-3 times per year, approximately aligned with the North American academic calendar. We hope to release next around early June 2019, e.g. v2019.06.

See Looking forward for more information on what to expect in the next release.

Looking forward

The next release of garage will focus primarily on two related goals: PyTorch support and completely-revamped component APIs. These are linked because gracefully supporting more than one framework requires well-defined interfaces for the sampler, logger, snapshotter, RL agent, and other components.

For TensorFlow algorithms development, we are focusing on adding a full suite of pixel-oriented RL algorithms to the TensorFlow tree, and on adding meta-RL algorithms to and associated new interfaces. We will also finish removing the custom layers library from the TensorFlow tree, and replacing it with code based on vanilla TensorFlow, a new abstraction called Model(inspired by the torch.nn.Module interface). We will also finish removing the custom garage.tf.distributions library and replacing it with fully-differentiable components from tensorflow-probability.

For PyTorch algorithms development, we hope to add garage support to a fork of rlkit, to prove the usefulness of our tooling for different algorithm libraries.

You can expect to see several more packages split from garage (e.g. the TensorFlow algorithm suite and experiment runner/sampler/logger), along with many API changes which make it easier to use those components independently from the garage codebase.

Contributors to this release

Assets 2
Loading