Skip to content

Latest commit

 

History

History

sim2real

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
envs
envs
 
 
README.md
README.md
 
 
TD3.py
TD3.py
 
 
test.py
test.py
 
 
train.py
train.py
 
 
utils.py
utils.py
 
 

README.md

Error in user YAML: (<unknown>): did not find expected key while parsing a block mapping at line 1 column 1
---
id: sim2real
title: Running sim2real for arm
sidebar_label: [Advanced] Sim2Real
---

Note This tutorial is tested to work only with Python2.7 version on PyRobot

What is Sim2Real?

In this section we will train a simple RL policy which will learn inverse kinematics for the arm using RL(TD3). The input to the RL agent is state(joint angles of arm) & goal location(x,y,z) and the control action is the change in each joint angles to achieve the desired goal.

For TD3 implementation , we used the publicly available code.

Here is a demo video showing what one can accomplish through this tutorial.

<iframe class="doc_vid" src="https://www.youtube.com/embed/oE6YI4lSCgE" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

Installation

  • Activate the virtual environment
source ~/pyenv_pyrobot/bin/activate
  • Install Dependencies
pip install gym
pip install pybullet
  • Start by either training the agent from scratch or downloading pretrained model

Train the agent (this might take around an hour)

cd ~/low_cost_ws/src/pyrobot/examples/sim2real
python train.py --exp_name expID01 --valid_goals --save_models

Download pretrained model

cd ~/low_cost_ws/src/pyrobot/examples/sim2real
wget -O log.zip https://www.dropbox.com/s/8cttp55odd7e79o/log.zip?dl=0
unzip log.zip
rm log.zip
  • Launch the real robot (open a new terminal）
roslaunch locobot_control main.launch use_arm:=true
  • Test the above trained policy on the real robot
source ~/pyenv_pyrobot/bin/activate
cd ~/low_cost_ws/src/pyrobot/examples/sim2real
python test.py --use_real_robot --valid_goals --directory ./log/expID01/pytorch_models --file_name LocoBotEnv-v0_0

Acknowledgments

The TD3 implementation we have used publicaly available code for Addressing Function Approximation Error in Actor-Critic Methods paper.

@article{DBLP:journals/corr/abs-1802-09477,
 author    = {Scott Fujimoto and
              Herke van Hoof and
              Dave Meger},
 title     = {Addressing Function Approximation Error in Actor-Critic Methods},
 journal   = {CoRR},
 volume    = {abs/1802.09477},
 year      = {2018},
 url       = {http://arxiv.org/abs/1802.09477},
 archivePrefix = {arXiv},
 eprint    = {1802.09477},
 timestamp = {Mon, 13 Aug 2018 16:47:21 +0200},
 biburl    = {https://dblp.org/rec/bib/journals/corr/abs-1802-09477},
 bibsource = {dblp computer science bibliography, https://dblp.org}
}