Transformer in Transformer as Backbone for Deep Reinforcement Learning

Paper:
old version: Transformer in Transformer as Backbone for Deep Reinforcement Learning
new version: PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning. AAMAS 2024 (full paper with oral presentation)

Code:

1. The two folders, DT_TIT and PPO_TIT_and_CQL_TIT, contain the old version;
2. The two folders, RL_Foundation_Mujoco_including_DT_MGDT_GATO_and_TIT and RLFoundation_BabyAI_including_DT_GATO_and_TIT, contain the new version;
3. We recommend the readers to use the new version, which has a satisfactory performance and good file structure (thus, is easy to modify to design new algorithms). Thanks to Zhiwei Xu for the contribution of this new version.

Cite

Please cite our paper as:

@inproceedings{mao2024PDiT,
  title={PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning},
  author={Mao, Hangyu and Zhao, Rui and Li, Ziyue and Xu, Zhiwei and Chen, Hao and Chen, Yiqun and Zhang, Bin and Xiao, Zhen and Zhang, Junge and Yin, Jiangjin},
  booktitle={Proceedings of the 23rd International Conference on Autonomous Agents and MultiAgent Systems},
  year={2024}
}

and cite the preliminary study as:

@article{mao2022transformer,
  title={Transformer in Transformer as Backbone for Deep Reinforcement Learning},
  author={Mao, Hangyu and Zhao, Rui and Chen, Hao and Hao, Jianye and Chen, Yiqun and Li, Dong and Zhang, Junge and Xiao, Zhen},
  journal={arXiv preprint arXiv:2212.14538},
  year={2022}
}

License

MIT