A Survey on Large Language Model-Based Game Agents

🔥 Must-read papers for LLM-based Game agents.

💫 Continuously update on a weekly basis. (last update: 2024/10/27)

Content

• A Survey on Large Language Model-Based Game Agents
  • Adventure Games
  • Crafting & Exploration Games
  • Simulation Games
  • Competition Games
  • Cooperation Games
  • Communication (Conversational) Games
  • Action Games

Adventure Games

Text Adventure Games

• [2019/09] Interactive Fiction Games: A Colossal Adventure AAAI 2020 [paper] [code]
• [2020/10] ALFWorld: Aligning Text and Embodied Environments for Interactive Learning ICLR 2021 [paper][code]
• [2022/03] ScienceWorld: Is your Agent Smarter than a 5th Grader? EMNLP 2022 [paper] [code]
• [2022/10] ReAct: Synergizing Reasoning and Acting in Language Models ICLR 2023 [paper] [code]
• [2023/03] Reflexion: Language Agents with Verbal Reinforcement Learning NeurIPS 2023 [paper] [code]
• [2023/04] Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions arXiv [paper]
• [2023/05] SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks NeurIPS 2023 [paper] [code]
• [2023/10] FireAct: Toward Language Agent Fine-tuning arXiv [paper][code]
• [2023/11] ADaPT: As-Needed Decomposition and Planning with Language Models arXiv [paper][code]
• [2024/02] Soft Self-Consistency Improves Language Model Agents arXiv [paper][code]
• [2024/02] Empowering Large Language Model Agents through Action Learning arXiv [paper][code]
• [2024/03] KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents arXiv [paper][code]
• [2024/03] Language Guided Exploration for RL Agents in Text Environments arXiv [paper][code]
• [2024/03] Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents ACL 2024 [paper][code]
• [2024/04] Learning From Failure: Integrating Negative Examples When Fine-tuning Large Language Models as Agent arXiv[paper][code]
• [2024/04] ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy [paper]
• [2024/05] Agent Planning with World Knowledge Model arXiv [paper][code]
• [2024/05] THREAD: Thinking Deeper with Recursive Spawning arXiv [paper]
• [2024/06] Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement arXiv [paper][code]
• [2024/06] STARLING: Self-supervised Training of Text-based Reinforcement Learning Agent with Large Language Models arXiv [paper][code]

Video Adventure Games

• [2023/09] Motif: Intrinsic Motivation from Artificial Intelligence Feedback ICLR 2024 [paper] [code]
• [2024/03] Cradle: Empowering Foundation Agents Towards General Computer Control arXiv [paper][code]
• [2024/03] Playing NetHack with LLMs: Potential & Limitations as Zero-Shot Agents arXiv [paper] [code]

Crafting & Exploration Games

MineCraft

• [2023/02] Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents NeurIPS 2023 [paper][code]
• [2023/03] Plan4MC: Skill Reinforcement Learning and Planning for Open-World Minecraft Tasks FMDM@NeurIPS2023 [paper][code]
• [2023/05] Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory arXiv [paper]
• [2023/05] VOYAGER: An Open-Ended Embodied Agent with Large Language Models FMDM@NeurIPS2023 [paper][code]
• [2023/10] LLaMA Rider: Spurring Large Language Models to Explore the Open World arXiv [paper][code]
• [2023/10] Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds ICLR 2024 [paper]
• [2023/11] JARVIS-1: Open-world Multi-task Agents with Memory-Augmented Multimodal Language Models arXiv [paper][code]
• [2023/11] See and Think: Embodied Agent in Virtual Environment arXiv [paper][code]
• [2023/12] MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception CVPR 2024 [paper][code]
• [2023/12] Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft arXiv [paper]
• [2023/12] Creative Agents: Empowering Agents with Imagination for Creative Tasks arXiv [paper][code]
• [2024/02] RL-GPT: Integrating Reinforcement Learning and Code-as-policy arXiv [paper]
• [2024/03] MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control arXiv [paper][code]
• [2024/07] Odyssey: Empowering Agents with Open-World Skills. arXiv [paper][code]

Crafter

• [2023/02] Guiding Pretraining in Reinforcement Learning with Large Language Models ICML 2023 [paper]
• [2023/05] SPRING: Studying Papers and Reasoning to play Games NeurIPS 2023 [paper]
• [2023/06] OMNI: Open-endedness via Models of human Notions of Interestingness arXiv [paper][code]
• [2023/09] AdaRefiner: Refining Decisions of Language Models with Adaptive Feedback arXiv [paper]
• [2024/03] EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents arXiv [paper]
• [2024/04] AgentKit: Flow Engineering with Graphs, not Coding arXiv [paper][code]
• [2024/04] World Models with Hints of Large Language Models for Goal Achieving arXiv [paper]
• [2024/07] Enhancing Agent Learning through World Dynamics Modeling arXiv [paper]
• [2024/10] Mars: Situated Inductive Reasoning in an Open-World Environment NeurIPS 2024 [paper]

Simulation Games

Human/social Simulation

• [2023/04] Generative Agents: Interactive Simulacra of Human Behavior UIST 2023 [paper][code]
• [2023/08] AgentSims: An Open-Source Sandbox for Large Language Model Evaluation arXiv [paper]
• [2023/10] Humanoid Agents: Platform for Simulating Human-like Generative Agents arXiv [paper]
• [2023/10] Lyfe Agents: Generative agents for low-cost real-time social interactions arXiv [paper]
• [2023/10] SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents arXiv [paper][code]
• [2024/03] SOTOPIA-$\pi$: Interactive Learning of Socially Intelligent Language Agents arXiv [paper][code]
• [2024/09] Altera: Building Digital Humans [website]

Embodied Simulation

• [2022/01] Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents ICML 2022 [paper][code]
• [2022/12] LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models ICCV 2023 [paper]
• [2023/05] Language Models Meet World Models: Embodied Experiences Enhance Language Models NeurIPS 2023 [paper][code]
• [2023/10] Octopus: Embodied Vision-Language Programmer from Environmental Feedback arXiv [paper] [code]
• [2024/01] True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning arXiv[paper][code]

Other Simulation

• [2024/01] CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents ICLR 2024 [paper][code]

Competition Games

• [2022/10] Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task ICLR 2023 [paper]
• [2023/06] ChessGPT: Bridging Policy Learning and Language Modeling NeurIPS 2023 [paper][code]
• [2023/08] Are ChatGPT and GPT-4 Good Poker Players?--A Pre-Flop Analysis arXiv [paper]
• [2023/09] Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4 arXiv [paper]
• [2023/12] Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach arXiv [paper][code]
• [2024/01] PokerGPT: An End-to-End Lightweight Solver for Multi-Player Texas Hold'em via Large Language Model arXiv [paper]
• [2024/01] SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language models arXiv [paper]
• [2024/02] PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models arXiv [paper][code]
• [2024/02] Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization arXiv [paper][code]
• [2024/03] Embodied LLM Agents Learn to Cooperate in Organized Teams arXiv [paper]
• [2024/08] Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information arXiv [paper]

Cooperation Games

• [2023/07] Building Cooperative Embodied Agents Modularly with Large Language Models ICLR 2024 [paper][code]
• [2023/09] MindAgent: Emergent Gaming Interaction arXiv [paper]
• [2023/10] Evaluating Multi-agent Coordination Abilities in Large Language Models arXiv [paper]
• [2023/12] LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination arXiv [paper]
• [2024/02] S-Agents: Self-organizing Agents in Open-ended Environments arXiv [paper]
• [2024/03] ProAgent: Building Proactive Cooperative Agents with Large Language Models AAAI 2024 [paper]
• [2024/03] Can LLM-Augmented Autonomous Agents Cooperate?, An Evaluation of Their Cooperative Capabilities through Melting Pot arXiv [paper]
• [2024/03] Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation arXiv[paper]
• [2024/05] Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration arXiv[paper][code]

Communication (Conversational) Games

• [2022/12] Human-Level Play in the Game of Diplomacy by Combining Language Models with Strategic Reasoning Science [paper]
• [2023/08] GameEval: Evaluating LLMs on Conversational Games arXiv [paper][code]
• [2023/09] Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf arXiv [paper]
• [2023/10] Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game arXiv [paper]
• [2023/10] Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation arXiv [paper]
• [2023/10] AvalonBench: Evaluating LLMs Playing the Game of Avalon FMDM@NeurIPS2023 [paper][code]
• [2023/10] LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay arXiv [paper]
• [2023/10] Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models arXiv [paper][code]
• [2023/11] War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars arXiv [paper][code]
• [2023/11] clembench: Systematic Evaluation of Chat-Optimized Language Models as Conversational Agents EMNLP 2023 [paper]
• [2023/12] Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis AAAI 2024 [paper]
• [2023/12] Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Game arXiv [paper]
• [2023/12] Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games [paper]
• [2024/02] Enhance Reasoning for Large Language Models in the Game Werewolf arXiv [paper]
• [2024/02] What if LLMs Have Different World Views: Simulating Alien Civilizations with LLM-based Agents arXiv [paper]
• [2024/02] Can Large Language Model Agents Simulate Human Trust Behaviors? arXiv [paper]
• [2024/02] Large Language Models Fall Short: Understanding Complex Relationships in Detective Narratives arXiv [paper]
• [2024/04] Self-playing Adversarial Language Game Enhances LLM Reasoning [paper][code]
• [2024/06] PLAYER: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games arXiv[paper]
• [2024/07] AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game arXiv [paper]

Action Games

• [2023/02] Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning ICML 2023 [paper][code]
• [2024/03] Cradle: Empowering Foundation Agents Towards General Computer Control arXiv [paper][code]
• [2024/03] Will GPT-4 Run DOOM? arXiv [paper][code]
• [2024/03] Evaluate LLMs in Real Time with Street Fighter III GitHub [code]
• [2024/07] Baba Is AI: Break the Rules to Beat the Benchmark ICML 2024 [paper]
• [2024/08] Atari-GPT: Investigating the Capabilities of Multimodal Large Language Models as Low-Level Policies for Atari Games arXiv [paper]
• [2024/09] Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Case arXiv [paper] [code]
• [2024/08] AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game arXiv [paper]

Citation

If you find this repository useful, please cite our paper:

@misc{hu2024survey,
      title={A Survey on Large Language Model-Based Game Agents}, 
      author={Sihao Hu and Tiansheng Huang and Fatih Ilhan and Selim Tekin and Gaowen Liu and Ramana Kompella and Ling Liu},
      year={2024},
      eprint={2404.02039},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}

Contact

If you discover any papers that are suitable but not included, please contact Sihao Hu (sihaohu@gatech.edu).