wordle-rl

RL for Wordle

Running the app

1. docker-compose up from the main project directory

2. Go to the jupyter lab url that is created

3. Run the stablebaselines3_dqn_demo.ipynb notebook to demo training a stable-baselines DQN agent on this Wordle environment.

Tools I used

• OpenAI Gym
• stable-baselines3
• TF Agents

Other tools that may be interesting

• Rllib (Ray RL library for distributed RL)
• Tensorforce (I guess like community supported take on TF Agents)
• Reverb (separate service to manage replay buffers; made by DeepMind)

Resources and useful links

Wordle related resources

• wordle word lists (https://gist.github.com/cfreshman) Note: these are not the same as Wordle's current lists since NYTimes bought it

• Solving Wordle with information theory (youtube)

Tutorials for RL implementation

• Making a custom openai gym env (stablebaselines tutorial)

• Making a custom openai gym env (blog post)

• TF-Agents DQN Tutorial

• TF Agents DQN Tutorial (youtube)

Resources for large discrete action space problem

• RL in Large Discrete Action Spaces

• Hindsight Experience Replay Buffers

• Blog post explanation of HER buffer

• Soft Actor-Critic (paper)

• OpenAI Spinning Up: Soft Actor-Critic

• SOFT ACTOR-CRITIC FOR DISCRETE ACTION SETTINGS (paper)

• Adapting Soft Actor Critic for Discrete Action Spaces (blog)

General RL knowledge

• Intro to RL textbook