This repository is the official implementation of Deep Variance Weighting for the Maze experiments in Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice.
- We modified ShinRL repository (commit 09f4ae2).
- You can see the implementation of M-DQN with DVW in shinrl/solvers/discrete_vi/solver.py
Install dependencies by:
# make sure you are in Variance-Weighted-MDVI/Deep-Variance-Weighting-Maze
poetry install
You can test if everything works by:
poetry run python experiments/run_generative.py --weight_mode dvw --seed 0 --maze_seed 1 --maze_eps 0.1 --iteration 10000 --maze_size 25 --num_pitfall 8 --num_samples_target 3 --is_munchausen
Run bash experiments/start_job.bash
(Optional) experiments/results.zip
contains the results of bash experiments/start_job.bash
. Unzip the zip-file inside the experiments folder to skip experiments.
Run poetry run python experiments/plot_all_results.py
The figures will be saved in experiments/results/0.1-3/optimality-gap.png
and experiments/results/0.1-10/optimality-gap.png
.