Skip to content

Latest commit

 

History

History
42 lines (35 loc) · 1.56 KB

README.md

File metadata and controls

42 lines (35 loc) · 1.56 KB
Raw

Q-Learning-LFA

This repository contains the source code to reproduce all the numerical experiments as described in the paper "Finite-Sample Analysis of Nonlinear Stochastic Approximation with Applications in Reinforcement Learning".

Here's a BibTeX entry that you can use to cite it in a publication:

@article{chen2019finite,
  title={Finite-sample analysis of nonlinear stochastic approximation with applications in reinforcement learning},
  author={Chen, Zaiwei and Zhang, Sheng and Doan, Thinh T and Clarke, John-Paul and Theja Maguluri, Siva},
  journal={arXiv e-prints},
  pages={arXiv--1905},
  year={2019}
}

Requirements

  • Python (>= 3.7)
  • Numpy (>= 1.19.1)

Usage

Constant Step Size

  1. Show convergence of Q-learning with linear function approximation for .
cd constant_step_size
python convergence.py
  1. Show exponentially fast convergence of Q-learning with linear function approximation for .
cd constant_step_size
python rate_of_convergence.py

Diminishing Step Sizes

Show convergence rate of Q-learning with linear function approximation for using diminishing step sizes .

cd diminishing_step_size
python rate_of_convergence.py

Maintainer