Deep Q-Learning for Lunar Lander

This repository contains an implementation of Deep Q-Learning (DQN) to solve the Lunar Lander environment using PyTorch and OpenAI Gym.

Deep Q-Learning (DQN) is a type of reinforcement learning (RL) algorithm. Reinforcement learning is a subfield of machine learning where an agent learns to make decisions by interacting with an environment. The agent receives feedback in the form of rewards or penalties based on the actions it takes in the environment. The goal of the agent is to learn a policy, a mapping from states to actions, that maximizes the cumulative reward over time.

Overview

1. Neural Network Architecture (`Network` class):
   • The neural network is implemented using PyTorch and consists of three fully connected layers.
   • The input size is determined by the `state_size` parameter, and the output size is equal to the number of possible actions (`action_size`).
   • Rectified Linear Unit (ReLU) activation functions are used between layers to introduce non-linearity.
2. Agent (`Agent` class):
   • The DQN agent interacts with the environment, collecting experiences and updating the Q-network.
   • The agent uses experience replay to store and sample experiences, which helps break correlations in the training data.
   • Target networks are used to stabilize the training process.
   • The `soft_update` method is employed to perform a soft update of the target Q-network parameters.
   • The agent's behavior is epsilon-greedy, where it selects a random action with probability epsilon to encourage exploration.

Dependencies

• Python 3.x
• PyTorch
• NumPy
• OpenAI Gym
• gymnasium (Lunar Lander environment)

Note: The gymnasium dependencies works properly under Linux Environment. So, clone this repository on a Linux Machine. If you are in Windows Machine, consider using WSL. I have built the model from WSL and it works perfectly well.

Minimum Configuration for Running the Model:

Note: If this config is not getting matched with your machine, then consider using Google Colab.

• CPU: Intel Core i3 or AMD Ryzen 5 (For Apple, Any M series Chipset!)
• GPU: Integrated Graphics is perfectly fine (If you have GPU, you need to install NVIDIA CUDA for Faster Training)
• RAM: Atleast 8GB (If you are in Linux or Mac Machine), 16 GB is needed for Windows User (For Running the model in WSL)
• Disk space Requried: Atleast 5GB of Disk Space (For Installing the Packages)

Instructions

1. Install the required dependencies:

   pip install -r requirements.txt

2. Run the train.py script to train the DQN agent:

   python3 AI_Runner.py

Results

The agent's performance is tracked over episodes, and training stops when the environment is considered solved (average score >= 250 over the last 100 episodes).

The training progress and average scores are printed to the console during training. Additionally, a video of the trained agent playing the game is saved as video.mp4 for visual inspection.

Video Reference:

Lunar Lander Video for 200 pts in Average
Lunar Lander Video for 250 pts in Average