Adaption for Reinforcement Learning

In my masters class "GPU-Programming" we used this console pacman to implement a Q-learning algorithm for reinforcement learning. We had alot of fun doing everything but GPU programming in fact. Anyway, we modified the Makefile to make it possible to call Cuda-C functions from the Original C-File. You can find an excample how to do that with the function "getAction()" called in pacman.c, defined in agent.cu and linked via agent.h. We also reduced the delays, sped the game up and removed the starting animation for a faster learning process. Can be reverted though, just look at the forked original.

So, Reinforcement Learning Pacman?

Basically what we did is quantizising the states Pacman can be in. Since there is actually alot, we reduced the possible States to the 2 - Neighbourhood.

Also we have just 4 different possibilities per field: Nothing,wall, item, ghost. Also we use the information if Pacman is invinvible at the moment. Sessions are recorded in a .dat File (training.dat). We recorded several Sessions already (Sessions_1-5.dat). We actually wanted to do the evaluation in Cuda C, but since we did not have enough time, we quickly did it in Python. Script is Model_Evaluation.py. This Script creates a matrix with the Reward for every seen state and conducted action. This are saved in .dat Files with the quantized States as Filename. So for every seen state we have a .dat file. Also we wrote a bash script, to start pacman several times. Filename is train.

How to get started

1. Install ncurses
2. Make install
3. Make
4. Use ./prog for letting pacman play once and ./train for letting pacman play x (50 at the moment) times
5. Watch pacman and have fun!

Future Notes

Pacman is still pretty bad at the moment. Some students in the future might continue this project. There are still alot of optimization possibilities. One might be to include the Python script in the bash file to do evaluation every time after pacman has played. Might also implement a neural network or do some parallelisation on the Window Drawing or the training.

Original Part of Readme

Pacman For Console

Okay, so basically, I got tired of enabling flash on my browser so that I could play Pacman. That, and I was extremely bored one night. So I decided to make my own Pacman... for Console.

Licensing Information

See COPYING for details on the GNU/GPL version 2.

Compile/Install/Run

To make... gee... let's see... type make.
To install... type make install.
To run... type pacman [level_#] where # is 1-9, for a premade level,
or type pacmac [level_file_name] for a custom level you made.
To uninstall... type make uninstall.
i.e.:

make && su -c "make install"
pacman 3                                           # Start @ level 3
pacman /usr/local/share/pacman/Levels/level07.dat  # Play only level 7
echo ":-( I don't like it." && make uninstall      # Uninstall :-(

The ASCII art

• C - Pacman - That's you.
• & - Ghosts - Boo.
• . - Pellet - Yummy. Collect all of them to pass to the next level.
• * - Power pellet - Makes you invincible for a short while.
• Wall - No one can walk through it.
• Blocker - A.K.A. Ghost wall. Only the Ghosts can walk through this.

13 Basic Playing Rules

In case you don't know the rules of Pacman (rules I programmed in), here are most of them:

1. Pacman must collect all the pellets of food in the maze. 1 point per pellet.
2. Big pellets make Pacman invincible for a short amount of time.
3. If Pacman touches a ghost without being invincible, you die and lose 1 life.
4. If Pacman touches a ghost while invincible, the ghost is sent back to his starting point.
5. Points are awarded for each ghost eaten in a row while invincible: 20, 40, 80, 160 (10*2^x).
6. Pacman cannot go through the Ghost Walls (Blockers).
7. Ghosts, cannot turn completely around unless there is no other option.
8. While Pacman is invincible, Ghosts will be a bit slower and tend to stay away from him.
9. While Pacman is NOT invincible, Ghosts will tend to come toward him.
10. Pacman starts with 3 extra lives, once all three are gone, the game is over.
11. Extra lives are awarded at 1000 points, 2000, 4000, 8000... (500*2^[x]).
12. If any character reaches a border of the maze, he will be transported to the opposite side.
13. Each character can only go in one X or Y direction at a time.

The keys used are UP, DOWN, LEFT, RIGHT or W, S, A, D.

To make your own levels, see Levels/README.

Contact Information

Send comments and levels you have made to doctormike@gmail.com.

There is a homepage.