Pacman

Teaching a computer to play Pacman better than humans.

The Idea

How It Started

In my first few months at IBM, my department VP came to me with a request. He wanted a quick, fun demo to show clients the power of AI. The catch? Many of IBM's clients aren't tech-savvy, so it needed to be entertaining and relatable - something that would click immediately.

A fully playable Pacman AI felt like the perfect fit. Everyone knows Pacman. Watching an AI learn to play it, make mistakes, and eventually master the game tells a compelling story about machine learning without needing a technical background to appreciate it.

The Requirements

The requirements were simple on paper:

Simple to describe, but getting there was a journey.

Map Generation

I didn't want to build Pacman from scratch, so I started with an open-source Pygame repo. The game represented maps as a 19x22 matrix where each index encoded walls, pellets, ghosts, vulnerable ghosts, power-ups, and portals.

Training

For map generation, I built a tile-based system powered by an LSTM (Long Short-Term Memory) network.

Each map is represented as a grid of numbers - a 19x22 matrix where each number encodes a different tile type:

The key insight was treating each row of the map as a token - like a word in a sentence. Row 1 predicts Row 2, which predicts Row 3, and so on. The LSTM learns the "grammar" of valid Pacman maps: where walls should connect, where corridors open up, how to balance pellet distribution.

Here's what a map looks like in this format:

18 25 25 25 25 25 19 50 26 12 26 50 18 25 25 25 25 25 19
26 14 14 14 14 14 26 50 26 14 26 50 26 14 14 14 14 14 26
26 15 21 25 22 14 16 25 17 14 16 25 17 14 21 25 22 15 26
26 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 26
27 25 25 22 14 18 25 30 22 14 21 30 25 19 14 21 25 25 27
11 10 10 10 14 16 25 17 14 14 14 16 25 17 14 10 10 10 11
30 25 25 22 14 14 14 14 14 23 14 14 14 14 14 21 25 25 30
26 14 14 14 14 23 14 18 25 27 25 19 14 23 14 14 14 14 26
26 14 23 14 21 17 14 16 25 25 25 17 14 16 22 14 23 14 26
26 14 26 14 14 14 14 14 14 33 14 14 14 14 14 14 26 14 26
26 14 20 14 21 19 14 18 25 13 25 19 14 18 22 14 20 14 26
26 14 14 14 14 26 14 26 34 35 36 26 14 26 14 14 14 14 26    <- Ghosts (34, 35, 36)
26 14 18 22 14 26 14 16 25 25 25 17 14 26 14 21 19 14 26
26 14 20 14 14 26 14 14 14 40 14 14 14 26 14 14 20 14 26    <- Pacman (40)
26 14 14 14 18 27 22 14 18 25 19 14 21 27 19 14 14 14 26
27 25 22 14 26 14 14 14 26 50 26 14 14 14 26 14 21 25 27
11 10 10 14 20 14 23 14 26 50 26 14 23 14 20 14 10 10 11
30 25 22 14 14 14 20 14 16 25 17 14 20 14 14 14 21 25 30
26 14 14 14 23 14 14 14 14 14 14 14 14 14 23 14 14 14 26
26 15 21 25 17 14 18 25 19 10 18 25 19 14 16 25 22 15 26
26 14 14 14 14 14 26 50 26 10 26 50 26 14 14 14 14 14 26
16 25 25 25 25 25 17 50 26 12 26 50 16 25 25 25 25 25 17

Throw in some randomness during generation, and the model produces unique, playable maps every time. The LSTM learned to create mazes that are both structurally valid and fun to play.

The Agent

Deep Q-Learning

The AI agent itself was a Deep Q-Network (DQN). The idea is straightforward: the agent plays thousands of games, getting rewarded for good moves (eating pellets, avoiding ghosts) and penalized for bad ones (getting caught). Over time, it learns which actions lead to winning.

The Struggle

This was the main struggle of the project. I just couldn't get the accuracy up.

Two problems:

When each iteration takes two days, you learn to dread the moment you realize something isn't working.

The Breakthrough

The "aha" moment came when I stopped thinking about what the game looks like and started thinking about what Pacman actually needs to know to survive.

I projected the game state down to a 21-dimensional vector capturing only the essentials:

This change was transformative. Training time dropped dramatically, and the agent finally started learning real strategies. The model could now consistently win games and even optimize for high scores.

Training Progression

Watching the AI learn over time was one of the most rewarding parts of this project.

Early Training

In the beginning, the agent has no idea what it's doing. It moves randomly, frequently running straight into ghosts or getting stuck in corners. Frustrating to watch, but this random exploration is essential for learning.

Mid Training

After a few thousand episodes, things start clicking. The agent learns to avoid ghosts most of the time and actively seeks out pellets. It still makes mistakes, but you can see the foundation of a strategy forming. It's no longer just wandering. It's hunting.

Final Model

The fully trained agent is a joy to watch. It navigates the maze efficiently, dodges ghosts with precision, and knows exactly when to grab a power pellet and turn the tables. It wins consistently and makes decisions that feel almost human.

Availability

The demo isn't hosted publicly and the code is IBM property, but a demo is available on request.

More work by Michael Rubenstein