The Idea
How It Started
In my first few months at IBM, my department VP came to me with a request. He wanted a quick, fun demo to show clients the power of AI. The catch? Many of IBM's clients aren't tech-savvy, so it needed to be entertaining and relatable - something that would click immediately.
A fully playable Pacman AI felt like the perfect fit. Everyone knows Pacman. Watching an AI learn to play it, make mistakes, and eventually master the game tells a compelling story about machine learning without needing a technical background to appreciate it.
The Requirements
The requirements were simple on paper:
- Generate random playable maps so every demo feels fresh
- Build an agent that could play and win consistently
Simple to describe, but getting there was a journey.
Map Generation
I didn't want to build Pacman from scratch, so I started with an open-source Pygame repo. The game represented maps as a 19x22 matrix where each index encoded walls, pellets, ghosts, vulnerable ghosts, power-ups, and portals.
Training
For map generation, I built a tile-based system powered by an LSTM (Long Short-Term Memory) network.
Each map is represented as a grid of numbers - a 19x22 matrix where each number encodes a different tile type:
- 14 = Pellet
- 15 = Power Pellet
- 25 = Horizontal Wall
- 26 = Vertical Wall
- 33-36 = Ghosts (red, blue, pink, orange)
- 40 = Pacman
The key insight was treating each row of the map as a token - like a word in a sentence. Row 1 predicts Row 2, which predicts Row 3, and so on. The LSTM learns the "grammar" of valid Pacman maps: where walls should connect, where corridors open up, how to balance pellet distribution.
Here's what a map looks like in this format:
18 25 25 25 25 25 19 50 26 12 26 50 18 25 25 25 25 25 19
26 14 14 14 14 14 26 50 26 14 26 50 26 14 14 14 14 14 26
26 15 21 25 22 14 16 25 17 14 16 25 17 14 21 25 22 15 26
26 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 26
27 25 25 22 14 18 25 30 22 14 21 30 25 19 14 21 25 25 27
11 10 10 10 14 16 25 17 14 14 14 16 25 17 14 10 10 10 11
30 25 25 22 14 14 14 14 14 23 14 14 14 14 14 21 25 25 30
26 14 14 14 14 23 14 18 25 27 25 19 14 23 14 14 14 14 26
26 14 23 14 21 17 14 16 25 25 25 17 14 16 22 14 23 14 26
26 14 26 14 14 14 14 14 14 33 14 14 14 14 14 14 26 14 26
26 14 20 14 21 19 14 18 25 13 25 19 14 18 22 14 20 14 26
26 14 14 14 14 26 14 26 34 35 36 26 14 26 14 14 14 14 26 <- Ghosts (34, 35, 36)
26 14 18 22 14 26 14 16 25 25 25 17 14 26 14 21 19 14 26
26 14 20 14 14 26 14 14 14 40 14 14 14 26 14 14 20 14 26 <- Pacman (40)
26 14 14 14 18 27 22 14 18 25 19 14 21 27 19 14 14 14 26
27 25 22 14 26 14 14 14 26 50 26 14 14 14 26 14 21 25 27
11 10 10 14 20 14 23 14 26 50 26 14 23 14 20 14 10 10 11
30 25 22 14 14 14 20 14 16 25 17 14 20 14 14 14 21 25 30
26 14 14 14 23 14 14 14 14 14 14 14 14 14 23 14 14 14 26
26 15 21 25 17 14 18 25 19 10 18 25 19 14 16 25 22 15 26
26 14 14 14 14 14 26 50 26 10 26 50 26 14 14 14 14 14 26
16 25 25 25 25 25 17 50 26 12 26 50 16 25 25 25 25 25 17
Throw in some randomness during generation, and the model produces unique, playable maps every time. The LSTM learned to create mazes that are both structurally valid and fun to play.
The Agent
Deep Q-Learning
The AI agent itself was a Deep Q-Network (DQN). The idea is straightforward: the agent plays thousands of games, getting rewarded for good moves (eating pellets, avoiding ghosts) and penalized for bad ones (getting caught). Over time, it learns which actions lead to winning.
The Struggle
This was the main struggle of the project. I just couldn't get the accuracy up.
Two problems:
- I was originally passing the entire 19x22 matrix as the input to the neural network
- Training took around 2 days on my work laptop, making tweaks and adjustments nearly impossible
When each iteration takes two days, you learn to dread the moment you realize something isn't working.
The Breakthrough
The "aha" moment came when I stopped thinking about what the game looks like and started thinking about what Pacman actually needs to know to survive.
I projected the game state down to a 21-dimensional vector capturing only the essentials:
- Walls around Pacman - which directions are blocked?
- Distances to pellets - where's the closest food?
- Distances to ghosts - how close is danger?
- Distances to vulnerable ghosts - any easy targets?
- Trapped flag - is there any escape route, or is Pacman cornered?
- Current velocity - which direction is Pacman moving?
This change was transformative. Training time dropped dramatically, and the agent finally started learning real strategies. The model could now consistently win games and even optimize for high scores.
Training Progression
Watching the AI learn over time was one of the most rewarding parts of this project.
Early Training
In the beginning, the agent has no idea what it's doing. It moves randomly, frequently running straight into ghosts or getting stuck in corners. Frustrating to watch, but this random exploration is essential for learning.
Mid Training
After a few thousand episodes, things start clicking. The agent learns to avoid ghosts most of the time and actively seeks out pellets. It still makes mistakes, but you can see the foundation of a strategy forming. It's no longer just wandering. It's hunting.
Final Model
The fully trained agent is a joy to watch. It navigates the maze efficiently, dodges ghosts with precision, and knows exactly when to grab a power pellet and turn the tables. It wins consistently and makes decisions that feel almost human.
Availability
The demo isn't hosted publicly and the code is IBM property, but a demo is available on request.