Google's Gemini outperforms Pokémon Blue with assistance

Google AI Model Beats 29-Year-Old Video Game

Google’s most expensive AI model seems to have crossed a major milestone: Beating a 29-year-old video game.

Last night, Google CEO Sundar Pichai posted triumphantly on X, “What a finish! Gemini 2.5 Pro just completed Pokémon Blue!”

To be clear, the Gemini Plays Pokemon livestream was created by (in his own words) “a 30-year-old software engineer unaffiliated with Google” who goes by Joel Z. But Google executives have been cheering the effort on.

AI Models Playing Pokémon

Why Pokémon? Back in February, Anthropic highlighted progress that its Claude AI models were making in “Pokémon Red,” writing that Claude’s “extended thinking and agent training” gives it “a major boost” on “more unexpected” tasks, like playing a classic game. There’s even a Claude Plays Pokemon Twitch channel that Joel Z cited as an inspiration.

Despite its progress, Claude does not appear to have beaten “Pokémon Red” yet. Does that mean Gemini is objectively better at the game? On his Twitch page, Joel Z urged viewers, “Please don’t consider this a benchmark for how well an LLM can play Pokemon. You can’t really make direct comparisons — Gemini and Claude have different tools and receive different information.”

Gemini Plays Pokémon Development

Joel Z acknowledged that there were other “dev interventions” to help Gemini complete the game, but insisted that it’s not cheating.

“My interventions improve Gemini’s overall decision-making and reasoning abilities,” he says. “I don’t give specific hints — there are no walkthroughs or direct instructions for particular challenges like Mt. Moon. The only thing that comes even close is letting Gemini know that it needs to talk to a Rocket Grunt twice to obtain the Lift Key, which was a bug that was later fixed in Pokemon Yellow.”

Plus, he said, “Gemini Plays Pokémon is still actively being developed, and the framework continues to evolve.”