Claude AI Can Now Play Pokémon—And It’s Winning

It turns out robot lawnmowers and ChatGPT are not the only ones that can play video games.

Anthropic said on Tuesday that Claude’s latest version, 3.7 Sonnet, can play the classic video game Pokémon.

In a thread posted to X, Anthropic said an early version of Claude 3.7 Sonnet could defeat opponents within hours of playing Pokémon.

“The results were striking. Within hours, Claude defeated Brock. Days later, it trounced Misty. Progress that older models had little hope of achieving,” Anthropic wrote. “Turns out extended thinking is super effective.”

According to Anthropic, Claude 3.7 Sonnet keeps notes in its knowledge base, observes the screen, and employs function calls to click buttons and navigate the game.

In addition to screenshots, Anthropic linked to a Twitch channel called “ClaudePlaysPokemon” showing Claude playing the game.

What made defeating the Pokémon opponents possible, Anthropic said, was Claude 3.7 Sonnet’s ability to plan its next moves and adapt its strategies, where previous models like Claude 3.5 Sonnet would wander or get stuck in a loop.

“With a few tools to help it see the screen a bit better, Claude acts as an agent, applying its abilities to a novel task,” Anthropic wrote. “In this, we start to see glimmers of AI systems that tackle challenges with increasing competence, not just through training but with generalized reasoning.”

Claude 3.7 Sonnet is the latest AI model to play video games successfully. Last March, researchers used ChatGPT to play classic first-person shooter Doom, managing to get to the last room in the game once.

That same month, Google DeepMind launched its Scalable Instructable Multiworld Agent (SIMA). This generalist AI, capable of performing various tasks such as text generation, image analysis, and translation, was trained to play video games such as No Man’s Sky, Teardown, and Valheim.

“Our AI agent doesn’t need access to a game's source code, nor bespoke APIs,” Google DeepMind wrote. “It requires just two inputs: the images on screen and simple, natural-language instructions provided by the user.”

Edited by Sebastian Sinclair