The source code for the AI benchmark ClaudePlaysPokemon has been released. ClaudePlaysPokemon is a benchmark to show how agents work and can generalize, it was made to see how a AI model not trained on Pokemon can use general thinking to play the game.

What I personally would like to see is the open source community taking a small local model like Gemma3 27b and finetuning it on annotated screenshots explaining it what tiles can be cut which ones can only be jumped over from one side etc and maybe general game knowledge from Bulbapedia. This would be a good way to show if a finetuned specialized small model can out perform a general big model.

Source: https://github.com/davidhershey/ClaudePlaysPokemonStarter

Twitch: https://www.twitch.tv/claudeplayspokemon