Skip to content

Comment on So how well is Claude playing Pokémon?

Comments

A notable error in the article:

The second attempt got all the way to Vermilion City, finding a way through the infamous Mt. Moon maze and achieving two badges, so pretty close to the benchmark.

It did not make it to Vermillion City: it got Misty's badge (with some fun battle RNG), then got stuck in Cerulean City and could not get out: the next objective was to go north to Bill's House to get the S.S Anne ticket which is required before going to Vermillion City, but it just couldn't do that.

Given the amount of loops in this livestream, I am somewhat skeptical of that benchmark results chart. There's no way it somehow made it to Vermillion, beat the S.S Anne for HM Cut, and also beat Surge with the relative amount of actions implied by the chart.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.