Skip to content

Comment on Nvidia AVO scores 100% on the ARC-AGI-3 interactive reasoning benchmarkparent

Comments

GPT-4(?) was capable of beating Pokemon 18 months ago but models only became capable of beating it without a harness in the last six months...?

GPT-4 was decidedly not capable of beating Pokemon 18 months ago. I doubt it would be able to complete a single level. I don't think people realize how large the advances in model capabilities have been. GPT-4 in a modern harness is absolutely horrendous compared to modern models.

I doubt it would be able to complete a single level

Have you ever played pokemon?

Yes. Presumably you're referring to my use of the word "level". I mean here basically every "level" as denoted by the order of locations and places on the town map that you get (which is usually +- some other locations how game runners refer to different sections of the game).

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.