The problem is that the benchmarks essentially test "is this data in the training set?" This methodology will never achieve any form of AI that is useful past a search engine.
A true AI shouldn't have any sort of informational knowledge about any domain, as it should be able to figure out how to get that knowledge (which includes building internal simulation if necessary), when asked a question.
Comments
Are all of the ai model benchmarks just made up? https://epoch.ai/data/ai-benchmarking-dashboard https://ai.azure.com/explore/benchmarks
The problem is that the benchmarks essentially test "is this data in the training set?" This methodology will never achieve any form of AI that is useful past a search engine.
A true AI shouldn't have any sort of informational knowledge about any domain, as it should be able to figure out how to get that knowledge (which includes building internal simulation if necessary), when asked a question.