Skip to content

Comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language Modelparent

Comments

Human priors are a feature, not a bug. ... Being forced to learn everything end-to-end, from scratch, is a severe limitation.

They're neither. When performing very human-adjacent tasks, it will certainly put the ML algorithm at a disadvantage compared to us.

But for non-human adjacent tasks, say interpreting what a sequence of amino acids actually means, we can expect the computer to absolutely crush us because our stupid human heuristics take us absolutely nowhere, cause us to see patterns that aren't there, etc. etc.

Regardless, this is irrelevant to the original point that I was making, which is that comparing the performance of DL on human adjacent tasks to the amount of time it takes a human to learn the same task is misguided because you are ignoring the million-year long optimization process to get there.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.