Skip to content

Comment on Megatron-Turing NLG 530B, the World’s Largest Generative Language Modelparent

Comments

Yes for CPU, no for RAM. You can buy a computer with terabytes of RAM just fine. It's just expensive.

Expensive is a really relative term right about now... 64gb DDR4 LRDIMMs could be had for about 250$ each on eBay before the chip shortage. While that price is a "good-ish deal", it really wasn't unheard of. A search I just did returns more than a few hits...

Now compare to single GPU prices...

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.