Expensive is a really relative term right about now... 64gb DDR4 LRDIMMs could be had for about 250$ each on eBay before the chip shortage. While that price is a "good-ish deal", it really wasn't unheard of. A search I just did returns more than a few hits...
Comments
CPU in kilohertz then megahertz then gigahertz then it stopped.
RAM in kilobytes then megabytes then gigabytes then it stopped.
Yes for CPU, no for RAM. You can buy a computer with terabytes of RAM just fine. It's just expensive.
Expensive is a really relative term right about now... 64gb DDR4 LRDIMMs could be had for about 250$ each on eBay before the chip shortage. While that price is a "good-ish deal", it really wasn't unheard of. A search I just did returns more than a few hits...
Now compare to single GPU prices...
Those are material science and physical limitations.
Number of parameters in a neural network is not really limited that way, doing useful compute with it is a different matter