Looks like Mac Minis had a 64 GB model (hard to get now) for roughly $1,600. You could get a 3 of these (192GB) & thanks to the Thunderbolt 5 connection, they out performed any single board AMD for a similar price that had more memory. If you tried to do so something similar with Nvidia GPU you're looking at a lot of watts, cooling & a much faster experience but only 48 GB of memory.
Comments
This made me go down a rabbit hole.
Looks like Mac Minis had a 64 GB model (hard to get now) for roughly $1,600. You could get a 3 of these (192GB) & thanks to the Thunderbolt 5 connection, they out performed any single board AMD for a similar price that had more memory. If you tried to do so something similar with Nvidia GPU you're looking at a lot of watts, cooling & a much faster experience but only 48 GB of memory.
You can layer the LLMs across multiple GPUs, you can absolutely have 192gb of Nvidia dGPU memory with the right MOBO and CPU combo.
I was trying to compare equal in price