Comment on Which H100 Instance to Train Nanochat – Benchmarking PCIe, SXM, and NVLparentComments−k2soOP6moYeah, for a single GPU inference, considering the higher VRAM and FP4 support on the RTX 6000, it should fit larger models as well than the H100.
Comments
Yeah, for a single GPU inference, considering the higher VRAM and FP4 support on the RTX 6000, it should fit larger models as well than the H100.