Comment on Unified Memory, Explained: Why Mini PCs Can Run 70B Models a Big GPU Can'tparentComments−entrope2moThose are the ratios for memory bandwidth, but the GPUs have a much higher ratio for compute, and that affects prefill rate / TTFT, right?
Comments
Those are the ratios for memory bandwidth, but the GPUs have a much higher ratio for compute, and that affects prefill rate / TTFT, right?