The new laptop has only 32GB DDR5 and a RTX 4070 with 8 GB GDDR6 on Xubuntu, so Gemini recommended qwen2.5. I don’t think I want to run anything larger because as you said it’d run on CPU and system RAM. As it is, the 14B model will still spill over some and not entirely fit into the GPU.
Comments
The new laptop has only 32GB DDR5 and a RTX 4070 with 8 GB GDDR6 on Xubuntu, so Gemini recommended qwen2.5. I don’t think I want to run anything larger because as you said it’d run on CPU and system RAM. As it is, the 14B model will still spill over some and not entirely fit into the GPU.