Well consumer hardware can run something in the order of ~50B quantized at a "reasonable" price today, we'd need about 5 or 6 doublings to run something that would be GPT 4 tier at 1T+. So, it would need to continue for roughly a decade at least?
Current models are horrendously inefficient though, so with architectural improvements we'll have something of that capability far sooner on weaker hardware.
Comments
Well consumer hardware can run something in the order of ~50B quantized at a "reasonable" price today, we'd need about 5 or 6 doublings to run something that would be GPT 4 tier at 1T+. So, it would need to continue for roughly a decade at least?
Current models are horrendously inefficient though, so with architectural improvements we'll have something of that capability far sooner on weaker hardware.