Comment on Cerebras’s giant chip will smash deep learning’s speed barrierparentComments−Veedrac6yGPT-2 uses attention, which is very memory hungry to train, so probably won't work well. But I agree with your overall point.
Comments
GPT-2 uses attention, which is very memory hungry to train, so probably won't work well. But I agree with your overall point.