Comment on New attention mechanisms that outperform standard multi-head attentionparentComments−ttul2yI’m certainly not a domain expert, but one thing I have read repeatedly about Transformers is that not all tricks scale the same.
Comments
I’m certainly not a domain expert, but one thing I have read repeatedly about Transformers is that not all tricks scale the same.