Hugsun on New attention mechanisms that outperform standard multi-head attention · Hackerly

Comment on New attention mechanisms that outperform standard multi-head attention

I'm surprised they don't mention grouped query attention at all.