PyTorch 2.2: FlashAttention-v2 integration, AOTInductorpytorch.org 15 pointsegnehots2 years ago1 commentSaveHideCopy link On HNComments−two_in_one2ynow supports FlashAttention-2, yielding around 2x speedupstorch.compile improvementsso far 2.1 didn't work well with MoE GPT, at least in my implementation, due to dynamism in data flow. will check how 2.2 does
Comments
so far 2.1 didn't work well with MoE GPT, at least in my implementation, due to dynamism in data flow. will check how 2.2 does