Comment on How has DeepSeek improved the Transformer architecture?parentComments−cma1yFlash attention was also a set of common techniques in other areas of optimized software, yet the big guys weren't doing the optimizations when it came out and it significantly improved everything.−whimsicalism1yyes, i agree that low-level & infra work is where a lot of deepseek's improvement came from
Comments
Flash attention was also a set of common techniques in other areas of optimized software, yet the big guys weren't doing the optimizations when it came out and it significantly improved everything.
yes, i agree that low-level & infra work is where a lot of deepseek's improvement came from