Comment on Retentive Network: A Successor to Transformer for Large Language ModelsComments−fgfm3yThe repo the paper is pointing to indicates that the code will be released within ~1 week! If the sheer difference in VRAM requirements and latency holds up, this will seriously be a major breakthrough for LLM architectures!Can't wait to try it out−deimos0x023yhttps://github.com/Jamie-Stirling/RetNet
Comments
The repo the paper is pointing to indicates that the code will be released within ~1 week! If the sheer difference in VRAM requirements and latency holds up, this will seriously be a major breakthrough for LLM architectures!
Can't wait to try it out
https://github.com/Jamie-Stirling/RetNet