No, you can make multiple passes across the data set. There's no reason that you have to use quadratic attention. I point to the shape of our own brains, which are limited in size. Yet, through multiple passes over information and random access, which we guide, we're able to process very large systems.
Comments
No, you can make multiple passes across the data set. There's no reason that you have to use quadratic attention. I point to the shape of our own brains, which are limited in size. Yet, through multiple passes over information and random access, which we guide, we're able to process very large systems.