So in an effective Mamba query the question goes at the end, after input data? I thought that the question should go at the beginning, so it can decide which information in the data is relevant.
I could be wrong, as I haven't used Mamba, but it seems to remain similar to transformers in that it doesn't "decide" anything and streams tokens to follow the existing ones; attention isn't a thing in the same way, but recency does still have impact. To that end, putting context after the question makes it more likely to follow the context, not the question.
Comments
So in an effective Mamba query the question goes at the end, after input data? I thought that the question should go at the beginning, so it can decide which information in the data is relevant.
I could be wrong, as I haven't used Mamba, but it seems to remain similar to transformers in that it doesn't "decide" anything and streams tokens to follow the existing ones; attention isn't a thing in the same way, but recency does still have impact. To that end, putting context after the question makes it more likely to follow the context, not the question.