Comment on StreamingLLM: tiny tweak to KV LRU improves long conversationsparentComments−joshspankit2yYou’re right: A lot of the conversation can be condensed, especially if there are enough cues for the AI to arrive in the same “neuronal neighborhood” as the previous conversation.
Comments
You’re right: A lot of the conversation can be condensed, especially if there are enough cues for the AI to arrive in the same “neuronal neighborhood” as the previous conversation.