Maybe we just need models that can reference spans by start:end range. Then they can pass arguments by reference instead of explicit quotation. We can use these spans as answers in extractive QA tasks, or as arguments for a code block, or to construct a graph from pointers, and do graph computation. If we define a "hide span" operation the LLM could dynamically open and close its context, which could lead to context size reduction. Basically - add explicit indexing to context memory, and make it powerful, the LLM can act like a CPU.
Comments
Maybe we just need models that can reference spans by start:end range. Then they can pass arguments by reference instead of explicit quotation. We can use these spans as answers in extractive QA tasks, or as arguments for a code block, or to construct a graph from pointers, and do graph computation. If we define a "hide span" operation the LLM could dynamically open and close its context, which could lead to context size reduction. Basically - add explicit indexing to context memory, and make it powerful, the LLM can act like a CPU.