Skip to content

Comment on LLMLingua: Compressing Prompts for Faster Inferencingparent

Comments

I would think you’d need to unescape the new lines and tabs and have a task for the model to perform with it.

Maybe try prefixing it with “summarize the following text” before compression.

Otherwise I’m not sure how it would judge what it’s important. Honestly I’m not sure what ChatGPT would do if you copied the text from this page uncompressed without asking it do something

Edit: pasting uncompressed it summarizes the discussion.

I think this solution isn’t well suited for this kind of task. It seems like you’d want to compress instructions, system prompts and memory. With a big block of text with no prior context you’re essentially relying on the smaller model to decide what’s important without enough information to judge.

Worth some more experimentation for sure

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.