Comment on Prompt caching for cheaper LLM tokensparentComments−babelfish8moThink of it as a very useful prefix match. If all of your threads start with the same system prompt, you will reap benefits from prompt caching.
Comments
Think of it as a very useful prefix match. If all of your threads start with the same system prompt, you will reap benefits from prompt caching.