This is a good question, because the context is limited and the more you use, the more you pay.
One technique is to use embeddings, and then only provide a subset that is close to the prompt embedding. The hard part here is chunking and deciding what close means. One example, when including a function in the context, do I need to include the body, or is the doc string and signature sufficient?
Comments
How does it decide which files?
This is a good question, because the context is limited and the more you use, the more you pay.
One technique is to use embeddings, and then only provide a subset that is close to the prompt embedding. The hard part here is chunking and deciding what close means. One example, when including a function in the context, do I need to include the body, or is the doc string and signature sufficient?