Show HN: ReliableGPT run 200 GPT-4 requests in parallel
colab.research.google.comreliableGPT handles GPT-4 429 rate limit errors
it optimally allocate jobs for incoming requests to maximize usage of Requests/min and Token/min limits
reliableGPT handles GPT-4 429 rate limit errors
it optimally allocate jobs for incoming requests to maximize usage of Requests/min and Token/min limits
Comments
Looks like this will only be effective for short prompts/responses, eg if you have 4k tokens in your prompt, you can only fire 10 requests/minute with 40k token/minute rate limit
Why not using just a fetch with retry function ?
It is, in fact, a try-catch. https://github.com/BerriAI/reliableGPT/blob/main/reliablegpt...