Comment on GPT-3.5/4 response times are linear with output tokensparentComments−Tostino3yYeah... But for every new token you generate, you need to take that into account, along with all prior generated tokens and input provided by user for generating the next one.
Comments
Yeah... But for every new token you generate, you need to take that into account, along with all prior generated tokens and input provided by user for generating the next one.