That would be pretty useful. I'm working on getting chatgpt to classify a dataset. So basically I use the same big prompt for a bunch of different small texts and ask chatgpt to generate the class label. Something like initializing the prompt state sounds good. Basically trade more processing time for more memory usage. Who know maybe openai is doing such optimization from their side