OpenAI Responses API has WebSocket mode, which can be used instead of SSE, which works very well and feels like a leap forward in terms of performance.
I have been building on it over the past month holding WebSocket sessions on workers warm, and command routing using NATS JetStream. With this, it has made using sidecar threads for a main thread very simple, as the worker treats them similar.
Comments
OpenAI Responses API has WebSocket mode, which can be used instead of SSE, which works very well and feels like a leap forward in terms of performance.
https://developers.openai.com/api/docs/guides/websocket-mode
I have been building on it over the past month holding WebSocket sessions on workers warm, and command routing using NATS JetStream. With this, it has made using sidecar threads for a main thread very simple, as the worker treats them similar.