Wouldn’t this be bad for marketing reasons? If people see the LLM output just instantly changes with each word or character they type it would cease to appear as some kind of “intelligence” and just feel like nothing more than a glorified autosuggest? Tweak a few words here and there to try to modify the output in subtle ways?

It seems for people to perceive it as true AI they must send off some prompt, watch it think deep while a loader spins, and then read a response.