Comment on Mini – The Minimal LanguageComments−wilg3yConlangs are fun, but what will probably happen is we will evolve more efficient communication protocols as bandwidth increases via multimedia communication, Neuralink, etc.−riku_iki3yLLMs will talk to each other using embeddings.−Rhapso3yFTR LLM embeddeds are a lot less efficient than ASCII for communication and are likely to stay that way.−riku_iki3yabsolutely not. Transformer layers already communicate using embeddings, and ASCII would be absolutely less efficient there.−Rhapso3yAnd how many bits are in an embedded vector?−riku_iki3y12k for gpt3.−riku_iki3yIt is not bits, but weights−Rhapso3ySo somehow ascii is less information dense than 12k 32-bit floats per token?
Comments
Conlangs are fun, but what will probably happen is we will evolve more efficient communication protocols as bandwidth increases via multimedia communication, Neuralink, etc.
LLMs will talk to each other using embeddings.
FTR LLM embeddeds are a lot less efficient than ASCII for communication and are likely to stay that way.
absolutely not. Transformer layers already communicate using embeddings, and ASCII would be absolutely less efficient there.
And how many bits are in an embedded vector?
12k for gpt3.
It is not bits, but weights
So somehow ascii is less information dense than 12k 32-bit floats per token?