GPT 4 gets it right. I understand that LLMs have some very rough emergent properties, like arbitrary arithmetic (but its often incorrect). I don't really understand how its able to base64 decode values it has never seen though. I'm using unique snippets from my own private source code.
Consider this (which is correct, unless I managed to include any brainfarts in my reasoning) – in the mapping between Base64 and ASCII, there are no unknown inputs. There’s no XOR or anything like that going on. It’s a pure mapping of byte values to byte values, just with an alternative representation. So there’s really no secret sauce in there, not even any obfuscation going on. Unlike AES or even a simple Vigenere cipher.
I don't really understand how its able to base64 decode values it has never seen though
You are surprised by this but not by its ability to translate languages? You can train a base 64 decoder and encoder neural network trivially, so it is very easy to see how it could have one lurking among all of its many parts give it has likely seen many millions of base64 encoding examples on the internet.
Comments
GPT 4 gets it right. I understand that LLMs have some very rough emergent properties, like arbitrary arithmetic (but its often incorrect). I don't really understand how its able to base64 decode values it has never seen though. I'm using unique snippets from my own private source code.
Consider this (which is correct, unless I managed to include any brainfarts in my reasoning) – in the mapping between Base64 and ASCII, there are no unknown inputs. There’s no XOR or anything like that going on. It’s a pure mapping of byte values to byte values, just with an alternative representation. So there’s really no secret sauce in there, not even any obfuscation going on. Unlike AES or even a simple Vigenere cipher.
Perhaps I should have written bit patterns instead of bit values (:
You are surprised by this but not by its ability to translate languages? You can train a base 64 decoder and encoder neural network trivially, so it is very easy to see how it could have one lurking among all of its many parts give it has likely seen many millions of base64 encoding examples on the internet.
I don't know why this surprises me but translating natural languages doesn't. I think it is the sustained accuracy.
Wouldn't it be 100% accurate if it was trained on base64