It's quite inaccurate. It makes frequent mistakes especially in unknown words like names, typos, and swear words. And at some length, it hallucinates the rest of the content.
So it seems to be able to "feel" the answer out there but it's not directly translating it. That's far more fascinating than if it were decoding while hooked to something else.
The decoded message is: "Tell EA-namir: Nanni sends the following message: When you came, you said to me as follows: 'I will give Gimp-Sin (when he comes) five shekels of silver.' You took your garments, even the garments of your back, just as you had promised, to give the messenger of Gimp-Sin (when he came) five shekels of silver, and then send my messenger. They grasped me, took me away and made me enter for the presence of Bel-Adda (the king)."
Errors: "EA-namir" instead of "Ea-nasir", "Gimp-Sin" instead of "Gimil-Sin", "five shekels of silver" instead of "fine quality copper ingots".
"Tell Ea-nasir: Nanni sends the following message: When you came, you said to me as follows : "I will give Gimil-Sin (when he comes) fine quality copper ingots." You left then but you did not do what you promised me. You put ingots which were not good before my messenger (Sit-Sin) and said: "If you want to take them, take them; if you do not want to take them, go away!" What do you take me for, that you treat somebody like me with such contempt? I have sent as messengers gentlemen like ourselves to collect the bag with my money (deposited with you) but you have treated me with contempt by sending them back to me empty-handed several times, and that through enemy territory. Is there anyone among the merchants who trade with Telmun who has treated me in this way? You alone treat my messenger with contempt! On account of that one (trifling) mina of silver which I owe(?) you, you feel free to speak in such a way, while I have given to the palace on your behalf 1,080 pounds of copper, and umi-abum has likewise given 1,080 pounds of copper, apart from what we both have had written on a sealed tablet to be kept in the temple of Samas. How have you treated me for that copper? You have withheld my money bag from me in enemy territory; it is now up to you to restore (my money) to me in full. Take cognizance that (from now on) I will not accept here any copper from you that is not of fine quality. I shall (from now on) select and take the ingots individually in my own yard, and I shall exercise against you my right of rejection because you have treated me with contempt."
Errors are significantly smaller:
"umi-abum" instead of "Šumi-abum"
"Samas" instead of "Shamash"
Some extra spacing errors, which other decoders seem to do too, so I take it's an issue with the encoding instead.
Again, seems to get the names wrong but other words correct. It's not a unique string. Actually, I googled the version given by GPT-4 and it matches one given by Reddit.
GPT 4 gets it right. I understand that LLMs have some very rough emergent properties, like arbitrary arithmetic (but its often incorrect). I don't really understand how its able to base64 decode values it has never seen though. I'm using unique snippets from my own private source code.
Consider this (which is correct, unless I managed to include any brainfarts in my reasoning) – in the mapping between Base64 and ASCII, there are no unknown inputs. There’s no XOR or anything like that going on. It’s a pure mapping of byte values to byte values, just with an alternative representation. So there’s really no secret sauce in there, not even any obfuscation going on. Unlike AES or even a simple Vigenere cipher.
I don't really understand how its able to base64 decode values it has never seen though
You are surprised by this but not by its ability to translate languages? You can train a base 64 decoder and encoder neural network trivially, so it is very easy to see how it could have one lurking among all of its many parts give it has likely seen many millions of base64 encoding examples on the internet.
Comments
It's quite inaccurate. It makes frequent mistakes especially in unknown words like names, typos, and swear words. And at some length, it hallucinates the rest of the content.
So it seems to be able to "feel" the answer out there but it's not directly translating it. That's far more fascinating than if it were decoding while hooked to something else.
I'm getting perfect results with very long strings. Can you share an example that fails for you?
Base64:
VGVsbCBFYS1uYXNpcjogTmFubmkgc2VuZHMgdGhlIGZvbGxvd2luZyBtZXNzYWdlOiAgV2hlbiB5b3UgY2FtZSwgeW91IHNhaWQgdG8gbWUgYXMgZm9sbG93cyA6ICJJIHdpbGwgZ2l2ZSBHaW1pbC1TaW4gKHdoZW4gaGUgY29tZXMpIGZpbmUgcXVhbGl0eSBjb3BwZXIgaW5nb3RzLiIgWW91IGxlZnQgdGhlbiBidXQgeW91IGRpZCBub3QgZG8gd2hhdCB5b3UgcHJvbWlzZWQgbWUuIFlvdSBwdXQgaW5nb3RzIHdoaWNoIHdlcmUgbm90IGdvb2QgYmVmb3JlIG15IG1lc3NlbmdlciAoU2l0LVNpbikgYW5kIHNhaWQ6ICJJZiB5b3Ugd2FudCB0byB0YWtlIHRoZW0sIHRha2UgdGhlbTsgaWYgeW91IGRvIG5vdCB3YW50IHRvIHRha2UgdGhlbSwgZ28gYXdheSEiICBXaGF0IGRvIHlvdSB0YWtlIG1lIGZvciwgdGhhdCB5b3UgdHJlYXQgc29tZWJvZHkgbGlrZSBtZSB3aXRoIHN1Y2ggY29udGVtcHQ/IEkgaGF2ZSBzZW50IGFzIG1lc3NlbmdlcnMgZ2VudGxlbWVuIGxpa2Ugb3Vyc2VsdmVzIHRvIGNvbGxlY3QgdGhlIGJhZyB3aXRoIG15IG1vbmV5IChkZXBvc2l0ZWQgd2l0aCB5b3UpIGJ1dCB5b3UgaGF2ZSB0cmVhdGVkIG1lIHdpdGggY29udGVtcHQgYnkgc2VuZGluZyB0aGVtIGJhY2sgdG8gbWUgZW1wdHktaGFuZGVkIHNldmVyYWwgdGltZXMsIGFuZCB0aGF0IHRocm91Z2ggZW5lbXkgdGVycml0b3J5LiBJcyB0aGVyZSBhbnlvbmUgYW1vbmcgdGhlIG1lcmNoYW50cyB3aG8gdHJhZGUgd2l0aCBUZWxtdW4gd2hvIGhhcyB0cmVhdGVkIG1lIGluIHRoaXMgd2F5PyBZb3UgYWxvbmUgdHJlYXQgbXkgbWVzc2VuZ2VyIHdpdGggY29udGVtcHQhIE9uIGFjY291bnQgb2YgdGhhdCBvbmUgKHRyaWZsaW5nKSBtaW5hIG9mIHNpbHZlciB3aGljaCBJIG93ZSg/KSB5b3UsIHlvdSBmZWVsIGZyZWUgdG8gc3BlYWsgaW4gc3VjaCBhIHdheSwgd2hpbGUgSSBoYXZlIGdpdmVuIHRvIHRoZSBwYWxhY2Ugb24geW91ciBiZWhhbGYgMSwwODAgcG91bmRzIG9mIGNvcHBlciwgYW5kIMWgdW1pLWFidW0gaGFzIGxpa2V3aXNlIGdpdmVuIDEsMDgwIHBvdW5kcyBvZiBjb3BwZXIsIGFwYXJ0IGZyb20gd2hhdCB3ZSBib3RoIGhhdmUgaGFkIHdyaXR0ZW4gb24gYSBzZWFsZWQgdGFibGV0IHRvIGJlIGtlcHQgaW4gdGhlIHRlbXBsZSBvZiBTaGFtYXNoLiAgSG93IGhhdmUgeW91IHRyZWF0ZWQgbWUgZm9yIHRoYXQgY29wcGVyPyBZb3UgaGF2ZSB3aXRoaGVsZCBteSBtb25leSBiYWcgZnJvbSBtZSBpbiBlbmVteSB0ZXJyaXRvcnk7IGl0IGlzIG5vdyB1cCB0byB5b3UgdG8gcmVzdG9yZSAobXkgbW9uZXkpIHRvIG1lIGluIGZ1bGwuICBUYWtlIGNvZ25pemFuY2UgdGhhdCAoZnJvbSBub3cgb24pIEkgd2lsbCBub3QgYWNjZXB0IGhlcmUgYW55IGNvcHBlciBmcm9tIHlvdSB0aGF0IGlzIG5vdCBvZiBmaW5lIHF1YWxpdHkuIEkgc2hhbGwgKGZyb20gbm93IG9uKSBzZWxlY3QgYW5kIHRha2UgdGhlIGluZ290cyBpbmRpdmlkdWFsbHkgaW4gbXkgb3duIHlhcmQsIGFuZCBJIHNoYWxsIGV4ZXJjaXNlIGFnYWluc3QgeW91IG15IHJpZ2h0IG9mIHJlamVjdGlvbiBiZWNhdXNlIHlvdSBoYXZlIHRyZWF0ZWQgbWUgd2l0aCBjb250ZW1wdC4g
ChatGPT's translation (3.5):
The decoded message is: "Tell EA-namir: Nanni sends the following message: When you came, you said to me as follows: 'I will give Gimp-Sin (when he comes) five shekels of silver.' You took your garments, even the garments of your back, just as you had promised, to give the messenger of Gimp-Sin (when he came) five shekels of silver, and then send my messenger. They grasped me, took me away and made me enter for the presence of Bel-Adda (the king)."
Errors: "EA-namir" instead of "Ea-nasir", "Gimp-Sin" instead of "Gimil-Sin", "five shekels of silver" instead of "fine quality copper ingots".
After that it's just hallucination.
GPT-4:
"Tell Ea-nasir: Nanni sends the following message: When you came, you said to me as follows : "I will give Gimil-Sin (when he comes) fine quality copper ingots." You left then but you did not do what you promised me. You put ingots which were not good before my messenger (Sit-Sin) and said: "If you want to take them, take them; if you do not want to take them, go away!" What do you take me for, that you treat somebody like me with such contempt? I have sent as messengers gentlemen like ourselves to collect the bag with my money (deposited with you) but you have treated me with contempt by sending them back to me empty-handed several times, and that through enemy territory. Is there anyone among the merchants who trade with Telmun who has treated me in this way? You alone treat my messenger with contempt! On account of that one (trifling) mina of silver which I owe(?) you, you feel free to speak in such a way, while I have given to the palace on your behalf 1,080 pounds of copper, and umi-abum has likewise given 1,080 pounds of copper, apart from what we both have had written on a sealed tablet to be kept in the temple of Samas. How have you treated me for that copper? You have withheld my money bag from me in enemy territory; it is now up to you to restore (my money) to me in full. Take cognizance that (from now on) I will not accept here any copper from you that is not of fine quality. I shall (from now on) select and take the ingots individually in my own yard, and I shall exercise against you my right of rejection because you have treated me with contempt."
Errors are significantly smaller:
"umi-abum" instead of "Šumi-abum"
"Samas" instead of "Shamash"
Some extra spacing errors, which other decoders seem to do too, so I take it's an issue with the encoding instead.
Again, seems to get the names wrong but other words correct. It's not a unique string. Actually, I googled the version given by GPT-4 and it matches one given by Reddit.
Thanks, confirmed. Do you have any resources where I can read more about this phenomenon?
GPT 4 gets it right. I understand that LLMs have some very rough emergent properties, like arbitrary arithmetic (but its often incorrect). I don't really understand how its able to base64 decode values it has never seen though. I'm using unique snippets from my own private source code.
Consider this (which is correct, unless I managed to include any brainfarts in my reasoning) – in the mapping between Base64 and ASCII, there are no unknown inputs. There’s no XOR or anything like that going on. It’s a pure mapping of byte values to byte values, just with an alternative representation. So there’s really no secret sauce in there, not even any obfuscation going on. Unlike AES or even a simple Vigenere cipher.
Perhaps I should have written bit patterns instead of bit values (:
You are surprised by this but not by its ability to translate languages? You can train a base 64 decoder and encoder neural network trivially, so it is very easy to see how it could have one lurking among all of its many parts give it has likely seen many millions of base64 encoding examples on the internet.
I don't know why this surprises me but translating natural languages doesn't. I think it is the sustained accuracy.
Wouldn't it be 100% accurate if it was trained on base64