Java, NT, .NET, "wide" C and C++ and a few others from the same time frame ended up with WTF-16 because surrogate pairs didn't exist when they were designed. They were designed with UCS-2, which is a fixed-length encoding. Unicode 2.0 then extended that to be variable length (16/32-bit) using surrogate pairs and that's where all the systems come from which don't validate surrogate pairs.
"Wide" characters in C and C++ do not have a standardized width. On most non-Windows platforms they are actually four bytes so can cover the entirety of Unicode and then some in one wchar_t. Not that that's a more reasonable encoding for most applications than WTF-16.
Comments
Java, NT, .NET, "wide" C and C++ and a few others from the same time frame ended up with WTF-16 because surrogate pairs didn't exist when they were designed. They were designed with UCS-2, which is a fixed-length encoding. Unicode 2.0 then extended that to be variable length (16/32-bit) using surrogate pairs and that's where all the systems come from which don't validate surrogate pairs.
Is that still true with dotnet? Especially since it has a surrogate pair api and a rune api? https://learn.microsoft.com/de-de/dotnet/api/system.char.iss... https://learn.microsoft.com/en-us/dotnet/api/system.text.run...
"Wide" characters in C and C++ do not have a standardized width. On most non-Windows platforms they are actually four bytes so can cover the entirety of Unicode and then some in one wchar_t. Not that that's a more reasonable encoding for most applications than WTF-16.
Where did they think Cuneiform was going to fit? Even with unihan the BMP was getting consumed fast.
In the beginning Unicode was only meant to be used for extant languages afaik.
Yes, the clusterfuck that is CJK unification would not have been done if it was known that Unicode was going to go beyond 16-bit anyway.
Surrogate pairs existed then, they just ignored them