I disagree. UTF-8 is the right default. Pure ASCII strings are rarely needed in modern software. Unless you know that you need ASCII, your strings should support unicode.
Perhaps so, and that's a good development imho. However, the most notorious of all, HTTP, has not afaik. Different encodings have been suggested including UTF-8. I'm not entirely sure why those proposals weren't implemented, but the dot-com bubble at the time might've had something to do with it.
The first networked computers predate unicode by several decades. Back then, the word length could be 8 but not neccesarily. All sorts of different encodings existed and a given common standard wasn't yet agreed upon
It's not like internet standards don't know there are other languages, it's just that they documented how things were done at the time. Some legacy has remained ever since.
Sure, but most code doesn't interact with raw packets. Legacy, ascii-only internet standards probably make up much less than 1% of new lines of code written today. Programming languages should support this use case. But not at the expense of the other 99% of software.
Unicode-aware strings are the right choice for 99% of code. The last 1% should be a special case.
Yes, plenty of times. But I’m usually using raw byte arrays via protobuf or something similar. ASCII only text protocols are rare. Most devs don’t reimplement them from scratch.
I’d wager most code is application code, where UTF8 strings are a great choice.
Comments
I disagree. UTF-8 is the right default. Pure ASCII strings are rarely needed in modern software. Unless you know that you need ASCII, your strings should support unicode.
Internet Messages have been ASCII since Internet Messages.
Most internet protocols these days have shifted to UTF-8.
Perhaps so, and that's a good development imho. However, the most notorious of all, HTTP, has not afaik. Different encodings have been suggested including UTF-8. I'm not entirely sure why those proposals weren't implemented, but the dot-com bubble at the time might've had something to do with it.
True, but humans have been speaking non English languages since humans.
The first networked computers predate unicode by several decades. Back then, the word length could be 8 but not neccesarily. All sorts of different encodings existed and a given common standard wasn't yet agreed upon
It's not like internet standards don't know there are other languages, it's just that they documented how things were done at the time. Some legacy has remained ever since.
Sure, but most code doesn't interact with raw packets. Legacy, ascii-only internet standards probably make up much less than 1% of new lines of code written today. Programming languages should support this use case. But not at the expense of the other 99% of software.
Unicode-aware strings are the right choice for 99% of code. The last 1% should be a special case.
Raw packets? Legacy? Have you've ever talked to anything over a socket?
You don't need a raw socket to get into trouble. You also don't need a "legacy" protocol.
Yes, plenty of times. But I’m usually using raw byte arrays via protobuf or something similar. ASCII only text protocols are rare. Most devs don’t reimplement them from scratch.
I’d wager most code is application code, where UTF8 strings are a great choice.