PHP devs are so slow they just adopted utf8 and see its glory.
I myself "UTF8 or die!"d a long time ago and discovered it was not a good idea.
I will forget the problem of the parsing of the nth character, the string length vs the memory used, the canonization of strings for comparison. And go directly to 2 problems:
* There exists cases in which latin1 & utf8 are mangled in a same string. (ex http/SIP headers are latin1 and content can be utf8 and you may want to store a full HTTP/SIP transaction verbatim for debugging purpose), and it can store in iso-latin3 (code table for esperanto to be sarcastic), but will explode in utf8 unless you rencode it (B64)
* tools are partly UTF8 compliant: mysql (which is as good as PHP in terms of quality) is clueless about UTF8 (hint index and collation), and PHP too https://bugs.php.net/bug.php?id=18556 <--- THIS BUG TOOK 10 YEARS TO BE CLOSED
The whole point is developer don't understand the organic nature of culture, and especially of its writing and the diversity of culture.
They think that because some rules applies in their language it also applies in others: BUT
* PHP devs: lowercase of I is not always i (it can be i without a dot). It took 10 years to the dev to find where their bug was!
* shortening a memory representation does not always shorten its graphical representation (apples bug with sms in arabian)
* sort orders are not immutable (collation not only can vary from language to language but also according to the administrative context (ex: proper name in french))
* inflexions are hell and text size for error varies a lot (hence the unstability of windows 95 in french because error message where copied in a reserved page and the fixed size was less than the whole size of the domain's corpus... hence any contiguous block in memory (lower xor upper bound) would have its memory potentially corrupted)).
My point is UTF8 is not hell. Real world is complex. And it becomes hell when some dumb devs thinking that by manipulating strings that represents any language they know about any language.. and apply universal rules that are not.
Some problems can be solved by ignoring them. But with culture it is not the case.
And actually, unicode SUX because it is US centrics
* computers should be able to store all our pasts books and make them available for the future, even in verbatim mode. But unicode HAS not archeological character sets like THIS https://fr.wikipedia.org/wiki/S_long I don't care about the USA lack of history. I see no use in the computer if it requires to sacrifice our histories and cultures,
* https://modelviewculture.com/pieces/i-can-text-you-a-pile-of... some people cannot even use it in their own language
Unicode suffers a lot of problem plus a conceptual one; it is has immutable characters AND directives (change the direction of the writing, apply ligatures)... that not only will create security concerns (one of the funniest being the possibility by adding a string to reedit silently text in on an effector (screen or printer))... We are introducing type setting rules in unicode.
For those who have used tex since a long time, the non separation of the almost programmatical typography and the graphens is like not separing the model and the controller.
Which actually also calls for the view (the effection) and thus the fonts. Having the encoding of the Slong does not tell you what it looks like unless you have a canonical representation of the codepoint as a graphem.
And since we are printing/creating document for juridical purpose we may like to control the view that ensures that the mapping of the string representation will not alter graphical representation in a way that can compromise its meaning. If someone signs in a box you don't want the signature to alter the representation anywhere or worse without notice.
The devil lies in the detail. Unicode is a Babel tower that may well crash for the same reason as in the bible: hubris.
Comments
PHP devs are so slow they just adopted utf8 and see its glory.
I myself "UTF8 or die!"d a long time ago and discovered it was not a good idea.
I will forget the problem of the parsing of the nth character, the string length vs the memory used, the canonization of strings for comparison. And go directly to 2 problems:
* There exists cases in which latin1 & utf8 are mangled in a same string. (ex http/SIP headers are latin1 and content can be utf8 and you may want to store a full HTTP/SIP transaction verbatim for debugging purpose), and it can store in iso-latin3 (code table for esperanto to be sarcastic), but will explode in utf8 unless you rencode it (B64)
* tools are partly UTF8 compliant: mysql (which is as good as PHP in terms of quality) is clueless about UTF8 (hint index and collation), and PHP too https://bugs.php.net/bug.php?id=18556 <--- THIS BUG TOOK 10 YEARS TO BE CLOSED
The whole point is developer don't understand the organic nature of culture, and especially of its writing and the diversity of culture.
They think that because some rules applies in their language it also applies in others: BUT
* PHP devs: lowercase of I is not always i (it can be i without a dot). It took 10 years to the dev to find where their bug was! * shortening a memory representation does not always shorten its graphical representation (apples bug with sms in arabian) * sort orders are not immutable (collation not only can vary from language to language but also according to the administrative context (ex: proper name in french)) * inflexions are hell and text size for error varies a lot (hence the unstability of windows 95 in french because error message where copied in a reserved page and the fixed size was less than the whole size of the domain's corpus... hence any contiguous block in memory (lower xor upper bound) would have its memory potentially corrupted)).
My point is UTF8 is not hell. Real world is complex. And it becomes hell when some dumb devs thinking that by manipulating strings that represents any language they know about any language.. and apply universal rules that are not.
Some problems can be solved by ignoring them. But with culture it is not the case.
And actually, unicode SUX because it is US centrics
* computers should be able to store all our pasts books and make them available for the future, even in verbatim mode. But unicode HAS not archeological character sets like THIS https://fr.wikipedia.org/wiki/S_long I don't care about the USA lack of history. I see no use in the computer if it requires to sacrifice our histories and cultures, * https://modelviewculture.com/pieces/i-can-text-you-a-pile-of... some people cannot even use it in their own language
Unicode suffers a lot of problem plus a conceptual one; it is has immutable characters AND directives (change the direction of the writing, apply ligatures)... that not only will create security concerns (one of the funniest being the possibility by adding a string to reedit silently text in on an effector (screen or printer))... We are introducing type setting rules in unicode.
For those who have used tex since a long time, the non separation of the almost programmatical typography and the graphens is like not separing the model and the controller.
Which actually also calls for the view (the effection) and thus the fonts. Having the encoding of the Slong does not tell you what it looks like unless you have a canonical representation of the codepoint as a graphem.
And since we are printing/creating document for juridical purpose we may like to control the view that ensures that the mapping of the string representation will not alter graphical representation in a way that can compromise its meaning. If someone signs in a box you don't want the signature to alter the representation anywhere or worse without notice.
The devil lies in the detail. Unicode is a Babel tower that may well crash for the same reason as in the bible: hubris.