There is fundamental a difference between Unicode and UTF-8/16/32. Unicode says nothing at all about how strings should be stored in memory. It is just a set of symbols and code-points. On the other hand, UTF-8/16/32 are implementations of Unicode. See http://en.wikipedia.org/wiki/Unicode#Mapping_and_encodings for others.
Statements like "first 256 code points in Unicode map to Latin-1" make little sense. They are true if you s/Unicode/UTF-8/g. However, they are not true for other encodings such as ones that use 2 or 4 bytes per character. There 'abcd' is not 4 bytes, but 8 or 16.
> Statements like "first 256 code points in Unicode map to Latin-1" make little sense.
That's not true. Latin-1 is both a character set and an encoding, so "the first 256 Unicode code points map to the corresponding Latin-1 characters" is a reasonable statement.
You can also say "the first 128 Unicode code points, when encoded in UTF-8, are equal to the corresponding Latin-1 encoding".
My point is that in Python land it is silly to call u"" strings, "Unicode" strings. Unicode strings are strings in UTF-8/16/32 and a bunch of lesser-used encodings. For that matter "" could be used as a Unicode string as long as it's only ASCII. What the docs should be talking about is ASCII vs UTF-16, not ASCII/Latin-1 vs Unicode. This starts making a difference when questions like "How much memory is consumed by this string?" or "What characters can I not store in Python?" are asked. In this light, Python 3 makes a big improvement: it has immutable byte arrays and it has encoded strings.
Comments
There is fundamental a difference between Unicode and UTF-8/16/32. Unicode says nothing at all about how strings should be stored in memory. It is just a set of symbols and code-points. On the other hand, UTF-8/16/32 are implementations of Unicode. See http://en.wikipedia.org/wiki/Unicode#Mapping_and_encodings for others.
Statements like "first 256 code points in Unicode map to Latin-1" make little sense. They are true if you s/Unicode/UTF-8/g. However, they are not true for other encodings such as ones that use 2 or 4 bytes per character. There 'abcd' is not 4 bytes, but 8 or 16.
> Statements like "first 256 code points in Unicode map to Latin-1" make little sense.
That's not true. Latin-1 is both a character set and an encoding, so "the first 256 Unicode code points map to the corresponding Latin-1 characters" is a reasonable statement.
You can also say "the first 128 Unicode code points, when encoded in UTF-8, are equal to the corresponding Latin-1 encoding".
My point is that in Python land it is silly to call u"" strings, "Unicode" strings. Unicode strings are strings in UTF-8/16/32 and a bunch of lesser-used encodings. For that matter "" could be used as a Unicode string as long as it's only ASCII. What the docs should be talking about is ASCII vs UTF-16, not ASCII/Latin-1 vs Unicode. This starts making a difference when questions like "How much memory is consumed by this string?" or "What characters can I not store in Python?" are asked. In this light, Python 3 makes a big improvement: it has immutable byte arrays and it has encoded strings.
Obligatory http://www.joelonsoftware.com/articles/Unicode.html