Eventually you have to deal with content that contains your separator characters, however obscure. So essentially you have two choices:
A. use some "weird" separators and hope those don't appear in your input
B. bite the bullet and escape and parse properly
Option A is perfectly reasonable for one-offs, where you can handle exceptional cases or know they won't occur because you know what's in the data. However for reusable code, you need option B, which means not using `cut` to parse CSV files, for instance (since commas can occur inside double-quoted strings). In that case, what's the benefit of using USV over an existing, more common, format?
Orthogonal to escaping, the choice is what characters to use for unit separator and record separator.
If the data are for machines only, then for me the choice of characters doesn't matter. If the data are potentially for reading or editing, such as by a programmer, then my choice is to prefer typically-visible characters over typically-invisible characters and/or zero-width characters (e.g. ASV a.k.a. DEL a.k.a. ASCII 30 & 31).
My choice of USV is thus because U+241F and U+241E are visible, and also in Unicode they are semantically meaningful.
I'm still not sure what the value is over CSV, which also has visible delimiters. It's true that you have to establish/enforce a specific convention around escaping and quoting, since CSV has historical variation here. But it would make more sense to me to encourage any particular consistent handling of CSV, rather than yet another entirely new separator. At least some tools already support CSV, whereas nothing currently supports USV, as far as I know.
Comments
Eventually you have to deal with content that contains your separator characters, however obscure. So essentially you have two choices:
A. use some "weird" separators and hope those don't appear in your input
B. bite the bullet and escape and parse properly
Option A is perfectly reasonable for one-offs, where you can handle exceptional cases or know they won't occur because you know what's in the data. However for reusable code, you need option B, which means not using `cut` to parse CSV files, for instance (since commas can occur inside double-quoted strings). In that case, what's the benefit of using USV over an existing, more common, format?
Yes you're exactly right about escaping.
Orthogonal to escaping, the choice is what characters to use for unit separator and record separator.
If the data are for machines only, then for me the choice of characters doesn't matter. If the data are potentially for reading or editing, such as by a programmer, then my choice is to prefer typically-visible characters over typically-invisible characters and/or zero-width characters (e.g. ASV a.k.a. DEL a.k.a. ASCII 30 & 31).
My choice of USV is thus because U+241F and U+241E are visible, and also in Unicode they are semantically meaningful.
Glad we agree on escaping.
I'm still not sure what the value is over CSV, which also has visible delimiters. It's true that you have to establish/enforce a specific convention around escaping and quoting, since CSV has historical variation here. But it would make more sense to me to encourage any particular consistent handling of CSV, rather than yet another entirely new separator. At least some tools already support CSV, whereas nothing currently supports USV, as far as I know.