That assumes you control both the producer and consumer. But if you're doing CSV, it's likely because you're looking to integrate with someone else's system. So you have to deal with whatever they're doing that they call CSV. And if "they" are "all your customers", you're going to encounter every weird quirk of different system's CSV parsing from that guy who just used
String.split(",").map(it.replace("\"\"", "\""))
to Spark insisting that backslash escapes exist in CSV.
For extra fun consider the German-Speaking word where CSV files are actually Semicolon-Separated but everyone still calls them CSV and looks at you like you drooled on yourself when you point out that “;” is not a “,”.
This appears to be because we use “,” as decimal separator and were too dense to learn how to use " properly in CSV.
How do you know the variant? If the user is just using an "import csv" on the web form, how do you know? You can't even ask the user because it's not even clear they'll know, it's just the CSV they got from Jira, their other vendor, whatever.
And then the other direction is even worse, when you emit a RFC compliant CSV and the other party complains their batch job chokes on "" escaped quotes, etc., so you end up holding a mapping of clients -> "CSV" formats
Comments
That assumes you control both the producer and consumer. But if you're doing CSV, it's likely because you're looking to integrate with someone else's system. So you have to deal with whatever they're doing that they call CSV. And if "they" are "all your customers", you're going to encounter every weird quirk of different system's CSV parsing from that guy who just used
to Spark insisting that backslash escapes exist in CSV.That’s the real problem. That those system claim the read or write CSV but in reality it’s a bastard form of CSV.
Never had a problem when both sides know the rules.
But I doubt those other system even support ASCII separated value files.
It’s all bastard CSV out here in the real world.
For extra fun consider the German-Speaking word where CSV files are actually Semicolon-Separated but everyone still calls them CSV and looks at you like you drooled on yourself when you point out that “;” is not a “,”.
This appears to be because we use “,” as decimal separator and were too dense to learn how to use " properly in CSV.
Bastard CSVs as far as the eye can see.
It's easier in most cases to just use a parser flexible enough for you to specify whatever variant the producer actually emitted.
How do you know the variant? If the user is just using an "import csv" on the web form, how do you know? You can't even ask the user because it's not even clear they'll know, it's just the CSV they got from Jira, their other vendor, whatever.
And then the other direction is even worse, when you emit a RFC compliant CSV and the other party complains their batch job chokes on "" escaped quotes, etc., so you end up holding a mapping of clients -> "CSV" formats