Lossless encoding of doubles is a long solved problem. Use hexadecimal floating point format (%a with printf in C99, inherited by most high-level languages that sit on top of C). The problem is platforms that still don't support C99.
I was wondering why he said that CSV was not capable of supporting full precision for doubles. Are you saying that the problem is not with CSV per se, but instead the C libc printf function? I'd have thought that modern standard libraries would do this right with the usual decimal %f and a large enough precision: http://pubs.opengroup.org/onlinepubs/009695399/functions/pri...
The trouble is that (a) the standard recommends but does not require that all binary-to-decimal and decimal-to-binary conversions be correctly rounded and (b) even if it did require correctly-rounded conversions, the precision needs to be absolutely enormous to guarantee that every double is exactly representable in the chosen format (which makes it wildly wasteful), otherwise you'd get different results depending on the rounding mode in effect at the time of conversion back to double.
Comments
That doesn't solve the lossless encoding of doubles problem.
Lossless encoding of doubles is a long solved problem. Use hexadecimal floating point format (%a with printf in C99, inherited by most high-level languages that sit on top of C). The problem is platforms that still don't support C99.
I was wondering why he said that CSV was not capable of supporting full precision for doubles. Are you saying that the problem is not with CSV per se, but instead the C libc printf function? I'd have thought that modern standard libraries would do this right with the usual decimal %f and a large enough precision: http://pubs.opengroup.org/onlinepubs/009695399/functions/pri...
The trouble is that (a) the standard recommends but does not require that all binary-to-decimal and decimal-to-binary conversions be correctly rounded and (b) even if it did require correctly-rounded conversions, the precision needs to be absolutely enormous to guarantee that every double is exactly representable in the chosen format (which makes it wildly wasteful), otherwise you'd get different results depending on the rounding mode in effect at the time of conversion back to double.
It is absolutely a solved problem, but printing hex floats into JSON or a CSV isn't going to be great when you need to re-import the data somewhere.