Skip to content

Comment on Large integer precision error in Bash command output renderingparent

Comments

It seems like it, but it can't be only that. A float64 representation of an int in the range 2^58 to 2^59 should be rounded to multiples of 2^6, i.e. 348555896224571968 as you found (3.48555896224571968E17) (the final digit 9 in the math.log2() expression was lost, it's 2^58 not 2^54) The unexpected output (according to the bugreport) arises from javascript, it does NOT round like everything else for reasons I don't understand. It seems to prefer rounding to arbitrary multiples of 10 in my limited testing.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.