Skip to content

Comment on Ask HN: largest number you've had to store in an application?

Comments

I built an aviation mapping engine that used fixed point math for lat/lon representation. It basically used an entire 32bit signed int and gave a guaranteed minimal resolution of 3 feet. It would quite often go to the ragged edge of what you can fit in a 32 bit quantity.

The fun bit was that you had to use 2 combined registers (ARM) to represent a 64bit quantity so you could do multiplication and keep all your precision.

Oh, for the luxury of a FPU...

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.