One thing I always wondered about is why are we using floating point arithmetic at all, instead of fixed point math with explicitly specified ranges (say, "here I need 20 bits for the integer part and 44 for the fractional part")? What is the practical value of having a floating point that would justify dealing with all that complexity and conceptual problems they introduce?
Have you used fixed-point arithmetic much? You need to specify the width and the position of the dot for each intermediate variable (for instance, multiplying two numbers that have 20 bits for the integer part may require 40 bits for the integer part of the result). Since the operands in 20.44 format had only 64 bits of precision, it only makes sense to represent the result to 64 bits, so the format of this result should probably be 40.24.
If you don't do this, your program may still work but you are wasting space. For instance, if the multiplication of your two 20.44 numbers does not overflow the 20.44 format, good for you, but it means that between the two operands, you were wasting 20 bits in total for leading zeroes that carried no information.
Consider floating-point as a way not to have to make all these decisions at compile-time and still use space extremely efficiently to represent as much precision as possible.
Also: when people screw with FP, 99% of the time the answer is still vaguely sane or makes it very obvious that something went wrong. When people screw up with fixed point, 99% of the time the result is indistinguishable from noise.
Because of hardware, and the fact that fixed point doesn't solve the core problems with floating point as it stands today. Its hard enough to get good stable/fast floating point hardware, much less adding the complexities of variable sized words, with variable sized mantissa/exponents.
Really, what your probably asking for is decimal floating point. Which, has its own set of limitations when implemented in hardware.
And if you happen to have an application where what you really need is more precision there are a number of bignum libraries like GMP (https://gmplib.org/) which provide that. But, throwing more precision at a problem doesn't alleviate the need to do error estimation.
Dynamic range. A double-precision variable has the same precision at 1e+10 as at 1e-10, your proposal does not. This is not a big problem if you have a good idea about the numbers in your application (e.g., digital signal processing), but it is a big problem if you write a matrix library (or anything else which deals with real numbers) for arbitrary inputs.
But OTOH having "the same precision" means loosing accuracy. This is a common case in computer games, especially with outdoor worlds like space simulators. When you move away from the world center, you start loosing significant digits on the fractional side and your calculations soon turn into nonsense. When you go far enough, the difference between two consecutive numbers can get as high as 0.1, at which point everything falls apart.
KSP scene has even a fun name for it, Deep Space Kraken [0].
This can be of course mitigated by subdividing the world (octrees and stuff) and stacking coordinate frames (though for some reason nobody seems to be doing that), but you said that floats are good "for arbitrary inputs". And if you have small and big floats in your computations, the result might quickly stop making any sense.
I admit I'm confused about the topic, any clarification is welcome.
Any particular fixed point format will have slightly more precision/accuracy at the edges than a floating point format with the same number of bits, but then it will hit a brick wall and stop working entirely. And it's easy for it to have less precision/accuracy in practice because it's hard to keep each intermediate result at the exact right exponent.
Fixed point won't save you from the space kraken. Instead of going so deep into space that consecutive numbers gradually become .1 apart, your ship will spontaneously explode a fiftieth of that distance out.
It's probably much easier to use two coordinate frames than to properly use fixed point numbers. Then a ship can run physics relative to its center of mass or a nearby planet/moon and have very high precision/accuracy.
On one hand FP lets you program without thinking too hard about the numerical properties of your program - until you hit a gotcha. On the other hand FP is pretty powerful for many domains if you're a FP wizard. Like physics simulation.
There have been other approaches besides fixed that people have tried out, such as interval arithmetic and exact (rational) arithmetic, but they all have their own problems.
Comments
One thing I always wondered about is why are we using floating point arithmetic at all, instead of fixed point math with explicitly specified ranges (say, "here I need 20 bits for the integer part and 44 for the fractional part")? What is the practical value of having a floating point that would justify dealing with all that complexity and conceptual problems they introduce?
Have you used fixed-point arithmetic much? You need to specify the width and the position of the dot for each intermediate variable (for instance, multiplying two numbers that have 20 bits for the integer part may require 40 bits for the integer part of the result). Since the operands in 20.44 format had only 64 bits of precision, it only makes sense to represent the result to 64 bits, so the format of this result should probably be 40.24.
If you don't do this, your program may still work but you are wasting space. For instance, if the multiplication of your two 20.44 numbers does not overflow the 20.44 format, good for you, but it means that between the two operands, you were wasting 20 bits in total for leading zeroes that carried no information.
Consider floating-point as a way not to have to make all these decisions at compile-time and still use space extremely efficiently to represent as much precision as possible.
Also: when people screw with FP, 99% of the time the answer is still vaguely sane or makes it very obvious that something went wrong. When people screw up with fixed point, 99% of the time the result is indistinguishable from noise.
Because of hardware, and the fact that fixed point doesn't solve the core problems with floating point as it stands today. Its hard enough to get good stable/fast floating point hardware, much less adding the complexities of variable sized words, with variable sized mantissa/exponents.
Really, what your probably asking for is decimal floating point. Which, has its own set of limitations when implemented in hardware.
And if you happen to have an application where what you really need is more precision there are a number of bignum libraries like GMP (https://gmplib.org/) which provide that. But, throwing more precision at a problem doesn't alleviate the need to do error estimation.
Dynamic range. A double-precision variable has the same precision at 1e+10 as at 1e-10, your proposal does not. This is not a big problem if you have a good idea about the numbers in your application (e.g., digital signal processing), but it is a big problem if you write a matrix library (or anything else which deals with real numbers) for arbitrary inputs.
But OTOH having "the same precision" means loosing accuracy. This is a common case in computer games, especially with outdoor worlds like space simulators. When you move away from the world center, you start loosing significant digits on the fractional side and your calculations soon turn into nonsense. When you go far enough, the difference between two consecutive numbers can get as high as 0.1, at which point everything falls apart.
KSP scene has even a fun name for it, Deep Space Kraken [0].
This can be of course mitigated by subdividing the world (octrees and stuff) and stacking coordinate frames (though for some reason nobody seems to be doing that), but you said that floats are good "for arbitrary inputs". And if you have small and big floats in your computations, the result might quickly stop making any sense.
I admit I'm confused about the topic, any clarification is welcome.
[0] - http://wiki.kerbalspaceprogram.com/wiki/Deep_Space_Kraken
Any particular fixed point format will have slightly more precision/accuracy at the edges than a floating point format with the same number of bits, but then it will hit a brick wall and stop working entirely. And it's easy for it to have less precision/accuracy in practice because it's hard to keep each intermediate result at the exact right exponent.
Fixed point won't save you from the space kraken. Instead of going so deep into space that consecutive numbers gradually become .1 apart, your ship will spontaneously explode a fiftieth of that distance out.
It's probably much easier to use two coordinate frames than to properly use fixed point numbers. Then a ship can run physics relative to its center of mass or a nearby planet/moon and have very high precision/accuracy.
On one hand FP lets you program without thinking too hard about the numerical properties of your program - until you hit a gotcha. On the other hand FP is pretty powerful for many domains if you're a FP wizard. Like physics simulation.
There have been other approaches besides fixed that people have tried out, such as interval arithmetic and exact (rational) arithmetic, but they all have their own problems.