enriquto on Can LLMs do randomness?

Comment on Can LLMs do randomness?parent

floating point operations like addition are not commutative

maybe you meant associative? Floating point addition is commutative: a+b is always equal to b+a for any values of a and b. It is not associative, though: a+(b+c) is in general different to (a+b)+c, think what happens if a is tiny and b,c are huge, for example.

david-gpu 1y

Sorry, yes, I meant associative. Thanks for the important correction.

To think that I used to do this for a living...

simulator5g 1y

How is that any different? 1+(2+3) = 6

(1+2)+3 = 6

0.000001+(200000+300000) = 500000.000001

(0.000001+200000)+300000 = 500000.000001

david-gpu 1y

You need to take it a step further, since e.g. 64-bit floats have a ton of mantissa bits.

Here's an example in python3.

    >>> "{:.2f}".format(1e16 + (1 + 1))
    '10000000000000002.00'
    >>> "{:.2f}".format((1e16 + 1) + 1)
    '10000000000000000.00'

enriquto 1y

take b and c with opposite signs

Comments