Comment on Microsoft researchers developed a hyper-efficient AI model that can run on CPUsparentComments−prvc1yThere are about 1.58 (i.e. log_2(3)) bits per digit, so they just applied the constant function that maps the reals to 1 to it.−ufocia1y1.58 is still more than 1 in general unless the parameters are corelated. At 1 bit it seems unlikely that you could pack/unpack independent parameters reliably without additional data.−falcor841yI like that as an explanation, but then every system is 1-bit, right? It definitely would simplify things.
Comments
There are about 1.58 (i.e. log_2(3)) bits per digit, so they just applied the constant function that maps the reals to 1 to it.
1.58 is still more than 1 in general unless the parameters are corelated. At 1 bit it seems unlikely that you could pack/unpack independent parameters reliably without additional data.
I like that as an explanation, but then every system is 1-bit, right? It definitely would simplify things.