It's probably not really more about getting every item in 64 bits than about memory use. That allows one to index each item without doing further bit twiddling: you can simply declare the data as
unsigned int64_t *datapoints;
Because of that, I think I would have combined two 5-valued fields (need 3 bits each) rating and user area into a single 25-valued one (fits in 5 bits), and stored the others as proper bit fields. That gains you the single bit you need to put each item in a 64 bit value, and only complicates accessing two of the fields.
That's how I would have done it too. He had 65 bits to start with, so just combining two fields with the trick in the article would have been enough to save that one bit. Or one could have just extracted one boolean field into a separate bit array (even simpler.) Since the article makes a big deal about getting down to 61 bits, I assume every bit saved helped and that they were all packed together with no padding. It's a lot of assumptions.
Of course, it needs benchmarking, but "one int64_t and a boolean" is bad, cache-wise, so I would expect it to slow down the code more than putting those two fields in 5 bits.
Also, I doubt he needed to pack below 64 bits, as the article states "which fits into 61 bits (8 bytes).".
It depends, as always, if you access it sequentially, iterating two separate arrays vs one is similar in performance, the prefetcher will help with both. If it's random access, then you really want things to fit into 64 bits.
Comments
It's probably not really more about getting every item in 64 bits than about memory use. That allows one to index each item without doing further bit twiddling: you can simply declare the data as
Because of that, I think I would have combined two 5-valued fields (need 3 bits each) rating and user area into a single 25-valued one (fits in 5 bits), and stored the others as proper bit fields. That gains you the single bit you need to put each item in a 64 bit value, and only complicates accessing two of the fields.That's how I would have done it too. He had 65 bits to start with, so just combining two fields with the trick in the article would have been enough to save that one bit. Or one could have just extracted one boolean field into a separate bit array (even simpler.) Since the article makes a big deal about getting down to 61 bits, I assume every bit saved helped and that they were all packed together with no padding. It's a lot of assumptions.
Of course, it needs benchmarking, but "one int64_t and a boolean" is bad, cache-wise, so I would expect it to slow down the code more than putting those two fields in 5 bits.
Also, I doubt he needed to pack below 64 bits, as the article states "which fits into 61 bits (8 bytes).".
It depends, as always, if you access it sequentially, iterating two separate arrays vs one is similar in performance, the prefetcher will help with both. If it's random access, then you really want things to fit into 64 bits.