You already said it, but just wanted to back you up: The MessagePack benchmark is completely useless!
It consists of serializing 3 integers and a string 200,000 times.
MessagePack defines 27 different types¹ (excluding reserved ones) with variable bit length for the type identifier, length somewhat correlated with frequency of use.
A benchmark should therefor test real life data and a lot of it.
Their inability to produce such benchmark makes me question the sanity of splitting up e.g. the type marker for “array” into 3 different types depending on the size of the array — this adds complexity, so it would be good to know what exactly the authors based this design choice on, hopefully not that it made it faster to serialize a 3 element array 200,000 times.
Comments
You already said it, but just wanted to back you up: The MessagePack benchmark is completely useless!
It consists of serializing 3 integers and a string 200,000 times.
MessagePack defines 27 different types¹ (excluding reserved ones) with variable bit length for the type identifier, length somewhat correlated with frequency of use.
A benchmark should therefor test real life data and a lot of it.
Their inability to produce such benchmark makes me question the sanity of splitting up e.g. the type marker for “array” into 3 different types depending on the size of the array — this adds complexity, so it would be good to know what exactly the authors based this design choice on, hopefully not that it made it faster to serialize a 3 element array 200,000 times.
¹ http://wiki.msgpack.org/display/MSGPACK/Format+specification...