As you can see, lwpb and fast-python-pb are neck and neck. And I should point out that lwpb isn't using C++ codegen at all, just the compiled schema in the .proto file. Of course, if completeness of implementation was the critical thing, you'd probably want to stay closer to google's official implementation. There's a lot of the google implementation that I never use though, like the RPC stuff.
Also notable that both lwpb and fast-python-pb outperform cPickle by almost 3x. It would be interesting to know why a portable, cross-language serialization format beats out the language-specific one.
> It would be interesting to know why a portable, cross-language serialization format beats out the language-specific one.
Because protobuf parses messages with a fixed schema in a very structured format. Pickle, OTOH, is an interpreted bytecode microlanguage used to describe arbitrary python objects (for instance, pickle can call python functions: http://nadiana.com/python-pickle-insecure)
Also, pickle supports references (so if an object is referenced two times in the same serialization stream, it is
serialized only once) and this has a cost at serialization time (need to keep the set of seen references)
Perl's Storable can also serialize code references which get eval'ed during deserialization, and can also serialize multiple references once, though you have to be more explicit about that. And Storable is still 2x-3x faster than the already quite fast JSON::XS.
It would seem the bytecode interpreter architecture in Pickle is the limiting factor. If anybody has some good profiling data on Pickle though, I'd love to see it.
It would be interesting to know why a portable, cross-language serialization format beats out the language-specific one.
Odds are that it is because the language-specific one supports features that the portable one does not. A feature I'd be particularly suspicious of is, "Did we already encounter this data structure and serialize it?" Supporting that feature means tracking a LOT of information during the serialization process, whether or not you get to use it.
Comments
Thanks Robby, got the benchmark working, was trying to do a homedir install of fast-python-pb earlier.
I added a couple more tests to the benchmark, here are the results:
As you can see, lwpb and fast-python-pb are neck and neck. And I should point out that lwpb isn't using C++ codegen at all, just the compiled schema in the .proto file. Of course, if completeness of implementation was the critical thing, you'd probably want to stay closer to google's official implementation. There's a lot of the google implementation that I never use though, like the RPC stuff.Also notable that both lwpb and fast-python-pb outperform cPickle by almost 3x. It would be interesting to know why a portable, cross-language serialization format beats out the language-specific one.
Here's a fork with the patched benchmark code: https://github.com/acg/fast-python-pb
> It would be interesting to know why a portable, cross-language serialization format beats out the language-specific one.
Because protobuf parses messages with a fixed schema in a very structured format. Pickle, OTOH, is an interpreted bytecode microlanguage used to describe arbitrary python objects (for instance, pickle can call python functions: http://nadiana.com/python-pickle-insecure)
Also, pickle supports references (so if an object is referenced two times in the same serialization stream, it is serialized only once) and this has a cost at serialization time (need to keep the set of seen references)
Perl's Storable can also serialize code references which get eval'ed during deserialization, and can also serialize multiple references once, though you have to be more explicit about that. And Storable is still 2x-3x faster than the already quite fast JSON::XS.
It would seem the bytecode interpreter architecture in Pickle is the limiting factor. If anybody has some good profiling data on Pickle though, I'd love to see it.
(c)pickle uses a string protocol by default which is the most portable (even across Python versions and platforms). You need to specify another protocol version for best performance. See http://docs.python.org/library/pickle.html#data-stream-forma...
The times should be similar to lwpb then.
Edit2: Oh, and for JSON you should use http://pypi.python.org/pypi/simplejson
Yup, simplejson is much faster than the standard json.
I think you'll find py-yajl to be faster than any of the other Python JSON modules: https://github.com/rtyler/py-yajl
It would be interesting to know why a portable, cross-language serialization format beats out the language-specific one.
Odds are that it is because the language-specific one supports features that the portable one does not. A feature I'd be particularly suspicious of is, "Did we already encounter this data structure and serialize it?" Supporting that feature means tracking a LOT of information during the serialization process, whether or not you get to use it.