d0mine on Speed tests for *json and cPickle in Python

Comment on Speed tests for *json and cPickle in Python

Python 2.7 on Ubuntu:

  $ python -mtimeit -s "from json import dumps; 
  d = {
      'foo': 'bar',
      'food': 'barf',
      'good': 'bars',
      'dood': 'wheres your car?',
      'wheres your car': 'dude?',
  }
  " "dumps(d)"
  100000 loops, best of 3: 6.89 usec per loop

  $ python -mtimeit -s "from cPickle import dumps; 
  d = {
      'foo': 'bar',
      'food': 'barf',
      'good': 'bars',
      'dood': 'wheres your car?',
      'wheres your car': 'dude?',
  }
  " "dumps(d)"
  100000 loops, best of 3: 7.78 usec per loop

So `json` seems faster than `cPickle`. Right? Wrong!:

  $ python -mtimeit -s "from cPickle import dumps; 
  d = {
      'foo': 'bar',
      'food': 'barf',
      'good': 'bars',
      'dood': 'wheres your car?',
      'wheres your car': 'dude?',
  }
  " "dumps(d, -1)"
  100000 loops, best of 3: 3.59 usec per loop

trafficlight 15y

What does the "-1" dump argument mean in 3rd example?

thezilch 15y

The equivalent of using pickle.HIGHEST_PROTOCOL or pickle.dumps(obj, protocol=pickle.HIGHEST_PROTOCOL) -- any negative value will force highest protocol.

kingkilr 15y

It says to use the highest optimization level.

d0mine 15y

No. It says use the highest available protocol.

The highest available protocol is not necessarily the fastest. There might be other than pure speed reasons to introduce the next Pickle protocol.

Comments