Skip to content

Commit 67730ec

Browse files
committed
perf: use pickle.loads / pickle.dumps when self.(un)pickler is the default.
The serialize / deserialize paths construct a BytesIO and a Pickler or Unpickler instance per call to support a user-overridable pickler class. When self.(un)pickler is pickle.(Un)Pickler -- the default -- this is equivalent to pickle.dumps / pickle.loads, which are implemented in C (_pickle) and skip the Python-level allocation. Microbench (round-trip a small dict): BytesIO+Unpickler.load 3.08us vs pickle.loads 1.59us, ~1.94x.
1 parent 388ffae commit 67730ec

1 file changed

Lines changed: 11 additions & 8 deletions

File tree

bmemcached/protocol.py

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
import six
1515
from six import binary_type, text_type
1616

17-
from bmemcached.compat import long
17+
from bmemcached.compat import long, pickle
1818
from bmemcached.exceptions import AuthenticationNotSupported, InvalidCredentials, MemcachedException
1919
from bmemcached.utils import str_to_bytes
2020

@@ -375,10 +375,13 @@ def serialize(self, value, compress_level=-1):
375375
value = str(value)
376376
else:
377377
flags |= self.FLAGS['object']
378-
buf = BytesIO()
379-
pickler = self.pickler(buf, self.pickle_protocol)
380-
pickler.dump(value)
381-
value = buf.getvalue()
378+
if self.pickler is pickle.Pickler:
379+
value = pickle.dumps(value, self.pickle_protocol)
380+
else:
381+
buf = BytesIO()
382+
pickler = self.pickler(buf, self.pickle_protocol)
383+
pickler.dump(value)
384+
value = buf.getvalue()
382385

383386
if compress_level != 0 and len(value) > self.COMPRESSION_THRESHOLD:
384387
if compress_level is not None and compress_level > 0:
@@ -418,9 +421,9 @@ def deserialize(self, value, flags):
418421
elif flags & FLAGS['long']:
419422
return long(value)
420423
elif flags & FLAGS['object']:
421-
buf = BytesIO(value)
422-
unpickler = self.unpickler(buf)
423-
return unpickler.load()
424+
if self.unpickler is pickle.Unpickler:
425+
return pickle.loads(value)
426+
return self.unpickler(BytesIO(value)).load()
424427

425428
if six.PY3:
426429
return value.decode('utf8')

0 commit comments

Comments
 (0)