Use external package betterproto-rust-codec for better (de-)serialization performance#545
Use external package betterproto-rust-codec for better (de-)serialization performance#545cetanu merged 2 commits intodanielgtaylor:masterfrom 124C41p:rust-codec
betterproto-rust-codec for better (de-)serialization performance#545Conversation
cetanu
left a comment
cetanu
left a comment
There was a problem hiding this comment.
Looks okay to me, will only be used if the optional package is installed, correct?
cetanu
left a comment
cetanu
left a comment
There was a problem hiding this comment.
Actually, this introduces two new try/except for every call to bytes, is that right?
|
I see your point. Is it better now? |
| # monkey patch (de-)serialization functions of class `Message` | ||
| # with functions from `betterproto-rust-codec` if available | ||
| try: | ||
| import betterproto_rust_codec | ||
|
|
||
| def __parse_patch(self: T, data: bytes) -> T: | ||
| betterproto_rust_codec.deserialize(self, data) | ||
| return self | ||
|
|
||
| def __bytes_patch(self) -> bytes: | ||
| return betterproto_rust_codec.serialize(self) | ||
|
|
||
| Message.parse = __parse_patch | ||
| Message.__bytes__ = __bytes_patch | ||
| except ModuleNotFoundError: | ||
| pass |
There was a problem hiding this comment.
I'd personally just do something along the lines of
HAS_ACCELERATOR = cast(Literal[False], importlib.util.find_spec("betterproto_rust_codec"))
if HAS_ACCELERATOR:
import betterproto_rust_codec
class Message:
def parse():...
def __bytes__():...
if HAS_ACCELERATOR:
parse = betterproto_rust_codec.deserialize # edit this so it returns self
__bytes__ = betterproto_rust_codec.serializebecause this reduces the number of call frames and looks significantly less hacky
There was a problem hiding this comment.
I don't really understand why, but directly assigning __bytes__ = betterproto_rust_codec.serialize does not work for some reason. So editing deserialize would probably not help either.
What do you think about this approach?
class Message:
if HAS_ACCELERATOR:
def parse(self, data):
...
else:
def parse(self, data):
...I think this might be even more transparent.
There was a problem hiding this comment.
I find that problematic because it's another level of indentation which is annoying especially with black. If the function isn't binding as a descriptor just leave it as is
3 participants
This PR is an alternative to #520. It uses the same Rust based extension module to improve (de-)serialization performance of
betterprotosignificantly. However, instead of integrating the extension into thebetterprotoproject, I published it to PyPi separately, and only referenced it here as an optional dependency. For this reason, this branch is ready to use as it is.