[SharedCache] Rework metadata serialization to reduce memory overhead#6127
Closed
bdash wants to merge 1 commit into
Closed
[SharedCache] Rework metadata serialization to reduce memory overhead#6127bdash wants to merge 1 commit into
bdash wants to merge 1 commit into
Conversation
Contributor
Author
bdash
force-pushed
the
dsc-serialization
branch
from
e92f5bb to
687626c
Compare
api/MetadataSerializable.hpp is removed in favor of including core/MetadataSerializable.hpp. Both headers defined types with the same name leading to One Definition Rule violations and surprising behavior. The serialization and deserialization context are now created on-demand during serialization rather than being a member of `MetadataSerializable`. This reduces the size of every serializable object by ~220 bytes. The context is passed explicitly as an argument to `Serialize` / `Deserialize`. As a result, `Serialize` / `Deserialize` can now be free functions rather than member functions. Since `MetadataSerializable` is not used for dynamic dispatch, the virtual methods are removed and the class is updated to be a class template using CRTP. This allows delegating to the derived class's `Load` and `Store` methods without the additional size overhead of the vtable pointer in every serializable object. These changes reduce the memory footprint of Binary Ninja after loading the macOS shared cache and loading a single dylib from it from 8.3GB to 4.6GB.
bdash
force-pushed
the
dsc-serialization
branch
from
687626c to
453ed0d
Compare
Contributor
|
Merged via ce88a94a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
api/MetadataSerializable.hpp is removed in favor of including core/MetadataSerializable.hpp. Both headers defined types with the same name leading to One Definition Rule violations and surprising behavior.
The serialization and deserialization context are now created on-demand during serialization rather than being a member of
MetadataSerializable. This reduces the size of every serializable object by ~220 bytes.The serialization / deserialization context is passed explicitly as an argument to
Serialize/Deserializerather than it being found via member variable onthis. As a result,Serialize/Deserializecan now be free functions rather than member functions.The
Serializeoverloads are updated to consistently accept their values by const reference rather than by value. This avoids an unnecessary copy of the values during serialization.Since
MetadataSerializableis not used for dynamic dispatch, the virtual methods are removed and the class is updated to be a class template using CRTP. This allows delegating to the derived class'sLoadandStoremethods without the additional size overhead of the vtable pointer in every serializable object.These changes reduce the memory footprint after loading the macOS shared cache and loading a single dylib from it from 8.3GB to 4.6GB:
Before:
After: