Add Python/JS/Rust/Go ports and modularize the C++ Blob library into cpp/ - #8
Merged
Conversation
… package Split the monolithic src/msgpack_blob.cpp (~2,250 lines) into five logical translation units — decode, encode, json, mutate, iterate — sharing private internals via msgpack_blob_detail.hpp. The public API (msgpack_blob.hpp) is unchanged and the SQLite extension is untouched. The C++ Blob library now lives in its own self-contained cpp/ package (the C++ sibling of the python/, js/, rust/ and go/ ports) with a CMakeLists.txt that builds standalone (cmake -B build cpp) or via add_subdirectory(cpp) from the repository root. The root build keeps the C++ <-> SQLite interop test. Also adds the cross-language test-vector generator (cpp/tests/gen_blob_vectors.cpp, CTest target blob_vectors_gen) and the generated tests/vectors/blob_vectors.json that every language port is validated against. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Native, zero-dependency re-implementations of the C++ Blob API in four languages, each exposing the same Blob / Builder / Value / Iterator API and producing byte-identical msgpack output, so blobs round-trip freely between SQL, C++, Python, JavaScript, Rust and Go.
- python/ pure stdlib (30 tests)
- js/ TypeScript, ESM + .d.ts, 64-bit ints via bigint (19 tests)
- rust/ no deps, exact %g float formatting via {:.*e} (22 tests)
- go/ stdlib only (20 tests)
Every port replays the shared tests/vectors/blob_vectors.json generated from the C++ reference, plus API/round-trip and non-UTF-8 regression tests. Float formatting and 64-bit integer handling were validated against the C reference across tens of thousands of randomized values.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the main README (new Language libraries section with a C++ reference row), the per-language READMEs, CONTRIBUTING.md and SECURITY.md to reflect the cpp/ package layout and the new ports, and extend .gitignore for Node, TypeScript, Rust and Go build artifacts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI previously built and tested only the C/C++ targets via CTest. Add dedicated parallel jobs so every language port is exercised on each push and pull request: - python — python -m unittest discover -s tests - js — npm ci, tsc type-check + build, node --test - rust — cargo fmt --check, cargo clippy -D warnings, cargo test - go — gofmt check, go vet, go test ./... Also adds a standalone cpp/ package build job and a step that regenerates the shared vectors and fails if tests/vectors/blob_vectors.json has drifted from the C++ reference. Reformats one regression test so cargo fmt --check passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactors the standalone C++ Blob API into a self-contained
cpp/package and adds four native, zero-dependency ports — Python, TypeScript/JS, Rust, and Go — each producing byte-identical MessagePack output. Blobs now round-trip freely between the SQLite extension, C++, Python, JavaScript, Rust and Go.The SQLite extension is untouched (
src/msgpack.c,src/sqlite3.c,include/sqlite3*.h).What changed
C++ library (
cpp/)src/msgpack_blob.cpp(~2,250 lines) into five logical modules —decode,encode,json,mutate,iterate— sharing private internals viamsgpack_blob_detail.hpp. Public API (msgpack_blob.hpp) is unchanged.cpp/package (sibling ofpython/,js/,rust/,go/) with aCMakeLists.txtthat builds standalone (cmake -B build cpp) or viaadd_subdirectory(cpp)from the root. The root build keeps the C++ ↔ SQLite interop test.New language ports
python/js/.d.ts, 64-bit ints viabigintrust/go/Each mirrors the same
Blob/Builder/Value/IteratorAPI.Cross-language byte-identity
A C++ generator (
cpp/tests/gen_blob_vectors.cpp, CTest targetblob_vectors_gen) emits a shared vector file (tests/vectors/blob_vectors.json) that every port replays — coveringfrom_json,to_json/pretty, typed encoding, mutation, merge-patch, extraction,array_length, and iteration.Notable engineering
%g) was the hardest part — no native%gin Rust/JS. Reproduced C's%.17g/%.7gexactly: JS via BigInt round-half-to-even, Rust via{:.*e}, Go viastrconv. Validated 0 mismatches across 35k+ values per language, plus a 50k-value end-to-endto_jsoncross-check vs the C++ reference.bigint, Rusti64/u64, Go native).from_jsonnumbers, 15kmerge_patch, 15kset/remove, and a 50k-record adversarial panic-fuzz through every public method (0 panics).Bugs found & fixed during review
to_jsoncorrupted non-UTF-8 string bytes (lossyTextDecoder) — replaced with a byte-preserving codec.%gexponent off-by-one on values like1e-7— rewritten with exact BigInt arithmetic.Testing
ctest16/16 under both Makefiles and Ninja (CI generator); standalonecpp/build 2/2.blob_vectors_gen.Docs
Main README (Language libraries section + C++ reference row), per-language READMEs,
CONTRIBUTING.md,SECURITY.md, and.gitignoreall updated for thecpp/layout and new ports.