Conversation
Add .size() to allow for parsing NDJSON files.
…we now use an internal function for ASCII speedups)
…segfault with extremely deep documents (copying the standard JSON module's behavior)
…nother Document and support an implicit temporary Document.
…t versions of the Document() constructor. This fixes ambiguity with passing strings - is Document(...) an encoded JSON string, or a python string to be converted to a literal single element?
…a Document(). Disable the GC when building the DOM. Faster ASCII check.
…13 doesn't follow sys.setrecursionlimit *anyways*. Switch to a simple depth counter limited to 1024.
…c pause/resume (prelude to free threading support)
…lves, and if its seekable we can do a single resize. Significant RSS and parse time reduction (~20% each)
… global interpreter changes we just don't want to be part of - it can be called by users directly.
There was a problem hiding this comment.
Pull request overview
This PR cuts a v5.0.0 release that significantly expands the public API and performance characteristics of yyjson, adding a streaming (SAX) parser, new C-implemented module functions, improved path/file-like handling, and updated docs/packaging to ship these features.
Changes:
- Add
yyjson.sax()streaming parser (bounded-memory windowed read) plusSAXHandlerhelper and extensive tests. - Refactor/optimize DOM conversion path (shared immutable/mutable converter template, key caching, ASCII fast-path), plus improved error reporting and new
DocumentAPIs (from_obj,from_json, chainingfreeze/thaw,bytes_read, patch coercion). - Ship typing support and update packaging/docs/CI for v5.0.0 (Python 3.14 classifiers, docs build job, changelog integration).
Reviewed changes
Copilot reviewed 28 out of 32 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| yyjson/sax.h | Declares module-level sax() method table export. |
| yyjson/sax.c | Implements streaming SAX reader binding and source adapters. |
| yyjson/py.typed | Marks package as typed for type checkers. |
| yyjson/pathlib.h | Declares cached pathlib.Path globals for fast Path checks. |
| yyjson/element_to_primitive.h | Template for fast immutable/mutable DOM→Python conversion. |
| yyjson/document.h | Exposes shared helpers (unicode_from_str, fopen_path) and module methods. |
| yyjson/document.c | Major refactor: faster conversion, stream/path parsing, new Document APIs/semantics. |
| yyjson/binding.c | Registers sax() + doc methods and initializes Decimal/Path globals. |
| yyjson/init.pyi | Updates stubs for new APIs, flags, and streaming parser. |
| yyjson/init.py | Exposes new public surface (sax, loads, SAXHandler, etc.) and improves docs. |
| tests/test_shim.py | Adds regression tests for dumps/dump semantics and bytes non-serializability. |
| tests/test_serialization.py | Adds regression test for Decimal.__str__ raising (no segfault). |
| tests/test_sax.py | Comprehensive SAX parser test suite across source types and edge cases. |
| tests/test_paths.py | Cross-platform non-ASCII Path handling tests for Document, loads, and sax. |
| tests/test_patch.py | Tests patch coercion, invalid argument handling, and non-mutation of patch docs. |
| tests/test_numbers.py | Adjusts tests for int<->str digit limit guard on some interpreters. |
| tests/test_flags.py | Validates new Reader/Writer flag values and behavior. |
| tests/test_document.py | Adds tests for new Document APIs, unicode correctness, recursion limits, and bytes_read. |
| README.md | Updates project positioning (JSON/JSON5 + SAX streaming feature). |
| pyproject.toml | Bumps version to 5.0.0, updates build config, package data, CI wheel matrix. |
| MANIFEST.in | Includes new headers/sources, typing markers, and stubs in sdist. |
| docs/index.rst | Documents JSON5 flags and new streaming SAX parsing feature. |
| docs/development.rst | Updates dev instructions, adds profiling guidance, stricter docs build command. |
| docs/conf.py | Adds intersphinx + myst-parser and nitpick ignores for stricter CI builds. |
| docs/changelog.md | Includes top-level CHANGELOG into Sphinx docs. |
| docs/api.rst | Sets currentmodule for API docs. |
| CHANGELOG.md | Introduces v5.0.0 changelog entries for added/changed/fixed items. |
| .gitignore | Ignores uv.lock. |
| .github/workflows/release.yml | Updates actions versions, adds docs job, updates cibuildwheel matrix/version. |
| .bumpversion.cfg | Switches bumpversion target to pyproject.toml and bumps to 5.0.0. |
Comments suppressed due to low confidence (1)
yyjson/binding.c:52
- Module init over-increments references for
decimal/pathlibobjects (bothPyImport_ImportModule()andPyObject_GetAttrString()already return new references), and several failure paths returnNULLwithoutPy_DECREF(m)or cleaning up globals. This can leak references and leave partially-initialized globals if import fails.
Keep exactly one owned reference for each global and ensure the module object and any previously-created globals are decref'd on failure.
// We need to pre-import the Decimal module to have it available globally.
YY_DecimalModule = PyImport_ImportModule("decimal");
if (YY_DecimalModule == NULL) {
return NULL;
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+209
to
+217
| n = PyNumber_AsSsize_t(r, NULL); | ||
| Py_DECREF(r); | ||
| if (n < 0) { | ||
| if (!PyErr_Occurred()) | ||
| PyErr_SetString(PyExc_ValueError, "readinto() returned < 0"); | ||
| s->pc->error = true; | ||
| return YYJSON_SAX_SOURCE_ERROR; | ||
| } | ||
| return (size_t)n; |
Comment on lines
+559
to
+567
| got = PyNumber_AsSsize_t(r, NULL); | ||
| Py_DECREF(r); | ||
| if (got < 0) { | ||
| if (!PyErr_Occurred()) | ||
| PyErr_SetString(PyExc_ValueError, "readinto() returned < 0"); | ||
| goto error; | ||
| } | ||
| if (got == 0) break; | ||
| len += (size_t)got; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.