Skip to content

v5.0.0#18

Open
TkTech wants to merge 42 commits into
mainfrom
41_fixes
Open

v5.0.0#18
TkTech wants to merge 42 commits into
mainfrom
41_fixes

Conversation

@TkTech

@TkTech TkTech commented Apr 29, 2025

Copy link
Copy Markdown
Owner

No description provided.

TkTech added 10 commits October 30, 2025 06:00
Add .size() to allow for parsing NDJSON files.
…we now use an internal function for ASCII speedups)
…segfault with extremely deep documents (copying the standard JSON module's behavior)
…nother Document and support an implicit temporary Document.
…t versions of the Document() constructor. This fixes ambiguity with passing strings - is Document(...) an encoded JSON string, or a python string to be converted to a literal single element?
@TkTech TkTech changed the title v4.1.0 v5.0.0 Jul 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR cuts a v5.0.0 release that significantly expands the public API and performance characteristics of yyjson, adding a streaming (SAX) parser, new C-implemented module functions, improved path/file-like handling, and updated docs/packaging to ship these features.

Changes:

  • Add yyjson.sax() streaming parser (bounded-memory windowed read) plus SAXHandler helper and extensive tests.
  • Refactor/optimize DOM conversion path (shared immutable/mutable converter template, key caching, ASCII fast-path), plus improved error reporting and new Document APIs (from_obj, from_json, chaining freeze/thaw, bytes_read, patch coercion).
  • Ship typing support and update packaging/docs/CI for v5.0.0 (Python 3.14 classifiers, docs build job, changelog integration).

Reviewed changes

Copilot reviewed 28 out of 32 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
yyjson/sax.h Declares module-level sax() method table export.
yyjson/sax.c Implements streaming SAX reader binding and source adapters.
yyjson/py.typed Marks package as typed for type checkers.
yyjson/pathlib.h Declares cached pathlib.Path globals for fast Path checks.
yyjson/element_to_primitive.h Template for fast immutable/mutable DOM→Python conversion.
yyjson/document.h Exposes shared helpers (unicode_from_str, fopen_path) and module methods.
yyjson/document.c Major refactor: faster conversion, stream/path parsing, new Document APIs/semantics.
yyjson/binding.c Registers sax() + doc methods and initializes Decimal/Path globals.
yyjson/init.pyi Updates stubs for new APIs, flags, and streaming parser.
yyjson/init.py Exposes new public surface (sax, loads, SAXHandler, etc.) and improves docs.
tests/test_shim.py Adds regression tests for dumps/dump semantics and bytes non-serializability.
tests/test_serialization.py Adds regression test for Decimal.__str__ raising (no segfault).
tests/test_sax.py Comprehensive SAX parser test suite across source types and edge cases.
tests/test_paths.py Cross-platform non-ASCII Path handling tests for Document, loads, and sax.
tests/test_patch.py Tests patch coercion, invalid argument handling, and non-mutation of patch docs.
tests/test_numbers.py Adjusts tests for int<->str digit limit guard on some interpreters.
tests/test_flags.py Validates new Reader/Writer flag values and behavior.
tests/test_document.py Adds tests for new Document APIs, unicode correctness, recursion limits, and bytes_read.
README.md Updates project positioning (JSON/JSON5 + SAX streaming feature).
pyproject.toml Bumps version to 5.0.0, updates build config, package data, CI wheel matrix.
MANIFEST.in Includes new headers/sources, typing markers, and stubs in sdist.
docs/index.rst Documents JSON5 flags and new streaming SAX parsing feature.
docs/development.rst Updates dev instructions, adds profiling guidance, stricter docs build command.
docs/conf.py Adds intersphinx + myst-parser and nitpick ignores for stricter CI builds.
docs/changelog.md Includes top-level CHANGELOG into Sphinx docs.
docs/api.rst Sets currentmodule for API docs.
CHANGELOG.md Introduces v5.0.0 changelog entries for added/changed/fixed items.
.gitignore Ignores uv.lock.
.github/workflows/release.yml Updates actions versions, adds docs job, updates cibuildwheel matrix/version.
.bumpversion.cfg Switches bumpversion target to pyproject.toml and bumps to 5.0.0.
Comments suppressed due to low confidence (1)

yyjson/binding.c:52

  • Module init over-increments references for decimal/pathlib objects (both PyImport_ImportModule() and PyObject_GetAttrString() already return new references), and several failure paths return NULL without Py_DECREF(m) or cleaning up globals. This can leak references and leave partially-initialized globals if import fails.

Keep exactly one owned reference for each global and ensure the module object and any previously-created globals are decref'd on failure.

  // We need to pre-import the Decimal module to have it available globally.
  YY_DecimalModule = PyImport_ImportModule("decimal");
  if (YY_DecimalModule == NULL) {
    return NULL;
  }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread yyjson/sax.c
Comment on lines +209 to +217
n = PyNumber_AsSsize_t(r, NULL);
Py_DECREF(r);
if (n < 0) {
if (!PyErr_Occurred())
PyErr_SetString(PyExc_ValueError, "readinto() returned < 0");
s->pc->error = true;
return YYJSON_SAX_SOURCE_ERROR;
}
return (size_t)n;
Comment thread yyjson/document.c
Comment on lines +559 to +567
got = PyNumber_AsSsize_t(r, NULL);
Py_DECREF(r);
if (got < 0) {
if (!PyErr_Occurred())
PyErr_SetString(PyExc_ValueError, "readinto() returned < 0");
goto error;
}
if (got == 0) break;
len += (size_t)got;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants