v5.0.0

.bumpversion.cfg

@@ -1,6 +1,8 @@
-[bumpversion]
-current_version = 4.0.5
-commit = True
-tag = True
-
-[bumpversion:file:setup.py]
+[bumpversion]
+current_version = 5.0.0
+commit = True
+tag = True
+
+[bumpversion:file:pyproject.toml]
+search = version = "{current_version}"
+replace = version = "{new_version}"

.github/workflows/release.yml

@@ -19,7 +19,7 @@ jobs:
       - uses: actions/checkout@v4.1.7
 
       - name: Setting up Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v6
         with:
           python-version: 3.13
 
@@ -37,15 +37,31 @@ jobs:
         run: |
           pytest
 
-      - name: Ensuring documentation builds
-        run: |
-          cd docs && make clean && make html
-
       - uses: actions/upload-artifact@v4.6.0
         with:
           name: dist-sdist
           path: dist/*.tar.gz
 
+  docs:
+    name: Building documentation
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4.1.7
+
+      - name: Setting up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: '3.13'
+
+      - name: Installing release dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[release]"
+
+      - name: Building documentation
+        run: |
+          sphinx-build -W -n --keep-going -b html docs docs/_build/html
+
   build_wheels:
     needs: [sdist]
     name: "[${{ strategy.job-index }}/${{ strategy.job-total }}] py${{ matrix.py }} on ${{ matrix.os }}"
@@ -54,12 +70,12 @@ jobs:
       fail-fast: true
       matrix:
         os: [ubuntu-latest, windows-latest, macos-14]
-        py: ["cp39", "cp310", "cp311", "cp312", "cp313", "pp39", "pp310"]
+        py: ["cp39", "cp310", "cp311", "cp312", "cp313", "cp314", "pp311"]
 
     steps:
       - uses: actions/checkout@v4.1.7
 
-      - uses: actions/setup-python@v5
+      - uses: actions/setup-python@v6
         name: Setting up Python
         with:
           python-version: '3.13'
@@ -71,12 +87,12 @@ jobs:
           platforms: all
 
       - name: Build & test wheels
-        uses: pypa/cibuildwheel@v2.22.0
+        uses: pypa/cibuildwheel@v3.2.1
         env:
           CIBW_ARCHS_LINUX: auto aarch64 ppc64le s390x
           CIBW_ARCHS_MACOS: x86_64 arm64 universal2
           CIBW_BUILD: "${{ matrix.py }}-*"
-          CIBW_TEST_SKIP: "*_arm64 *_universal2:arm64"
+          CIBW_TEST_SKIP: "*_aarch64 *_ppc64le *_s390x"
 
       - uses: actions/upload-artifact@v4.6.0
         with:
@@ -108,7 +124,7 @@ jobs:
       - uses: actions/checkout@v3
 
       - name: Setting up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v6
         with:
           python-version: 3.13
 

.gitignore

@@ -10,3 +10,4 @@ _build/
 dist/
 .idea
 .vscode/settings.json
+uv.lock

CHANGELOG.md

@@ -0,0 +1,95 @@
+# Changelog
+
+## v5.0.0 (unreleased)
+
+### Added
+
+- A SAX-style streaming parser, `yyjson.sax()`. It parses inputs of unlimited
+  size with bounded memory - a fixed sliding window plus the nesting depth -
+  calling optional methods on a handler object for each token, and can stop
+  early when a handler returns `False`. Sources can be `str`, `bytes`, a
+  `pathlib.Path`, or a binary file-like object.
+- `Document()` accepts binary file-like objects and parses them directly into
+  a document.
+- The module-level `loads()` is now implemented in C and accepts `str`,
+  `bytes`, `bytearray`, or `pathlib.Path`.
+- `Document.from_obj()` and `Document.from_json()` classmethods - explicit
+  counterparts to the type-guessing constructor, resolving the `Document(str)`
+  ambiguity between "parse this JSON text" and "build from this Python
+  string".
+- `Document.patch()` accepts any `Document`-constructible value (a `dict`,
+  `list`, JSON text, or `Path`), wrapping it in a temporary `Document`.
+- `Document.bytes_read`: the number of input bytes consumed by the parse
+  (useful with `ReaderFlags.STOP_WHEN_DONE`, e.g. for NDJSON).
+- `SAXHandler`, an optional base class for `sax()` handlers: subclass it and
+  override only the events you need, with IDE completion and static type
+  checking.
+- `sax()` accepts any object exporting a contiguous byte buffer
+  (`bytearray`, `memoryview`, `mmap`, ...) as a zero-copy source. The buffer
+  is pinned for the duration of the parse; resizing it from a handler
+  callback raises `BufferError`.
+- Type annotations: a `py.typed` marker and `.pyi` stubs for the C extension.
+- New `ReaderFlags`: `JSON5` (full JSON5 parsing), `ALLOW_BOM`, and the
+  individual `ALLOW_EXT_NUMBER`, `ALLOW_EXT_ESCAPE`, `ALLOW_EXT_WHITESPACE`,
+  `ALLOW_SINGLE_QUOTED_STR`, and `ALLOW_UNQUOTED_KEY` extensions. JSON5 is
+  supported by the DOM readers only; `sax()` ignores non-standard flags.
+- New `WriterFlags`: `FP_TO_FLOAT` and `WriterFlags.fp_to_fixed(precision)`
+  for controlling floating-point output, `LOWERCASE_HEX` for lowercase
+  `\uXXXX` escapes, and `ALLOW_INVALID_UNICODE`.
+- Python 3.14 and PyPy 3.11 wheels.
+
+### Changed
+
+- Major performance work on the read path - object-key caching, direct
+  document-tape walking, an ASCII fast path, and presized containers.
+  `loads()` and `Document.as_obj` are now competitive with the fastest
+  Python JSON libraries.
+- Parsing from file-like objects and `Path`s now reads into a single owned
+  buffer parsed in place: roughly 20% faster with roughly 20% lower peak
+  memory.
+- Parse errors from `Document()` and `loads()` now include the position:
+  line, column and byte offset for in-memory and stream inputs, byte offset
+  for file paths.
+- Dictionary keys must be strings when serializing; other key types now raise
+  `TypeError` instead of producing undefined behaviour.
+- Tuples are serialized as JSON arrays.
+- Converting documents nested deeper than 1024 container levels raises
+  `RecursionError` instead of risking a C stack overflow.
+- `flags` and `default` arguments are keyword-only. (They always were at
+  runtime; the documentation and type stubs now agree.)
+- `Document.freeze()` and `Document.thaw()` return the document itself
+  (previously `None`), allowing chaining such as
+  `Document(obj).freeze().dumps()`.
+- `Document.patch()` no longer converts its patch argument between the
+  frozen and thawed representations in place; the argument comes back
+  exactly as it went in.
+- Opening a file that is missing or unreadable via `Document(Path)`,
+  `loads(Path)`, or `sax(Path)` raises `OSError` (e.g. `FileNotFoundError`)
+  carrying the filename, instead of a generic `ValueError`.
+- A non-blocking stream returning `None` from `read()`/`readinto()` ("no
+  data available yet") raises `BlockingIOError` instead of being treated as
+  end-of-input and silently truncating the document.
+- Converting thawed (mutable) documents to Python objects uses the same
+  optimized path as frozen documents; `patch()` results convert roughly 30%
+  faster and the 1024-level nesting limit now applies uniformly.
+- Wheels are no longer built against the limited API (required by the new
+  ASCII fast path), and EOL PyPy 3.9/3.10 wheels are no longer produced.
+- The vendored yyjson was upgraded to 0.12.0.
+
+### Fixed
+
+- A UTF-8 corruption bug on platforms where `char` is signed.
+- A possible segfault when parsing extremely deep documents.
+- A crash when `Document.patch()` was called with a non-`Document` value.
+- A memory leak when serializing integers larger than 64 bits.
+- A memory leak when `Document.__init__()` was invoked more than once on the
+  same object.
+- Non-ASCII file paths now work on Windows: `Document(Path)` and
+  `sax(Path)` open paths through the wide-character API instead of the ANSI
+  codepage.
+- A segfault when serializing a `Decimal` subclass whose `__str__` raises.
+- An allocation failure during `freeze()`/`thaw()` could destroy the
+  document's contents while reporting success; it now raises `MemoryError`
+  with the document intact. Other out-of-memory conditions during
+  serialization now raise `MemoryError` instead of silently dropping data
+  or raising `SystemError`.

MANIFEST.in

@@ -5,4 +5,10 @@ include yyjson/memory.c
 include yyjson/memory.h
 include yyjson/document.c
 include yyjson/document.h
-include yyjson/decimal.h
+include yyjson/element_to_primitive.h
+include yyjson/decimal.h
+include yyjson/pathlib.h
+include yyjson/sax.c
+include yyjson/sax.h
+include yyjson/py.typed
+include yyjson/__init__.pyi

README.md

@@ -5,7 +5,7 @@
 
 ![py_yyjson Logo](misc/logo_small.png)
 
-Fast and flexible Python JSON parsing built on the excellent [yyjson][]
+Fast and flexible Python JSON/JSON5 parsing built on the excellent [yyjson][]
 project.
 
 ![GitHub Sponsors](https://img.shields.io/github/sponsors/tktech)
@@ -30,6 +30,9 @@ project.
   objects.
 - **Traceable**: `yyjson` uses Python's memory allocator by default, so you can
   trace memory leaks and other memory issues using Python's built-in tools.
+- **SAX-style parsing**: An optional SAX-style parser allowing you to read
+  JSON documents of unlimited size with bounded memory usage, typically 2-5x
+  faster than `ijson`.
 
 ## Documentation
 

docs/api.rst

@@ -1,6 +1,8 @@
 API
 ---
 
+.. currentmodule:: yyjson
+
 .. note::
 
     When creating and manipulating a :class:`Document`, it's important to keep

docs/changelog.md

@@ -0,0 +1,2 @@
+```{include} ../CHANGELOG.md
+```

docs/conf.py

@@ -32,7 +32,21 @@
     "sphinx.ext.viewcode",
     "sphinx.ext.todo",
     "sphinx.ext.doctest",
+    "sphinx.ext.intersphinx",
     "sphinx_copybutton",
+    "myst_parser",
+]
+
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/3", None),
+}
+
+# `:type foo: callable, optional` in the C docstrings produces py:class
+# lookups for words that aren't classes; ignore those so the build can run
+# with -n (nitpicky) and -W (warnings are errors) in CI.
+nitpick_ignore = [
+    ("py:class", "callable"),
+    ("py:class", "optional"),
 ]
 
 # Add any paths that contain templates here, relative to this directory.

docs/development.rst

@@ -1,29 +1,37 @@
 Development
 ===========
 
-To install development requirements, use:
+To build the extension and install development requirements, use:
 
   pip install -e ".[test]"
 
-You can then build the project using ``python setup.py develop``. Keep in
-mind since this is a C extension, you'll need a C compiler installed. On
-Windows, you can use Visual Studio Community Edition. On Linux, you can
-use GCC or Clang. You'll also need to re-run it each time you make a
-change to a ``.c`` or ``.h`` file to recompile.
+Keep in mind since this is a C extension, you'll need a C compiler installed.
+On Windows, you can use Visual Studio Community Edition. On Linux and macOS,
+you can use GCC or Clang. You'll also need to re-run the install each time
+you make a change to a ``.c`` or ``.h`` file to recompile.
 
 To run the tests, just type ``pytest``. To prepare for a release or to rebuild
 documentation, you need a few extra dependencies:
 
     pip install -e ".[release]"
 
-You can then rebuild the documentation by running ``make html`` within the
-``docs/`` directory.
+You can then rebuild the documentation with the same strict build CI uses:
 
+    sphinx-build -W -n -b html docs docs/_build/html
 
-Why not Poetry?
----------------
 
-Poetry is a great tool, but it's not a good fit for this project. Poetry
-is designed to manage Python projects, and this project is a C extension
-with a Python wrapper. Poetry's current support for building C extensions
-is a hack which may change at any time.
+Profiling
+---------
+
+Performance is a core feature of this project, not an afterthought - being
+several times faster than the builtin ``json`` module is most of the reason
+py_yyjson exists. Changes to the hot paths (parsing, DOM-to-Python
+conversion, and serialization) should come with before/after numbers, and
+"obvious" improvements should be measured rather than assumed: more than one
+past optimization turned out to be a wash once benchmarked.
+
+Benchmarks intentionally live outside of this repository in
+`json_benchmark <https://github.com/tktech/json_benchmark>`_, which compares
+py_yyjson against other popular JSON libraries with a single harness and a
+corpus of real-world documents. If you're proposing a performance change,
+run it before and after.