feat(tests): add pytest-codeblocks for documentation code snippet CI#198
Open
haoyu-haoyu wants to merge 3 commits intodotimplement:mainfrom
Open
feat(tests): add pytest-codeblocks for documentation code snippet CI#198haoyu-haoyu wants to merge 3 commits intodotimplement:mainfrom
haoyu-haoyu wants to merge 3 commits intodotimplement:mainfrom
Conversation
Add automated testing infrastructure for documentation code examples using pytest-codeblocks: - Add pytest-codeblocks>=0.17.0 to dev dependencies - Add [tool.pytest.ini_options] with codeblocks marker - Add doc snippet test step to CI workflow - Mark 56 Python code blocks across 11 doc files with skip markers (these require healthchain package + external services to run) The CI step runs with continue-on-error initially so snippets can be incrementally enabled as they are made self-contained. Usage: uv run pytest --codeblocks docs/ # Test all doc snippets uv run pytest --codeblocks docs/quickstart.md # Test specific file To make a snippet testable, remove its <!-- pytest-codeblocks:skip --> marker and ensure it runs standalone (no external dependencies). Closes dotimplement#164
- Fix skip marker format: replace legacy `pytest-codeblocks:skip` with current `pytest.mark.skip` syntax (56 markers updated) - Restrict CI doc test to Python 3.12 only (pytest-codeblocks not verified for 3.13) - Narrow CI scope to quickstart + cookbook + tutorials (avoids executing shell blocks in reference docs) - Update pyproject.toml comment with correct skip syntax
- Add skip markers to 3 missed cookbook files (format_conversion, index, setup_fhir_sandboxes) — 10 Python blocks - Add skip markers to all bash/sh/shell blocks in scoped docs to prevent pytest-codeblocks from executing install/setup scripts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add automated testing infrastructure for documentation code examples using
pytest-codeblocks, so code snippets stay in sync with the evolving codebase.Changes
New dependency
pytest-codeblocks>=0.17.0,<0.18added to dev dependenciesCI integration
.github/workflows/ci.ymlrunspytest --codeblocksondocs/quickstart.md,docs/cookbook/, anddocs/tutorials/continue-on-error: trueso snippets can be incrementally enabledSkip markers
<!--pytest.mark.skip-->(these require healthchain package + external services)pytest configuration
[tool.pytest.ini_options]withcodeblocksmarker inpyproject.tomlUsage
To make a snippet testable, remove its
<!--pytest.mark.skip-->marker and ensure it runs standalone.Design decisions
quickstart,cookbook, andtutorialsare tested — reference/API docs are excluded to avoid executing auto-generated content.Closes #164