Skip to content

[codex] Add diagnostic metadata regression tests#32

Open
xianxiye wants to merge 5 commits into
jackjin1997:mainfrom
xianxiye:codex/diagnostic-metadata-tests
Open

[codex] Add diagnostic metadata regression tests#32
xianxiye wants to merge 5 commits into
jackjin1997:mainfrom
xianxiye:codex/diagnostic-metadata-tests

Conversation

@xianxiye

@xianxiye xianxiye commented Jul 6, 2026

Copy link
Copy Markdown

Summary

Adds deterministic regression tests for build.py diagnostic metadata generation so future build changes do not break the JSON/logd contract required for PR review.

Changes

  • Added tests/test_build_diagnostics.py using standard-library unittest.
  • Covers successful metadata with commit id, module summaries, and a single diagnostic_logd path.
  • Covers logd generation failure behavior where diagnostic_logd_error is populated and no valid archive is claimed.
  • Covers chunked/multi-file .logd references and decrypt command metadata.
  • Committed required diagnostic artifacts:
    • diagnostic/build-0618b98e.json
    • diagnostic/build-0618b98e-part001.logd
    • diagnostic/build-0618b98e-part002.logd

Testing

  • python -m py_compile build.py tests/test_build_diagnostics.py
  • python -m unittest discover -s tests -v
  • PYTHONIOENCODING=utf-8 PYTHONUTF8=1 python3 build.py -m backend
    • Generated the required diagnostic files for commit 0618b98e.
    • Backend build failed locally because cargo is not installed on this Windows machine.

Diagnostic decrypt command:

cat diagnostic/build-0618b98e-part001.logd diagnostic/build-0618b98e-part002.logd > diagnostic/build-0618b98e.logd
encryptly unpack diagnostic/build-0618b98e.logd <outdir> --password 328ee58da38110829a7b

Checklist

  • Relevant modules affected by these changes build locally
  • Tests pass locally
  • Diagnostic build log is committed in this PR
  • Documentation has been updated, if applicable
  • Configuration or schema changes are documented, if applicable
  • No generated build artifacts are committed, except the required diagnostic build log
  • Changes are scoped to the PR purpose and avoid unrelated cleanup
  • Security, privacy, and error-handling implications have been considered

  • I would like to request that my diagnostic build log is removed before merging

Refs #3

Summary by CodeRabbit

  • New Features

    • Improved diagnostic report packaging to support larger files and chunked attachments.
    • Added richer diagnostic metadata, including build status, artifact references, and decryption details.
  • Tests

    • Added regression coverage for successful, failed, and chunked diagnostic report generation.

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR increases the --max-file-size value in the encryptly pack command within build.py from 10000 to 35840, adds a sample diagnostic JSON artifact recording a failed build module, and introduces regression tests for build_diagnostic_report covering success, logd failure, and chunked report scenarios.

Changes

Build diagnostics changes

Layer / File(s) Summary
Encryptly pack max-file-size update
build.py
The --max-file-size argument in the encryptly pack invocation is increased from 10000 to 35840.
Sample diagnostic artifact
diagnostic/build-0618b98e.json
A new diagnostic JSON file records commit info, chunked .logd paths, decrypt configuration, and a failed backend module with a Windows command-not-found error.
build_diagnostic_report regression tests
tests/test_build_diagnostics.py
New test module and three test cases verify report fields for successful runs, logd encryption failures, and chunked logd references, plus a unittest entry point.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the main change: adding diagnostic metadata regression tests.
Description check ✅ Passed The description follows the required template and includes Summary, Changes, Testing, Checklist, and artifact details.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
build.py (1)

564-578: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Magic number lacks unit/rationale comment.

"max-file-size" bumped from 10000 to 35840 with no comment on units or why this specific value was chosen. Consider a named constant (similar to DIAGNOSTIC_CHUNK_SIZE) with a brief comment.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build.py` around lines 564 - 578, The pack command in subprocess.run uses a
hard-coded max-file-size value with no explanation, so replace the literal in
build.py with a named constant near the relevant logic, similar to
DIAGNOSTIC_CHUNK_SIZE, and add a brief comment stating the units and why that
size is chosen. Update the encryptly_bin pack invocation to reference that
constant so the rationale is easy to find and maintain.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@build.py`:
- Around line 564-578: The pack command in subprocess.run uses a hard-coded
max-file-size value with no explanation, so replace the literal in build.py with
a named constant near the relevant logic, similar to DIAGNOSTIC_CHUNK_SIZE, and
add a brief comment stating the units and why that size is chosen. Update the
encryptly_bin pack invocation to reference that constant so the rationale is
easy to find and maintain.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3029c08f-37c3-42e2-bd07-0a7087b4aec5

📥 Commits

Reviewing files that changed from the base of the PR and between 1462fe7 and af06fd5.

⛔ Files ignored due to path filters (2)
  • tools/encryptly/windows-arm64/encryptly.exe is excluded by !**/*.exe
  • tools/encryptly/windows-x64/encryptly.exe is excluded by !**/*.exe
📒 Files selected for processing (8)
  • build.py
  • diagnostic/build-0618b98e-part001.logd
  • diagnostic/build-0618b98e-part002.logd
  • diagnostic/build-0618b98e.json
  • tests/test_build_diagnostics.py
  • tools/encryptly/linux-arm64/encryptly
  • tools/encryptly/linux-x64/encryptly
  • tools/encryptly/macos-arm64/encryptly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant