-
-
Notifications
You must be signed in to change notification settings - Fork 4
fix: SDK audit — correctness fixes and improved drift detection #15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -71,49 +71,229 @@ jobs: | |||||
| name: audit-report | ||||||
| path: specs/audit-report.md | ||||||
|
|
||||||
| - name: Build issue body | ||||||
| - name: Classify severity and build issue | ||||||
| id: classify | ||||||
| if: steps.fetch.outputs.changes == 'true' | ||||||
| run: | | ||||||
| SEVERITY="low" | ||||||
| DATE=$(date -u +%Y-%m-%d) | ||||||
|
|
||||||
| # --- Parse diff report for high/medium signals --- | ||||||
| DIFF_FILE="specs/diff-report.md" | ||||||
| NEW_ENDPOINTS=0 | ||||||
| REMOVED_ENDPOINTS=0 | ||||||
| CHANGED_ENDPOINTS=0 | ||||||
| SCHEMA_CHANGES=0 | ||||||
|
|
||||||
| if [ -f "$DIFF_FILE" ]; then | ||||||
| # Count sections that have content (not just "No ... endpoints/changes") | ||||||
| if ! grep -q "No new endpoints" "$DIFF_FILE"; then | ||||||
| NEW_ENDPOINTS=$(grep -c '^\- \*\*' "$DIFF_FILE" 2>/dev/null || echo 0) | ||||||
| fi | ||||||
| if ! grep -q "No removed endpoints" "$DIFF_FILE"; then | ||||||
| REMOVED_ENDPOINTS=1 | ||||||
| fi | ||||||
| if ! grep -q "No changed endpoints" "$DIFF_FILE"; then | ||||||
| CHANGED_ENDPOINTS=1 | ||||||
| fi | ||||||
| if ! grep -q "No schema changes" "$DIFF_FILE"; then | ||||||
| SCHEMA_CHANGES=1 | ||||||
| fi | ||||||
| fi | ||||||
|
|
||||||
| # --- Parse audit report for drift signals --- | ||||||
| AUDIT_FILE="specs/audit-report.md" | ||||||
| MISSING_ENDPOINTS=0 | ||||||
| BODY_DRIFT=0 | ||||||
| PARAM_DRIFT=0 | ||||||
| CODE_ISSUES=0 | ||||||
|
|
||||||
| if [ -f "$AUDIT_FILE" ]; then | ||||||
| MISSING_ENDPOINTS=$(grep -oP 'Missing from SDK: \K[0-9]+' "$AUDIT_FILE" | head -1 || echo 0) | ||||||
| [ -z "$MISSING_ENDPOINTS" ] && MISSING_ENDPOINTS=0 | ||||||
|
|
||||||
| if ! grep -q "No request body drift" "$AUDIT_FILE"; then | ||||||
| BODY_DRIFT=1 | ||||||
| fi | ||||||
| if ! grep -q "No query/path parameter drift" "$AUDIT_FILE"; then | ||||||
| PARAM_DRIFT=1 | ||||||
| fi | ||||||
| CODE_ISSUES=$(grep -oP 'Code issues found: \K[0-9]+' "$AUDIT_FILE" | head -1 || echo 0) | ||||||
| [ -z "$CODE_ISSUES" ] && CODE_ISSUES=0 | ||||||
| fi | ||||||
|
|
||||||
| # --- Determine severity --- | ||||||
| # High: removed endpoints, missing endpoints, breaking schema changes, body drift | ||||||
| if [ "$REMOVED_ENDPOINTS" -gt 0 ] || [ "$MISSING_ENDPOINTS" -gt 0 ] || [ "$BODY_DRIFT" -gt 0 ]; then | ||||||
| SEVERITY="high" | ||||||
| # Medium: new endpoints, changed endpoints, param drift, schema changes, real code issues | ||||||
| elif [ "$NEW_ENDPOINTS" -gt 0 ] || [ "$CHANGED_ENDPOINTS" -gt 0 ] || [ "$PARAM_DRIFT" -gt 0 ] || [ "$SCHEMA_CHANGES" -gt 0 ]; then | ||||||
|
||||||
| elif [ "$NEW_ENDPOINTS" -gt 0 ] || [ "$CHANGED_ENDPOINTS" -gt 0 ] || [ "$PARAM_DRIFT" -gt 0 ] || [ "$SCHEMA_CHANGES" -gt 0 ]; then | |
| elif [ "$NEW_ENDPOINTS" -gt 0 ] || [ "$CHANGED_ENDPOINTS" -gt 0 ] || [ "$PARAM_DRIFT" -gt 0 ] || [ "$SCHEMA_CHANGES" -gt 0 ] || [ "$CODE_ISSUES" -gt 0 ]; then |
Copilot
AI
Mar 23, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These sed -n '/## Request Body Drift/,/^## /p' (and similar) ranges will stop immediately because the end pattern ^## matches the same header line as the start pattern. As a result the issue body will include only the section header, not the section content. Use an end pattern that matches the next specific section header (or a range that excludes the first ^## match).
Copilot
AI
Mar 23, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The EXTRA_SECTION extraction uses sed -n '/## Extra SDK Methods/,/^## /p', which will also stop on the start header and likely capture only a single line. This prevents the <details> block from containing the actual list; extract until the next known section header (e.g., “## Missing Exports”) instead.
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,5 @@ | ||||||
| from dataclasses import dataclass | ||||||
| from enum import Enum | ||||||
| from typing import Optional, Union | ||||||
|
||||||
| from typing import Optional, Union | |
| from typing import Union |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
NEW_ENDPOINTScounter is computed by grepping for- **across the entire diff report, which will also count bullets in other sections (Removed Endpoints, Schema Changes, Deprecations). This can inflate the “new endpoints” count and summary; consider scoping the count to just the “## New Endpoints” section (e.g., by extracting that section first).