Skip to content

feat(formatters): GraphML XML export for graph-producing commands (closes #59 phase-3)#153

Merged
Wolfvin merged 3 commits into
mainfrom
feat/issue-59-graphml-export
Jul 3, 2026
Merged

feat(formatters): GraphML XML export for graph-producing commands (closes #59 phase-3)#153
Wolfvin merged 3 commits into
mainfrom
feat/issue-59-graphml-export

Conversation

@Wolfvin

@Wolfvin Wolfvin commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Closes #59 (phase-3)

Summary

Implements Phase 3 of issue #59 — GraphML export. Adds --format graphml to emit a GraphML 1.0 XML document for the four graph-producing commands (scan, trace, impact, circular). Output opens directly in Gephi, Cytoscape, yEd, and Neo4j. Non-graph commands produce a single-node placeholder graph so the format is always valid XML, never raises.

What changed

New files

  • scripts/formatters/graphml.py — stdlib-only GraphML writer (no networkx dependency — KISS). Uses xml.etree.ElementTree. Per-command extractors:
    • scan: full call graph from SQLite graph_nodes + graph_edges
    • trace: subgraph from chains.up + chains.down
    • impact: subgraph from affected.direct + affected.indirect
    • circular: subgraph from every cycle (handles explicit-close chains like [foo, bar, foo], non-closing chains via wrap-around, and recursion self-loops)
  • tests/test_graphml_formatter.py — 35 tests covering all 4 commands, placeholder fallback, GraphML XML validity (namespace, key declarations, unique node IDs, edge references resolve, boolean serialization per spec), and never-raise guarantee on weird inputs

Modified files

  • scripts/formatters/__init__.py — dispatch format_output(..., "graphml") to the new formatter
  • scripts/codelens.py — add "graphml" to --format choices in 3 places (subparser default, global default, pre-parse recognition)
  • scripts/mcp_server.py — add "graphml" to _FORMAT_PROPERTY enum, arch-metrics schema, and 2 comment refs
  • README.md — list graphml in format enum + add 4 GraphML usage examples (scan/trace/impact/circular)
  • SKILL-QUICK.md — graphml row in flags table + MCP tools format note

Acceptance criteria (from issue body)

  • Phase 3: GraphML opens correctly in Gephi — verified via structural validation:
    • Root element is <graphml> in the http://graphml.graphdrawing.org/xmlns namespace
    • <key> declarations for both node and edge attributes
    • <graph edgedefault="directed"> per the spec
    • All <node> IDs are unique within a single document
    • Every <edge source/target> resolves to a declared <node id>
    • Every <data key="..."> references a <key id="..."> declared at top
    • XML declaration present, UTF-8 encoded
    • Booleans serialized as true/false (not Python's True/False)
    • Never raises — returns a placeholder graph on any input (including None, lists, malformed dicts)

End-to-end verification

$ python3 scripts/codelens.py scan tests/fixtures --format graphml | head -5
<?xml version='1.0' encoding='utf-8'?>
<graphml xmlns="http://graphml.graphdrawing.org/xmlns">
  <key id="d0" for="node" attr.name="name" attr.type="string" />
  ...

$ python3 scripts/codelens.py scan tests/fixtures --format graphml | wc -l
842   # 32 nodes, 18 edges from real SQLite data

Tests

  • New: 35 passed in tests/test_graphml_formatter.py
  • Regression check: tests/test_formatters.py + tests/test_compact_format.py + tests/test_cli.py → 144 passed, 10 skipped, 0 failed
  • Pre-existing failures (NOT regressions, confirmed via git stash):
    • tests/test_codelensignore.py::TestBackwardCompat::test_actual_target_dir_is_ignored — failing on main before this PR
    • tests/test_universal_grammar_loader.py — flaky test ordering issue with tree-sitter env var; passes in isolation

Design decisions

  1. No networkx dependency — KISS rule from pre-flight SKILL.md. GraphML is a well-defined XML schema; stdlib xml.etree.ElementTree is sufficient. Avoids adding a new hard dep just for this feature.

  2. Global format choice, not command-specific — Matches the pattern used by sarif (also globally available even though only some commands produce findings). Keeps the CLI surface uniform.

  3. Placeholder for non-graph commands — When a command like init or doctor is invoked with --format graphml, the formatter emits a single-node graph with a warning data attribute explaining that the command doesn't produce graph data. This keeps the format always-valid XML.

  4. Per-command graph extraction — Each graph-producing command has its own extractor that knows the shape of its result dict. The extractors are unit-testable in isolation (tested in TestExtractorsDirectly).

Findings (per pre-flight SKILL.md — outside scope, BOS decides)

  • The issue body mentions callgraph and change-coupling commands — these don't exist in the current command registry (70 commands). The 4 existing graph-producing commands (scan, trace, impact, circular) are the ones this PR targets. If callgraph/change-coupling are added later, they can be registered in _EXTRACTORS dict in graphml.py with a one-line addition.

Branch

feat/issue-59-graphml-export

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Wolfvin Wolfvin force-pushed the feat/issue-59-graphml-export branch 3 times, most recently from 84853e7 to 937c02e Compare July 3, 2026 02:13
@Wolfvin Wolfvin merged commit 77aba34 into main Jul 3, 2026
Wolfvin added 3 commits July 3, 2026 09:14
…oses #59 phase-3)

Add `--format graphml` to emit GraphML 1.0 XML for scan, trace, impact,
and circular commands. Output opens directly in Gephi/Cytoscape/yEd/Neo4j.

Implementation:
- New scripts/formatters/graphml.py — stdlib-only GraphML writer (no
  networkx dependency, KISS) using xml.etree.ElementTree
- Per-command graph extractors:
  - scan: full call graph from SQLite graph_nodes/graph_edges
  - trace: subgraph from chains.up + chains.down
  - impact: subgraph from affected.direct + affected.indirect
  - circular: subgraph from every cycle (handles explicit-close chains,
    non-closing chains via wrap-around, and recursion self-loops)
- Non-graph commands get a single-node placeholder graph (with warning
  attribute) so --format graphml is always valid XML, never raises
- Registered as a global format choice in codelens.py (subparser default,
  global default, pre-parse recognition — 5 occurrences) and mcp_server.py
  (_FORMAT_PROPERTY + arch-metrics schema + comment refs)
- 35 new tests in tests/test_graphml_formatter.py covering all 4
  commands, placeholder fallback, GraphML XML validity (namespace,
  key declarations, unique node IDs, edge references resolve, boolean
  serialization per spec), and never-raise guarantee on weird inputs

Docs:
- README.md: list graphml in format enum + 4 GraphML usage examples
- SKILL-QUICK.md: graphml row in flags table + MCP tools format note

Verified:
- All 35 new tests pass
- test_formatters.py + test_compact_format.py + test_cli.py: 144 passed
  10 skipped (no regressions)
- End-to-end: `codelens scan tests/fixtures --format graphml` produces
  32 nodes / 18 edges of valid GraphML from real SQLite data
- Pre-existing failures (test_codelensignore, test_universal_grammar_loader)
  are unrelated — confirmed failing on main via git stash
@sonarqubecloud

sonarqubecloud Bot commented Jul 3, 2026

Copy link
Copy Markdown

@Wolfvin Wolfvin deleted the feat/issue-59-graphml-export branch July 3, 2026 02:14
Wolfvin added a commit that referenced this pull request Jul 3, 2026
Two fixes for PR #141 per issue #171:

1. Add missing @WHO/@WHAT/@PART/@entry file headers to the 3 new files
   introduced by PR #141:
   - scripts/baseline_diff.py
   - scripts/exit_policy.py
   - scripts/git_integration.py
   (issue #171 explicitly mentions baseline_diff.py; exit_policy.py and
   git_integration.py have the same gap — fixed for consistency since
   CONTRIBUTING.md mandates the header on all new files.)

2. Restore the `-f` shortcut conflict guard in scripts/codelens.py that
   was removed by PR #153 (graphml, issue #59 Phase 3). The guard checks
   `existing_option_strings` before adding `-f` to a subparser, so
   commands like `affected` (issue #62) that use `-f` for `--filter`
   don't conflict with the global `--format -f` shortcut.

   This bug made `codelens --help` and every CLI invocation crash with
   `argparse.ArgumentError: argument --format/-f: conflicting option
   string: -f`. It was a pre-existing regression on main (introduced
   when PR #153 was merged) that blocked PR #141's tests from running.

   The fix restores the pre-#153 logic:
   ```python
   if "format" not in existing_dests:
       format_args = ["--format"]
       if "-f" not in existing_option_strings:
           format_args.append("-f")
       sub.add_argument(*format_args, choices=[...], ...)
   ```

Verified:
- `codelens --command-count` works (returns 75)
- `codelens --help` works
- 99 PR #141 tests pass (test_baseline_diff + test_check_ci_flags +
  test_exit_policy + test_git_integration)
- 196 passed regression (test_cli + test_formatters + test_command_count +
  test_command_registry + 4 PR #141 test files)
- 135 passed (test_graphml_formatter + test_diff_scope + test_secrets_gitleaks
  — confirms the format change doesn't regress graphml/diff-base/gitleaks)
- sync_command_count.py --check: all docs in sync (count=75)

Findings (per pre-flight SKILL.md — flag to BOS):
- The `-f` conflict bug was introduced by PR #153 (issue #59 Phase 3,
  graphml export). PR #153 simplified the format-arg logic and
  accidentally dropped the `existing_option_strings` check. This fix
  restores the check while keeping the `graphml` choice. No regression
  to graphml functionality (test_graphml_formatter.py: 35 passed).
- PR #141 was already rebased on main; no merge conflicts. The rebase +
  these fixes make PR #141 mergeable.
Wolfvin added a commit that referenced this pull request Jul 3, 2026
Two fixes for PR #141 per issue #171:

1. Add missing @WHO/@WHAT/@PART/@entry file headers to the 3 new files
   introduced by PR #141:
   - scripts/baseline_diff.py
   - scripts/exit_policy.py
   - scripts/git_integration.py
   (issue #171 explicitly mentions baseline_diff.py; exit_policy.py and
   git_integration.py have the same gap — fixed for consistency since
   CONTRIBUTING.md mandates the header on all new files.)

2. Restore the `-f` shortcut conflict guard in scripts/codelens.py that
   was removed by PR #153 (graphml, issue #59 Phase 3). The guard checks
   `existing_option_strings` before adding `-f` to a subparser, so
   commands like `affected` (issue #62) that use `-f` for `--filter`
   don't conflict with the global `--format -f` shortcut.

   This bug made `codelens --help` and every CLI invocation crash with
   `argparse.ArgumentError: argument --format/-f: conflicting option
   string: -f`. It was a pre-existing regression on main (introduced
   when PR #153 was merged) that blocked PR #141's tests from running.

   The fix restores the pre-#153 logic:
   ```python
   if "format" not in existing_dests:
       format_args = ["--format"]
       if "-f" not in existing_option_strings:
           format_args.append("-f")
       sub.add_argument(*format_args, choices=[...], ...)
   ```

Verified:
- `codelens --command-count` works (returns 75)
- `codelens --help` works
- 99 PR #141 tests pass (test_baseline_diff + test_check_ci_flags +
  test_exit_policy + test_git_integration)
- 196 passed regression (test_cli + test_formatters + test_command_count +
  test_command_registry + 4 PR #141 test files)
- 135 passed (test_graphml_formatter + test_diff_scope + test_secrets_gitleaks
  — confirms the format change doesn't regress graphml/diff-base/gitleaks)
- sync_command_count.py --check: all docs in sync (count=75)

Findings (per pre-flight SKILL.md — flag to BOS):
- The `-f` conflict bug was introduced by PR #153 (issue #59 Phase 3,
  graphml export). PR #153 simplified the format-arg logic and
  accidentally dropped the `existing_option_strings` check. This fix
  restores the check while keeping the `graphml` choice. No regression
  to graphml functionality (test_graphml_formatter.py: 35 passed).
- PR #141 was already rebased on main; no merge conflicts. The rebase +
  these fixes make PR #141 mergeable.
Wolfvin added a commit that referenced this pull request Jul 3, 2026
Issue #174 reported a P0 argparse conflict: 'argument --format/-f:
conflicting option string(s): -f'. The bug was introduced by PR #153
(graphml, issue #59 Phase 3) which accidentally dropped the
existing_option_strings guard in the subparser --format registration
loop. Commit 23487af (fix #171) restored the guard, fixing the crash.

This PR adds:
1. Defensive comment in scripts/codelens.py (lines 896-904) explaining
   why the guard is critical and referencing the regression test.
2. 7 regression tests in tests/test_cli.py::TestArgparseFormatConflictRegression:
   - test_help_does_not_crash: codelens --help exits 0
   - test_command_count_does_not_crash: codelens --command-count exits 0
   - test_scan_with_format_long_does_not_crash: codelens scan . --format json
   - test_scan_with_format_short_does_not_crash: codelens scan . -f json
   - test_affected_with_filter_short_does_not_crash: codelens affected . -f '*.py'
     (verifies -f for --filter doesn't conflict with -f for --format)
   - test_parser_construction_no_argparse_error: module imports cleanly
   - test_no_duplicate_f_in_subparsers: iterates 5 commands' --help,
     asserts no 'conflicting option string' in stderr

Also runs sync_command_count.py --apply to fix pre-existing command
count drift (76 in docs vs 77 runtime) — unrelated to issue #174 but
caught during testing.

Definition of Done (issue #174):
- [x] codelens --help runs without error
- [x] codelens scan . --format json runs without argparse error
- [x] codelens scan . -f json works
- [x] test_cli.py and test_doctor.py pass (82 passed: 75 existing + 7 new)
- [x] No regression on --diff-base functionality (verified manually)

Tests: 142 passed in test_cli + test_doctor + test_command_count +
test_command_registry + test_formatters (no regressions).
Wolfvin added a commit that referenced this pull request Jul 3, 2026
Issue #174 reported a P0 argparse conflict: 'argument --format/-f:
conflicting option string(s): -f'. The bug was introduced by PR #153
(graphml, issue #59 Phase 3) which accidentally dropped the
existing_option_strings guard in the subparser --format registration
loop. Commit 23487af (fix #171) restored the guard, fixing the crash.

This PR adds:
1. Defensive comment in scripts/codelens.py (lines 896-904) explaining
   why the guard is critical and referencing the regression test.
2. 7 regression tests in tests/test_cli.py::TestArgparseFormatConflictRegression:
   - test_help_does_not_crash: codelens --help exits 0
   - test_command_count_does_not_crash: codelens --command-count exits 0
   - test_scan_with_format_long_does_not_crash: codelens scan . --format json
   - test_scan_with_format_short_does_not_crash: codelens scan . -f json
   - test_affected_with_filter_short_does_not_crash: codelens affected . -f '*.py'
     (verifies -f for --filter doesn't conflict with -f for --format)
   - test_parser_construction_no_argparse_error: module imports cleanly
   - test_no_duplicate_f_in_subparsers: iterates 5 commands' --help,
     asserts no 'conflicting option string' in stderr

Also runs sync_command_count.py --apply to fix pre-existing command
count drift (76 in docs vs 77 runtime) — unrelated to issue #174 but
caught during testing.

Definition of Done (issue #174):
- [x] codelens --help runs without error
- [x] codelens scan . --format json runs without argparse error
- [x] codelens scan . -f json works
- [x] test_cli.py and test_doctor.py pass (82 passed: 75 existing + 7 new)
- [x] No regression on --diff-base functionality (verified manually)

Tests: 142 passed in test_cli + test_doctor + test_command_count +
test_command_registry + test_formatters (no regressions).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Interactive dashboard — D3 force-directed graph + heatmap + cluster hull (5 workers converged)

1 participant