Skip to content

feat(parser): migrate qwen provider#761

Draft
mariusvniekerk wants to merge 1 commit into
provider-pifrom
provider-qwen
Draft

feat(parser): migrate qwen provider#761
mariusvniekerk wants to merge 1 commit into
provider-pifrom
provider-qwen

Conversation

@mariusvniekerk

@mariusvniekerk mariusvniekerk commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Qwen uses a nested project/chats JSONL source shape, so it is a good next provider facade slice after the shallow and directory JSONL migrations. Moving it behind a concrete provider keeps discovery, lookup, fingerprinting, and parse output explicit without introducing another source framework.

Legacy discovery accepts any one-level .jsonl file under chats while raw-session lookup still validates the requested ID before matching filename-derived IDs. The provider keeps that asymmetry, symlinked project directory and file behavior, project hints, and existing parser normalization intact.

@roborev-ci

roborev-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown

roborev: Combined Review (4d719cd)

No issues found.


Panel: ci_default_security | Synthesis: codex | Members: codex_default (claude-code/default, done, 4m13s), codex_security (claude-code/security, done, 49s) | Total: 5m2s

@roborev-ci

roborev-ci Bot commented Jun 20, 2026

Copy link
Copy Markdown

roborev: Combined Review (186dc79)

No issues found.


Panel: ci_default_security | Synthesis: codex | Members: codex_default (claude-code/default, done, 3m45s), codex_security (claude-code/security, done, 1m0s) | Total: 4m45s

@roborev-ci

roborev-ci Bot commented Jun 21, 2026

Copy link
Copy Markdown

roborev: Combined Review (f82fab1)

Summary verdict: one medium correctness issue should be fixed before relying on Qwen provider shadow comparisons.

Medium

  • internal/parser/qwen_provider.go:113 - Qwen is opted into ProviderMigrationShadowCompare, but its source set does not enable Hash, so provider.Fingerprint leaves Hash empty and qwenProvider.Parse stores an empty Session.File.Hash. The legacy processQwen path computes and stores a SHA-256 hash, so real shadow comparisons will report every parsed Qwen session as different. The new shadow test masks this by copying the empty observation hash into the legacy session.
    • Fix: Add Hash: true to the Qwen JSONLSourceSetOptions and update the Qwen shadow/provider tests to assert a non-empty fingerprint hash that matches the legacy file hash.

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 5m21s), codex_security (codex/security, done, 1m2s) | Total: 6m30s

@mariusvniekerk

mariusvniekerk commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

@roborev-ci

roborev-ci Bot commented Jun 21, 2026

Copy link
Copy Markdown

roborev: Combined Review (6b3c03f)

Review verdict: one Medium issue remains; no High or Critical findings.

Medium

  • internal/parser/qwen_provider.go:113: The Qwen source set does not enable content hashing, so qwenProvider.Parse leaves Session.File.Hash empty during normal provider observation. Legacy processQwen computes and stores the file hash, and this PR opts Qwen into shadow comparison, so successful Qwen sessions will produce caller-level shadow mismatches and would lose file_hash if promoted to provider-authoritative.
    • Fix: Set Hash: true in JSONLSourceSetOptions for Qwen, or otherwise compute the same SHA-256 hash before returning the parse result, and add a caller-level shadow test that compares through processFile.

Panel: ci_default_security | Synthesis: codex, 6s | Members: codex_default (codex/default, done, 3m17s), codex_security (codex/security, done, 1m52s) | Total: 5m15s

@roborev-ci

roborev-ci Bot commented Jun 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (ae746c8)

No issues found.


Panel: ci_default_security | Synthesis: codex | Members: codex_default (claude-code/default, done, 7m52s), codex_security (claude-code/security, done, 1m57s) | Total: 9m49s

@roborev-ci

roborev-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown

roborev: Combined Review (6f4b43e)

Summary verdict: one medium regression blocks full Qwen provider parity; no security issues were found.

Medium

  • internal/parser/types.go:335 - Dropping Qwen's DiscoverFunc/FindSourceFunc leaves provider-unaware paths blind to unsynced Qwen sessions. resolveRawSessionID still probes disk only through FindSourceFunc, and SSH remote directory resolution only includes agents with DiscoverFunc, so session usage qwen:<id> and remote sync discovery regress for Qwen.
    • Fix: Add provider-backed probing to those call sites, or keep Qwen compatibility hooks until all downstream discovery/lookup paths use providers.

Panel: ci_default_security | Synthesis: codex, 7s | Members: codex_default (codex/default, done, 5m15s), codex_security (codex/security, done, 1m33s) | Total: 6m55s

Qwen uses a nested project/chats JSONL source shape, so it is a good next provider facade slice after the shallow and directory JSONL migrations. Moving it behind a concrete provider keeps discovery, lookup, fingerprinting, and parse output explicit without introducing another source framework.

Legacy discovery accepts any one-level .jsonl file under chats while raw-session lookup still validates the requested ID before matching filename-derived IDs. The provider keeps that asymmetry, symlinked project directory and file behavior, project hints, and existing parser normalization intact.

Validation: go fmt ./...; go test -tags "fts5" ./internal/parser -run TestQwenProvider -count=1; go test -tags "fts5" ./internal/parser -count=1; make test-short; go vet ./...; git diff --check

test(parser): opt qwen into provider shadow

Qwen now has a concrete facade provider on this branch, so its migration mode should enter shadow comparison instead of remaining an additive legacy-only provider.

Lower provider opt-ins stay inherited and later branches remain responsible for their own concrete providers.

Validation: go test -tags "fts5" ./internal/parser -run TestProviderMigrationModes -count=1; go test -tags "fts5" ./internal/parser -count=1; go vet ./...; git diff --check

test(sync): compare qwen shadow parity

Qwen is shadow-compared on this branch, so add the source-level migration proof that provider observation matches ParseQwenSession for its nested project/chats layout.

This keeps reviewers focused on behavioral parity while later branches continue migrating their own provider shapes.

Validation: go test -tags "fts5" ./internal/parser ./internal/sync -run 'TestObserveProviderSourceMatchesQwenLegacyParser|TestQwenProvider|TestParseQwen' -count=1; go fmt ./...; go vet ./...; git diff --check; go test -tags "fts5" ./internal/parser ./internal/sync -count=1; ./custom-gcl run --config .golangci.nilaway.yml ./internal/parser/... ./internal/sync/...

refactor(parser): fold qwen into provider

Qwen already had a concrete provider, but the branch still kept exported legacy parser/source functions and legacy sync dispatch. That left the migration additive instead of making the provider shape authoritative.\n\nMove Qwen parsing behind the provider method, remove the registry callbacks and sync processor/classifier, and replace the shadow comparison with provider API coverage plus a guard that the old entrypoints stay gone.\n\nValidation: go test -tags "fts5" ./internal/parser -run 'TestQwen|TestParseQwenSession' -count=1 -v; go test -tags "fts5" ./internal/sync -run 'TestEngine_ClassifyPathsQwenSession|TestProviderMigration|TestObserveProvider|TestSyncSingle.*Qwen|TestQwen' -count=1 -v; go test -tags "fts5" ./internal/parser ./internal/sync ./cmd/agentsview -count=1; go vet ./...; git diff --check

fix(parser): thread ctx through qwen source lookups
@roborev-ci

roborev-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown

roborev: Combined Review (dba161b)

High-severity compile failure; medium-severity Qwen hash regression. No security issues reported.

High

  • internal/parser/qwen_provider_test.go:47 - The new test references sourceDisplayPaths and sourceProjects, but those helpers are not defined in the parser package, so the test package will not compile. Add shared/local helper functions for these projections or inline the extracted slices in the test.

Medium

  • internal/parser/qwen_provider.go:23 - The Qwen source set does not enable content hashing, so qwenParseFile will never receive req.Fingerprint.Hash during normal provider processing. The legacy path computed and persisted File.Hash, so this migration will clear file_hash for Qwen sessions. Add withContentHashing() to newQwenSourceSet and cover the real Fingerprint + Parse path in the provider test.

Panel: ci_default_security | Synthesis: codex, 9s | Members: codex_default (codex/default, done, 4m1s), codex_security (codex/security, done, 1m10s) | Total: 5m20s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant