fix(discovery): honour type_source='manual' in the UniFi reconcile path (#3011) - #3185
Conversation
reconcileDiscoveredAsset wrote asset_type unconditionally on both the UPDATE and the ON CONFLICT branch, so the next UniFi sync clobbered a type a user had set by hand. The manual write itself landed correctly (type_source='manual' stuck) — it was simply overwritten on the next ingest, which is why the value appeared to "revert on refresh". The agent-scan path (discoveryWorker.processResults) already implements this precedence; the UniFi path was never given it. Two guards: - type_source='manual' now suppresses the asset_type write. The lookup selects type_source for the UPDATE branch; the ON CONFLICT branch cannot know the colliding row's type_source at build time, so the guard is expressed in SQL (`case when type_source = 'manual' then asset_type else excluded.asset_type end`). type_source is set only on the INSERT side so a conflict never resets an existing row to 'auto'. - The UniFi classifier only recognises network gear (switch/AP/gateway/ firewall) and returns 'unknown' for everything else — Protect cameras and doorbells, exactly the models reported. That is "no opinion", not a classification, so it no longer writes either type column. This also stops the sync erasing a classification made by another source. detected_asset_type is now recorded whenever the sync does recognise a device, so "reset to auto" has a value to restore — previously a UniFi-created asset had nothing to fall back to. No schema change: discovered_assets is already registered in the RLS, org/device cascade, and tenant-export-policy contracts, and both columns already exist. Closes #3011 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deploying breeze with
|
| Latest commit: |
95f85be
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://d20a286a.breeze-9te.pages.dev |
| Branch Preview URL: | https://fix-3011-unifi-manual-asset.breeze-9te.pages.dev |
…t Postgres Review follow-ups on the #3011 fix. - The UPDATE branch decided precedence in JS from a value read by an earlier SELECT. type_source can change in between, and a user saving a manual type mid-sync is exactly the race this issue is about — so that branch had the same bug in a narrower window. Both branches now use one SQL expression, evaluated by Postgres at write time. The lookups no longer need to project type_source at all. - Write sets were typed `Record<string, unknown>`. Drizzle iterates the table's columns and silently DROPS set keys that don't name one, and DbExecutor is deliberately loose, so a mis-keyed column would compile, pass the mocked tests and no-op at runtime. They are now PgUpdateSetSource<typeof discoveredAssets>, which rejects the typo at compile time. - excluded.asset_type is derived from the column definition rather than hardcoded, so a rename cannot desync the two halves of the CASE. - The unit test asserted the guard by flattening the sql template's literal chunks. That drops every column reference, so an INVERTED guard rendered byte-identical and passed. Assertions now render through PgDialect; both mutants (inverted branches, wrong guard column) fail. The conflict arbiter is asserted too, and the mac-less ip-fallback path is covered. - Added unifiAssetTypeSource.integration.test.ts. A mock cannot verify any of what makes this work — that a table-qualified reference in DO UPDATE SET reads the pre-update row, that the bind parameter unifies with the discovered_asset_type enum, that excluded resolves, that the (org_id, ip_address) arbiter is inferred. Five cases against real Postgres, including the ON CONFLICT branch driven through a simulated race; four fail against origin/main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Review run: Findings: 5 raised as real and consequential → all addressed in 072dc8f; 0 outstanding. 4 further findings triaged as pre-existing/out-of-scope and recorded as follow-ups in the PR body rather than fixed here. Addressed:
Triaged as follow-ups (detail in the PR body): Tests: Status: review-clean, awaiting maintainer merge. Not merging and not closing #3011 — the reporter should confirm the manual type sticks across a sync first. |
…ivy on all open PRs (#3212) ## Why **GHSA-5p4m-2wfm-xmqj / CVE-2026-59870** — quadratic CPU consumption in js-yaml's `!!omap` resolution (3.x and 4.x), rated **HIGH**, fixed in **4.3.1** / 3.15.1. The advisory entered Trivy's vulnerability DB at ~02:25 UTC on 2026-08-07. From that moment every PR whose scan resolved after the DB refresh went red on **both** `Trivy Filesystem Scan` and `Trivy Image Scan` (Web image). PRs scanned before the refresh are still green, which is why the failure looked selective rather than global — it is not. This blocks all six remaining v0.104 PRs (#3184, #3185, #3186, #3194, #3195, #3196) and will block every future PR and main until it lands. These are genuine scan-step failures, not the GitHub Actions outage that hit the earlier batch — the jobs fail at `Run blocking Trivy filesystem scan` / `Scan Web image`, with every prior step green. ## What One `pnpm.overrides` entry. js-yaml 4.3.0 arrives transitively through the Astro / expressive-code docs toolchain; nothing in the repo depends on it directly. The override is **upper-bounded to `<5.0.0`**. Without that bound it resolves to 5.2.2, a major-version jump for `@astrojs/markdown-remark`, `@astrojs/starlight` and `@expressive-code/core`. Bounding it keeps the change a 4.3.0 → 4.3.1 patch bump, matching the convention already used for `undici` and `@babel/core`. The tree's other two js-yaml copies need no action: **3.15.1** is already the fixed 3.x release named in the advisory, and **5.2.2** is unaffected. ## Note on the diff The lockfile carries one hunk unrelated to js-yaml: `anymatch@3.1.3`'s picomatch pin moves 4.0.5 → 4.0.4. That is pre-existing drift between `package.json` and the committed lockfile on main which a fresh resolve normalizes — not something this change introduces. Left as the resolver produced it rather than hand-editing the lockfile into an inconsistent state. Co-authored-by: Todd Hebebrand <todd@lanternops.io> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Closes #3011
Root cause
reconcileDiscoveredAssetinapps/api/src/services/unifi/unifiSyncService.tswroteasset_typeunconditionally on both the UPDATE branch and theON CONFLICT DO UPDATEbranch, and never consultedtype_source.The manual write was never lost —
PATCH /discovery/assets/:idcorrectly stampedtype_source='manual', and the next UniFi sync overwroteasset_typeon top of it. That is exactly the reporter's evidence:type_sourcestill readsmanual,asset_typehas reverted, andlast_seen_at/unifi_devices.updated_atbump together on the revert.The agent-scan path (
discoveryWorker.processResults) already implements this precedence. The UniFi path was never given it — the original2026-06-28-discovered-asset-type-source.sqldesign doc has no UniFi coverage at all.unifiTelemetryService(the agent-collector path) never writesasset_type, so it was not a contributor. This is the only write site, verified by grep acrossservices/unifi/,jobs/unifi*.tsandroutes/unifi/.The fix
1. A manual type outranks telemetry classification — enforced in SQL on both write paths.
<proposed>is a bind parameter on the UPDATE path andexcluded.asset_typeon the conflict path. The guard is deliberately not a JS decision: on the conflict path we never read the colliding row, and on the update pathtype_sourcecan change between the SELECT and the UPDATE — a user saving a manual type mid-sync is precisely the race this issue is about.type_sourceis written only on the INSERT side, so a collision can never reset an existing row to'auto'.2. An unclassified device no longer writes either type column.
assetType()only maps UniFi network gear (usw/uap/ugw/udm/ufg) and returns'unknown'for everything else — including the G6 180, G4 Doorbell Pro and G3 Pro named in the issue. That return means "this sync has no opinion", not "the device is of unknown type", so stamping it back would erase a classification made by another source. This is why the reporter sawunknownspecifically.This is slightly beyond the literal ask — flagging it explicitly. It is required for the correctness of the change itself: without it, the newly-added
detected_asset_typewrite would clobber a better detection from the agent-scan path. It does not implement camera auto-identification, which stays with #2199.3.
detected_asset_typeis now recorded whenever the sync does recognise a device, mirroringdiscoveryWorker. Previously a UniFi-created asset had it NULL, so "reset to auto" (coalesce(detected_asset_type, asset_type)) had nothing to restore.Supporting hardening from review: write sets are typed
PgUpdateSetSource<typeof discoveredAssets>(drizzle silently drops set keys that don't name a column, andDbExecutoris deliberately loose — a mis-keyed column previously compiled, passed the mocked tests, and no-opped at runtime), andexcluded.asset_typeis derived from the column definition so a rename cannot desync the CASE.Relationship to #2199
#2199 notes a "manual overrides still win" requirement. This PR establishes that precedence rule in the UniFi path so #2199 builds on it rather than reimplementing it.
Tenancy
No schema or migration change —
type_sourceanddetected_asset_typealready exist (2026-06-28-discovered-asset-type-source.sql).discovered_assetsis already registered inCORE_ORG_CASCADE_DELETE_ORDER,CORE_DEVICE_CASCADE_DELETE_TABLESandCORE_TENANT_EXPORT_POLICY(both columns already classifiedincluded), and is RLS shape 1 (directorg_id, auto-discovered). No registration work applies.The UniFi ingest runs under
withSystemDbAccessContext, so RLS is bypassed on this path — the override guard is necessarily application-level, which is what this PR adds.Verification
Real Postgres —
apps/api/src/__tests__/integration/unifiAssetTypeSource.integration.test.ts(new, 5 cases). A mock cannot verify what makes this work: that a table-qualified reference insideDO UPDATE SETreads the pre-update row, that the bind parameter unifies with thediscovered_asset_typeenum, thatexcludedresolves, that the(org_id, ip_address)arbiter is inferred. Covers manual-preserved and auto-applied on both the UPDATE path and the ON CONFLICT path (the latter driven through a simulated race that makes the row invisible to the lookups but present for the arbiter), plus the unclassified-device case. 4 of the 5 fail againstorigin/main; the fifth is a positive control.Unit — 6 new cases in
unifiSyncService.test.ts. The guard assertions render throughPgDialectrather than flatteningsqltemplate chunks; the earlier chunk-flattening version passed an inverted guard, and both mutants (inverted branches, wrong guard column) now fail. Conflict arbiter and the mac-less IP-fallback path are covered.Suites —
vitest run src/services/unifi src/jobs/unifiTelemetryWorker.test.ts src/routes/unifi src/routes/discovery.test.ts src/jobs/discoveryWorker.test.ts→ 11 files, 186 passed. Integration:unifiAssetTypeSource+unifiCollectorUpsert→ 7 passed.tsc --noEmitclean.Known follow-ups (not in this PR)
discoveryWorkerstill stamps'unknown'and, worse,inferAssetTypeFromVendormapsubiquiti → access_pointwhile this sync setsmanufacturer: 'Ubiquiti'on every asset it touches — so an agent scan can rewrite a correctly-classified UniFi switch toaccess_pointand flapdetected_asset_typebetween runs. Pre-existing and orthogonal to [Discovery] Manual Asset Type is overwritten on the next UniFi ingest — type_source='manual' is not honoured by the reconcile path #3011 (that path does honourtype_source='manual'), but the "no opinion" semantics is only half-adopted across the two writers.detected_asset_typestays NULL,coalescefalls back to the current value). Pre-existing; naturally resolved by [Discovery] Auto-identify cameras and UniFi devices (hostname heuristics, OUI coverage, UniFi Protect sync) #2199 adding camera mappings.discovered_assets.updated_atis not bumped by the UniFi sync (no$onUpdate, not in the write set), unlikediscoveryWorker. Pre-existing.🤖 Generated with Claude Code