Drift Report enhancements: per-parameter confidence, enrichment, systemic gaps, annotations - #60
Merged
juemerson-at-purestorage merged 17 commits intoJul 27, 2026
Conversation
…ec extraction
Get-PfbSpecCapabilities threw away everything but a bare property name string when
extracting request-body properties (Get-PfbSchemaPropertyNames, line 221 in the old
code) and a query parameter's name (Get-PfbSpecCapabilities, line 265-266). That's
the reason readOnly never survives into the capability map or the drift report
today: the information is discarded at the very first extraction step, before it
reaches anything downstream.
Add Get-PfbSchemaPropertyDetails as the one schema walker (Get-PfbSchemaPropertyNames
is now a thin wrapper over it -- nothing else in the repo calls the old function, so
a second parallel allOf/$ref walker would only drift from the first). It resolves the
same allOf/$ref chains at the *schema* level and keeps each property's own node
instead of discarding it, exposing Name/ReadOnly/Deprecated/Type/Format/Required.
Deliberately does NOT follow a property's own "$ref" to look for these attributes
(only its own directly-declared keys). Resolving into it is the natural
implementation choice, and it changes the answer: `direction` on
POST /file-system-replica-links is a bare $ref to `_direction`, which is itself
readOnly, but `direction`'s own node has no sibling readOnly key. Following it would
flip `direction` from an actionable gap to read-only and shift the fb2.27 baseline
from 226 read-only / 402 addable / 33 enum-ready to 227 / 401 / 32 -- the only field
on the whole surface where it matters, so it's easy to get wrong silently.
Get-PfbSpecCapabilities gains ReadOnlyBodyProperties, DeprecatedBodyProperties,
BodyPropertyDetails, and ParameterComponents (a { paramName: componentName } map
recovered from each parameter's own "$ref" string, not from the resolved parameter
object). ParameterComponents deliberately covers the same parameter set as the
existing Parameters output (all `in:` locations), not just query params, so a future
consumer can zip them together per endpoint. An inline parameter with no $ref simply
has no entry (never emitted as null/'', indistinguishable from a bug downstream); a
parameter name that resolves to two different components on one operation (not
expected -- 0 occurrences across all of fb2.27) is reported via Write-Warning and
resolved deterministically to the alphabetically-first component.
All of this is additive: Parameters/BodyProperties keep their existing name, type,
and contents unchanged, and every existing test in PfbSpecTools.Tests.ps1 passes
with zero edits. Verified against tools/specs/fb2.27.json: PATCH /certificates's
read-only set is exactly {issued_by, issued_to, realms, status, valid_from,
valid_to}; GET /file-systems's context_names parameter resolves to component
Context_names_get, PATCH /file-systems's to Context_names.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ndings
Code review of the Task 2 spec-details walker found six issues.
Important 1 (controller ruling on a plan conflict -- byte-identical wins for
BodyProperties): routing Get-PfbSchemaPropertyNames through
Get-PfbSchemaPropertyDetails's now-sorted output flipped BodyProperties from
spec-traversal order to alphabetical. Set contents are identical (0 differences
over all 632 fb2.27 operations), but tools/Build-PfbCapabilityMap.ps1 inserts into
an [ordered]@{} in traversal order, and 124 of 632 endpoints in the committed
Data/PfbCapabilityMap.json already have non-alphabetical bodyProperties -- sorting
would have churned ~1100 keys in a tracked file Task 3 is about to touch, burying
its additive change in reordering noise. Fixed by extracting the shared walk setup
into Get-PfbSchemaPropertyWalkAccumulators: Get-PfbSchemaPropertyDetails still sorts
its own (detail-record) output by Name, but Get-PfbSchemaPropertyNames now reads
straight off the Dictionary's traversal-order keys instead of piping through the
sorted details. Re-verified byte-identical (including order) against the pre-Task-2
implementation across all 632 fb2.27 operations, plus confirmed the scalar-vs-array
return shape (single-property bodies still return a plain String, not Object[]) --
24 real fb2.27 operations exercise that shape in both old and new code.
Important 2: the sort-invariant on Get-PfbSchemaPropertyDetails's output had no
test -- deleting Sort-Object Name still passed 35/35. Added an assertion on the
ResourcePatch fixture (traversal order id/name/enabled/status/ref_only, not
alphabetical, so it actually bites), plus a companion assertion pinning
Get-PfbSchemaPropertyNames to that SAME fixture's traversal order -- the pair is
what stops either ordering behaviour silently flipping back. Both mutation-tested
(each fails alone without affecting the other, confirming they're independent).
Minor 3: ParameterComponents is populated in explicitly sorted-key order now,
rather than relying on Dictionary enumeration order as an implicit (if reliable)
.NET implementation detail -- it gets serialized into a tracked JSON file
downstream.
Minor 4: added a comment on the readOnly/deprecated/type/format reads explaining
why every test fixture for this function must use PSCustomObject, never @{} --
PSObject.Properties.Name on a plain Hashtable exposes the dictionary's own members
(Keys, Values, Count, ...), never a real key named "readOnly".
Minor 5: strengthened the parameter-collision test's warning assertion from
Should -Not -BeNullOrEmpty (satisfied by any warning at all) to
-Match 'multiple different components'. Mutation-tested: the old assertion would
have passed even with an unrelated warning message; the new one catches it.
Minor 6: renamed Get-PfbSchemaDetailsWalk to Add-PfbSchemaPropertyNodes -- it used
a "Get" verb but returns nothing, mutating its two accumulator arguments in place.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… capability map
Task 3 of the drift-report-actionable plan: surfaces the per-property detail
Task 2 taught the spec parser to extract, in the checked-in capability
manifest, so the drift report can tell "actually addable" gaps from
"read-only, don't bother" ones.
readOnlyBodyProperties uses last-seen-wins (assigned unconditionally each
ascending-version iteration), NOT the first-sight guard that parameters/
bodyProperties use. readOnly is not monotonic: 58 of 1264 analysed
(endpoint, body-field) pairs flip somewhere across fb2.0-2.27, mostly
read-only -> writable. Persisting first-sight would have silently suppressed
11 genuinely settable fields from the actionable gap list -- including
PATCH /api-clients|max_role, one of the fields this whole effort exists to
surface. A noisy gap is annoying; a missing gap is unfindable. Verified
against all 11 negative-canary fields: none appear in the regenerated
readOnlyBodyProperties. deprecatedBodyProperties and parameterComponents get
the same last-seen-wins treatment for the same reason (parameterComponents
in particular describes the CURRENT wire shape, not "introduced in version
X" -- a version number on it would be meaningless). All three keys are
emitted only when non-empty to keep the manifest lean: an unconditional
"[]"/"{}" on every one of the ~520 operations with no read-only fields (or
the ~1 endpoint with no $ref parameters) would blow the ~15KB growth budget
roughly 14x for zero information gain -- confirmed by measurement (readOnly
alone: +13.46KB, matching budget almost exactly; see report for the full
parameterComponents size finding).
The rebuild is capped at -MaxVersion 2.27 (new parameter, compared
numerically via the existing Major/Minor int parsing -- never a string
compare, since '2.9' would otherwise sort above '2.27'). tools/specs/ now
also holds a cached fb2.28.json, but every acceptance number in this plan
(226 read-only / 402 addable / 33 enum-ready / 13 phantom, and this task's
own 632-endpoint/28-version checks) is calibrated against 2.0-2.27. An
uncapped rebuild would silently ingest 2.28 and move those numbers, making a
real regression indistinguishable from an input change. Adopting 2.28 is a
separate, deliberate decision for a later PR.
schemaVersion stays at 1 (purely additive; pinned by
Tests/Build-PfbApiDriftReport.Tests.ps1 and Tests/Get-PfbCapabilityMap.Tests.ps1).
Existing minVersion/parameters/bodyProperties values are verified
byte-identical across all 632 endpoints. Determinism verified: two
consecutive builds to separate paths are SHA256-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…int overrides Follow-up to de752d0: that commit's per-endpoint parameterComponents key grew the manifest +186.87 KB against a ~15KB budget (a 12x overshoot), because nearly every parameter is $ref'd to one of a small set of shared components (Filter, Limit, Sort, ...). Measured on the committed manifest: 4102 total (endpoint, parameter) -> component pairs collapse to just 224 distinct pairs and 179 distinct parameter names -- ~90% pure duplication. Replaces the per-endpoint parameterComponents key with a top-level parameterComponentDefaults table ({ paramName: componentName }, the most frequent component per parameter name across all endpoints, ties broken ALPHABETICALLY for determinism regardless of endpoint processing order) plus a per-endpoint parameterComponentOverrides key (emitted only when non-empty) holding just the 299 (endpoint, parameter) pairs that actually differ from their default, across 250 endpoints. Resolution contract (documented in both the script's comment block and tools/README.md): override wins if present, else default, else no known component. The subtle case this design requires and that the round-trip test exists to catch: a parameter declared inline (no "$ref", so genuinely no component) can still share its NAME with some other endpoint's $ref'd component, and would therefore inherit that unrelated default if simply omitted from overrides. The builder emits an explicit JSON null override for these 3 cases (of 4109 total parameter declarations in fb2.0-2.27) so "absent" (check the default) and "explicitly none" (null) stay distinguishable. Verified with a dedicated mutation test: disabling the null-override code path breaks both the null-override assertion and the round-trip assertion. Added a committed round-trip test (Tests/Build-PfbCapabilityMap.Tests.ps1) that reconstructs every (endpoint, param) -> component pair from defaults + overrides against a hand-authored synthetic ground truth and asserts on the reconstructed set, not just counts -- this is the test that would have caught a dropped override; mutation-tested by dropping the null-override loop (2 assertions failed as expected) and separately by dropping the alphabetical tie-break secondary sort (3 assertions failed as expected, since a stable sort without it would return the wrong majority-tie winner and mis-set two endpoints' override/no-override status). Additionally ran an independent, from-scratch verification script (not committed -- ad hoc confidence check) that re-derives the (endpoint, param) -> component ground truth directly from all 28 cached specs without reusing any of the builder's own code, then diffs it against reconstruction from the regenerated manifest: 4109 pairs checked, 0 mismatches. Net result: manifest is 298,693 bytes, +45.82 KB over the pre-Task-3 baseline (251,777 bytes) -- readOnlyBodyProperties/deprecatedBodyProperties unchanged from de752d0 (+13.46 KB, on budget), parameterComponentDefaults (179 entries) + parameterComponentOverrides (302 entries across 250 endpoints, 3 of them explicit null) add the remaining ~32 KB, close to the ~25 KB dedup estimate. generatedFrom still 28 versions ending 2.27, endpointCount still 632, schemaVersion still 1. Determinism reconfirmed: three independent builds (two scratch + the committed file) are SHA256-identical. All 11 negative-canary fields re-verified absent from every readOnlyBodyProperties. GET /file-systems|context_names resolves to Context_names_get via an OVERRIDE; PATCH /file-systems|context_names resolves to Context_names via the DEFAULT -- confirming these two are exactly the default/override pair the brief described. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dd MaxVersion numeric-compare regression
The parameterComponentOverrides emission check at line 302 read
$overrides.Count, but $overrides is an [ordered]@{} keyed by literal API
query-parameter names -- and "count" is a real parameter name in this API
(parameterComponentDefaults.count = "Ping_count" in the committed manifest).
A hashtable key literally named count/keys/values shadows the matching
.Count/.Keys/.Values *member*, silently returning that key's value instead.
This is a known, previously-hit bug class in this exact codebase (it's why
neighbouring code in this same function already uses get_Keys()/get_Count()
defensively) -- line 302 was the one holdout, inconsistent with line 306 three
lines below it. It was dormant only because "count" has exactly one component
occurrence today, so no endpoint's override set could yet contain a "count"
key; a future spec revision that gives "count" a second, divergent component
would silently drop that endpoint's entire parameterComponentOverrides block.
Fixed proactively before it could bite, mirroring the rest of the function.
Added a synthetic regression test forcing exactly this shape (a "count"
parameter with a global default plus one endpoint overriding it via an
inline, no-$ref declaration -- the null-override case, the worst case for
the bug since `$null -gt 0` is False). Mutation-tested: reverting to bare
.Count fails exactly the 2 assertions exercising this scenario; restoring
the fix passes all 38 tests in the file again.
Also closed a gap in the existing -MaxVersion cap test: it only used
single-digit-minor versions (9.0/9.1/9.2), which sort identically whether
compared as strings or as parsed integers, so it could not have caught a
regression to string comparison. '2.9' vs '2.10' do disagree ('2.9' sorts
above '2.10' as a string but below it numerically) -- this exact string-
vs-numeric mistake has already cost real time twice in this effort. Added a
dedicated 2.9/2.10 case; mutation-tested by swapping the cap filter to a
string comparison, which fails the new test (2 assertions) while leaving the
original 9.0/9.1/9.2 test green, proving that test alone was not sufficient
coverage. Restored the numeric comparison; all tests pass again.
No functional/output change: both bugs were dormant on real data. Verified
by regenerating Data/PfbCapabilityMap.json (via -MaxVersion 2.27) to a
scratch path and confirming its SHA256 is byte-identical to the committed
file, so the manifest itself is untouched by this commit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s enum join Task 5 needs to join each addable body-property gap to its documented enum values (Reports/PfbValueEnumMap.json, keyed <OwnerSchema>.<field>) via Resolve-PfbFieldValueEnum. The existing cmdlet-name-derived resource hint (Get-PfbResourceHint) only reaches 14 of the expected 33 matches against fb2.27, because cmdlet names and declaring schema names diverge (e.g. Update-PfbNfsExportRule -> hint "NfsExportRule", but the real schema is NfsExportPolicyRuleBase*, which the prefix match never finds). Reaching 33 requires the actual declaring schema, which can only be recovered by walking the request body's allOf/$ref chain -- exactly what Get-PfbSchemaPropertyDetails already does. Rather than add a second schema walker for Task 5 (rejected per this plan's "one walker, not two"), thread owner tracking through the existing Add-PfbSchemaPropertyNodes recursion: each property now also carries OwnerSchema, the nearest enclosing named component ($ref'd from #/components/schemas/<Name>) whose own "properties" block declares it, inherited across anonymous/inline allOf branches and resolved outermost-wins on the rare multi-owner collision (2 of ~3960 schema/property pairs in fb2.27: POST /array-connections' "encrypted" and "throttle"). $null (never '') marks "no named owner" so downstream code can tell that apart from a bug. The PIN (never follow a property's own $ref) still holds -- owner comes from the chain being walked into, not from dereferencing the property's own ref -- verified against POST /file-system-replica-links/direction (ReadOnly=false, OwnerSchema _replicaLinkBuiltIn) and the full Pester suite. Every existing field (Name, ReadOnly, Deprecated, Type, Format, Required) and BodyProperties' traversal order are unchanged; this is purely additive. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…→ 2) The Get-PfbSchemaPropertyDetails doc comment incorrectly stated 0 occurrences of multi-owner properties (where two named schemas both declare the same property name) across the fb2.27 spec. The task's own real-spec verification measured exactly 2 occurrences: POST /array-connections' encrypted and throttle properties, each with candidate owners ArrayConnection and ArrayConnectionPost. Updated the doc comment to reflect the real measurement and concrete examples. This fixes a contradiction where a shipped doc comment contradicted the task's own verification report. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…et inventory Get-PfbCmdletParameterInventory now records Line ($p.Extent.StartLineNumber) alongside the File it already carried. Needed so Get-PfbParameterCoverageGaps's upcoming per-parameter confidence annotation can point a reader at an exact file:line for every unresolved parameter, instead of just a filename to search. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…r confidence Get-PfbParameterCoverageGaps used to gate an ENTIRE endpoint on one boolean: any cmdlet parameter with a non-Typed (AttributesOnly/TypedUnresolved) surface discarded the whole endpoint into a notVerified bucket, throwing away every gap -- including the ones on parameters that resolved cleanly. Measured on this repo's own Public/ tree, ~70% of those non-Typed classifications turned out to be tool blindness (three since-fixed parser gaps), not real gaps, and the failure mode was silence: the report just got quieter, and quieter read as better. Per-parameter confidence replaces the gate. Gaps are now ALWAYS computed; uncertainty is carried per-endpoint as `confidence` (level high/ partial, unresolvedParameters with file:line, escapeHatchOnly, a caveat). Verified against the real Public/ tree: the 56 endpoints this newly surfaces account for 454 previously-invisible fields, while the pre-existing 376-endpoint/897-query/402-body/226-read-only population is reproduced byte-for-byte once isolated back out by confidence level. Output is also split three ways instead of one flat MissingParameters list: MissingQueryParameters, MissingBodyProperties (addable), and ReadOnlyFields (body fields read-only in the newest ANALYSED spec, i.e. the capability map's own generatedFrom[-1] -- never whichever spec happens to be newest on disk, which can be ahead of the map). The newest analysed spec is re-parsed once (Get-PfbSpecCapabilities) and passed in as -CurrentSpecCapabilities purely to exclude "phantom" fields: the capability map's parameters/bodyProperties dictionaries accumulate every field ever seen and never record removal, so a field withdrawn from the API in a later version still lingers there forever. 13 real (endpoint, field) pairs hit this on fb2.27 today, e.g. PATCH /certificates|id (read-only 2.0-2.19, removed 2.20+) -- without this check they would misreport as addable body gaps for a field that no longer exists to add. Also fixes a keys/count/values-shadowing bug: a plain Hashtable's .Keys property is shadowed by a KEY literally named "keys" (PowerShell's dictionary-key-as-member adapter resolves .Keys to that key's VALUE instead of the real key collection). This is the fifth known instance of this bug class in this codebase. It corrupted DELETE /workloads/tags two ways at once: fabricating a phantom "2.23" field (the literal value of its "keys" parameter) while silently losing the endpoint's one genuine gap, context_names. Fixed generally, not just at that one call site: both the query and body field-name collections are now typed Dictionary[string,string] (immune to the adapter trick) accessed via .get_Keys(), with a regression test covering a field literally named "keys" on both the query and body side. tools/Build-PfbApiDriftReport.ps1's consumption of Get-PfbParameterCoverageGaps is updated to match: no more notVerifiedEndpoints, the new three-way split plus confidence block is serialized into the manifest, and the newest analysed spec is loaded once and threaded through as -CurrentSpecCapabilities. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…m/target
Task 5 of the drift-report-actionable plan: turns each of the 402 addable
(high-confidence) missing-body-property gaps from a bare field name into a
record a human can act on without re-deriving spec facts by hand --
{name, type, format, specRequired, synopsis, suggestedPowerShellType,
enumValues, enumStatus, target}.
The enum join goes through Resolve-PfbFieldValueEnum, never a bare wire-name
lookup, keyed by the field's OwnerSchema (Task 2b) as -ResourceHint -- the
old cmdlet-name-derived Get-PfbResourceHint only reaches 14 of 33 real
matches. Reproduces the pinned oracle exactly: 33 matched / 43
not-found-in-resource / 326 no-spec-enum-found over the 402 addable gaps,
including PATCH /certificates|certificate_type -> [appliance,external] and
POST /nfs-export-policies/rules|access ->
[root-squash,all-squash,no-squash].
suggestedPowerShellType exists specifically because the int64-vs-int32
distinction is real and silent: 37 of the 402 fields are type:integer,
format:int64, and mapping bare "type":"integer" to [int] without checking
format would silently clip/overflow every one of them the first time a
caller passed a value above 2^31-1. Raw type/format are always emitted
alongside so a human can override the suggestion.
Two deliberate asymmetries, both documented in the script's own .NOTES,
tools/README.md, and each function's doc comment:
- missingQueryParameters/readOnlyFields stay bare strings -- ~896 enriched
query-gap records would roughly double the artifact for no consumer.
- Enrichment itself is gated on confidence.level -eq 'high'. A
partial-confidence endpoint's gap list can contain false positives, and
attaching a fully-worked-out type/synopsis/enum/target to a field that
might not even be a real gap would overstate a confidence the endpoint's
own caveat is telling the reader not to have -- every field name still
appears unchanged, only the extra metadata is withheld. This gating is
also why the real numbers land at exactly 402/33/43/326: the same join
over all addable gaps regardless of confidence (605) gives a different,
non-oracle split.
target carries insertion-point COORDINATES ONLY, never a diff (decision 12):
{file, paramBlockLine, payloadVariable, assignmentStyle, hasAttributes}, via
the new Get-PfbCmdletBodyInsertionTarget (tools/lib/PfbCmdletParamTools.ps1).
assignmentStyle reflects the target cmdlet's OWN dominant convention
('index'/'literal'/'attributesOnly'/'unknown') so a human adding a field
matches the file's existing pattern rather than inventing a new one.
specRequired is spec metadata only and must never be read as
[Parameter(Mandatory)] -- stated explicitly per the repo's recorded
Should-Throw/Mandatory-prompt hazard.
Two real-data-only bugs found and fixed during verification (both would
have shipped invisibly against synthetic fixtures alone):
- $missingBodyProperties = if (...) {...} else {...} wrapped @() inside
each branch instead of around the whole if/else -- the classic
assign-a-statement's-output hazard already flagged elsewhere in this
codebase. An empty result read as 0 elements in-memory but round-tripped
through JSON into a phantom one-element array with a blank name,
inflating the real high-confidence addable total from 402 to 682.
- OwnerSchema is usually itself allOf-composed with no properties at its
own top level (_certificateBase, NfsExportPolicyRuleBase,
ActiveDirectoryPatch, SmbSharePolicyRule all measured this way against
fb2.27) -- a naive single-hop schema.properties.field lookup returned
$null for nearly every real synopsis/array-item-type. Fixed with a
bounded helper that searches the owner's own inline allOf branches (never
crossing into a $ref'd branch, never a second decision-3 walker).
Tests: 61 new/rewritten cases across the three touched Describe files, each
new invariant mutation-tested (confidence-gating, int64 branch, resource-hint
sentinel, empty-hashtable-literal-bootstrap, allOf-owner recursion, and the
if/else array-collapse fix) -- every mutation broke its specific assertion,
confirmed, then reverted. Full suite: 559 passed / 1 known pre-existing
failure (Build-PfbValueEnumMap fb2.28-vs-2.27 mismatch, not in scope) / 2
skipped.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…nnotations + injection detection (Task 6) Task 6 of the drift-report-actionable plan (issue dmann000#31): turns the per-endpoint parameter/body-property gap lists into actionable, decision-ready output. - Get-PfbSystemicGaps collapses MissingQueryParameters/MissingBodyProperties across every endpoint into one finding per distinct wire name, with a deduplicated EndpointCount (an endpoint hitting the same name in both lists, e.g. POST /workloads/placement-recommendations|placement_names, counts once). - Get-PfbConventionStrength ranks each missing name by how many distinct cmdlets already expose it as a resolved ('Typed') parameter -- a mechanical batch fix (names=306, ids=218) vs. an architectural decision (context_names=0). - docs/drift-annotations.json is a small checked-in annotation channel (design decisions, prior conclusions, live-testing hazards), loaded via Get-PfbDriftAnnotations/Find-PfbDriftAnnotation. - Get-PfbCentralInjectionSites ASTs Private/*.ps1 for the same `$var['key'] = <rhs>` shape already used for Public/ wire-name resolution, classifying each site as parameter-sourced or server-or-internal-derived. Get-PfbDerivedNonActionableParameters/Get-PfbNonActionableParameters reduce that to a per-key verdict: continuation_token is now DERIVED (Strength 'structural', since Private/Invoke-PfbApiRequest.ps1 line 227 sources it from $response, never a parameter) instead of hand-written, while X-Request-ID/ offset stay hardcoded (no Private/ injection site exists for either). The six Add-PfbCommonQueryParams-injected keys (filter/sort/limit/total_only/names/ids) are confirmed parameter-sourced and never enter the exclusion list. - tools/Build-PfbApiDriftReport.ps1 wires Get-PfbNonActionableParameters in place of the old hardcoded $script:PfbNonActionableParameters reference for -ExcludedFields -- the only report-script change; no Task 7 output work (no manifest keys, no Markdown section) is included here. Investigated the 253-vs-252 discrepancy between this task's real-data measurement and the plan's corrected ledger figure: DELETE /workloads/tags has a real capability-map parameter literally named `keys` alongside `context_names`. The old (pre-Task-4) Get-PfbParameterCoverageGaps merged parameters/bodyProperties into a plain `@{}` Hashtable and read `.Keys` on it -- which a literal `keys` entry silently shadows, returning that entry's VALUE (a version string) instead of the real key collection, corrupting this one endpoint's entire missing-field computation and hiding its real context_names gap. Task 4's typed-Dictionary/get_Keys() rewrite eliminated that bug class as a side effect, so this endpoint is now correctly counted -- 253 is the honest, reproducible figure; 252 was an undercounted artifact of a bug that predates this task. Confirmed empirically (Hashtable.Keys shadowing repro) and by diffing old-code-real-data vs new-code-real-data endpoint sets, which converge on exactly this one endpoint. This finishes work a prior agent started in this same worktree (background process died mid-task before verification/commit) -- the ~1023 lines of lib/test diff and docs/drift-annotations.json are substantially that agent's work, reviewed, verified against real data, and completed here (the 253 investigation, final full-suite run, and mutation testing were done in this session). Tests: full suite 597 passed / 1 known pre-existing failure (Build-PfbValueEnumMap fb2.28-vs-2.27 mismatch, out of scope) / 2 skipped. Three new invariants mutation-tested live (systemic-gaps EndpointCount dedup, ContainsKey-guard classification, any-parameter-sourced exclusion rule) -- each mutation broke its target assertion, confirmed, then reverted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…d generatedFrom split into the drift report (Task 7) Wires Task 4/5/6's building blocks into the actual report output: Get-PfbSystemicGaps and Get-PfbConventionStrength are now aggregated (high-confidence gaps only) and emitted as new top-level JSON keys (systemicGaps/conventionStrength), each systemic finding and parameter-gap row carries matching docs/drift-annotations.json entries, and a new phantomFieldCount surfaces how many (endpoint, field) pairs are silently excluded by phantom-field filtering. generatedFrom is split into analysedVersions (from the capability map) and availableSpecVersions (from tools/specs/ on disk), with a versionDivergenceWarning when they disagree -- verified live: 28 vs 29 versions today. The Markdown report is restructured in the required section order (How to read this report -> Summary -> Systemic gaps -> Parameter gaps -> Read-only fields -> Uncovered endpoints -> ValidateSet drift -> New ValidateSet candidates), reproduces the decision-6 false-positive procedure verbatim, and gives partial-confidence rows a visible marker plus a Partial-confidence detail appendix with file:line for each unresolved parameter. Also fixes update-api-capability-map.yml's driftSummary, which mislabeled an endpoint count as "parameter gaps"; it now reports the endpoint count under an accurate label alongside honest field-level counts and the new systemicGaps count. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… add determinism test, regenerate drift report for real tools/README.md gained six new subsections covering the generatedFrom split, the notVerified->confidence norm reversal (decision 5) with the old rationale preserved, the decision-6 false-positive procedure, systemicGaps/conventionStrength, the two correct phantomFieldCount scopes (34 full population / 13 high-confidence subset), and the REST 2.17 cliff (corrected framing: a $ref consolidation where resolved read-only pairs rose 219->262, not a capability loss, per the plan's own progress-log correction). Tests/Build-PfbApiDriftReport.Tests.ps1: added an end-to-end determinism round-trip test (run the builder twice, assert byte-identical JSON+Markdown -- no prior test in this file actually did this), the 11 corrected regression canaries + 4 spot-checks from the task brief, and a "nothing vanishes" invariant test verifying every capability-map candidate field lands in exactly one of reported/phantom-excluded, arithmetically over (endpoint, list, field) triples cross-checked against the real generated JSON on disk. Reports/PfbApiDriftReport.json/.md regenerated for real against the unmodified Data/PfbCapabilityMap.json + tools/specs/ + Public/Private tree -- the committed report predated Task 4's query/body split entirely (still had a flat missingParameters list and a bare notVerifiedEndpoints count with no detail), so this is the first true end-to-end regeneration since early in this effort. All acceptance figures reproduced exactly on the real data. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ings
Doc/test-only fixes from the final whole-branch review of the API-drift-report
plan, zero behavior change:
- correct stale "252" to "253" in Get-PfbSystemicGaps's comment-based help
- hoist $enrichmentSpec into the file-level BeforeAll in
PfbApiDriftTools.Tests.ps1 so the Get-PfbBodyPropertyEnrichment tests pass
under a filtered/reordered Pester run, not just the full suite
- escape the annotation match string before using it in Find-PfbDriftAnnotation's
-like lookup, so a future match value containing */?/[ is treated literally
- name the one known, persistent Build-PfbValueEnumMap.Tests.ps1 failure (and its
fb2.28-cache-vs-2.27-pin cause) directly in tools/README.md's Tests section
instead of a dangling forward reference
- soften the versionDivergenceWarning wording so it reads as an expected
consequence of the deliberate spec pin, not an imperative to fix a bug;
regenerate Reports/PfbApiDriftReport.{json,md} for real (diff confirms only the
warning text changed, no pinned figures moved)
- add a test that the internal _currentParamComponents/_currentParamsWithoutComponent
bookkeeping keys never leak into the serialized capability map
- add -SinceVersion tests for the body-property/read-only side of
Get-PfbParameterCoverageGaps, mirroring the existing query-side tests
- document the silent index/literal tiebreak in
Get-PfbCmdletBodyInsertionTarget (comment-only)
- assert decision-6 procedure step 3 in the "How to read this report"
Markdown test, alongside the existing steps 1/2/4 assertions
Every new/changed test was mutation-tested (break the invariant, confirm the
specific test fails, then restore). Full suite: 632 passed, 1 pre-existing
failure (Build-PfbValueEnumMap.Tests.ps1's fb2.28-cache issue, unrelated to
this plan), no new failures.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…-only scope limitation Records that systemicGaps/conventionStrength aggregate over high-confidence endpoints only, deliberately, per the false-positive procedure -- but that this silently omits 54 field names that appear only on partial-confidence rows, and undercounts names that do appear (context_names is 253 in systemicGaps but 289 across all confidence levels). Follow-up tracked separately; not fixed here since it would re-baseline every pinned figure this effort measured. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…r PS 5.1 ConvertFrom-Json has no -Depth parameter on Windows PowerShell 5.1 (added in PS6) -- Get-PfbDriftAnnotations called it unconditionally, breaking the Windows PowerShell 5.1 CI job (ParameterBindingException on every test that touches docs/drift-annotations.json). Apply the same version-gated guard already used by Private/Get-PfbCapabilityMap.ps1 and Private/Get-PfbVersionMap.ps1 for the identical gotcha. This file's shape (schemaVersion + a flat annotation array) is far shallower than 5.1's own recursion limit, same as those two. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…hecks Every failing assertion crossed the function-call boundary with a bare, un-array-wrapped result whenever exactly one annotation matched. PowerShell always unwraps a single-element pipeline result on return regardless of the function's own internal @() wrapping -- PowerShell 7+ masks this because it adds an automatic .Count/.Length property to every object (including bare scalars), but Windows PowerShell 5.1 has no such property, so .Count on the unwrapped scalar silently returns $null instead of 1. Confirmed live under real Windows PowerShell 5.1 (installed locally): reproduced "Expected 1, but got $null" on every affected assertion before this fix, then reran the full file (97 passed / 0 failed / 9 skipped -- the 9 are a pre-existing, intentional PS7+-only skip unrelated to this fix) and the full suite (461 passed, 23 pre-existing failures confined to Connect-PfbArray/Get-PfbApiTokenViaSsh/New-PfbJwtToken -- files this branch never touches, caused by a local Microsoft.PowerShell.Security module-load conflict, not present in the clean CI runner) after. Production code was never affected -- both real call sites in tools/Build-PfbApiDriftReport.ps1 already wrap Find-PfbDriftAnnotation in @(...). This was a test-file-only gap. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2 tasks
juemerson-at-purestorage
added a commit
that referenced
this pull request
Jul 27, 2026
…ap, and field-cmdlet map
Data/PfbCapabilityMap.json, Reports/PfbApiDriftReport.{json,md},
Reports/PfbValueEnumMap.json, Reports/PfbValueEnumReconciliation.md, and
Reports/PfbFieldCmdletMap.{json,md} were stale relative to both PR #60 (drift
report enrichment/confidence work) and REST 2.28, which had already been
fetched into the local tools/specs/ cache but never folded into these
committed artifacts. Regenerated all four via their tools/Build-Pfb*.ps1
scripts against the current 29-version spec cache -- no manual edits.
Notable deltas: capability map now spans 2.0-2.28 (632 endpoints); drift
report gaps grew modestly (432->433 endpoints, 605->609 addable body props,
1002->1004 query params, 1 new ValidateSet candidate); field-cmdlet map goes
from matched:0 to matched:1 (Get-PfbPolicyAllMember -MemberType) with
not-found-in-resource 2->3 -- the full 4-candidate population of typed
parameters with any spec-documented enum at all, confirming the
2026-07-27 field-cmdlet-map-resource-hint-design.md finding that a better
resource hint can't move the remaining no-spec-enum-found candidates.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns
Reports/PfbApiDriftReport.{json,md}from a flat list of ~1538 undifferentiated field names into a triaged, machine-actionable work queue. Progresses issue #31 (the drift report is the tool that will drive closing those gaps).readOnly/deprecated/parameterComponentsinto the capability map (schema walker consolidated to one, decision 3)notVerified/fullyMappedgate with per-parameter confidence (high/partial), so a gap is never silently suppressed because one parameter has an unresolved surface (the false-positive contract, decision 6)ValidateSetvalues, and insertion-point coordinates (never a diff/patch)systemicGaps) and ranks by existing convention strength (conventionStrength), turning hundreds of per-endpoint rows into a handful of real decisionsdocs/drift-annotations.json) for recorded design decisions, prior conclusions, and live-testing hazardscontinuation_token) from a live AST scan ofPrivate/instead of a hand-maintained listReports/PfbApiDriftReport.{json,md}for real, with a determinism round-trip test and a "nothing vanishes" accounting invariantAcceptance-figure re-baseline (read this before comparing to old numbers)
Deleting the old all-or-nothing gate (decision 5) surfaces 56 previously-hidden partial-confidence endpoints. The total moves from ~1538 fields / 896 query / 628 body / 226 read-only to 1979 / 1002 / 605 (addable+read-only split further into 372 read-only) / 34 phantom (13 on the high-confidence-only subset). Both figures are correct measurements of different populations -- 1979 is not a regression, it's the point of this PR.
Known limitation (tracked separately)
systemicGaps/conventionStrengthaggregate over high-confidence endpoints only (documented intools/README.md), which undercounts some field names (context_namesis 253 there vs. 289 across all confidence levels) and omits 54 names that appear only on partial-confidence rows. Follow-up: #59.Live-array verification
Data/PfbCapabilityMap.jsonis runtime-loaded byPrivate/Get-PfbCapabilityMap.ps1and this branch changes its shape/size, so I verified against FB-A (REST 2.26): the larger capability map loads correctly, a normal call with no version-gated fields succeeds, and a call using a real 2.27-only field correctly throwsAssert-PfbApiCapability's version-gate error.CI
The Windows PowerShell 5.1 job caught two real cross-version bugs during this PR's own CI runs (both fixed, both verified locally against real PS 5.1 before the final push):
ConvertFrom-Jsonhas no-Depthparameter on 5.1, and a single-match function result silently loses.Counton 5.1 (PS7+ masks this with an automaticCountproperty PS 5.1 doesn't have).Test plan