[fix] dataconv: map json.Number to Int/Float and decode JSON exactly (LET-28)#167
Conversation
Two related JSON-number gaps, both surfaced by the diagnosis in #128 (thanks @phughes-scwx): 1. Marshal had no json.Number case, so a json.Number fell through to "unrecognized type" and errored. A caller decoding raw JSON into map[string]interface{} with dec.UseNumber() (the way to keep the int/float distinction that a plain json.Unmarshal collapses to float64) could not then Marshal it. Add the case, mapping by literal form: an integer literal -> Int (big.Int beyond int64), otherwise -> Float. This matches json.decode and serial; mapping to a string (as proposed in #128) would make Marshal the only converter that turns a number into a non-number and break value == 6. Also accept *big.Int (the inverse of Unmarshal, so the round-trip stays exact). 2. UnmarshalStarlarkJSON decoded WITHOUT UseNumber and re-derived int-ness from float64, silently saturating integers past 2^53 (it is on the lib/http request-body path, so real JSON ids were affected). Decode with UseNumber and the same number-preserving logic; impose single-document strictness the previous json.Unmarshal had. interface.go doc updated: numbers now map by literal form (whole floats like 6.0 become Float, not Int — no test pinned the old heuristic). Test-first: TestMarshal gains json.Number/*big.Int cases and TestUnmarshalStarlarkJSONNumberFidelity pins exact big-int decoding. Requirement: LET-28. Refs #128. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 22 |
| Duplication | 1 |
🟢 Coverage 98.15% diff coverage · +0.04% coverage variation
Metric Results Coverage variation ✅ +0.04% coverage variation (-1.00%) Diff coverage ✅ 98.15% diff coverage Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (433eae0) 7692 7276 94.59% Head commit (fc3a6c9) 7744 (+52) 7328 (+52) 94.63% (+0.04%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#167) 54 53 98.15% Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #167 +/- ##
==========================================
+ Coverage 93.39% 93.42% +0.03%
==========================================
Files 49 49
Lines 6177 6206 +29
==========================================
+ Hits 5769 5798 +29
+ Misses 260 259 -1
- Partials 148 149 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Closes the JSON-number gaps surfaced by the diagnosis in #128 — thanks @phughes-scwx for the clear write-up and repro.
What
Two related gaps in how
dataconvturns JSON numbers into Starlark values:Marshalerrored onjson.Number. It had nojson.Numbercase, so the value fell through tounrecognized type. A caller decoding raw JSON intomap[string]interface{}withdec.UseNumber()— the way to keep the int/float distinction that a plainjson.Unmarshalcollapses tofloat64— could not thenMarshalit.UnmarshalStarlarkJSONlost precision. It decoded withoutUseNumber()and re-derived int-ness fromfloat64, silently saturating integers past 2^53. It is on thelib/httprequest-body path, so real JSON ids were affected.Fix
Marshalmapsjson.Numberby literal form: an integer literal →Int(with*big.Intbeyond int64), otherwise →Float. This matchesjson.decode,serial, and dataconv's ownGoToStarlarkViaJSON. Mapping to a string (as proposed in Add support forjson.Numbertodataconv.Marshal#128) would makeMarshalthe only converter that turns a number into a non-number and would breakvalue == 6/arithmetic in scripts.Marshalalso now accepts*big.Int(the inverse ofUnmarshal, so a round-trip stays exact).UnmarshalStarlarkJSONdecodes withUseNumber()and the same number-preserving logic, and re-imposes the single-document strictness the previousjson.Unmarshalhad.Note on @phughes-scwx / #128
We can't merge #128 directly (the string mapping), but the gap it identified is real and we've landed it here with the Int/Float mapping and credited the report. @phughes-scwx — does this implementation work for your use case? Happy to adjust.
Behavior change
With the unified literal-form rule, a whole float literal like
6.0now decodes toFloat(previouslyUnmarshalStarlarkJSON's heuristic returnedInt). Common cases (6→Int,6.5→Float, large ints→exact) are unchanged; no test pinned the old6.0→int behavior.interface.godoc updated accordingly.Test-first
TestMarshalgainsjson.Number/*big.Intcases;TestUnmarshalStarlarkJSONNumberFidelitypins exact big-int decoding (including the dict/HTTP-body path). Fail before the fix.Verification
go test -race -count=2 ./...(incl.lib/http),go vet,gofmt -lclean, Dockergolang:1.19race run green.Requirement: LET-28 · Refs #128