Skip to content

[fix] dataconv: map json.Number to Int/Float and decode JSON exactly (LET-28)#167

Merged
vt128 merged 1 commit into
masterfrom
fix/let-28-dataconv-jsonnumber
Jun 13, 2026
Merged

[fix] dataconv: map json.Number to Int/Float and decode JSON exactly (LET-28)#167
vt128 merged 1 commit into
masterfrom
fix/let-28-dataconv-jsonnumber

Conversation

@vt128

@vt128 vt128 commented Jun 13, 2026

Copy link
Copy Markdown
Member

Closes the JSON-number gaps surfaced by the diagnosis in #128 — thanks @phughes-scwx for the clear write-up and repro.

What

Two related gaps in how dataconv turns JSON numbers into Starlark values:

  1. Marshal errored on json.Number. It had no json.Number case, so the value fell through to unrecognized type. A caller decoding raw JSON into map[string]interface{} with dec.UseNumber() — the way to keep the int/float distinction that a plain json.Unmarshal collapses to float64 — could not then Marshal it.

  2. UnmarshalStarlarkJSON lost precision. It decoded without UseNumber() and re-derived int-ness from float64, silently saturating integers past 2^53. It is on the lib/http request-body path, so real JSON ids were affected.

Fix

  • Marshal maps json.Number by literal form: an integer literal → Int (with *big.Int beyond int64), otherwise → Float. This matches json.decode, serial, and dataconv's own GoToStarlarkViaJSON. Mapping to a string (as proposed in Add support for json.Number to dataconv.Marshal #128) would make Marshal the only converter that turns a number into a non-number and would break value == 6/arithmetic in scripts. Marshal also now accepts *big.Int (the inverse of Unmarshal, so a round-trip stays exact).
  • UnmarshalStarlarkJSON decodes with UseNumber() and the same number-preserving logic, and re-imposes the single-document strictness the previous json.Unmarshal had.

Note on @phughes-scwx / #128

We can't merge #128 directly (the string mapping), but the gap it identified is real and we've landed it here with the Int/Float mapping and credited the report. @phughes-scwx — does this implementation work for your use case? Happy to adjust.

Behavior change

With the unified literal-form rule, a whole float literal like 6.0 now decodes to Float (previously UnmarshalStarlarkJSON's heuristic returned Int). Common cases (6→Int, 6.5→Float, large ints→exact) are unchanged; no test pinned the old 6.0→int behavior. interface.go doc updated accordingly.

Test-first

TestMarshal gains json.Number/*big.Int cases; TestUnmarshalStarlarkJSONNumberFidelity pins exact big-int decoding (including the dict/HTTP-body path). Fail before the fix.

Verification

go test -race -count=2 ./... (incl. lib/http), go vet, gofmt -l clean, Docker golang:1.19 race run green.

Requirement: LET-28 · Refs #128

Two related JSON-number gaps, both surfaced by the diagnosis in #128
(thanks @phughes-scwx):

1. Marshal had no json.Number case, so a json.Number fell through to
   "unrecognized type" and errored. A caller decoding raw JSON into
   map[string]interface{} with dec.UseNumber() (the way to keep the
   int/float distinction that a plain json.Unmarshal collapses to
   float64) could not then Marshal it. Add the case, mapping by literal
   form: an integer literal -> Int (big.Int beyond int64), otherwise
   -> Float. This matches json.decode and serial; mapping to a string
   (as proposed in #128) would make Marshal the only converter that
   turns a number into a non-number and break value == 6. Also accept
   *big.Int (the inverse of Unmarshal, so the round-trip stays exact).

2. UnmarshalStarlarkJSON decoded WITHOUT UseNumber and re-derived
   int-ness from float64, silently saturating integers past 2^53 (it is
   on the lib/http request-body path, so real JSON ids were affected).
   Decode with UseNumber and the same number-preserving logic; impose
   single-document strictness the previous json.Unmarshal had.

interface.go doc updated: numbers now map by literal form (whole floats
like 6.0 become Float, not Int — no test pinned the old heuristic).

Test-first: TestMarshal gains json.Number/*big.Int cases and
TestUnmarshalStarlarkJSONNumberFidelity pins exact big-int decoding.
Requirement: LET-28. Refs #128.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@codacy-production

codacy-production Bot commented Jun 13, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 22 complexity · 1 duplication

Metric Results
Complexity 22
Duplication 1

View in Codacy

🟢 Coverage 98.15% diff coverage · +0.04% coverage variation

Metric Results
Coverage variation +0.04% coverage variation (-1.00%)
Diff coverage 98.15% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (433eae0) 7692 7276 94.59%
Head commit (fc3a6c9) 7744 (+52) 7328 (+52) 94.63% (+0.04%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#167) 54 53 98.15%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@codecov

codecov Bot commented Jun 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.77419% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.42%. Comparing base (433eae0) to head (fc3a6c9).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
dataconv/helper.go 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #167      +/-   ##
==========================================
+ Coverage   93.39%   93.42%   +0.03%     
==========================================
  Files          49       49              
  Lines        6177     6206      +29     
==========================================
+ Hits         5769     5798      +29     
+ Misses        260      259       -1     
- Partials      148      149       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vt128 vt128 merged commit a4bb960 into master Jun 13, 2026
12 checks passed
@vt128 vt128 deleted the fix/let-28-dataconv-jsonnumber branch June 13, 2026 07:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant