Skip to content

feat: add JSON and NDJSON input/output format support#118

Merged
vmvarela merged 4 commits intomasterfrom
issue-96/json-ndjson-formats
May 1, 2026
Merged

feat: add JSON and NDJSON input/output format support#118
vmvarela merged 4 commits intomasterfrom
issue-96/json-ndjson-formats

Conversation

@vmvarela
Copy link
Copy Markdown
Owner

Summary

  • Implements JSON and NDJSON input and output format support #96 (JSON/NDJSON formats) incorporating the format plugin architecture from Format plugin architecture for multi-format I/O #95
  • Adds -I/--input-format and -O/--output-format flags with csv, json, ndjson values; --json alias preserved for backward compatibility
  • JSON array input: reads entire stdin, parses with std.json, inserts rows with type-aware SQLite bindings
  • NDJSON input: streams line-by-line, creates table schema from first object, inserts with transactions
  • JSON/NDJSON output: execQuery dispatches to printJsonRow/printNdjsonRow per result row
  • 14 new integration tests (48–61) covering roundtrips, null values, nested objects, empty arrays, --columns, and error paths

Notes

  • All JSON-sourced columns are created as TEXT in SQLite; integer/float/bool/null values are bound via their native SQLite bind functions
  • Serialized array/object values use SQLITE_STATIC with a deferred_allocs list to keep allocations alive past sqlite3_step
  • --header remains mutually exclusive with non-CSV output formats

- Add InputFormat and OutputFormat enums (csv, json, ndjson)
- Add -I/--input-format and -O/--output-format flags; --json alias preserved
- Add loadJsonArray: reads full JSON array from stdin, inserts into SQLite
- Add loadNdjsonInput: streams NDJSON line-by-line into SQLite
- Add bindJsonValue: maps std.json.Value variants to SQLite bind calls
- Add insertRowFromJson: binds a JSON object row by column name
- Add printNdjsonRow: writes one NDJSON object per result row
- Add printJsonRow/printNdjsonRow output path in execQuery
- Use SQLITE_STATIC for all string bindings; deferred_allocs tracks
  serialized array/object strings so they outlive sqlite3_step
- Add 14 integration tests (48-61) covering JSON/NDJSON roundtrips,
  null values, nested objects, empty arrays, --columns, error paths
@github-actions github-actions Bot added type:feature New functionality type:chore Maintenance, refactoring, tooling labels Apr 29, 2026
vmvarela added 3 commits May 1, 2026 10:37
…ules

Move @cImport to src/c.zig so all modules share a single opaque sqlite3
type. Extract bindJsonValue, insertRowFromJson, readLine, loadJsonArray,
loadNdjsonInput, writeJsonString, printJsonRow and printNdjsonRow from
main.zig into src/json.zig. Fix ziglint violations in json.zig (Z023,
Z017, Z026).
@vmvarela vmvarela merged commit f2e650e into master May 1, 2026
4 checks passed
@vmvarela vmvarela deleted the issue-96/json-ndjson-formats branch May 1, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:chore Maintenance, refactoring, tooling type:feature New functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant