fix(frontend): drop unnamed CSV columns so the dev UI validates (#109)#113
Closed
Mandyx22 wants to merge 1 commit into
Closed
fix(frontend): drop unnamed CSV columns so the dev UI validates (#109)#113Mandyx22 wants to merge 1 commit into
Mandyx22 wants to merge 1 commit into
Conversation
The frontend kept uploaded file content verbatim for both the in-browser validator and the download zip. With finding #2's library fix, generate() drops R-style unnamed row-index columns from variableMeasured, but the unchanged CSV still carried that column — so a dataset that generated fine failed validation with CSV_COLUMN_MISSING_FROM_METADATA and the zip shipped an invalid file. Normalize uploaded CSV content once in handleProcess (shared normalizeDataContent helper reusing the library's stripUnnamedColumns) before it is generated, validated, and zipped, so all three see the same bytes. Well-formed CSVs pass through unchanged. Completes finding #2 of #109 on the frontend side. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 4b290e9 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Author
jodeleeuw
pushed a commit
that referenced
this pull request
Jun 18, 2026
#109) R's write.csv(row.names=TRUE) prepends an unnamed row-index column (empty-string header). It can't appear in variableMeasured (Psych-DS requires a name), so generate() drops it from the metadata — but the on-disk CSV kept it, failing validation with CSV_COLUMN_MISSING_FROM_METADATA. Put the strip in the shared data-file path so the CLI and frontend behave identically (layered on #103's buildPsychDSDataFiles): - New exported stripUnnamedColumns helper; generate() strips parsed data up front with a single warning (keeps variableMeasured clean + standalone use safe). - buildPsychDSDataFiles strips the main table: a clean CSV keeps its exact bytes (verbatim mainContent), a dirty one is re-serialised from cleaned rows. - CLI (rename-plan + non-plan paths) and frontend now feed parsed mainRows, so the written/zipped/validated CSV always matches the metadata. Supersedes #112 and #113 (the CLI verbatim-write rewrite and the frontend-only normalizeData helper) with one shared-builder implementation. Fixes finding #2 of #109. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jodeleeuw
pushed a commit
that referenced
this pull request
Jun 18, 2026
#109) (#114) R's write.csv(row.names=TRUE) prepends an unnamed row-index column (empty-string header). It can't appear in variableMeasured (Psych-DS requires a name), so generate() drops it from the metadata — but the on-disk CSV kept it, failing validation with CSV_COLUMN_MISSING_FROM_METADATA. Put the strip in the shared data-file path so the CLI and frontend behave identically (layered on #103's buildPsychDSDataFiles): - New exported stripUnnamedColumns helper; generate() strips parsed data up front with a single warning (keeps variableMeasured clean + standalone use safe). - buildPsychDSDataFiles strips the main table: a clean CSV keeps its exact bytes (verbatim mainContent), a dirty one is re-serialised from cleaned rows. - CLI (rename-plan + non-plan paths) and frontend now feed parsed mainRows, so the written/zipped/validated CSV always matches the metadata. Supersedes #112 and #113 (the CLI verbatim-write rewrite and the frontend-only normalizeData helper) with one shared-builder implementation. Fixes finding #2 of #109. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Completes finding #2 of #109 on the frontend side.
The library fix (#112) makes
generate()drop R-style unnamed row-index columns fromvariableMeasured. But the frontend kept each uploaded file's original content verbatim and fed that to both the in-browser validator (Review.tsx) and the download zip — so a dataset that generated cleanly still failed validation withCSV_COLUMN_MISSING_FROM_METADATA, and the downloaded zip shipped an invalid CSV. Same caller-owns-the-CSV gap the CLI fix closed.How
normalizeDataContent(src/normalizeData.ts) that re-uses the library'sstripUnnamedColumns+objectsToCSVto drop empty/whitespace-named columns from CSV content (column order preserved). Well-formed CSVs and JSON pass through unchanged; unparseable CSV is left untouched.DataUpload.handleProcessnormalizes content once when buildingfileTexts, so generate, in-browser validation, and the zip all see the same bytes.Testing
normalizeDataContent; full frontend suite green (40 tests).eslintclean on the new/changed files;tsc --noEmitclean.Targets
fix/unnamed-leading-column(#112), notmain, because it imports the newstripUnnamedColumnsexport added there. Merge #112 first, then re-target this tomain(or merge after #112 lands).🤖 Generated with Claude Code