fix(metadata,cli,frontend): drop unnamed columns in the shared builder (#109)#114
Merged
Merged
Conversation
🦋 Changeset detectedLatest commit: 840dc2b The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
This was referenced Jun 16, 2026
#109) R's write.csv(row.names=TRUE) prepends an unnamed row-index column (empty-string header). It can't appear in variableMeasured (Psych-DS requires a name), so generate() drops it from the metadata — but the on-disk CSV kept it, failing validation with CSV_COLUMN_MISSING_FROM_METADATA. Put the strip in the shared data-file path so the CLI and frontend behave identically (layered on #103's buildPsychDSDataFiles): - New exported stripUnnamedColumns helper; generate() strips parsed data up front with a single warning (keeps variableMeasured clean + standalone use safe). - buildPsychDSDataFiles strips the main table: a clean CSV keeps its exact bytes (verbatim mainContent), a dirty one is re-serialised from cleaned rows. - CLI (rename-plan + non-plan paths) and frontend now feed parsed mainRows, so the written/zipped/validated CSV always matches the metadata. Supersedes #112 and #113 (the CLI verbatim-write rewrite and the frontend-only normalizeData helper) with one shared-builder implementation. Fixes finding #2 of #109. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
9770ac0 to
840dc2b
Compare
jodeleeuw
added a commit
that referenced
this pull request
Jun 18, 2026
…changes The DataUpload component gained a parseJsonData-based join-key preflight (#115) and a buildPsychDSDataFiles conversion step (#103/#114) since these tests were written. Expand the @jspsych/metadata mock (parseJsonData, parseCSV, buildPsychDSDataFiles, deriveFallbackBase, isValidPsychDSDataFilename, PSYCHDS_IGNORE_*) and the metadata stub (getExtractedArrays/Objects, getArrayJoinKeys) so the join-key chooser and session-sync paths exercise correctly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jodeleeuw
added a commit
that referenced
this pull request
Jun 18, 2026
* test(frontend): add unit tests for remaining page/UI components Covers DataUpload, Variables, Authors, JsonViewer, PageHeader, PreviewDrawer, Sidebar, ProjectInfo, and AppShell (201 tests total). Closes #9. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(frontend): update DataUpload tests for merged preflight/builder changes The DataUpload component gained a parseJsonData-based join-key preflight (#115) and a buildPsychDSDataFiles conversion step (#103/#114) since these tests were written. Expand the @jspsych/metadata mock (parseJsonData, parseCSV, buildPsychDSDataFiles, deriveFallbackBase, isValidPsychDSDataFilename, PSYCHDS_IGNORE_*) and the metadata stub (getExtractedArrays/Objects, getArrayJoinKeys) so the join-key chooser and session-sync paths exercise correctly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Josh de Leeuw <josh.deleeuw@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Re-cut of finding #2 of #109, layered on #103's shared
buildPsychDSDataFiles. Supersedes #112 and #113.R's
write.csv(defaultrow.names = TRUE) prepends an unnamed row-index column, so the CSV header starts with a bare comma — an empty-string column name. It can't appear invariableMeasured(Psych-DS requires a name), sogenerate()drops it from the metadata, but the on-disk CSV kept it → validation failed withCSV_COLUMN_MISSING_FROM_METADATA.How
The strip lives in the shared data-file path so the CLI and frontend behave identically by construction (no per-caller helper):
stripUnnamedColumnshelper.generate()strips empty/whitespace-only columns from the parsed data up front, with a single warning instead of per-row spam — keepsvariableMeasuredclean and standalone library use safe.buildPsychDSDataFilesstrips the main table before emitting it: a clean CSV keeps its exact bytes (verbatimmainContent); a file with an unnamed column is re-serialised from the cleaned rows. Sidecars are already clean (they come fromgenerate()'s stripped extraction).mainRows: the CLI (rename-plan and non-plan write paths) and the frontendrunGeneratenowparseCSVthe CSV input, so the written / zipped / in-browser-validated CSV always matches the metadata.This replaces #112's CLI verbatim-write rewrite and #113's frontend-only
normalizeData.tswith one shared-builder implementation.Testing
stripUnnamedColumnsunit;buildPsychDSDataFiles(dirty CSV re-serialised, clean CSV byte-for-byte, JSON rows);generate()strip (one warning, not per-row); CLIprocessDirectoryon both write paths; frontendrunGenerateparse→build path.tsc --noEmitclean; no new eslintanyintroduced.,subject_id,trial_type,trial_index,rt,response: written header drops the empty column and matchesvariableMeasured; noCSV_COLUMN_MISSING_FROM_METADATA. (The residualtime_elapsederror is finding Incorporating Psych-DS validator with the metadata module #1 / fix(metadata): register jsPsych system variables lazily (#109) #110, not on this branch.)Targets
fix/frontend-missing-datafile(#103), notmain, because it builds onbuildPsychDSDataFiles. Merge #103 first, then re-target this tomain.🤖 Generated with Claude Code