Skip to content

Move MODS and TFE XML out of SQL storage #44

@candyhazlett

Description

@candyhazlett

Description

mods_xml (:mediumtext) and tfe_xml (:text) are currently stored as SQL columns on core_files (migration 20260113171831_add_tapas_xq_fields_to_core_files.rb). @amclark42 flagged two concerns: character encoding issues (XML encoding declarations can conflict with MySQL's own encoding settings), and queryability (XML stored as a text column isn't queryable in any meaningful way). TAPAS already uses Active Storage for the TEI file; applying the same pattern to MODS and TFE output would be more consistent and avoid these issues. Surfaced in PR #29 review.

Steps

  1. Add has_one_attached :mods_file and has_one_attached :tfe_file (or equivalent) to CoreFile
  2. Update ProcessTeiFileJob to write MODS/TFE content as file attachments rather than SQL columns
  3. Update any other code that reads mods_xml or tfe_xml to use the new attachments
  4. Write a migration to remove the mods_xml and tfe_xml columns
  5. Update specs

Acceptance criteria

  • mods_xml and tfe_xml columns removed from the core_files table
  • MODS and TFE data stored via Active Storage attachments
  • ProcessTeiFileJob successfully writes and reads MODS/TFE data through the new mechanism
  • No character encoding issues
  • Existing job specs pass

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions