Skip to content

Big Sam job reliability fixes, dry-run, and email reports - #11

Merged
jayvarner merged 11 commits into
mainfrom
develop
Aug 7, 2026
Merged

Big Sam job reliability fixes, dry-run, and email reports#11
jayvarner merged 11 commits into
mainfrom
develop

Conversation

@jayvarner

@jayvarner jayvarner commented Aug 7, 2026

Copy link
Copy Markdown
Member

Summary

Brings main up to date with develop - main is currently missing all of this work, which is why the deployed ECS task is crash-looping (the SES boot-order fix included here is what actually resolves that).

  • Elasticsearch 8 refactor and related test/DB config fixes
  • LoadBigSamJob reliability pass: per-row transactions with rollback on failure, structured error/skip reporting instead of silently swallowed rescues, fixed several confirmed bugs found by auditing against the real production spreadsheet (case-sensitive flag columns, duplicate repository/collection labels, a copy-paste bug wiring the wrong collection to the third repository slot, a dead notes-field typo, a NoMethodError that could crash the whole job on certain recipient names, and blank spreadsheet IDs silently colliding into one letter record)
  • LoadBigSamJob#dry_run - runs the full pipeline and reports what would happen without persisting anything or touching Elasticsearch
  • Decomposed the 234-line, cyclomatic-complexity-78 process_letter into focused methods; the three near-duplicate repository/collection blocks collapsed into one table-driven method
  • SES email reports after every real load: a developer report and a plain-language owner report, sent via BigSamMailer
  • Fixes the ActionMailer/SES boot-order bug (ses_settings= NoMethodError) currently crashing the ECS task

Test plan

  • Full relevant spec suite green on develop (job, mailer, model specs - 50 examples)
  • Verified the SES fix directly by booting with RAILS_ENV=production and confirming ActionMailer::Base.delivery_method/ses_settings resolve without error
  • Redeploy ECS from main after merge and confirm the task boots past db:migrate

@jayvarner
jayvarner merged commit c2bdf2c into main Aug 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant