Feature/multi source scraper

[jannicaSD]

Summary

• Improved representatives scraper workflow
• Added multi-source scraping structure
• Fixed image download stream handling
• Updated env loading behavior for local CLI scripts
• Improved DB migration reliability with pgvector extension setup

What I changed

• scripts/scrape-representatives.ts
  • Refactored source handling
  • Fixed image write stream issue
  • Improved URL normalization and output handling
• lib/env.mjs
  • Load .env.local for CLI commands
• lib/db/migrate.ts
  • Ensure vector extension exists before migrations

Validation

• npm run db:generate
• npm run db:migrate (successful)
• npm run scrape:representatives (runs successfully)

Notes

• .env.local is ignored and not committed.

Known limitations

Provincial assembly URLs currently use a generic parser, so some sources may return zero records until source-specific selectors are added.
National Assembly source is the currently verified data source.

[vercel]

@jannicaSD is attempting to deploy a commit to the passion projects Team on Vercel.

A member of the Team first needs to authorize it.