Skip to content

Repository files navigation

3Things

Personalized weekly discovery letter for Dallas. Every Thursday, subscribers receive 3-4 picks — a community to join, an experience to try, a place to discover, and one nudge — matched to their goals, vibe, and neighborhood.

Previously "The Local Post" (hyperlocal news newsletter). Pivoted to discovery-focused personalization in M9-M15.

Quick start

Prerequisites

Setup

# Clone and install
git clone <repo-url>
cd the-local-post
npm install

# Configure environment
cp .env.example .env
# Edit .env with your API keys and database URL

# Run database migrations
# Paste src/db/schema.sql then src/db/migrations/001_3things_pivot.sql into Supabase SQL Editor

# Seed the locality graph (Dallas neighborhoods)
npx tsx scripts/seed-graph.ts

Run locally (development)

npm run dev
# → http://localhost:3000        Onboarding page (area → neighborhood → tenure → goals → vibe → email)
# → http://localhost:3000/api/*  API endpoints

Production

The app is deployed on Vercel at news.elilaird.com. Pushing to main triggers auto-deploy.

# Develop on dev branch
git checkout dev
# ... make changes, commit ...

# Deploy to production
git checkout main
git merge dev
git push
git checkout dev

Environment variables are configured in the Vercel dashboard (Settings → Environment Variables).

Onboard a subscriber

Via the landing page at news.elilaird.com (or http://localhost:3000 in dev), or via API:

curl -X POST http://localhost:3000/api/onboard \
  -H "Content-Type: application/json" \
  -d '{
    "email": "you@example.com",
    "name": "Your Name",
    "neighborhood": "Oak Lawn",
    "tenure": "few_months",
    "goals": ["meet_people", "stay_active"],
    "social_vibe": "small_group"
  }'

Pipeline CLI

The pipeline handles content ingestion, matching, letter generation, and email delivery. Run it with --mode to control which steps execute.

npx tsx scripts/run-pipeline.ts [--mode=MODE] [--force] [OPTIONS]

Modes

Mode Ingest Match Generate Send email Save HTML locally
full (default) Yes Yes Yes Yes No
ingest-only Yes No No No No
generate-only No Yes Yes No No
preview No Yes Yes No Yes (test-output/)
send-only No No No Yes (latest unsent) No

Examples

# Full pipeline — ingest, match, generate, and send
npx tsx scripts/run-pipeline.ts --mode=full --force

# Preview what the letter looks like without sending
npx tsx scripts/run-pipeline.ts --mode=preview --force

# Refresh the content pool without generating letters
npx tsx scripts/run-pipeline.ts --mode=ingest-only --force

# Generate letters from existing content pool
npx tsx scripts/run-pipeline.ts --mode=generate-only --force

# Send previously generated letters
npx tsx scripts/run-pipeline.ts --mode=send-only

# Target a specific subscriber
npx tsx scripts/run-pipeline.ts --mode=preview --subscriber=you@example.com --force

# Show all options
npx tsx scripts/run-pipeline.ts --help

Flags

Flag Effect
--mode=MODE Pipeline mode (see table above). Default: full
--force Bypass cadence, sent-today, and freshness checks
--subscriber=EMAIL Target a single subscriber
--node=NAME Target a single locale node (for ingestion)

Ingestion queries are automatically shaped by subscriber goals — no manual configuration needed. The pipeline aggregates goals across all active subscribers per neighborhood and builds targeted search queries.

How it works

Step 1: Load subscribers

Determines who needs a letter this run. --subscriber=email targets one person, --force includes all active/beta subscribers, otherwise getSubscribersDue() checks cadence and last send date.

Step 2: Ingestion (fills the content pool)

For each neighborhood that has subscribers, two adapters run in sequence. Subscriber goals are aggregated per node and passed to both adapters to guide what content gets searched for.

Serper adapter (src/sources/adapters/serper.ts)

  • Builds goal-driven search queries (e.g. subscribers with stay_active → "Oak Lawn Dallas run club fitness group yoga")
  • Runs queries through Serper API (Google search) → titles, snippets, URLs
  • Passes all snippets to Claude Sonnet (RESEARCH_MODEL) which extracts structured items (title, description, content_type, when/where/cost, is_recurring)

Claude deep adapter (src/sources/adapters/claude-deep.ts) — neighborhood-level only

  • Calls Claude Sonnet (RESEARCH_MODEL) with web_search tool enabled — Claude autonomously searches the web, reads pages, and finds content that Serper's structured search misses
  • Prompt is steered by subscriber goals
  • Returns items in the same format

Both adapters feed into runAdapter() (src/sources/registry.ts) which chains:

  • Normalize → maps raw items to ContentItem, generates headline hash
  • Dedup → checks headline hash against existing content_items in DB
  • Tag → rule-based regex (~27 patterns) assigns activity_tags (fitness, outdoor, creative, food, etc.) and vibe_tags (solo_friendly, beginner_friendly, small_group, free)
  • Validate → checks expiry dates
  • Store → INSERT into content_items table (Postgres + PostGIS)

Enrichment — items with short/missing descriptions get a one-time Claude Haiku (ENRICHMENT_MODEL) call to write a proper editorial description.

Manual JSON ingestion is also available for hand-curated content (data/manual-content.json).

Step 3: Matching (per subscriber)

matchContentToSubscriber() in src/matching/engine.ts:

  1. SQL query pulls candidates from content_items within radius (ST_DWithin), not expired, not sent to this subscriber in last 60 days
  2. Scores each item in application code across 7 dimensions:
Dimension Points Logic
Goal overlap 0–60 30 pts per matching activity tag via goal-to-activity map
Vibe fit 0–20 Subscriber's social_vibe maps to vibe_tag
Tenure fit 0–15 Bonus for beginner/solo-friendly when just_moved
Distance 0 to -30 -1 pt per 300m from subscriber's home_location
Freshness 0–10 Bonus if never recommended before
Quality source 0–15 Curated: 15, Partner: 10
Recurring bonus 0–5 For community-type items that are recurring
  1. Returns MatchedPicks: best community, best experience, best discovery, best overall (nudge)

Step 4: Generation (per cohort)

  1. Subscribers with identical top-4 item IDs + tenure + top goal are grouped into cohorts
  2. For each cohort, generateLetter() calls Claude Sonnet (GENERATION_MODEL) with the matched picks + subscriber profile
  3. Claude returns JSON with 4 sections — each with headline, body, and a "why this is for you" justification that references the subscriber's goals and tenure naturally
  4. JSON is parsed and rendered into inline-CSS HTML via the email template
  5. For remaining cohort members: the greeting is string-replaced with their name, same letter body
  6. Each subscriber gets an issues row with the HTML and content_ids (for 60-day dedup)

Step 5: Delivery

  1. Queries issues where sent_at IS NULL
  2. Sends each via Resend SDK
  3. Marks sent_at on success

Models summary

Step Model Env var What it does
Serper extraction Sonnet RESEARCH_MODEL Extracts structured items from search snippets
Claude deep search Sonnet RESEARCH_MODEL Agentic web search + extraction
Enrichment Haiku ENRICHMENT_MODEL Writes descriptions for thin items
Letter writing Sonnet GENERATION_MODEL Writes the personalized 4-pick letter

Idempotency

The pipeline is safe to re-run without --force: dedup prevents duplicate content items, generation skips subscribers who already have an issue today, and delivery only sends issues with sent_at IS NULL.

Test scripts

# Ingest content from a manual JSON file
npx tsx scripts/test-ingest.ts --source=manual --file=data/manual-content.json --node="Oak Lawn"

# Test matching for a subscriber (prints picks with scores and reasons)
npx tsx scripts/test-match.ts "you@example.com"

# Generate letter HTML for one subscriber (saves to test-output/)
npx tsx scripts/test-generate.ts "you@example.com"

# Send an HTML file as a test email via Resend
npx tsx scripts/test-send.ts "you@example.com" "test-output/letter-2026-03-20.html"

# Populate the locality graph
npx tsx scripts/seed-graph.ts

Environment variables

Required

ANTHROPIC_API_KEY=sk-ant-...       # Claude API (console.anthropic.com)
DATABASE_URL=postgresql://...       # Supabase connection string
RESEND_API_KEY=re_...              # Email delivery (resend.com)
FROM_EMAIL=hello@thelocal.post     # Verified sending address

Ingestion + providers

SERPER_API_KEY=                    # Required for Serper ingestion (serper.dev, 2,500 free)
RESEARCH_MODEL=claude-sonnet-4-20250514   # LLM for Serper extraction + Claude deep search
DEEP_LOCAL_QUERIES_PER_NODE=5      # Max web searches per neighborhood in Claude deep mode

Pipeline

PIPELINE_CONCURRENCY=10            # Parallel generation calls
ACTIVE_RESEARCH_LEVELS=city,neighborhood  # Which graph levels to research
GENERATION_MODEL=claude-sonnet-4-20250514  # LLM for letter writing
CONTENT_POOL_RADIUS_METERS=8000    # Default search radius for content matching (~5 miles)
ENRICHMENT_MODEL=claude-haiku-4-5-20251001  # LLM for one-time item descriptions

Optional

LOG_LEVEL=info                     # debug | info | warn | error
GOOGLE_PLACES_API_KEY=             # Optional: Google Places Nearby Search
EVENTBRITE_API_KEY=                # Optional: venue event polling
MEETUP_OAUTH_TOKEN=                # Optional: GraphQL group event queries

Architecture

Two-layer system: geographic content pool + profile-based matching.

Ingest (per-node)  →  Match (per-subscriber)  →  Generate (per-cohort)  →  Deliver (per-subscriber)
 Serper + Claude       Scoring engine              Claude Sonnet            Resend

The core data structure is a locality DAG: metro → city → area → neighborhood. Content is ingested per-node and tagged with location, activity type, vibe, and tenure fit. Matching scores content against subscriber profiles (goals, vibe, tenure, distance). Subscribers with identical top picks share a generation call via cohort grouping.

See docs/ARCHITECTURE.md for full system design.

Deployment

Hosted on Vercel at news.elilaird.com. Email domain: thelocalpost.elilaird.com.

Component Where Trigger
Landing page Vercel (static CDN) Push to main
Onboarding API Vercel (serverless) Push to main
Pipeline Local machine Manual CLI

Branches:

  • dev — development (default working branch)
  • main — production (Vercel auto-deploys)

Project structure

src/
  sources/      — Content ingestion: adapters (manual, serper, claude-deep), normalize, dedup, tag, enrich, validate
  matching/     — Profile-based content scoring, matching engine, cohort grouping
  providers/    — LLM + search provider abstraction (Claude, Serper)
  db/           — Schema, migrations, query helpers
  graph/        — Locality graph types, traversal, node management
  research/     — Research agent and prompts (used by deep adapter for web search patterns)
  generate/     — Letter writer, context assembly, email template (table-based, inline CSS)
  deliver/      — Resend sender, tracking stubs
  feedback/     — Reply parsing stubs (Phase 2)
  pipeline/     — Orchestrator, scheduler, reporter
  api/          — Express server, onboarding endpoint
  shared/       — Config, logger, types, usage tracking
public/         — Onboarding page
data/           — Pre-seeded neighborhood graph JSON, manual content seed
scripts/        — CLI test scripts and pipeline runner
docs/           — Architecture, spec, decisions, design docs

Cost estimates

Subscribers Content pool Cohorts Ingestion Writing Email Total/week
100 ~150 items ~20 ~$1 ~$0.40 ~$0.40 ~$2
500 ~250 items ~60 ~$1 ~$1.20 ~$2 ~$4
1,000 ~350 items ~150 ~$1 ~$2.40 ~$4 ~$7

Content pool grows sublinearly (recurring communities and persistent discoveries accumulate). Writing costs scale with cohorts, not subscribers.

License

Private — all rights reserved.

About

Hyperlocal newsletter platform.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages