Skip to content

Comments

Telegram OSINT early-signal feed via Railway relay#249

Open
koala73 wants to merge 5 commits intomainfrom
feat/telegram-osint-railway
Open

Telegram OSINT early-signal feed via Railway relay#249
koala73 wants to merge 5 commits intomainfrom
feat/telegram-osint-railway

Conversation

@koala73
Copy link
Owner

@koala73 koala73 commented Feb 23, 2026

Adds true Telegram channel ingestion (MTProto via GramJS) to the existing Railway relay service and exposes it to the web app.

What’s included

  • Railway relay (scripts/ais-relay.cjs)
    • polls curated public Telegram channels using MTProto when TELEGRAM_* env vars are present
    • serves JSON feed at GET /telegram/feed?limit=50&topic=&channel=
    • /health now includes telegram status (enabled, channels, items, lastPollAt, lastError)
  • Vercel edge endpoint
    • GET /api/telegram-feed proxies to the Railway relay /telegram/feed (uses WS_RELAY_URL)
  • Product-managed curated list file
    • data/telegram-channels.json (full/tech/finance buckets)
  • Local helper
    • scripts/telegram/session-auth.mjs to generate TELEGRAM_SESSION (StringSession) for Railway

Config (Railway relay)

  • TELEGRAM_API_ID
  • TELEGRAM_API_HASH
  • TELEGRAM_SESSION
  • TELEGRAM_CHANNEL_SET=full|tech|finance

Config (Vercel)

  • WS_RELAY_URL (already used)

Notes

  • Telegram ingestion is disabled by default unless all TELEGRAM_* env vars are set.
  • Curated list is product-managed (not user-editable in UI).

Next steps (follow-up PR)

  • Add a "Telegram Early Signals" panel in the UI that calls /api/telegram-feed.
  • Add basic geo/topic enrichment + correlation hooks.

@vercel
Copy link

vercel bot commented Feb 23, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment Feb 23, 2026 6:39pm
worldmonitor-finance Ready Ready Preview, Comment Feb 23, 2026 6:39pm
worldmonitor-happy Ready Ready Preview, Comment Feb 23, 2026 6:39pm
worldmonitor-startup Ready Ready Preview, Comment Feb 23, 2026 6:39pm

Request Review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 434ecf30ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +177 to +181
if (!msg || !msg.id || !msg.message) continue;
const item = normalizeTelegramMessage(msg, channel);
newItems.push(item);
if (!telegramState.cursorByHandle[handle] || msg.id > telegramState.cursorByHandle[handle]) {
telegramState.cursorByHandle[handle] = msg.id;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Advance Telegram cursor for skipped non-text messages

The poll loop only updates cursorByHandle after passing if (!msg || !msg.id || !msg.message) continue;, so media-only posts (or other messages without message text) never advance the cursor. On channels that emit many non-text posts, each interval re-reads the same IDs, increasing API load/rate-limit risk and potentially preventing progression to newer text posts when the fetch limit window is saturated.

Useful? React with 👍 / 👎.

Comment on lines +208 to +210
pollTelegramOnce().catch(() => {});
setInterval(() => {
pollTelegramOnce().catch(() => {});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prevent overlapping Telegram poll executions

The interval starts a new pollTelegramOnce() on a fixed schedule without checking whether the previous poll is still running. Since each poll performs network calls per channel and an intentional per-channel delay, slower runs can overlap, causing concurrent reads/writes to shared cursor state and duplicate upstream requests that make Telegram throttling more likely.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant