feat(unrest): surface protest source links#3492
Conversation
|
@lspassos1 is attempting to deploy a commit to the World Monitor Team on Vercel. A member of the Team first needs to authorize it. |
Greptile SummaryThis PR threads Confidence Score: 4/5Safe to merge — all findings are non-blocking style/cleanup suggestions with no functional regressions. Only P2 findings: an empty container div edge case in MapPopup, a missing cap on the initial GDELT location entry, and an unbounded merge in the server-side deduplicator. The field is fully optional, URL sanitization is in place, and CI (typecheck, proto-freshness, unit, biome) passed. scripts/seed-unrest-events.mjs (initial location entry missing 5-URL cap), server/worldmonitor/unrest/v1/_shared.ts (unbounded sourceUrls merge) Important Files Changed
Sequence DiagramsequenceDiagram
participant GDELT as GDELT API
participant Seed as seed-unrest-events.mjs
participant Server as _shared.ts (deduplicateEvents)
participant API as /api/unrest/v1/list-unrest-events
participant Svc as src/services/unrest/index.ts
participant Popup as MapPopup.ts
GDELT->>Seed: GeoJSON features (properties.url, .source_url, etc.)
Seed->>Seed: extractGdeltSourceUrls() → normalizeSourceUrl() → uniqueSourceUrls()
Seed->>Seed: mergeSourceUrls() + slice(0,5) per location
Seed->>Server: UnrestEvent[] with sourceUrls[]
Server->>Server: deduplicateEvents() — merge sourceUrls (no cap)
Server->>API: ListUnrestEventsResponse (sourceUrls in each event)
API->>Svc: UnrestEvent (sourceUrls: string[])
Svc->>Svc: toSocialUnrestEvent() — sourceUrls?: string[] (omit if empty)
Svc->>Popup: SocialUnrestEvent with optional sourceUrls
Popup->>Popup: sanitizeUrl() + extractDomain() per URL
Popup->>Popup: render up to 3 links in popup, 1 link per cluster item
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| const sourceLinks = event.sourceUrls?.length | ||
| ? `<div class="popup-source-links">${event.sourceUrls.slice(0, 3).map((url, index) => { | ||
| const safeUrl = sanitizeUrl(url); | ||
| if (!safeUrl) return ''; | ||
| const domain = extractDomain(url) || `${t('popups.source')} ${index + 1}`; | ||
| return `<a class="popup-link" href="${safeUrl}" target="_blank" rel="noopener noreferrer nofollow">${escapeHtml(domain)} →</a>`; | ||
| }).join('')}</div>` | ||
| : ''; |
There was a problem hiding this comment.
event.sourceUrls?.length is truthy whenever the array is non-empty, but each item is still passed through sanitizeUrl and silently dropped (returning '') if it fails. If every URL in the array fails the client-side check the result is <div class="popup-source-links"></div> — an empty, invisible div injected into the popup. While harmless visually, it's cleaner to guard against this:
| const sourceLinks = event.sourceUrls?.length | |
| ? `<div class="popup-source-links">${event.sourceUrls.slice(0, 3).map((url, index) => { | |
| const safeUrl = sanitizeUrl(url); | |
| if (!safeUrl) return ''; | |
| const domain = extractDomain(url) || `${t('popups.source')} ${index + 1}`; | |
| return `<a class="popup-link" href="${safeUrl}" target="_blank" rel="noopener noreferrer nofollow">${escapeHtml(domain)} →</a>`; | |
| }).join('')}</div>` | |
| : ''; | |
| const filteredLinks = (event.sourceUrls ?? []).slice(0, 3).flatMap((url, index) => { | |
| const safeUrl = sanitizeUrl(url); | |
| if (!safeUrl) return []; | |
| const domain = extractDomain(url) || `${t('popups.source')} ${index + 1}`; | |
| return [`<a class="popup-link" href="${safeUrl}" target="_blank" rel="noopener noreferrer nofollow">${escapeHtml(domain)} →</a>`]; | |
| }); | |
| const sourceLinks = filteredLinks.length | |
| ? `<div class="popup-source-links">${filteredLinks.join('')}</div>` | |
| : ''; |
| event.sourceUrls = [...new Set([...(event.sourceUrls || []), ...(existing.sourceUrls || [])])]; | ||
| unique.set(key, event); | ||
| } else if (existing.sourceType === 'UNREST_SOURCE_TYPE_ACLED') { | ||
| existing.sources = [...new Set([...existing.sources, ...event.sources])]; | ||
| existing.sourceUrls = [...new Set([...(existing.sourceUrls || []), ...(event.sourceUrls || [])])]; | ||
| } else { | ||
| // Both GDELT: combine sources, upgrade confidence if 2+ sources | ||
| existing.sources = [...new Set([...existing.sources, ...event.sources])]; | ||
| existing.sourceUrls = [...new Set([...(existing.sourceUrls || []), ...(event.sourceUrls || [])])]; |
There was a problem hiding this comment.
sourceUrls cap in server-side deduplication
The seed script caps GDELT sourceUrls at 5 per location, but the server-side deduplicateEvents spreads both sides into a new Set with no bound at all three merge branches. Over many ACLED+GDELT merges the array can grow beyond the intended limit and bloat the API response payload. Consider adding .slice(0, 5) (or a shared constant) after each new Set(...) spread.
|
Follow-up pushed in Changes:
Validation:
|
* feat(unrest): surface protest source links * fix(unrest): cap protest source links
2 participants
Summary
Surfaces authoritative source links for unrest/protest events when GDELT provides a usable document URL. The map popup gets a compact source action so analysts can inspect the source behind a protest point instead of relying only on the normalized event text.
cc @koala73
Refs #131
Type of change
Affected areas
/api/*)Root cause
Unrest events already carried event context to the map, but the UI did not expose source links even when upstream GDELT rows included source URLs. That reduced map fidelity and made protest points harder to verify.
Changes
sourceUrlto the unrest event proto/type path and generated OpenAPI artifacts.MapPopupwhen a protest/unrest event has a valid URL.Validation
proto-freshness,unit,typecheck, andbiomepassed in fork CI.Risk
Low. The field is optional and the UI only renders the link when a source URL exists. Existing unrest events without source links keep the current popup behavior.