feat(osm-equity): wire real TIGER / Census ACS / Overpass data sources

[rlemke]

Replaces the offline generator's data with live sources, opt-in via FW_EQUITY_SOURCE=real (default stays offline so CI needs no network/keys).

• ResolveStudyTracts → Census TIGERweb REST (real tract polygons, MultiPolygon-aware).
• FetchCensusEquity → Census ACS 5-year API (CENSUS_API_KEY): income/rent-burden/race/education/English/vehicles/internet; one request per county, cached; ACS null sentinels dropped; no-data tracts raise explicitly.
• FetchRegionOSM → Overpass (buildings/highways/amenities + edit metadata), retry + fallback mirrors, fetched once and clipped per tract via a cached STRtree (fast over 250k+ features).
• FetchRegionFootprints honestly gates in real mode (has_reference=false; Open Buildings not yet wired) rather than fabricating.

Verified end-to-end on San Francisco (260 real tracts, 254k OSM features). A live test (TestRealSources, opt-in) exercises all three sources. 13 offline tests pass; ruff clean.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a live data mode (FW_EQUITY_SOURCE=real) to the osm-equity study, enabling integration with live APIs such as Census TIGERweb, Census ACS, and Overpass OSM. It also optimizes local tract clipping using a cached STRtree spatial index. The reviewer feedback focuses on improving the robustness and efficiency of these live integrations, specifically by handling missing geometries in GeoJSON features, properly checking for API error payloads and pagination limits in ArcGIS and Census API responses, and combining multiple Overpass queries into a single request to reduce network overhead.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

If any feature in the GeoJSON has a null or missing geometry, shape(f["geometry"]) will raise an exception. Additionally, if we were to filter out or skip features with invalid geometries inside the loop, the indices of reps and feats would become misaligned, causing clip_tract_osm to return incorrect features. Filtering feats beforehand to ensure all elements have a valid geometry prevents both potential crashes and index misalignment.

Suggested change

	feats = _read_geojson(region_osm_path)
	reps = []
	for f in feats:
	g = shape(f["geometry"])
	reps.append(g if g.geom_type == "Point" else g.representative_point())
	feats = [f for f in _read_geojson(region_osm_path) if f.get("geometry")]
	reps = []
	for f in feats:
	g = shape(f["geometry"])
	reps.append(g if g.geom_type == "Point" else g.representative_point())

[gemini-code-assist]

ArcGIS REST API queries can fail with an error payload in the JSON response (e.g., {"error": ...}) even if the HTTP status code is 200. Checking for the "error" key explicitly provides much better error diagnostics. Furthermore, pagination using a hardcoded limit of 800 to break the loop can fail if the server's maxRecordCount is lower than 800 (e.g., 500), causing features to be silently skipped. Using the standard "exceededTransferLimit" flag and incrementing the offset by the actual number of features returned (len(feats)) makes the pagination robust against any server-side limits.

        r = requests.get(_TIGER_TRACTS, params=params, headers=_UA, timeout=60)
        r.raise_for_status()
        res = r.json()
        if "error" in res:
            error_msg = res["error"].get("message", "Unknown error")
            raise RuntimeError(f"TIGERweb query failed: {error_msg}")
        feats = res.get("features", [])
        if not feats:
            break
        for f in feats:
            if not f.get("geometry"):
                continue
            geom = shape(f["geometry"])
            c = geom.centroid
            kpl, kpo = _km_per_deg(c.y)
            p = f.get("properties", {})
            tracts.append(
                {
                    "geoid": p.get("GEOID", ""),
                    "name": p.get("NAME", ""),
                    "geometry_wkt": geom.wkt,
                    "area_km2": round(geom.area * kpl * kpo, 4),
                }
            )
        if not res.get("exceededTransferLimit"):
            break
        offset += len(feats)

[gemini-code-assist]

Making multiple separate Overpass API requests (one for buildings/amenities and one for highways) is inefficient and increases the likelihood of hitting rate limits or experiencing timeouts. Overpass QL supports multiple output statements in a single query. Combining these into a single query and processing the results in a single pass reduces the network round-trips by 50%, significantly improving performance and reliability.

    # Combine buildings, amenities, and highways into a single Overpass query to reduce HTTP round-trips.
    q = (
        f"[out:json][timeout:180];("
        f'way["building"]({b});node["amenity"]({b});way["amenity"]({b});'
        f");out center meta tags;"
        f'way["highway"]({b});out geom meta tags;'
    )

    feats: list[dict] = []
    for el in _overpass(q):
        tags = el.get("tags", {}) or {}
        rec = _recency_days(el.get("timestamp"), now)
        if "highway" in tags:
            geom = el.get("geometry") or []
            if len(geom) < 2:
                continue
            coords = [[p["lon"], p["lat"]] for p in geom]
            feats.append(
                {
                    "type": "Feature",
                    "properties": {
                        "osm_kind": "road",
                        "highway": tags.get("highway", "road"),
                        "recency_days": rec,
                    },
                    "geometry": {"type": "LineString", "coordinates": coords},
                }
            )
        else:
            if el["type"] == "node":
                lon, lat = el.get("lon"), el.get("lat")
            else:
                c = el.get("center") or {}
                lon, lat = c.get("lon"), c.get("lat")
            if lon is None or lat is None:
                continue
            if "building" in tags:
                props = {"osm_kind": "building", "building": tags["building"], "recency_days": rec}
            else:
                props = {
                    "osm_kind": "poi",
                    "amenity": tags.get("amenity", "other"),
                    "recency_days": rec,
                }
                for k in ("name", "addr:street", "opening_hours"):
                    if tags.get(k):
                        props[k] = tags[k]
            feats.append(
                {
                    "type": "Feature",
                    "properties": props,
                    "geometry": {"type": "Point", "coordinates": [lon, lat]},
                }
            )

[gemini-code-assist]

The Census API can return error messages as a JSON dictionary (e.g., {"error": "..."}) or a string, rather than a list of lists, even on successful HTTP status codes. Attempting to access rows[0] directly without validating that the response is a list with at least two elements (header + data) can lead to unhandled KeyError, IndexError, or TypeError exceptions. Adding an explicit type and length check provides a much clearer error message.

    rows = r.json()
    if not isinstance(rows, list) or len(rows) < 2:
        raise RuntimeError(f"Invalid or empty response from Census API: {rows}")
    idx = {h: i for i, h in enumerate(rows[0])}