feat(viz): choropleth filled-region maps (geo + MapLibre) with optional geocode#4086
Merged
Conversation
…al geocode Add a `choropleth` chart type to `qsv viz`, adopting the two filled-region traces from dathere/plotly PR #2 (`Choropleth` on the token-free geo projection, `ChoroplethMap` on a MapLibre basemap). - `viz choropleth` colors regions by `--value`/`--agg` (or per-region row count). `--location-mode` matches regions by iso3 / usa-states / country-names / geojson-id; `--color-scale` picks the palette. - `--map` switches to the MapLibre `ChoroplethMap` with `--geojson` (local file or remote http(s) URL), `--feature-id-key` and `--style`. - `viz smart` auto-adds a per-country choropleth panel beside the point map when coordinates reverse-geocode to >= 2 countries (`PanelKind::Choropleth`; detection geocode-gated, render ungated). - `--geocode` (geocode feature) derives region codes from the local Geonames index: reverse from `--lat`/`--lon`, or forward from a place/country-name column. New `GeoRegion` + `reverse_geocode_regions` / `forward_geocode_regions` in `geocode.rs`; non-geocode builds emit a clean feature-required error. Re-pins plotly to the `feat/choropleth-maps` branch (zero Cargo.lock churn beyond the new traces). Adds 9 integration tests (+1 ignored geocode test), a gallery figure with `country_stats.csv`, and regenerates `docs/help/viz.md` and the `qsv-viz` MCP skill. Browser-verified all three render paths (ISO-3 geo, MapLibre `--map` via remote-URL geojson, geojson-id geo) paint filled regions, not just serialize. Builds green for all_features / viz-without-geocode / qsvlite / qsvdp; fmt + clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Up to standards ✅🟢 Issues
|
…xtent Addresses roborev job 3223 (two Medium findings): - `viz smart` built the per-country choropleth from the downsampled map coordinates, so datasets above MAX_SMART_POINTS showed sampled counts under a "count" label. Now aggregate from the full pre-downsample `core_lats`/`core_lons` set — the panel embeds only per-country aggregates (never the raw points), so accurate counts add no HTML weight. - `viz choropleth --map` never set an initial center/zoom, so custom or local GeoJSON (counties, cities) rendered at plotly's default whole-world view and was effectively invisible. Add `geojson_lat_lons` (descends only into geometry-bearing keys, so property arrays can't pollute the bounds) and reuse `map_center_zoom` to frame the MapLibre basemap to the GeoJSON extent. Adds `viz_choropleth_map_frames_ignore_properties` and center/zoom assertions to `viz_choropleth_map`. 136 viz tests pass; fmt + clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses roborev job 3224 (Medium): the GeoJSON-extent framing added for `viz choropleth --map` always computed the fit zoom with the default 1000x600 aspect, but static image exports honor `--width`/`--height` — so a narrow/tall or wide/short export could be zoomed for the wrong aspect and crop the GeoJSON extent. Thread `out_format` into `build_choropleth_plot` and extract the shared `fit_dims(flag_width, flag_height, out_format)` helper (de-duplicated from `build_map_plot`): image exports fit the requested dimensions, HTML keeps the representative default aspect. Adds the `fit_dims_honors_image_aspect_only` unit test (per-format dimension choice + proves `map_center_zoom` is aspect-sensitive, so the wrong dims would mis-zoom). 136 viz integration + 90 viz unit tests pass; fmt + clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses roborev branch review 3226 (two Medium findings): - The non-`--map` choropleth path loaded custom GeoJSON but never framed the `geo` projection to it, so `--location-mode geojson-id --geojson counties.json` opened at the whole-world view with local polygons effectively invisible. Now frame the projection to the GeoJSON extent (collected via `geojson_lat_lons` before the value moves into the trace), reusing the smart panel's `geo_framing` (albers-usa / natural-earth / mercator fit, padded + antimeridian-aware). Built-in iso3 / usa-states modes (which plotly auto-scopes) are untouched. - `viz smart` reverse-geocoded every core coordinate to build the choropleth panel, paying a second index load + an unbounded all-row pass even when the panel was later dropped. Gate it on the cheap min/max-extent span (`SMART_CHOROPLETH_MIN_SPAN_DEG`): a compact metro/region cluster is skipped outright (no geocode pass), while country-scale and global extents still run it — and unlike gating on the geocoded bounding-box corners, this also fires for global extents that wrap the antimeridian (where the corner metadata is suppressed). `build_smart_choropleth_panel`'s own 2-country check still drops wide single-country data. Adds `viz_choropleth_geojson_id_geo_framed`. 137 viz tests pass; fmt + clippy clean. Browser-verified the framed European geojson-id render. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Previously, `viz smart` on a wide US-only dataset paid the full per-row reverse-geocode pass only to find a single country (USA) and then drop the panel — wasted work and a missed opportunity. A US dataset's informative breakdown is per-STATE, not a single-country blob. Now decide the choropleth scope from the cheap extent bounds: when the core extent lies within the US bounding box (`extent_within_us`, no geocoding), aggregate per US state (`us_state_code` → `LocationMode:: UsaStates`) and title the panel "US states"; otherwise per country as before. US-states panels render with the albers-usa projection (CONUS + AK/HI insets) so they frame the US instead of sitting tiny on the world view — applied to the smart inline panel and, for consistency, the standalone `viz choropleth --location-mode usa-states` path (which was also unframed). The generous US box can include border/ocean points; non-US points simply resolve to no state and drop out of the per-state tally, so a mostly-US dataset still yields a clean state map. Compact single-metro data still short-circuits (no geocode pass); genuinely multi-country data still gets the per-country panel. Adds `extent_within_us` unit test + asserts the albers-usa projection in `viz_choropleth_usa_states`. 137 viz integration + 91 viz unit tests pass; fmt + clippy clean; browser-verified both the smart US-states panel and the standalone framing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-country (`Countries`) choropleth always drew on the whole-world natural-earth projection, so a region-confined dataset (Europe, Asia, etc.) showed its countries colored but tiny on a world map. Carry the source points' bounding-box corners on `PanelKind::Choropleth` and run them through the existing `geo_framing` at render time, so the panel frames to its own extent — generically, no hard-coded continent list: a European/Asian/Oceanian/South-American cluster fits to that region (mercator), a US extent resolves to the albers-usa composite, and global / multi-continent data correctly stays on the world projection. This also unifies the US-states special case (geo_framing returns albers-usa for within-US extents on its own). Near-hemispheric extents (e.g. all of Africa or pole-to-pole South America, beyond geo_framing's 90deg-lon / 45deg-lat "global" threshold) still render as world — a tunable threshold. Adds the `#[ignore]` `viz_smart_choropleth_frames_to_region` test (Western Europe → mercator + fitted axes). The fixture uses real newlines because rustfmt's string wrapping corrupted a `\n` escape at a line boundary. 137 viz + 91 unit tests pass; fmt + clippy clean; browser-verified the framed Europe render. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses roborev jobs 3227 and 3228 (both Medium): - 3227: custom GeoJSON framing reused geo_framing's 2.5% outlier trim, which could clip legitimate edge/island polygons (every GeoJSON vertex is intentional geometry). geo_framing now takes a `trim_frac`: custom GeoJSON frames from the FULL extent (0.0), point-derived panels keep MAP_FRAME_TRIM_FRAC. - 3228: the smart choropleth chose US-states purely from the broad US bounding box, which also covers Canada/Mexico/Caribbean — so a multi-country dataset inside that box rendered a US-states panel that silently dropped the non-US points. Scope is now decided from the reverse-geocoded countries: per-US-state only when every resolved country is the USA (and 2+ states exist), otherwise per-ISO-3-country. Drops the bbox-based extent_within_us / ChoroplethScope in favor of a single geocode pass + a shared `tally` helper. Adds `viz_choropleth_geojson_framing_keeps_edge_vertices` (CI-runnable) and `#[ignore]` `viz_smart_choropleth_us_bbox_multicountry_is_per_country` (US+Mexico → ISO-3 + MEX, not USA-states). 138 viz + 90 unit tests pass; fmt + clippy clean; non-geocode build green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses roborev job 3229 (Medium): the smart choropleth framed to the source-point bounding box, but the trace fills whole countries/states — so a city near a country's interior (e.g. Madrid) clipped that country (Spain) at the viewport edge. Add a typed `LayoutGeo` `fitbounds` to the dathere/plotly fork (GeoFitBounds::Locations/GeoJson) and frame the smart country choropleth with `fitbounds: "locations"` on a natural-earth geo, so Plotly fits the view to the union of the rendered region geometries rather than the points. A European/Asian/etc. cluster now shows whole countries with no edge clipping; global data stays world-scale. US-states keep the self-framing albers-usa composite (no fitbounds needed). Drops the point-extent corners from PanelKind::Choropleth (framing no longer derives from points). Cargo.lock re-pinned to the fork commit that adds the field. Updated viz_smart_choropleth_frames_to_region to assert fitbounds. 138 viz + 90 unit tests pass; fmt + clippy clean; browser-verified Spain/Italy/UK render uncropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tally Addresses roborev job 3230 (Low): `build_smart_choropleth_panel` derived the all-USA decision from the ISO-3 tally, which skips empty codes. A GeoRegion can carry a valid `iso2` (e.g. "MX") but an empty `iso3` when the country-info lookup fails, so such a non-US point would vanish from the tally and leave an all-"USA" set that wrongly renders US states, silently dropping the point. Decide `all_usa` from every resolved region's `iso2 == "US"` (set straight from the matched record); keep the ISO-3 tally only for the per-country rendering path. The job's Medium finding (point-based framing clips filled regions) was already resolved in 07d8c2c, which replaced point-extent framing with `geo.fitbounds = "locations"` and removed the frame corners entirely. 138 viz + 90 unit tests pass; fmt + clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add four gallery figures covering the new `viz` choropleth capabilities: - standalone `--location-mode usa-states` (US-state geo choropleth) - standalone `--map` MapLibre choropleth with `--geojson`/`--feature-id-key` - `viz smart` per-US-state auto-panel (all points resolve to US → albers-usa) - `viz smart` per-country auto-panel with `--dictionary infer` (LLM-labeled) New datasets: world_cities.csv, us_cities.csv, us_state_stats.csv, western_states.csv + western_states.geojson. The smart-dashboard figures embed as CDN-slimmed iframes; the LLM-dependent world choropleth reuses committed pregenerated HTML, while the deterministic US figure re-geocodes on regen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Picks up the dathere/plotly fix on feat/choropleth-maps: - GeoFitBounds gains a `False` variant (serializes as boolean `false`) - LayoutGeo now skips serializing unset (null) fields Both are safe for qsv viz: the `False` variant is additive (we build layouts fresh) and the null-omission only cleans up geo JSON. All 142 viz tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `viz smart` choropleth auto-panel is gated behind the `geocode` feature (reverse-geocoding lat/lon to regions). The README presented it as automatic for any lat/lon dataset; a viz-only build renders just the point map. Note the requirement in the smart-choropleth section and tag the world_cities.csv / us_cities.csv dataset rows with `geocode`, matching the delivery_stops.csv row. Addresses roborev #3234. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
choroplethchart type toqsv viz(issue #302), adopting the two filled-region traces from dathere/plotlyfeat/choropleth-maps:Choroplethon the token-free geo projection, andChoroplethMapon a MapLibre basemap.Scope
viz choropleth— colors regions by--value/--agg, or per-region row count when--valueis omitted.--location-modematches regions byiso3(default) /usa-states/country-names/geojson-id.--color-scaleselects the palette (defaultviridis).--mapswitches the token-free geo projection for a MapLibreChoroplethMapwith--geojson(local file or remote http(s) URL),--feature-id-key, and--style. The standalone--mapview now auto-frames to the GeoJSON extent (center + zoom) instead of opening at plotly's whole-world[0,0]default; the geo subplot self-frames viafitbounds.viz smart— auto-adds a choropleth panel beside the point map when coordinates reverse-geocode to ≥ 2 distinct regions (newPanelKind::Choropleth):fitbounds: "locations"(regions are never clipped).geocode-gated; the render path ships unconditionally.--geocode(requires thegeocodefeature) — derives region codes from the local Geonames index: reverse from--lat/--lon, or forward from a place/country-name column. AddsGeoRegion+reverse_geocode_regions/forward_geocode_regionstogeocode.rs. Non-geocode builds emit a clean "requires the geocode feature" error.Dependency
Re-pins the dathere plotly fork (
feat/choropleth-maps, commiteace43c3). On top of the newChoropleth/ChoroplethMaptraces it addsLayoutGeofitbounds(typedGeoFitBounds, incl. aFalsevariant) used to frame the smart choropleth, and makesLayoutGeoskip serializing unset (null) fields. Additive for qsv — all existing traces unchanged.Tests & docs
tests/test_viz.rs(12 CI-runnable + geocode#[ignore]tests run locally with the index present); 139 viz tests pass, 3 ignored, no regressions.--location-mode usa-states, standalone--map(MapLibre + custom GeoJSON), aviz smartper-US-state auto-panel, and aviz smartper-country world choropleth with--dictionary inferLLM-labeled fields — plus datasetscountry_stats.csv,us_state_stats.csv,western_states.csv(+western_states.geojson),world_cities.csv, andus_cities.csv(gallery now 34 figures).examples/viz/README.mdupdated and notes thegeocodebuild requirement for the smart choropleth auto-panel.docs/help/viz.mdand theqsv-vizMCP skill JSON; skillsCHANGELOG.mdUnreleased note added.Verification
--mapChoroplethMap (filled western US states on a MapLibre basemap from a custom GeoJSON), theviz smartper-US-state (albers-usa) and per-country (world,fitbounds) auto-panels all paint filled regions.cargo +nightly fmtand clippy clean.Follow-ups (non-blocking)
viz smarton geo data pays one extra reverse-geocode pass (engine cached, bounded ≤ 50k points) even when it resolves to < 2 regions and the panel is dropped.🤖 Generated with Claude Code