fix: route tile downloads through MapTiler to stop OSM 403 blocks

[erikleon]

Summary

Downloaded tiles from tile.openstreetmap.org are returning 403 Access Blocked because the tool violates OSM's Tile Usage Policy — specifically the prohibition on bulk downloading. This PR fixes the root cause rather than trying to mask it.

Core change: default osm/satellite/terrain sources now route through MapTiler (free tier, explicitly permits offline caching with an API key). Raw OSM remains available as an opt-in osm-direct source gated behind --i-understand-osm-policy with hard-capped concurrency (workers=2, delay≥1s).

Policy-compliant HTTP client:

• Real User-Agent: tdeck-maps/1.0 (<contact>) with contact info, plus Referer — required by MapTiler, OSM, and Nominatim policies
• Exponential backoff on 429/403/503, honors Retry-After
• Rejects non-image responses instead of silently saving error HTML as .png
• Aborts the whole run on persistent 403 from osm-direct with a pointer to the policy URL
• Defaults lowered: --max-workers 3 → 2, --delay 0.2 → 0.5

GUI (maps.html): adds a MapTiler key field + contact field, emits --maptiler-key/--contact/--i-understand-osm-policy in the generated command, and switches the Leaflet preview layer to MapTiler so panning the preview no longer hammers OSM's servers either.

Docs: README now explains the MapTiler key requirement up-front and why OSM direct is restricted.

Commits (bisectable)

1. chore: ignore tiles output and .DS_Store
2. fix: route tile downloads through MapTiler; restrict direct OSM — core client + CLI
3. feat(gui): add MapTiler key + contact fields; switch preview to MapTiler
4. docs: document MapTiler key requirement and OSM policy rationale

Test plan

• Syntax check: python3 -c "import ast; ast.parse(open('meshtastic_tiles.py').read())" passes
• --help renders new flags (--maptiler-key, --contact, --i-understand-osm-policy, new --source choices)
• Missing-key guard: python3 meshtastic_tiles.py --city "Portland" --source osm exits cleanly with "MapTiler key required" message
• osm-direct gate: refuses to run without --i-understand-osm-policy and prints the policy URL
• End-to-end: MAPTILER_KEY=<key> python3 meshtastic_tiles.py --city "Portland" --min-zoom 10 --max-zoom 11 — download completes, tiles are valid PNGs (needs reviewer's MapTiler key)
• GUI smoke: open maps.html, enter MapTiler key, verify preview loads from api.maptiler.com (check Network tab) and generated command includes --maptiler-key

Notes

• MapTiler's free tier (100k tile requests/month) is generous enough for the T-Deck offline use case. Get a key at https://maptiler.com/.
• The default --contact is hardcoded to the author's email (ekarwatowski@gmail.com); redistributors should override with their own via --contact or the GUI contact field.
• --source cycle (Thunderforest, required a key that was never wired up) has been removed since it was already broken.

[qodo-code-review]

Review Summary by Qodo

Route tile downloads through MapTiler; restrict direct OSM access

🐞 Bug fix ✨ Enhancement

Walkthroughs

Description

• Route tile downloads through MapTiler to comply with OSM policy
  - Default sources (osm/satellite/terrain) now use MapTiler API
  - Raw OSM available as osm-direct with policy gate and hard caps
• Implement policy-compliant HTTP client with proper headers and retries
  - Add User-Agent with contact info and Referer header
  - Exponential backoff on 429/403/503 with Retry-After support
  - Reject non-image responses instead of saving error HTML as PNG
• Update GUI to support MapTiler key and contact fields
  - Add MapTiler key input and contact email field
  - Switch preview layer to MapTiler to avoid hammering OSM servers
  - Update tile source dropdown with policy labels
• Lower default concurrency and increase delay for policy compliance
  - max-workers: 3 → 2, delay: 0.2 → 0.5 seconds

Diagram

flowchart LR
  A["OSM Policy<br/>Violation"] -->|"Bulk downloads<br/>prohibited"| B["403 Blocks"]
  C["MapTiler API<br/>Free tier"] -->|"Explicit offline<br/>caching support"| D["Default sources<br/>osm/satellite/terrain"]
  E["osm-direct<br/>Raw OSM"] -->|"Policy-gated<br/>--i-understand-osm-policy"| F["Hard-capped<br/>workers=2, delay≥1s"]
  D -->|"Requires key"| G["User provides<br/>MAPTILER_KEY"]
  H["Policy-compliant<br/>HTTP client"] -->|"User-Agent +<br/>Referer + Retry-After"| D
  H -->|"Rejects non-image<br/>responses"| I["Valid PNG/JPG tiles"]

Loading

File Changes

1. meshtastic_tiles.py Bug fix, enhancement, error handling +131/-45

MapTiler integration and policy-compliant HTTP client

• Add MapTiler API integration with key support for default sources (osm/satellite/terrain)
• Implement policy-compliant HTTP client with User-Agent, Referer, and exponential backoff
• Add osm-direct source gated behind --i-understand-osm-policy with enforced caps (workers≤2,
 delay≥1s)
• Reject non-image responses and raise PolicyBlockedError on persistent 403 from osm-direct
• Lower default --delay to 0.5s and --max-workers to 2; add --contact and --maptiler-key CLI args

meshtastic_tiles.py

---

2. maps.html Enhancement, configuration changes +55/-22

Add MapTiler key input and switch preview to MapTiler

• Add MapTiler key input field for preview layer and contact email field for User-Agent
• Switch preview tile layers from raw OSM/ArcGIS/OpenTopoMap to MapTiler API endpoints
• Update tile source dropdown to label MapTiler-backed sources and add osm-direct option
• Refactor command generation to emit --maptiler-key, --contact, and --i-understand-osm-policy flags
• Lower default workers to 2 and delay to 0.5s to match CLI defaults

maps.html

---

3. README.md 📝 Documentation +24/-5

Document MapTiler key requirement and OSM policy rationale

• Add quick-start instruction to obtain and export MAPTILER_KEY before generating tiles
• Explain why OSM direct is no longer default and link to OSM Tile Usage Policy
• Document MapTiler as the compliant default and osm-direct as policy-restricted opt-in
• Remove cycle source (Thunderforest) from examples as it was already broken
• Add section explaining API key requirement and policy rationale

README.md

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (3)   📘 Rule violations (0)   📎 Requirement gaps (0)

🐞\ ≡ Correctness (2) ☼ Reliability (1)

1. Metadata format mismatch 🐞 ≡

Description

For --source satellite, tiles are now saved with a .jpg extension but generate_metadata()
still writes "format": "png", so the output package metadata contradicts the actual tile files.
Any consumer that relies on metadata.json.format to determine tile decoding/paths can fail or
mis-handle satellite tiles.

Code

meshtastic_tiles.py[R171-181]

+    def _tile_extension(self, source):
+        return 'jpg' if source == 'satellite' else 'png'
+
    def download_tile(self, x, y, zoom, source="osm"):
-        """Download a single tile"""
+        """Download a single tile with policy-compliant retries."""
        url = self.get_tile_url(x, y, zoom, source)
-        
-        # Create directory structure
+
        tile_dir = self.output_dir / str(zoom) / str(x)
        tile_dir.mkdir(parents=True, exist_ok=True)
-        
-        tile_path = tile_dir / f"{y}.png"
-        
-        # Skip if tile already exists
+        tile_path = tile_dir / f"{y}.{self._tile_extension(source)}"
+

Evidence

download_tile() switches the satellite file extension to JPG via _tile_extension(), but
generate_metadata() still hardcodes the metadata format to PNG, creating an inconsistency within
the generated output directory.

meshtastic_tiles.py[160-205]
meshtastic_tiles.py[311-323]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Satellite tiles are written as `*.jpg` but `metadata.json` is generated with `"format": "png"`, making the output inconsistent.

### Issue Context
- `download_tile()` uses `_tile_extension()` to choose `jpg` for `source == 'satellite'`.
- `generate_metadata()` currently hardcodes `format` to `png` for all sources.

### Fix Focus Areas
- meshtastic_tiles.py[160-205]
- meshtastic_tiles.py[311-323]

### Suggested fix
- Set `metadata["format"]` dynamically based on the selected source (e.g., `self._tile_extension(source)`), OR
- If the downstream consumer requires PNG only, keep writing `.png` files and transcode JPEG responses to PNG before saving, and continue emitting `format: png` accordingly.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Preview loads without key 🐞 ☼

Description

maps.html initializes the Leaflet tile layer using a MapTiler URL even when previewKey is empty,
producing requests like ...?key= that will repeatedly fail while the user pans/zooms. This creates
unnecessary traffic/noise and makes the preview look broken until the user discovers and fills the
preview key field.

Code

maps.html[R360-377]

+        function tileUrl(style) {
+            const key = document.getElementById('previewKey')?.value || '';
+            const layers = {
+                osm:       `https://api.maptiler.com/maps/openstreetmap/{z}/{x}/{y}.png?key=${key}`,
+                satellite: `https://api.maptiler.com/tiles/satellite-v2/{z}/{x}/{y}.jpg?key=${key}`,
+                terrain:   `https://api.maptiler.com/maps/topo-v2/{z}/{x}/{y}.png?key=${key}`,
+            };
+            return layers[style] || layers.osm;
+        }
+
        // Initialize map
        function initMap() {
            map = L.map('map').setView([39.7392, -104.9903], 8); // Denver
-            
+
            // Add default tile layer
-            currentTileLayer = L.tileLayer(tileLayers.osm, {
-                attribution: '© OpenStreetMap contributors'
+            currentTileLayer = L.tileLayer(tileUrl('osm'), {
+                attribution: '© OpenStreetMap contributors · © MapTiler'
            }).addTo(map);

Evidence

tileUrl() defaults the key to an empty string when the input is blank, and initMap() immediately
creates a tile layer with that URL on page load.

maps.html[359-377]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The map preview issues unauthorized/invalid MapTiler tile requests on initial load because the preview key defaults to empty.

### Issue Context
This is preview-only, but Leaflet will keep attempting to fetch tiles as the user interacts with the map.

### Fix Focus Areas
- maps.html[359-377]

### Suggested fix
- If `previewKey` is empty, do not attach a tile layer; instead show a lightweight placeholder layer/message.
- When the user enters a key, create (or refresh) the tile layer once.
- Optionally, disable map interaction until a key is present if that matches the intended UX.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Preview key not encoded 🐞 ≡

Description

The preview key is interpolated directly into the MapTiler URL query string without URL-encoding, so
reserved characters (e.g., +, &, #) can break the request and cause hard-to-diagnose preview
failures. This is avoidable by encoding the key before building the tile URL.

Code

maps.html[R360-366]

+        function tileUrl(style) {
+            const key = document.getElementById('previewKey')?.value || '';
+            const layers = {
+                osm:       `https://api.maptiler.com/maps/openstreetmap/{z}/{x}/{y}.png?key=${key}`,
+                satellite: `https://api.maptiler.com/tiles/satellite-v2/{z}/{x}/{y}.jpg?key=${key}`,
+                terrain:   `https://api.maptiler.com/maps/topo-v2/{z}/{x}/{y}.png?key=${key}`,
+            };

Evidence

tileUrl() constructs ...?key=${key} directly from the input value rather than
encodeURIComponent(key).

maps.html[359-366]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The preview tile URL concatenates the user-provided key without URL encoding.

### Issue Context
Even if MapTiler keys are typically safe, encoding prevents edge-case breakage and avoids accidental parameter injection via `&`.

### Fix Focus Areas
- maps.html[359-366]

### Suggested fix
- Change `const key = ...value || ''` to `const key = encodeURIComponent((...value || '').trim())`.
- Keep the rest of the URL templates unchanged.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

ⓘ The new review experience is currently in Beta. Learn more