Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .idea/.gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions .idea/modules.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Empty file added .idea/tippecanoe-cartesian.iml
Empty file.
Empty file added .idea/vcs.xml
Empty file.
61 changes: 61 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Build & Test

```bash
make -j # build all binaries
make test # full test suite (includes geobuf)
make fewer-tests # faster subset (skips geobuf)
make indent # clang-format all source
make clean # remove build artifacts
make install # install to /usr/local (override with PREFIX=...)
```

Build flags: `BUILDTYPE=Debug` (default Release), custom `CC`, `CXX`, `CFLAGS`, `CXXFLAGS`, `LDFLAGS`.

Unit tests run via `./unit` (built from `unit.cpp`, tests `text.cpp`).

## What This Is

C++11 geospatial toolchain that converts GeoJSON/Geobuf/CSV into Mapbox Vector Tile (MVT) tilesets stored as MBTiles (SQLite) or directory trees. Core insight: at low zoom, drop *least-visible* features rather than simplifying geometry—preserving data density and regional texture.

## Executables

| Binary | Entry point | Purpose |
|--------|-------------|---------|
| `tippecanoe` | `main.cpp` | Main feature→tileset pipeline |
| `tippecanoe-decode` | `decode.cpp` | MBTiles/PBF → GeoJSON |
| `tippecanoe-enumerate` | `enumerate.cpp` | List tiles in an MBTiles |
| `tile-join` | `tile-join.cpp` | Merge/filter/subset MBTiles |
| `tippecanoe-json-tool` | `jsontool.cpp` | JSON sorting/CSV integration |
| `unit` | `unit.cpp` | Unit test runner |

## Architecture

**Processing pipeline** (all in `main.cpp`):
```
CLI parse → read input (geojson/geobuf/csv) → serialize features to temp files
→ build spatial index → traverse_zooms() → write_tile() per zoom
→ encode MVT/PBF → write MBTiles (mbtiles.cpp) or dirtiles (dirtiles.cpp)
```

**Key module responsibilities:**
- `geometry.cpp/hpp` — coordinate transforms, drawing command encoding, simplification
- `tile.cpp/hpp` — zoom traversal, per-tile feature collection
- `mvt.cpp/hpp` — Mapbox Vector Tile PBF encoding
- `mbtiles.cpp/hpp` — SQLite MBTiles I/O
- `serial.cpp/hpp` — feature serialization between pipeline stages
- `evaluator.cpp/hpp` — filter expression evaluation
- `projection.cpp` — Web Mercator and custom projections
- `jsonpull/` — streaming JSON parser (embedded library)
- `geojson.cpp`, `geobuf.cpp`, `geocsv.cpp` — input format parsers

**Limits** (defined in `main.hpp`): `MAX_ZOOM=24`, default tile max 500KB, max 200K features/tile.

**Parallelism**: `-P` flag enables multi-file parallel processing; uses `CPUS` (detected core count) and temp files for overflow.

## Test Structure

`tests/` holds 40+ fixture directories. Each test compares output tiles/JSON against a `stable/` baseline. To add a test, create a directory with input files and update the relevant test target in `Makefile`.
14 changes: 12 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ indent:
TESTS = $(wildcard tests/*/out/*.json)
SPACE = $(NULL) $(NULL)

test: tippecanoe tippecanoe-decode $(addsuffix .check,$(TESTS)) raw-tiles-test parallel-test pbf-test join-test enumerate-test decode-test join-filter-test unit json-tool-test allow-existing-test csv-test layer-json-test
test: tippecanoe tippecanoe-decode $(addsuffix .check,$(TESTS)) raw-tiles-test parallel-test pbf-test join-test enumerate-test decode-test join-filter-test unit json-tool-test allow-existing-test csv-test layer-json-test cartesian-test
./unit

suffixes = json json.gz
Expand All @@ -100,7 +100,17 @@ nogeobuf = tests/overflow/out/-z0.json $(wildcard tests/stringid/out/*.json)
geobuf-test: tippecanoe-json-tool $(addsuffix .checkbuf,$(filter-out $(nogeobuf),$(TESTS)))

# For quicker address sanitizer build, hope that regular JSON parsing is tested enough by parallel and join tests
fewer-tests: tippecanoe tippecanoe-decode geobuf-test raw-tiles-test parallel-test pbf-test join-test enumerate-test decode-test join-filter-test unit
fewer-tests: tippecanoe tippecanoe-decode geobuf-test raw-tiles-test parallel-test pbf-test join-test enumerate-test decode-test join-filter-test unit cartesian-test

cartesian-test: tippecanoe tippecanoe-decode
./tippecanoe -q -z5 --cartesian --cartesian-extent=0,0,100,100 -f -n cartesian-points -o tests/cartesian/points.mbtiles.check tests/cartesian/points.json
./tippecanoe-decode -x generator -x generator_options tests/cartesian/points.mbtiles.check > tests/cartesian/points.mbtiles.json.check
cmp tests/cartesian/points.mbtiles.json.check tests/cartesian/points.mbtiles.json
rm -f tests/cartesian/points.mbtiles.check tests/cartesian/points.mbtiles.json.check
./tippecanoe -q -z3 --cartesian --cartesian-extent=-50,-50,50,50 -f -n cartesian-shapes -o tests/cartesian/shapes.mbtiles.check tests/cartesian/shapes.json
./tippecanoe-decode -x generator -x generator_options tests/cartesian/shapes.mbtiles.check > tests/cartesian/shapes.mbtiles.json.check
cmp tests/cartesian/shapes.mbtiles.json.check tests/cartesian/shapes.mbtiles.json
rm -f tests/cartesian/shapes.mbtiles.check tests/cartesian/shapes.mbtiles.json.check

# XXX Use proper makefile rules instead of a for loop
%.json.checkbuf:
Expand Down
68 changes: 43 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,22 @@ Parallel processing will also be automatic if the input file is in Geobuf format

* `-s` _projection_ or `--projection=`_projection_: Specify the projection of the input data. Currently supported are `EPSG:4326` (WGS84, the default) and `EPSG:3857` (Web Mercator). In general you should use WGS84 for your input files if at all possible.

### Cartesian coordinate mode

* `--cartesian`: Treat input coordinates as raw Cartesian x,y values instead of geographic longitude/latitude. Requires `--cartesian-extent`.
* `--cartesian-extent=`_minx_`,`_miny_`,`_maxx_`,`_maxy_: Define the coordinate space bounds for Cartesian mode. Coordinates outside this extent will be clipped.

Use Cartesian mode for non-geographic data such as floor plans, game maps, or scientific coordinate systems. The projection is linear (no Mercator distortion). Aspect ratio is preserved — the larger dimension maps to the full tile space, and the shorter dimension is centered.

`tippecanoe-decode` automatically detects Cartesian tilesets from the stored metadata and outputs the original Cartesian coordinates. The CRS is reported as `"cartesian"` in decoded output. Cannot be combined with `--projection`.

Comment thread
idoaignostics marked this conversation as resolved.
Example:

```
tippecanoe --cartesian --cartesian-extent=0,0,1000,500 -z8 -f -o floor-plan.mbtiles floor-plan.json
tippecanoe-decode floor-plan.mbtiles # outputs Cartesian coordinates automatically
```

### Zoom levels

* `-z` _zoom_ or `--maximum-zoom=`_zoom_: Maxzoom: the highest zoom level for which tiles are generated (default 14)
Expand All @@ -372,31 +388,31 @@ or the map scale of a corresponding printed map,
this table shows the approximate precision and scale corresponding to various
`-z` options if you use the default `-d` detail of 12:

zoom level | precision (ft) | precision (m) | map scale
---------- | -------------- | ------------- | ---------
`-z0` | 32000 ft | 10000 m | 1:320,000,000
`-z1` | 16000 ft | 5000 m | 1:160,000,000
`-z2` | 8000 ft | 2500 m | 1:80,000,000
`-z3` | 4000 ft | 1250 m | 1:40,000,000
`-z4` | 2000 ft | 600 m | 1:20,000,000
`-z5` | 1000 ft | 300 m | 1:10,000,000
`-z6` | 500 ft | 150 m | 1:5,000,000
`-z7` | 250 ft | 80 m | 1:2,500,000
`-z8` | 125 ft | 40 m | 1:1,250,000
`-z9` | 64 ft | 20 m | 1:640,000
`-z10` | 32 ft | 10 m | 1:320,000
`-z11` | 16 ft | 5 m | 1:160,000
`-z12` | 8 ft | 2 m | 1:80,000
`-z13` | 4 ft | 1 m | 1:40,000
`-z14` | 2 ft | 0.5 m | 1:20,000
`-z15` | 1 ft | 0.25 m | 1:10,000
`-z16` | 6 in | 15 cm | 1:5000
`-z17` | 3 in | 8 cm | 1:2500
`-z18` | 1.5 in | 4 cm | 1:1250
`-z19` | 0.8 in | 2 cm | 1:600
`-z20` | 0.4 in | 1 cm | 1:300
`-z21` | 0.2 in | 0.5 cm | 1:150
`-z22` | 0.1 in | 0.25 cm | 1:75
| zoom level | precision (ft) | precision (m) | map scale |
|------------|----------------|---------------|---------------|
| `-z0` | 32000 ft | 10000 m | 1:320,000,000 |
| `-z1` | 16000 ft | 5000 m | 1:160,000,000 |
| `-z2` | 8000 ft | 2500 m | 1:80,000,000 |
Comment thread
idoaignostics marked this conversation as resolved.
| `-z3` | 4000 ft | 1250 m | 1:40,000,000 |
| `-z4` | 2000 ft | 600 m | 1:20,000,000 |
| `-z5` | 1000 ft | 300 m | 1:10,000,000 |
| `-z6` | 500 ft | 150 m | 1:5,000,000 |
| `-z7` | 250 ft | 80 m | 1:2,500,000 |
| `-z8` | 125 ft | 40 m | 1:1,250,000 |
| `-z9` | 64 ft | 20 m | 1:640,000 |
| `-z10` | 32 ft | 10 m | 1:320,000 |
| `-z11` | 16 ft | 5 m | 1:160,000 |
| `-z12` | 8 ft | 2 m | 1:80,000 |
| `-z13` | 4 ft | 1 m | 1:40,000 |
| `-z14` | 2 ft | 0.5 m | 1:20,000 |
| `-z15` | 1 ft | 0.25 m | 1:10,000 |
| `-z16` | 6 in | 15 cm | 1:5000 |
| `-z17` | 3 in | 8 cm | 1:2500 |
| `-z18` | 1.5 in | 4 cm | 1:1250 |
| `-z19` | 0.8 in | 2 cm | 1:600 |
| `-z20` | 0.4 in | 1 cm | 1:300 |
| `-z21` | 0.2 in | 0.5 cm | 1:150 |
| `-z22` | 0.1 in | 0.25 cm | 1:75 |

### Tile resolution

Expand Down Expand Up @@ -881,6 +897,8 @@ resolutions.
### Options

* `-s` _projection_ or `--projection=`*projection*: Specify the projection of the output data. Currently supported are EPSG:4326 (WGS84, the default) and EPSG:3857 (Web Mercator).
* `--cartesian`: Decode using Cartesian coordinates. Automatically detected from tileset metadata when decoding an MBTiles file created with `--cartesian`.
* `--cartesian-extent=`_minx_`,`_miny_`,`_maxx_`,`_maxy_: Override the Cartesian extent for decoding. Useful when decoding standalone PBF tiles created with `--cartesian`.
* `-z` _maxzoom_ or `--maximum-zoom=`*maxzoom*: Specify the highest zoom level to decode from the tileset
* `-Z` _minzoom_ or `--minimum-zoom=`*minzoom*: Specify the lowest zoom level to decode from the tileset
* `-l` _layer_ or `--layer=`*layer*: Decode only layers with the specified names. (Multiple `-l` options can be specified.)
Expand Down
41 changes: 40 additions & 1 deletion decode.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,10 @@ void decode(char *fname, int z, unsigned x, unsigned y, std::set<std::string> co
char *map = (char *) mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (map != NULL && map != MAP_FAILED) {
if (strcmp(map, "SQLite format 3") != 0) {
if (cartesian_mode && !cartesian_extent_set) {
fprintf(stderr, "%s: --cartesian requires --cartesian-extent when decoding a PBF tile\n", fname);
exit(EXIT_FAILURE);
}
if (z >= 0) {
std::string s = std::string(map, st.st_size);
handle(s, z, x, y, to_decode, pipeline, stats, state);
Expand Down Expand Up @@ -282,6 +286,9 @@ void decode(char *fname, int z, unsigned x, unsigned y, std::set<std::string> co
}
}

// Auto-detect Cartesian mode from MBTiles metadata (CLI flags override)
set_cartesian_from_metadata(db);

Comment thread
idoaignostics marked this conversation as resolved.
if (z < 0) {
int within = 0;

Expand Down Expand Up @@ -504,6 +511,8 @@ int main(int argc, char **argv) {
{"stats", no_argument, 0, 'S'},
{"force", no_argument, 0, 'f'},
{"exclude-metadata-row", required_argument, 0, 'x'},
{"cartesian", no_argument, 0, '~'},
{"cartesian-extent", required_argument, 0, '~'},
{0, 0, 0, 0},
};

Expand All @@ -518,7 +527,8 @@ int main(int argc, char **argv) {
}
}

while ((i = getopt_long(argc, argv, getopt_str.c_str(), long_options, NULL)) != -1) {
int option_index = 0;
while ((i = getopt_long(argc, argv, getopt_str.c_str(), long_options, &option_index)) != -1) {
switch (i) {
case 0:
break;
Expand Down Expand Up @@ -555,6 +565,35 @@ int main(int argc, char **argv) {
exclude_meta.insert(optarg);
break;

case '~': {
const char *opt = long_options[option_index].name;
if (strcmp(opt, "cartesian") == 0) {
cartesian_mode = true;
projection = get_projection("cartesian");
} else if (strcmp(opt, "cartesian-extent") == 0) {
if (sscanf(optarg, "%lf,%lf,%lf,%lf",
&cartesian_extent[0], &cartesian_extent[1],
&cartesian_extent[2], &cartesian_extent[3]) != 4) {
fprintf(stderr, "Can't parse Cartesian extent: %s\n", optarg);
exit(EXIT_FAILURE);
}
double width = cartesian_extent[2] - cartesian_extent[0];
double height = cartesian_extent[3] - cartesian_extent[1];
if (width <= 0 || height <= 0) {
fprintf(stderr, "--cartesian-extent must have positive width and height (got %g,%g,%g,%g)\n",
cartesian_extent[0], cartesian_extent[1], cartesian_extent[2], cartesian_extent[3]);
exit(EXIT_FAILURE);
}
cartesian_extent_set = true;
cartesian_mode = true;
projection = get_projection("cartesian");
} else {
Comment thread
idoaignostics marked this conversation as resolved.
fprintf(stderr, "Unrecognized option: --%s\n", opt);
usage(argv);
}
break;
}

default:
usage(argv);
}
Expand Down
68 changes: 57 additions & 11 deletions main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1103,13 +1103,16 @@ void choose_first_zoom(long long *file_bbox, std::vector<struct reader> &readers
// If the bounding box extends off the plane on either side,
// a feature wrapped across the date line, so the width of the
// bounding box is the whole world.
if (file_bbox[0] < 0) {
file_bbox[0] = 0;
file_bbox[2] = (1LL << 32) - 1;
}
if (file_bbox[2] > (1LL << 32) - 1) {
file_bbox[0] = 0;
file_bbox[2] = (1LL << 32) - 1;
// (Not applicable for Cartesian mode where coordinates don't wrap.)
if (!cartesian_mode) {
if (file_bbox[0] < 0) {
file_bbox[0] = 0;
file_bbox[2] = (1LL << 32) - 1;
}
if (file_bbox[2] > (1LL << 32) - 1) {
file_bbox[0] = 0;
file_bbox[2] = (1LL << 32) - 1;
}
}
if (file_bbox[1] < 0) {
file_bbox[1] = 0;
Expand Down Expand Up @@ -2300,14 +2303,14 @@ int read_input(std::vector<source> &sources, char *fname, int maxzoom, int minzo

double minlat = 0, minlon = 0, maxlat = 0, maxlon = 0, midlat = 0, midlon = 0;

tile2lonlat(midx, midy, maxzoom, &minlon, &maxlat);
tile2lonlat(midx + 1, midy + 1, maxzoom, &maxlon, &minlat);
projection->unproject(midx, midy, maxzoom, &minlon, &maxlat);
projection->unproject(midx + 1, midy + 1, maxzoom, &maxlon, &minlat);

midlat = (maxlat + minlat) / 2;
midlon = (maxlon + minlon) / 2;

tile2lonlat(file_bbox[0], file_bbox[1], 32, &minlon, &maxlat);
tile2lonlat(file_bbox[2], file_bbox[3], 32, &maxlon, &minlat);
projection->unproject(file_bbox[0], file_bbox[1], 32, &minlon, &maxlat);
projection->unproject(file_bbox[2], file_bbox[3], 32, &maxlon, &minlat);

if (midlat < minlat) {
midlat = minlat;
Expand Down Expand Up @@ -2485,6 +2488,7 @@ int main(int argc, char **argv) {
std::map<std::string, std::string> attribute_descriptions;
int exclude_all = 0;
int read_parallel = 0;
bool projection_explicitly_set = false;
int files_open_at_start;
json_object *filter = NULL;

Expand Down Expand Up @@ -2514,6 +2518,8 @@ int main(int argc, char **argv) {

{"Projection of input", 0, 0, 0},
{"projection", required_argument, 0, 's'},
{"cartesian", no_argument, 0, '~'},
{"cartesian-extent", required_argument, 0, '~'},

Comment thread
idoaignostics marked this conversation as resolved.
{"Zoom levels", 0, 0, 0},
{"maximum-zoom", required_argument, 0, 'z'},
Expand Down Expand Up @@ -2697,6 +2703,27 @@ int main(int argc, char **argv) {
}
} else if (strcmp(opt, "use-attribute-for-id") == 0) {
attribute_for_id = optarg;
} else if (strcmp(opt, "cartesian") == 0) {
if (projection_explicitly_set) {
fprintf(stderr, "%s: --cartesian cannot be combined with --projection/-s\n", argv[0]);
exit(EXIT_FAILURE);
}
cartesian_mode = true;
projection = get_projection("cartesian");
} else if (strcmp(opt, "cartesian-extent") == 0) {
if (projection_explicitly_set) {
fprintf(stderr, "%s: --cartesian-extent cannot be combined with --projection/-s\n", argv[0]);
exit(EXIT_FAILURE);
}
if (sscanf(optarg, "%lf,%lf,%lf,%lf",
&cartesian_extent[0], &cartesian_extent[1],
&cartesian_extent[2], &cartesian_extent[3]) != 4) {
fprintf(stderr, "%s: Can't parse Cartesian extent --%s=%s\n", argv[0], opt, optarg);
exit(EXIT_FAILURE);
}
cartesian_extent_set = true;
cartesian_mode = true;
projection = get_projection("cartesian");
} else {
fprintf(stderr, "%s: Unrecognized option --%s\n", argv[0], opt);
exit(EXIT_FAILURE);
Expand Down Expand Up @@ -2971,7 +2998,12 @@ int main(int argc, char **argv) {
break;

case 's':
if (cartesian_mode) {
fprintf(stderr, "%s: --projection/-s cannot be combined with --cartesian/--cartesian-extent\n", argv[0]);
exit(EXIT_FAILURE);
}
set_projection_or_exit(optarg);
projection_explicitly_set = true;
break;

case 'S':
Expand Down Expand Up @@ -3046,6 +3078,20 @@ int main(int argc, char **argv) {
}
}

if (cartesian_mode) {
if (!cartesian_extent_set) {
fprintf(stderr, "%s: --cartesian requires --cartesian-extent=minx,miny,maxx,maxy\n", argv[0]);
exit(EXIT_FAILURE);
}
double width = cartesian_extent[2] - cartesian_extent[0];
double height = cartesian_extent[3] - cartesian_extent[1];
if (width <= 0 || height <= 0) {
fprintf(stderr, "%s: --cartesian-extent must have positive width and height (got %g,%g,%g,%g)\n",
argv[0], cartesian_extent[0], cartesian_extent[1], cartesian_extent[2], cartesian_extent[3]);
exit(EXIT_FAILURE);
}
}

if (additional[A_HILBERT]) {
encode_index = encode_hilbert;
decode_index = decode_hilbert;
Expand Down
Loading