Skip to content

DO NOT MERGE — Integration with external mission provider ollebo.com (read-only JSON telemetry export) - #21

Open
mattiashem wants to merge 1 commit into
b14ckyy:developmentfrom
mattiashem:feat/json-telemetry-export
Open

DO NOT MERGE — Integration with external mission provider ollebo.com (read-only JSON telemetry export)#21
mattiashem wants to merge 1 commit into
b14ckyy:developmentfrom
mattiashem:feat/json-telemetry-export

Conversation

@mattiashem

Copy link
Copy Markdown

DO NOT MERGE — proposal / conversation starter

I know Kite is in feature-freeze for 1.0 and that only bugfix PRs are being merged. This is not a merge request. It's a worked example, in code, of what an integration with an external mission provider could look like — so there's something concrete to react to instead of an issue full of hand-waving. Happy for it to sit, be closed, or be picked up after 1.0. If there's no interest, just say so and I'll close it myself.

Disclosure up front: I build Ollebo, the service used as the example consumer here. I've tried to keep that bias out of the code — see "Why there's no Ollebo code in Kite" below.


What this adds

A read-only JSON telemetry export: one new Encoder (json) and one new OutputSink (http) inside the existing Telemetry Relay.

That's the whole idea. The relay already taps the app's own telemetry-* events backend-side, already has persisted config, a settings UI, connect/disconnect lifecycle, throughput stats and a reconfigure reconciler. So an external consumer can be added without touching a single producer — no changes to mavlink_proto, scheduler, or passive_telemetry — and it works for MAVLink, MSP and passive links alike, because it rides the shared normalized cache.

telemetry-* events ─► RelayHub tap ─► TelemetryCache ─► JsonEncoder ─► HttpSink ─► GET /api/v1/…
                                     (6 unified fields)  (rate-limited)  (snapshot + SSE)
Route Response
GET /api/v1/telemetry 200 application/json — most recent frame (503 before any telemetry arrives)
GET /api/v1/stream 200 text/event-stream — SSE, one data: record per frame
GET /api/v1/health 200 application/json{ ok, schema, missionId, hasData, streamClients }

Today the only telemetry egress is a binary FC protocol (LTM / MAVLink / CRSF / SmartPort). Anything that just wants "where is the aircraft, is it armed" has to implement a protocol parser first. This closes that gap.

Design notes (the non-obvious bits)

  • SSE, not WebSocket. The stream is one-way and read-only. A WebSocket handshake needs SHA-1 + base64 and client-frame unmasking, and tokio is built without net/rt-multi-thread, so axum/hyper/tungstenite aren't on the table without a real dependency change. SSE hand-rolls in ~40 lines on std::net::TcpListener — the same approach video/mjpeg_server.rs already takes. No new crates.
  • Loopback by default. A tracker relay binds 0.0.0.0 because reaching the LAN is its whole purpose. A telemetry API is different — it shouldn't be silently readable by everyone on a field network. LAN exposure is an explicit per-relay opt-in.
  • Rate-limited in the encoder. The relay paces on the attitude update, which on MAVLink can run at 10–50 Hz. JsonEncoder returns an empty Vec when called too soon and Relay::emit_set already early-returns on that, so no frame is written and the counters stay honest. Default 5 Hz.
  • A stalled consumer can't stall the relay. HttpSink::write runs on the Tauri event-listener thread that drives every relay's dispatch. So: per-client write timeout, dead clients dropped, no lock held across a socket write, and one thread per connection so a client that connects and goes silent can't block the accept loop.
  • Explicit DTO. The payload is a versioned camelCase struct, deliberately not a Serialize of the internal cache structs — those are snake_case internals that move with the frontend, and a public contract shouldn't be welded to them. armed is derived from armingFlags bit 2 so consumers don't decode a bitfield.
  • The mission id is a hard gate. Relay::build resolves it before constructing the sink, so with no mission id the relay is refused and no port is ever bound — off means off, not "serving untagged data". The UI flags the empty field, but the backend check is the authoritative one.

What Ollebo is, and how it fits

Ollebo is a platform for drone maps and live missions — "the all-in-one platform to host high-resolution maps, analyze assets, and stream real-time telemetry". Two halves:

  • Maps — upload imagery from a flight, it's processed into high-resolution orthophotos and elevation models you can annotate, share and compare over time.
  • Missions — a mission is a planned flight. You sketch the survey polygon on the map, set altitude and overlap, and it generates waypoints you can export to PX4 or ArduPilot. While you fly, telemetry streams onto a live mission map, either public or locked to a project team.

The base features are free — makers and drone hobbyists get the free tier, and the professionals paying for hosting fund it. It's built on open standards (MAVLink, MQTT), EU-hosted, and it is under heavy active development right now, so the integration surface is still moving and I'd rather shape it around what a real GCS actually wants to emit than guess.

Ingest is a plain HTTP PUT /event/<mission-key>; Ollebo replays it on the mission's live map over its own SSE stream. It already accepts MAVLink via MAVProxy on UDP 14550 — so in principle Kite's existing mavlink + udp relay can already feed it, with MAVProxy in the middle. The point of this PR is that a JSON export removes that middle entirely, for Ollebo and for anything else.

Why there's no Ollebo code in Kite

Deliberately: nothing in this diff mentions Ollebo. What's added is a vendor-neutral JSON export. Feeding Ollebo is then a ~40-line stdlib Python bridge that reads Kite's SSE stream and PUTs each frame — it lives in the design doc (docs/dev/active/JSON_TELEMETRY_API.md), not in the app.

That's the shape I'd argue for regardless of who's asking: Kite stays a GCS that serves telemetry, and consumers do their own pushing. Putting a third-party endpoint and a credential inside the app would be a much bigger ask, and a worse trade for the project.

Set a relay to JSON/HTTP with a mission id, connect, run the bridge, and the flight appears live on the Ollebo mission map.

If there's interest

I can start testing this properly — flying it against real hardware, on the Ollebo side and the Kite side, and reporting back. Ollebo's free tier means anyone here can try it without paying for anything. And if the JSON schema is wrong for what Kite wants to expose, I'd much rather change Ollebo than bend Kite around it.

Honest status

  • The Rust in this PR has never been compiled. I don't have a working toolchain on this machine (no cargo, and the Tauri Linux system deps aren't installed), so cargo check has not run against encoders/json.rs or output/http.rs. Treat the backend as reviewed-by-eye, not verified. I'll fix whatever a build turns up if this is worth pursuing.
  • The frontend does pass npm run -s check clean (549 files, 0 errors, 0 warnings).
  • It has not been driven end-to-end against a vehicle yet: the mission-id gate, the snapshot/SSE endpoints, the rate limit, the LAN bind guard and the stalled-client path are all unverified at runtime.
  • i18n keys added to en / de / fr.

Given all that, please read this as a design proposal with code attached, not as finished work.

🤖 Generated with Claude Code

Adds a `json` encoder and an `http` output sink to the Telemetry Relay, so
external services can consume live telemetry without implementing a binary FC
protocol. No producer code is touched — the relay already taps the app's own
`telemetry-*` events, so this works for MAVLink, MSP and passive links alike.

- encoders/json.rs: explicit versioned camelCase DTO (not the internal cache
  structs, so the public contract isn't coupled to internals), stamped with a
  mission id and a monotonic seq. Self-rate-limits by returning an empty frame
  set, which Relay::emit_set already treats as "nothing to send" — otherwise a
  50 Hz MAVLink attitude stream would emit 50 JSON frames/s.
- output/http.rs: loopback-by-default HTTP server. GET /api/v1/telemetry
  (snapshot), /api/v1/stream (SSE), /api/v1/health. Hand-rolled on
  std::net::TcpListener, as tokio is built without net/rt-multi-thread; same
  approach as video/mjpeg_server.rs. Per-client write timeout, per-connection
  thread, and no lock held across a socket write, so one stalled consumer can't
  stall the relay dispatch thread.
- The mission id is a hard gate: Relay::build resolves it before constructing
  the sink, so with none configured no port is ever bound.
- HTTP output requires the JSON protocol (SSE-wrapping binary frames would be
  garbage). The reverse is allowed: JSON out serial/tcp/udp is valid NDJSON.

DO NOT MERGE — proposal only; see the PR description.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mattiashem
mattiashem marked this pull request as draft July 14, 2026 09:38
@mattiashem
mattiashem marked this pull request as ready for review July 14, 2026 09:47
@b14ckyy

b14ckyy commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Interesting extension and I will definitely reject it right away. I will have a closer look on it. Its planned to have external APIs for other applications to connect to anyway so something like this could be a good starting point.

As you correctly stated, it will not be merged before final 1.0 But I will definitely consider it when the API implementation comes.

One note beforehand: this json data stream could be added as a relay output instead so the user has control over when its active and when not. At the moment the relay feature is gated by a live connection, so it won't output anything for a log replay but that can be changed if necessary.

@b14ckyy b14ckyy added the enhancement New feature or request label Jul 14, 2026
@mattiashem

Copy link
Copy Markdown
Author

I keep my fork up to date and see if I can get a full working test. I already have a mavProxy that I use with other GC tools, but it's nice to use the API directly at ollebo.com.

@b14ckyy

b14ckyy commented Jul 14, 2026

Copy link
Copy Markdown
Owner

If you go into your own fork under actions, there SHOULD be the release build action (afaik it gets copied with the fork).
So you can directly build it entirely in Github from your dev branch and need no local build environment.

@b14ckyy
b14ckyy changed the base branch from master to development July 29, 2026 07:10
@b14ckyy b14ckyy added this to the 1.1 milestone Jul 29, 2026
@b14ckyy

b14ckyy commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Build status

Ran your branch through the full check set on Windows (touched both new files to
force a real rebuild rather than trusting a warm cache):

  • cargo check — 0 errors, 0 warnings
  • cargo clippy — 7 warnings, all pre-existing, none in your files
  • cargo test --no-run — clean
  • npm run check — 549 files, 0 errors, 0 warnings

The "never compiled" caveat can come off. Windows only; nothing here is
platform-specific.

What I want changed

Wildcard CORS. Access-Control-Allow-Origin: * means any website open in any
browser on that machine can read live position while Kite runs, and there's no
Host check, so DNS rebinding reaches it without the LAN box ticked. Read-only is
true of the API, but position and armed state are exactly the sensitive part.
Drop the header if browser consumers aren't the target; otherwise an explicit
origin list plus a Host check.

The mission ID isn't a gate. Any non-empty string passes. "Genuinely off, not
merely serving untagged data" reads as a security property and will be treated as
one. It's a required field — call it that. Related: your doc warns against putting
the Ollebo key in it, which suggests the name invites the mistake. sourceId
would invite it less.

Lifecycle. The server appears on connect and vanishes on disconnect. A
consumer that has to poll through connection-refused isn't much better off than
one parsing MAVLink, so this decides whether the feature is actually useful.
Fixing it means lifting the server out of the relay lifecycle — bigger than the
rest of this PR.

The blocking concern is mine, not yours

dispatch() walks all relays sequentially on the Tauri listener thread while
holding the relay lock. There is no worker per relay. Our own TcpSink sets no
write timeout at all, so a client that stops reading blocks it indefinitely —
worse than your 2 s bound. You handled the case better than the code you
extended.

What remains on your side: a stalled SSE client still holds that thread for up to
2 s per client, serially, and HTTP consumers are long-lived and arbitrary in a way
tracker clients aren't. I'm fixing the shared path for 1.0; the per-relay worker
rework lands in 1.1, alongside this.

Smaller

  • No SSE keep-alive. Pacing is on attitude, so the stream goes silent when
    telemetry stops and intermediaries time it out. A periodic : ping fixes it.
  • Dead clients are only reaped on write, so zombies accumulate while nothing flows.
  • No cap on concurrent connections, one thread each.
  • pending() only clears with an SSE client attached, so a snapshot-polling
    consumer shows "waiting" forever in the UI.
  • The accept thread holds the port for up to 50 ms after Drop. The reconciler's
    120 ms pause covers it — noting it so it isn't rediscovered later.

Timing

1.0 is in freeze, so this can't land now. I moved the target to development,
which is correct under the branching model we adopted last week. The CORS
question is the one I'd like your thinking on first, since it decides what the API is for.

@b14ckyy

b14ckyy commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Oh and one small detail:

"JSON" in the output channel picker is not really fitting as a name there since its not clear what it actually does. and since its a universal API maybe we should call it like that. Something like "Kite API" or "Telemetry API"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants