Skip to content

feat: datacap direct mode (in-process sidecar replacement)#168

Open
myleshorton wants to merge 3 commits intomainfrom
feat/datacap-direct-mode
Open

feat: datacap direct mode (in-process sidecar replacement)#168
myleshorton wants to merge 3 commits intomainfrom
feat/datacap-direct-mode

Conversation

@myleshorton
Copy link
Contributor

Summary

  • Moves datacap sidecar logic in-process as an optional DeviceUsageStore for VPS deployments
  • Introduces DataCapAPI / ReportSink interfaces to decouple connection tracking from backend (HTTP sidecar vs gRPC direct)
  • Extracts reportingCore from conn.go / packet_conn.go eliminating ~150 lines of duplication
  • Adds mTLS client credentials for gRPC auth with CA-verified server cert
  • Local proto types in dcpb/ mirror lantern-cloud's DataCapService to avoid cross-repo dependency
  • Multi-region OCI packer builds (iad, fra, nrt, sin)

Both modes coexist: --datacap-url (legacy sidecar) and --datacap-grpc-api (new direct) are mutually exclusive flags.

Companion PR: lantern-cloud feat/datacap-direct-mode (mTLS listener + VPS cert provisioning)

Test plan

  • go build ./... passes
  • go test -race ./tracker/datacap/... passes (10 new tests for DeviceUsageStore)
  • Integration test with mock gRPC server
  • Deploy to test proxy, verify datacap tracking without sidecar container
  • Verify sidecar mode (--datacap-url) still works unchanged

🤖 Generated with Claude Code

Replace the Docker sidecar with an in-process DeviceUsageStore that
handles device state caching, batch usage uploads, and throttle
decisions directly via gRPC. Both sidecar (--datacap-url) and direct
(--datacap-grpc-api) modes coexist; the sidecar path is unchanged.

Key changes:
- DataCapAPI / ReportSink interfaces decouple connection tracking from backend
- DeviceUsageStore with singleflight-deduplicated state sync
- reportingCore extracted from conn.go / packet_conn.go (~150 lines removed)
- mTLS client credentials for gRPC auth (CA-verified server cert)
- Local proto types in dcpb/ to avoid lantern-cloud dependency
- Multi-region OCI packer builds (iad, fra, nrt, sin)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings March 13, 2026 16:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a “direct mode” datacap reporting path that runs aggregation + batch upload in-process (via gRPC to the cloud API), while preserving the existing “sidecar over HTTP” legacy mode. It also expands OCI image builds to multiple regions.

Changes:

  • Refactor datacap connection reporting to use a ReportSink abstraction shared by HTTP sidecar and the new in-process store.
  • Add DeviceUsageStore (direct mode) with background batching, sync, and a gRPC-backed DataCapAPI implementation.
  • Update CLI flags and image build automation to support direct-mode configuration and multi-region OCI images.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tracker/datacap/tracker.go Tracker now targets a ReportSink and adds direct-mode lifecycle wiring.
tracker/datacap/reporting.go New shared reporting/throttling core used by both Conn and PacketConn.
tracker/datacap/conn.go Conn refactored to embed reportingCore and report via ReportSink.
tracker/datacap/packet_conn.go PacketConn refactored similarly to Conn using reportingCore.
tracker/datacap/api.go Introduces ReportSink and DataCapAPI abstractions + DTOs for batching/sync.
tracker/datacap/client.go HTTP client now implements ReportSink via Report().
tracker/datacap/device_store.go New in-process store implementing ReportSink, batching uploads + syncing state.
tracker/datacap/device_store_test.go Unit tests for store batching, sync behavior, concurrency, and shutdown.
tracker/datacap/integration_test.go Updates tests to use Sink instead of Client in configs.
tracker/datacap/dcpb/datacap.proto New proto mirroring lantern-cloud DataCapService for isolated dependency.
tracker/datacap/dcpb/datacap.pb.go Generated protobuf types for datacap service.
tracker/datacap/dcpb/datacap_grpc.pb.go Generated gRPC client/server interfaces for datacap service.
cmd/grpc_datacap.go gRPC DataCapAPI implementation + TLS/mTLS connection helper.
cmd/cmd_run.go Adds direct-mode CLI flags and wiring; enforces mode exclusivity.
deploy/packer/lantern-box.pkr.hcl Splits OCI builds into per-region sources with region-specific subnet/AD vars.
.github/workflows/build-images.yaml Updates workflow inputs, OCI env vars, manifest parsing, and Vault updates per region.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

myleshorton and others added 2 commits March 13, 2026 10:40
- Make Stop() idempotent with sync.Once (prevents double-close panic)
- Wait for background loop to exit before final upload (prevents race)
- Use caller's ctx deadline for shutdown, fresh context for final upload
- Check all flag parsing errors instead of discarding with _
- Validate all mTLS paths are non-empty before attempting to load certs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Take the reviewed fixes from main: legacy env var fallbacks, oracle-oci
builder expansion, Vault secret merging, and missing-region warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants