Skip to content

sdk: add multi-language read-only SDKs for serviceability, telemetry, and revdist programs#2776

Merged
snormore merged 26 commits intomainfrom
snor/serviceability-telemetry-sdks
Feb 5, 2026
Merged

sdk: add multi-language read-only SDKs for serviceability, telemetry, and revdist programs#2776
snormore merged 26 commits intomainfrom
snor/serviceability-telemetry-sdks

Conversation

@snormore
Copy link
Contributor

@snormore snormore commented Feb 1, 2026

Summary of Changes

  • Add read-only Go, Python, and TypeScript SDKs for the serviceability, telemetry, and revenue distribution (revdist) onchain programs, each with typed deserialization, RPC clients, PDA derivation, and Rust-generated binary fixture tests for cross-language consistency
  • Add shared borsh-incremental library (Go, Python, TypeScript) providing cursor-based Borsh deserialization with backward-compatible trailing field defaults via tryRead* methods
  • Add enum string types for serviceability enums across all three languages, with a shared enum_strings.json fixture and bidirectional completeness checks to keep variants in sync
  • Add compat tests that deserialize live mainnet accounts via RPC (gated behind env vars)
  • Add CI workflow for SDK tests and npm/PyPI publish workflows for all three programs
  • Add sdk/README.md documenting the testing strategy and how to extend fixtures
  • Closes sdk: python implementation for reading serviceability and telemetry program state #2461

Testing Verification

  • Rust-generated binary fixtures verified by Go, Python, and TypeScript for all account types (serviceability: 12 fixtures, telemetry: 2, revdist: 5)
  • Comprehensive borsh-incremental unit tests: 75 (Go), 114 (Python), 105 (TypeScript)
  • Cross-language enum string fixture tests with bidirectional completeness check
  • PDA derivation tests in all three languages
  • Compat tests pass against live mainnet RPC for serviceability, telemetry, and revdist
  • make sdk-test and make sdk-compat-test pass

@snormore snormore force-pushed the snor/serviceability-telemetry-sdks branch from 87ea711 to adfee07 Compare February 1, 2026 16:12
@snormore snormore marked this pull request as ready for review February 2, 2026 13:55
@snormore snormore force-pushed the snor/serviceability-telemetry-sdks branch 4 times, most recently from 0d56ff2 to 0ea0bb7 Compare February 3, 2026 18:16
Copy link
Contributor

@karl-dz karl-dz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beast of a PR, but deserialization looks safe

Add convenience constructors that use default program IDs and RPC URLs
for each environment (mainnet-beta, testnet, devnet, localnet):

- Go: NewForEnv(env), NewMainnetBeta(), NewTestnet(), etc.
- Python: Client.from_env(env), Client.mainnet_beta(), etc.
- TypeScript: Client.forEnv(env), Client.mainnetBeta(), etc.

Also adds config.go to revdist Go SDK with ProgramID and RPC URL maps,
and updates serviceability Go constructors to create RPC clients
internally (matching telemetry SDK pattern).
Add assertions in from_bytes() methods to verify that all bytes up to
STRUCT_SIZE are accounted for during deserialization. This catches bugs
like missed fields or incorrect offset math, similar to Go's
TestStructSizes approach.
@snormore snormore force-pushed the snor/serviceability-telemetry-sdks branch from 8ab0cd6 to 3bf2ac2 Compare February 4, 2026 18:27
Rename package from doublezero-borsh-incremental to borsh-incremental
for both Python (PyPI) and TypeScript (npm). Add GitHub Actions release
workflow triggered by borsh-incremental/v* tags.
Add example fetch CLIs for serviceability, telemetry, and revdist SDKs
in Go, Python, and TypeScript. Each example demonstrates fetching and
displaying onchain data.

Also add sdk/Makefile with targets for running examples and tests.
Add DefensiveReader wrapper to borsh-incremental that uses try_read for
all operations, returning zero/empty defaults on insufficient data.
This matches Go's ByteReader behavior and makes Python/TypeScript SDKs
resilient to schema changes when new fields are added.

Update all SDKs (serviceability, telemetry, revdist) to use
DefensiveReader instead of IncrementalReader.

Also fix AccessPass to handle all enum variants (SOLANA_RPC,
SOLANA_MULTICAST_PUBLISHER, SOLANA_MULTICAST_SUBSCRIBER, OTHERS).
Add tests that verify DefensiveReader returns zero/empty defaults when
data is truncated or missing, rather than throwing errors. This ensures
SDKs remain resilient when new trailing fields are added to onchain
structs.
Add getProgramData compat tests to Python and TypeScript that fetch
and deserialize all program accounts including AccessPass, ensuring
we catch deserialization issues across all account types and variants.
- Add bun workspace at sdk level for TypeScript cross-package deps
- Update telemetry examples to use serviceability for device discovery
- Get current epoch from DZ Ledger RPC instead of approximation
- Add bs58 dependency to revdist package
- Update package.json files to use source entry points for dev
- Use bunx for tsc in Makefile
- Add missing AccessPassTypeTag enum values to TypeScript
@snormore snormore merged commit 726487c into main Feb 5, 2026
24 of 25 checks passed
@snormore snormore deleted the snor/serviceability-telemetry-sdks branch February 5, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sdk: python implementation for reading serviceability and telemetry program state

2 participants