Annotate ledger-api protos with google.api.http for proto-derived OpenAPI (3.5.9 baseline) - #4
Open
monsieurleberre wants to merge 5 commits into
Open
Conversation
…e options Demonstrates Option 4 from the C# SDK proposal appendix: annotate Canton's .proto files with google.api.http so the JSON HTTP surface can be derived from the protos directly, bypassing tapir's translation layer. Scope (one service end-to-end): - community/ledger-api-proto/.../v2/command_service.proto: imports google/api/annotations.proto and adds an option (google.api.http) block to each of the three RPCs (SubmitAndWait, SubmitAndWaitForTransaction, SubmitAndWaitForReassignment), matching the existing tapir routes. - All v2 .proto files (22) plus value.proto: add option go_package next to the existing csharp_namespace/java_package options. Required by every off-the-shelf OpenAPI generator in the ecosystem. Harmless to existing C#/Java/Scala codegen, which does not read this option. Verified with `buf build`: clean compile. Verified end-to-end OpenAPI generation with buf.build/grpc-ecosystem/openapiv2:v2.28.0: the output fixes all four type-system defects catalogued in the C# SDK proposal (numbered duplicate schemas, single-key oneOf wrappers, inline duplicated enums, untyped DAML payload fields). See peacefulstudio/documentation/dev-funding/openApi/upstream-fork-experiment.md for the full write-up.
Builds on the CommandService experiment by annotating every other v2 RPC
the JSON API exposes over HTTP, plus adding go_package to the admin/,
testing/, and interactive/ sub-packages so off-the-shelf OpenAPI
generators can resolve the full proto closure.
Annotated services (route source: community/ledger/ledger-json-api Js*Service.scala):
v2/
- CommandSubmissionService: Submit, SubmitReassignment
- CommandCompletionService: CompletionStream
- ContractService: GetContract
- EventQueryService: GetEventsByContractId
- PackageService: ListPackages, GetPackage,
GetPackageStatus, ListVettedPackages
- StateService: GetActiveContracts, GetActiveContractsPage,
GetConnectedSynchronizers, GetLedgerEnd,
GetLatestPrunedOffsets
- UpdateService: GetUpdates, GetUpdateByOffset,
GetUpdateById, GetUpdatesPage
- VersionService: GetLedgerApiVersion
v2/admin/
- IdentityProviderConfigService: 5 RPCs
- PackageManagementService: UploadDarFile, ValidateDarFile,
UpdateVettedPackages
- PartyManagementService: 7 HTTP-exposed RPCs
- UserManagementService: 9 RPCs
v2/interactive/
- InteractiveSubmissionService: 6 RPCs
Each annotation matches the existing tapir route (method + path + body
mapping) so the proto-derived OpenAPI is comparable to the current
tapir-emitted document.
Scope notes:
- Server-streaming RPCs (CompletionStream, GetActiveContracts,
GetUpdates) get only the canonical POST/list mapping; the WebSocket
GET form is a JSON-API-only convenience that has no representation in
google.api.http and remains in the AsyncAPI doc.
- Legacy JSON-only paths (/v2/updates/flats|trees|transaction-*,
/v2/authenticated-user, /v2/definitions/*) are not annotated — they
either lack a 1:1 proto RPC or are non-gRPC custom endpoints.
- Non-HTTP-exposed RPCs (TimeService, CommandInspectionService,
ParticipantPruningService, PackageManagementService.ListKnownPackages,
PartyManagementService.UpdatePartyIdentityProviderId) are deliberately
left unannotated.
Verified locally with `buf build community/ledger-api-proto/src/main/protobuf`:
clean compile.
The JSON API exposes this as GET with a JSON body containing a required
EventFormat message — non-standard REST, and rejected by
grpc-ecosystem/openapiv2 ("must not set request body when http method
is GET"). Switching to POST is the canonical mapping for a request
with a non-trivial body, and is the single intentional deviation from
the existing tapir route for the proto-derived OpenAPI.
Mirrors the experiment-evidence pattern from the single-service experiment, scaled to the full HTTP API: - openapi-3.0/openapi.yaml — OpenAPI 3.0.3 YAML generated via buf.build/community/google-gnostic-openapi:v0.7.0; same major version as the existing community/ledger/ledger-json-api/.../openapi.yaml, so the two can be diffed directly. - swagger-2.0/canton-ledger-api.swagger.yaml — Swagger 2.0 YAML via buf.build/grpc-ecosystem/openapiv2:v2.28.0; same plugin as the upstream CommandService experiment, merged across every annotated service. - coverage-comparison/ — sorted path inventories and the diff between the JSON API spec and the proto-derived spec, after normalizing path-parameter names (proto field names ↔ tapir placeholder names). 9 paths-only-in-JSON-API (all deliberate skips: 1 custom JSON-only endpoint + 8 deprecated legacy aliases), 0 paths-only-in-proto. - reproduce.sh — one-shot regen of every artefact in this directory. Internal to the Peaceful Studio fork. To be deleted before any upstream PR to digital-asset/canton.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-opens the annotation work from #2 against a pristine
release-line-3.5base.#2 was squash-merged by accident on 2026-07-26 — auto-merge had been armed on it since 2026-07-10 and fired the moment a base retarget made it mergeable. That left
release-line-3.5carrying our annotations instead of mirroring upstream. The branch has since been reset to pristine upstream1f9a483455307cfb7875ff120ff803500663eeac; the pre-reset head is preserved atarchive/release-line-3.5-at-cc150af.This PR is the same work, presented as a reviewable diff. Auto-merge is deliberately off.
Base and scope
release-line-3.5@1f9a4834— proto tree byte-identical to Maven3.5.9experiment/google-api-http-full-coverage@472b36a(archived atarchive/google-api-http-full-coverage-3.5.9)1f9a4834,behind_by: 0Content is metadata only:
option go_packageon 34 protos, plusimport "google/api/annotations.proto"andoption (google.api.http)on the 14 service protos — 52 annotated RPCs, zero new RPCs, messages, or fields.What changed relative to #2
The branch is rebased from its 3.4.11-era base onto
release-line-3.5@1f9a4834. At that baselineGetUpdatesPageandGetActiveContractsPageexist and are annotated, so all 52 RPCs are live — under the 3.4.11 pin a consumer could only carry 50.The design, the intentional deviations, and the deliberate omissions are otherwise unchanged from #2, which remains the reference for the rationale.
Known deviation
StateService.GetActiveContractsPageis annotatedPOSTwhile Canton's own JSON Ledger API exposes it asGET. GET-with-body is rejected bygrpc-ecosystem/openapiv2and disallowed by OpenAPI generally, and the request carries a requiredEventFormatmessage that does not flatten to query params. This is the only verb deviation in the spec.Why this fork exists
Advocacy-only; never fetched by any build.
peacefulstudio/canton-ledger-api-csharpexports these annotations as a standalone patch set over the range1f9a4834..472b36aand applies them to SHA-256-pinned Maven artifacts, so its supply chain depends on those two commit SHAs rather than on whether this branch is merged.Supersedes #2.