Skip to content

Commit a03304d

Browse files
refactor(runway): collapse orchestrator into single-service domain (#272)
## Summary ### Why? Runway only ever had a single service, so the per-domain `gateway/` + `orchestrator/` split wrapped it in an empty service layer. Making the runway domain *be* the service removes that indirection and matches how runway actually works — a consumer-only landing service with no gateway. ### What? Collapses the `orchestrator` service segment out of the runway domain: - `runway/orchestrator/controller/*` → `runway/controller/*` - `api/runway/orchestrator/{proto,protopb}` → `api/runway/{proto,protopb}` - `example/runway/orchestrator/{client,server}` → `example/runway/{client,server}` The proto package becomes `uber.runway` (was `uber.runway.orchestrator`) and the RPC service `RunwayOrchestrator` becomes `Runway`. Server/client identifiers, metric scope, and the `ServiceName` ping field follow suit. Build tooling (`tool/proto`, `Makefile` targets, `docker-compose.yml`, `Dockerfile`) and docs (`CLAUDE.md`, `runway/README.md`, runway workflow RFC) are updated, documenting the single-service-domain pattern. ## Test Plan - ✅ `make build` - ✅ `make proto` + `make gazelle` (idempotent) - ✅ runway tests: `//runway/...`, `//api/runway/...`, `//example/runway/...` Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent bbb00d7 commit a03304d

29 files changed

Lines changed: 456 additions & 457 deletions

BUILD.bazel

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ load("@gazelle//:def.bzl", "gazelle")
1010
# gazelle:resolve go github.com/uber/submitqueue/api/base/change/protopb //api/base/change/protopb
1111
# gazelle:resolve go github.com/uber/submitqueue/api/base/mergestrategy/protopb //api/base/mergestrategy/protopb
1212
# gazelle:resolve go github.com/uber/submitqueue/api/runway/messagequeue/protopb //api/runway/messagequeue/protopb
13-
# gazelle:resolve go github.com/uber/submitqueue/api/runway/orchestrator/protopb //api/runway/orchestrator/protopb
13+
# gazelle:resolve go github.com/uber/submitqueue/api/runway/protopb //api/runway/protopb
1414
# gazelle:resolve go github.com/uber/submitqueue/api/submitqueue/gateway/protopb //api/submitqueue/gateway/protopb
1515
# gazelle:resolve go github.com/uber/submitqueue/api/submitqueue/orchestrator/protopb //api/submitqueue/orchestrator/protopb
1616
# gazelle:resolve go github.com/uber/submitqueue/api/stovepipe/protopb //api/stovepipe/protopb

CLAUDE.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ submitqueue/ # repo root (Go module github.com/uber/submi
3939
├── api/ # Published wire contracts (cross-domain/external)
4040
│ ├── submitqueue/{gateway,orchestrator}/{proto,protopb}/ # RPC (proto)
4141
│ ├── stovepipe/{proto,protopb}/ # single-service RPC (proto) — no service segment yet
42+
│ ├── runway/{proto,protopb}/ # RPC (proto) — single-service domain, no service segment
4243
│ └── runway/messagequeue/ # external queue contracts (proto + protojson)
4344
├── platform/ # SHARED cross-domain packages — no domain deps
4445
│ ├── errs/, metrics/, consumer/, http/
@@ -52,20 +53,23 @@ submitqueue/ # repo root (Go module github.com/uber/submi
5253
│ └── core/ # SubmitQueue-internal shared infra (consumer wiring, request, topickey, …)
5354
├── stovepipe/ # Stovepipe domain (single Ping-only service for now)
5455
│ └── controller/ # Business logic (currently just Ping); entity/extension/core added as it grows
56+
├── runway/ # Runway domain (single service — the domain *is* the service)
57+
│ └── controller/ # Runway service controllers (consumes the merge queues; no gateway/orchestrator split)
5558
├── tool/ # Development and CI tooling
5659
├── example/
5760
│ ├── submitqueue/ # Runnable SubmitQueue servers/clients + Docker Compose
58-
│ └── stovepipe/ # Runnable Stovepipe servers/clients
61+
│ ├── stovepipe/ # Runnable Stovepipe servers/clients
62+
│ └── runway/ # Runnable Runway server/client + Docker Compose
5963
├── test/
6064
│ ├── e2e/submitqueue/ # End-to-end tests (full stack)
6165
│ ├── integration/ # Integration tests (platform/, submitqueue/, stovepipe/, …)
6266
│ └── testutil/ # Test utilities (ComposeStack, MySQL helpers)
6367
└── doc/ # Documentation
6468
```
6569

66-
The `platform/` tree holds code reused across domains (infrastructure, shared entities, shared extension contracts). Each **domain** (`submitqueue/`, `stovepipe/`, …) grows into the same internal layout (`gateway/`, `orchestrator/`, `entity/`, `extension/`, `core/`); a domain's own `core/` (e.g. `submitqueue/core/`) holds infra shared only between that domain's services. A domain may start smallerStovepipe is currently a single Ping-only service with just `controller/` (and a service-segment-free `api/stovepipe/`), adding the other layers as it gains real behavior.
70+
The `platform/` tree holds code reused across domains (infrastructure, shared entities, shared extension contracts). A multi-service **domain** (e.g. `submitqueue/`) keeps the same internal layout (`gateway/`, `orchestrator/`, `entity/`, `extension/`, `core/`); a domain's own `core/` (e.g. `submitqueue/core/`) holds infra shared only between that domain's services. A **single-service domain** collapses that splitthe domain *is* the service, so its controllers live directly under the domain root (e.g. `runway/controller/`, `stovepipe/controller/`) with no `gateway/`/`orchestrator/` segment, and its wire contract is service-segment-free (`api/{domain}/`). `runway` is a consumer-only landing service with no gateway; `stovepipe` is currently a single Ping-only service that can grow the other layers (`entity/`, `extension/`, `core/`) as it gains real behavior.
6771

68-
The `api/` tree holds **published** wire contracts — those depended on from outside the owning domain. RPC contracts live at `api/{domain}/{service}/` (`proto/` for `.proto` sources, `protopb/` for committed generated Go); a service package may hold multiple `.proto` files, all generating into the same `protopb/`. External message-queue contracts live at `api/{domain}/messagequeue/` (see Message Queue Contracts below). Internal queue contracts do **not** go here — they live under `{domain}/core/messagequeue/`.
72+
The `api/` tree holds **published** wire contracts — those depended on from outside the owning domain. RPC contracts live at `api/{domain}/{service}/` (`proto/` for `.proto` sources, `protopb/` for committed generated Go); for a single-service domain the service segment is dropped, so the contract lives directly at `api/{domain}/` (e.g. `api/runway/{proto,protopb}/`). A service package may hold multiple `.proto` files, all generating into the same `protopb/`. External message-queue contracts live at `api/{domain}/messagequeue/` (see Message Queue Contracts below). Internal queue contracts do **not** go here — they live under `{domain}/core/messagequeue/`.
6973

7074
### Platform notes
7175

@@ -86,7 +90,7 @@ Each service follows the same layout:
8690
└── {step}_test.go
8791
```
8892

89-
Wire contracts for a service live separately under `api/{domain}/{service}/` (see Project Layout): `proto/` holds `.proto` sources and `protopb/` holds the committed generated stubs.
93+
Wire contracts for a service live separately under `api/{domain}/{service}/` (see Project Layout): `proto/` holds `.proto` sources and `protopb/` holds the committed generated stubs. For a single-service domain the service root *is* the domain root (e.g. `runway/controller/`), and its wire contract lives at `api/{domain}/` (e.g. `api/runway/`) with no service segment.
9094

9195
### Controllers
9296

@@ -152,9 +156,9 @@ When in doubt, ask: *"If the next implementation were DynamoDB / Kafka / Bigtabl
152156
153157
Paths follow the directory layout: shared packages live under `platform/` at the repo root; domain code nests under `submitqueue/`, `stovepipe/`, and other domain folders.
154158
155-
- RPC Controllers: `github.com/uber/submitqueue/{domain}/{service}/controller` (e.g. `.../submitqueue/gateway/controller`)
156-
- Queue Controllers: `github.com/uber/submitqueue/{domain}/{service}/controller/{step}`
157-
- Proto (generated): `github.com/uber/submitqueue/api/{domain}/{service}/protopb`
159+
- RPC Controllers: `github.com/uber/submitqueue/{domain}/{service}/controller` (e.g. `.../submitqueue/gateway/controller`; single-service domains drop the `{service}` segment, e.g. `.../runway/controller`)
160+
- Queue Controllers: `github.com/uber/submitqueue/{domain}/{service}/controller/{step}` (single-service: `.../runway/controller/{step}`, e.g. `.../runway/controller/merge`)
161+
- Proto (generated): `github.com/uber/submitqueue/api/{domain}/{service}/protopb` (single-service: `.../api/{domain}/protopb`, e.g. `.../api/runway/protopb`)
158162
- Queue contracts: external `github.com/uber/submitqueue/api/{domain}/messagequeue`; internal `github.com/uber/submitqueue/{domain}/core/messagequeue`
159163
- Domain entities: `github.com/uber/submitqueue/{domain}/entity` (e.g. `.../submitqueue/entity`)
160164
- Domain extensions: `github.com/uber/submitqueue/{domain}/extension/{ext}[/{impl}]` (e.g. `.../submitqueue/extension/storage/mysql`)

Makefile

Lines changed: 25 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ STOVEPIPE_COMPOSE_FILE = example/stovepipe/docker-compose.yml
1919
STOVEPIPE_LOCAL_PROJECT = stovepipe
2020

2121
# Runway compose files
22-
RUNWAY_ORCHESTRATOR_COMPOSE_FILE = example/runway/orchestrator/server/docker-compose.yml
22+
RUNWAY_COMPOSE_FILE = example/runway/server/docker-compose.yml
2323

2424
# Fixed project name for local manual testing (tests use unique random names)
2525
RUNWAY_LOCAL_PROJECT = runway
@@ -35,7 +35,7 @@ GOIMPORTS_VERSION ?= v0.33.0
3535
# (the out_dir convention in tool/proto/BUILD.bazel) and copied back here. A
3636
# package may hold multiple .proto files (e.g. an RPC contract plus messagequeue
3737
# contracts); all generated stubs land in the same protopb/ dir.
38-
PROTO_PACKAGES = api/base/change api/base/mergestrategy api/base/messagequeue api/runway/messagequeue api/runway/orchestrator api/submitqueue/gateway api/submitqueue/orchestrator api/stovepipe
38+
PROTO_PACKAGES = api/base/change api/base/mergestrategy api/base/messagequeue api/runway/messagequeue api/runway api/submitqueue/gateway api/submitqueue/orchestrator api/stovepipe
3939

4040
# Set REPO_ROOT for docker-compose
4141
export REPO_ROOT := $(shell pwd)
@@ -51,7 +51,7 @@ define assert_clean
5151
fi
5252
endef
5353

54-
.PHONY: build build-all-linux build-runway-orchestrator-linux build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-linux check-gazelle check-mocks check-tidy clean clean-proto deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-fmt lint-license local-init-runway-queue-schema local-runway-orchestrator-start local-runway-orchestrator-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway-orchestrator run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
54+
.PHONY: build build-all-linux build-runway-linux build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-linux check-gazelle check-mocks check-tidy clean clean-proto deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-fmt lint-license local-init-runway-queue-schema local-runway-start local-runway-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-logs local-stovepipe-start local-stovepipe-stop mocks proto query-deps query-targets run-client-runway run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
5555

5656

5757
build: ## Build all services and examples
@@ -60,16 +60,16 @@ build: ## Build all services and examples
6060
@echo "Build complete!"
6161

6262
# Build Linux binaries required for Docker containers
63-
build-all-linux: build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-linux build-runway-orchestrator-linux ## Build all Linux binaries for Docker
63+
build-all-linux: build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-linux build-runway-linux ## Build all Linux binaries for Docker
6464
@echo "All Linux binaries ready for Docker"
6565

66-
build-runway-orchestrator-linux: ## Build Runway orchestrator Linux binary for Docker
67-
@echo "Building Runway orchestrator Linux binary for Docker..."
68-
@$(BAZEL) build --platforms=@rules_go//go/toolchain:linux_amd64 //example/runway/orchestrator/server:orchestrator
66+
build-runway-linux: ## Build Runway Linux binary for Docker
67+
@echo "Building Runway Linux binary for Docker..."
68+
@$(BAZEL) build --platforms=@rules_go//go/toolchain:linux_amd64 //example/runway/server:runway
6969
@mkdir -p .docker-bin
70-
@cp -f bazel-bin/example/runway/orchestrator/server/orchestrator_/orchestrator .docker-bin/runway-orchestrator 2>/dev/null || \
71-
cp -f bazel-bin/example/runway/orchestrator/server/orchestrator .docker-bin/runway-orchestrator
72-
@echo "Runway orchestrator Linux binary ready at .docker-bin/runway-orchestrator"
70+
@cp -f bazel-bin/example/runway/server/runway_/runway .docker-bin/runway 2>/dev/null || \
71+
cp -f bazel-bin/example/runway/server/runway .docker-bin/runway
72+
@echo "Runway Linux binary ready at .docker-bin/runway"
7373

7474
build-submitqueue-gateway-linux: ## Build Gateway Linux binary for Docker
7575
@echo "Building Gateway Linux binary for Docker..."
@@ -221,23 +221,23 @@ local-init-runway-queue-schema: ## Apply queue schema only (mysql-queue) for Run
221221
done
222222
@echo "✅ Runway queue schema applied successfully"
223223

224-
local-runway-orchestrator-start: build-runway-orchestrator-linux ## Start Runway orchestrator locally (orchestrator + MySQL queue)
225-
@echo "Starting Runway orchestrator with compose..."
226-
@$(COMPOSE) -f $(RUNWAY_ORCHESTRATOR_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) up -d --build --wait
224+
local-runway-start: build-runway-linux ## Start Runway locally (runway + MySQL queue)
225+
@echo "Starting Runway with compose..."
226+
@$(COMPOSE) -f $(RUNWAY_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) up -d --build --wait
227227
@echo "Applying queue schema to mysql-queue (no Runway app schema)..."
228228
@$(MAKE) -s local-init-runway-queue-schema
229229
@echo ""
230-
@echo "✅ Runway orchestrator is running!"
230+
@echo "✅ Runway is running!"
231231
@echo ""
232-
@$(COMPOSE) -f $(RUNWAY_ORCHESTRATOR_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) ps
232+
@$(COMPOSE) -f $(RUNWAY_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) ps
233233
@echo ""
234-
@echo "Runway orchestrator gRPC port: $$(docker port $(RUNWAY_LOCAL_PROJECT)-orchestrator-service-1 8080 2>/dev/null | cut -d: -f2 || echo 'unknown')"
235-
@echo "MySQL Queue port: $$(docker port $(RUNWAY_LOCAL_PROJECT)-mysql-queue-1 3306 2>/dev/null | cut -d: -f2 || echo 'unknown')"
234+
@echo "Runway gRPC port: $$(docker port $(RUNWAY_LOCAL_PROJECT)-runway-service-1 8080 2>/dev/null | cut -d: -f2 || echo 'unknown')"
235+
@echo "MySQL Queue port: $$(docker port $(RUNWAY_LOCAL_PROJECT)-mysql-queue-1 3306 2>/dev/null | cut -d: -f2 || echo 'unknown')"
236236

237-
local-runway-orchestrator-stop: ## Stop Runway orchestrator service
238-
@echo "Stopping Runway orchestrator services..."
239-
@$(COMPOSE) -f $(RUNWAY_ORCHESTRATOR_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) down
240-
@echo "Runway orchestrator services stopped."
237+
local-runway-stop: ## Stop Runway service
238+
@echo "Stopping Runway services..."
239+
@$(COMPOSE) -f $(RUNWAY_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) down
240+
@echo "Runway services stopped."
241241

242242
local-submitqueue-logs: ## View logs from all running services
243243
@$(COMPOSE) -f $(COMPOSE_FILE) -p $(SUBMITQUEUE_LOCAL_PROJECT) logs -f
@@ -304,7 +304,7 @@ local-stop: ## Stop all services (keep data)
304304
@echo "Stopping all services..."
305305
@$(COMPOSE) -f $(COMPOSE_FILE) -p $(SUBMITQUEUE_LOCAL_PROJECT) down
306306
@$(COMPOSE) -f $(STOVEPIPE_COMPOSE_FILE) -p $(STOVEPIPE_LOCAL_PROJECT) down
307-
@$(COMPOSE) -f $(RUNWAY_ORCHESTRATOR_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) down
307+
@$(COMPOSE) -f $(RUNWAY_COMPOSE_FILE) -p $(RUNWAY_LOCAL_PROJECT) down
308308
@echo "Services stopped. Data volumes preserved."
309309

310310
local-stovepipe-logs: ## View logs from the running Stovepipe service
@@ -363,9 +363,9 @@ run-client-submitqueue-orchestrator:
363363
run-client-stovepipe:
364364
@$(BAZEL) run //example/stovepipe/client:stovepipe -- -addr $(or $(SERVER_ADDR),localhost:8083) -message "$(or $(MESSAGE),ping)"
365365

366-
# Run runway orchestrator client (connects to any running runway orchestrator service)
367-
run-client-runway-orchestrator:
368-
@$(BAZEL) run //example/runway/orchestrator/client:orchestrator -- -addr $(or $(SERVER_ADDR),localhost:8086) -message "$(or $(MESSAGE),ping)"
366+
# Run runway client (connects to any running runway service)
367+
run-client-runway:
368+
@$(BAZEL) run //example/runway/client:runway -- -addr $(or $(SERVER_ADDR),localhost:8086) -message "$(or $(MESSAGE),ping)"
369369

370370
run-queue-admin: ## Run queue-admin CLI (use ARGS to pass arguments, e.g. make run-queue-admin ARGS="list-topics")
371371
@$(BAZEL) run //platform/extension/messagequeue/mysql/ctl -- $(ARGS)

0 commit comments

Comments
 (0)