Skip to content

Commit d43bb75

Browse files
committed
feat(stovepipe): wire gateway Ingest to the request log
## Summary ### Why? The Stovepipe gateway minted a SPID and published to the queue but persisted nothing, so a freshly ingested commit had no observable record until the orchestrator processed it. SubmitQueue's gateway records an `accepted` status synchronously on entry and owns the request log end to end; this brings the Stovepipe gateway to the same shape now that the entity stores exist. ### What? Wires the gateway to the gateway-owned `RequestLogStore` (persist + consume); identity stays the synthetic SPID and there is no new read RPC or proto change. - `Ingest` now records an `accepted` `RequestLog` synchronously (written straight to storage, before publishing to the start topic) so the status is visible the moment the RPC returns. A log-write failure aborts before anything is published. - Adds a gateway `log`-topic consumer (`stovepipe/gateway/controller/log`) that persists `RequestLog` entries — the gateway is the sole writer of the request log; the orchestrator only publishes log events for it to persist. - Adds the `log` topic key to `stovepipe/core/topickey`. - Rewires the example gateway server: opens the app DB (storage + SPID counter) and the queue, builds a registry that publishes `start` and consumes `log`, registers `Ingest` behind an InvalidArgument interceptor, and runs the log consumer with graceful shutdown. - Adds `local-init-stovepipe-schemas` (storage + counter to mysql-app, queue to mysql-queue) and points the gateway / full-stack local start targets at it. The `IngestController` and the log consumer both depend on the narrow `RequestLogStore` interface rather than the full `Storage` facade, since that is all either needs. ## Test Plan - `bazel test //stovepipe/...` — all targets pass (new gateway log consumer + updated ingest tests) - `bazel build //stovepipe/... //example/stovepipe/...` - `make fmt`, `make lint-license`, `make gazelle` — clean / idempotent
1 parent caf7e61 commit d43bb75

11 files changed

Lines changed: 568 additions & 41 deletions

File tree

Makefile

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ define assert_clean
5353
fi
5454
endef
5555

56-
.PHONY: build build-all-linux build-runway-orchestrator-linux build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-gateway-linux build-stovepipe-orchestrator-linux check-gazelle check-mocks check-tidy clean clean-proto deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-fmt lint-license local-init-runway-queue-schema local-runway-orchestrator-start local-runway-orchestrator-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-init-stovepipe-queue-schema local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-gateway-start local-stovepipe-orchestrator-start local-stovepipe-start mocks proto query-deps query-targets run-client-runway-orchestrator run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe-gateway run-client-stovepipe-orchestrator run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
56+
.PHONY: build build-all-linux build-runway-orchestrator-linux build-submitqueue-gateway-linux build-submitqueue-orchestrator-linux build-stovepipe-gateway-linux build-stovepipe-orchestrator-linux check-gazelle check-mocks check-tidy clean clean-proto deps e2e-test fmt gazelle integration-test integration-test-submitqueue-consumer integration-test-extensions integration-test-submitqueue-gateway integration-test-submitqueue-orchestrator license-fix lint lint-fmt lint-license local-init-runway-queue-schema local-runway-orchestrator-start local-runway-orchestrator-stop local-submitqueue-clean local-submitqueue-gateway-start local-submitqueue-gateway-stop local-init-submitqueue-schemas local-init-stovepipe-queue-schema local-init-stovepipe-schemas local-submitqueue-logs local-submitqueue-orchestrator-start local-submitqueue-orchestrator-stop local-submitqueue-ps local-submitqueue-restart local-submitqueue-start local-stop local-stovepipe-gateway-start local-stovepipe-orchestrator-start local-stovepipe-start mocks proto query-deps query-targets run-client-runway-orchestrator run-client-submitqueue-gateway run-client-submitqueue-orchestrator run-client-stovepipe-gateway run-client-stovepipe-orchestrator run-queue-admin test test-no-cache tidy tidy-bazel tidy-go help
5757

5858

5959
build: ## Build all services and examples
@@ -224,13 +224,31 @@ local-init-submitqueue-schemas: ## Manually apply all database schemas
224224
@echo "✅ All schemas applied successfully"
225225

226226
local-init-stovepipe-queue-schema: ## Apply queue schema only (mysql-queue) for Stovepipe compose stacks
227-
@echo "Applying queue schema to mysql-queue (Stovepipe; no app storage/counter schema yet)..."
227+
@echo "Applying queue schema to mysql-queue (Stovepipe; orchestrator example does not use app storage yet)..."
228228
@for file in platform/extension/messagequeue/mysql/schema/*.sql; do \
229229
echo " - Applying $$(basename $$file)..."; \
230230
docker exec -i $(STOVEPIPE_LOCAL_PROJECT)-mysql-queue-1 mysql -uroot -proot submitqueue < $$file 2>&1 | grep -v "Using a password" || true; \
231231
done
232232
@echo "✅ Stovepipe queue schema applied successfully"
233233

234+
local-init-stovepipe-schemas: ## Apply all Stovepipe database schemas (storage + counter to mysql-app, queue to mysql-queue)
235+
@echo "Applying storage schema to mysql-app..."
236+
@for file in stovepipe/extension/storage/mysql/schema/*.sql; do \
237+
echo " - Applying $$(basename $$file)..."; \
238+
docker exec -i $(STOVEPIPE_LOCAL_PROJECT)-mysql-app-1 mysql -uroot -proot submitqueue < $$file 2>&1 | grep -v "Using a password" || true; \
239+
done
240+
@echo "Applying counter schema to mysql-app..."
241+
@for file in platform/extension/counter/mysql/schema/*.sql; do \
242+
echo " - Applying $$(basename $$file)..."; \
243+
docker exec -i $(STOVEPIPE_LOCAL_PROJECT)-mysql-app-1 mysql -uroot -proot submitqueue < $$file 2>&1 | grep -v "Using a password" || true; \
244+
done
245+
@echo "Applying queue schema to mysql-queue..."
246+
@for file in platform/extension/messagequeue/mysql/schema/*.sql; do \
247+
echo " - Applying $$(basename $$file)..."; \
248+
docker exec -i $(STOVEPIPE_LOCAL_PROJECT)-mysql-queue-1 mysql -uroot -proot submitqueue < $$file 2>&1 | grep -v "Using a password" || true; \
249+
done
250+
@echo "✅ All Stovepipe schemas applied successfully"
251+
234252
local-init-runway-queue-schema: ## Apply queue schema only (mysql-queue) for Runway compose stacks
235253
@echo "Applying queue schema to mysql-queue (Runway; consumes the merge queues)..."
236254
@for file in platform/extension/messagequeue/mysql/schema/*.sql; do \
@@ -331,8 +349,8 @@ local-stovepipe-logs: ## View logs from all running Stovepipe services
331349
local-stovepipe-start: build-stovepipe-gateway-linux build-stovepipe-orchestrator-linux ## Start full Stovepipe stack (gateway + orchestrator + MySQL)
332350
@echo "Starting full Stovepipe stack with compose..."
333351
@$(COMPOSE) -f $(STOVEPIPE_STACK_COMPOSE_FILE) -p $(STOVEPIPE_LOCAL_PROJECT) up -d --build --wait
334-
@echo "Applying queue schema to mysql-queue (no Stovepipe app schema yet)..."
335-
@$(MAKE) -s local-init-stovepipe-queue-schema
352+
@echo "Applying database schemas..."
353+
@$(MAKE) -s local-init-stovepipe-schemas
336354
@echo ""
337355
@echo "✅ Full Stovepipe stack is running!"
338356
@echo ""
@@ -366,8 +384,8 @@ local-stovepipe-orchestrator-start: build-stovepipe-orchestrator-linux ## Start
366384
local-stovepipe-gateway-start: build-stovepipe-gateway-linux ## Start Stovepipe gateway locally (gateway + 2 MySQL databases)
367385
@echo "Starting Stovepipe gateway with compose..."
368386
@$(COMPOSE) -f $(STOVEPIPE_GATEWAY_COMPOSE_FILE) -p $(STOVEPIPE_LOCAL_PROJECT) up -d --build --wait
369-
@echo "Applying queue schema to mysql-queue (no Stovepipe app schema yet)..."
370-
@$(MAKE) -s local-init-stovepipe-queue-schema
387+
@echo "Applying database schemas..."
388+
@$(MAKE) -s local-init-stovepipe-schemas
371389
@echo ""
372390
@echo "✅ Stovepipe gateway is running!"
373391
@echo ""

example/stovepipe/gateway/server/BUILD.bazel

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,23 @@ go_library(
1212
visibility = ["//visibility:private"],
1313
deps = [
1414
"//api/stovepipe/gateway/protopb",
15+
"//platform/consumer",
16+
"//platform/errs",
17+
"//platform/errs/generic",
18+
"//platform/errs/mysql",
19+
"//platform/extension/counter/mysql",
20+
"//platform/extension/messagequeue",
21+
"//platform/extension/messagequeue/mysql",
22+
"//stovepipe/core/topickey",
23+
"//stovepipe/extension/storage/mysql",
1524
"//stovepipe/gateway/controller",
25+
"//stovepipe/gateway/controller/log",
26+
"@com_github_go_sql_driver_mysql//:mysql",
1627
"@com_github_uber_go_tally//:tally",
1728
"@org_golang_google_grpc//:grpc",
29+
"@org_golang_google_grpc//codes",
1830
"@org_golang_google_grpc//reflection",
31+
"@org_golang_google_grpc//status",
1932
"@org_uber_go_zap//:zap",
2033
],
2134
)

example/stovepipe/gateway/server/docker-compose.yml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Docker Compose for Stovepipe gateway manual testing
22
#
33
# Mirrors example/submitqueue/gateway/server/docker-compose.yml: same MySQL pair, healthchecks,
4-
# env wiring, and startup ordering. The Stovepipe gateway binary is Ping-only today
5-
# and does not open MySQL yet; variables are set so future work matches SubmitQueue.
4+
# env wiring, and startup ordering. The gateway opens the app DB for the request log and SPID
5+
# counter, and the queue DB to publish ingest requests and consume the log topic.
66
#
77
# IMPORTANT: Before running compose, build the Linux binary:
88
# make build-stovepipe-gateway-linux
@@ -12,7 +12,8 @@
1212
# Quick start:
1313
# make local-stovepipe-gateway-start
1414
#
15-
# After `up`, only the queue schema is applied (`local-init-stovepipe-queue-schema`).
15+
# After `up`, all schemas are applied (`local-init-stovepipe-schemas`): storage + counter to
16+
# mysql-app and the queue schema to mysql-queue.
1617

1718
services:
1819
# Application Database - Stores business data (requests, counters, etc.)

example/stovepipe/gateway/server/main.go

Lines changed: 174 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ package main
1616

1717
import (
1818
"context"
19+
"database/sql"
1920
"errors"
2021
"fmt"
2122
"net"
@@ -25,25 +26,44 @@ import (
2526
"syscall"
2627
"time"
2728

29+
_ "github.com/go-sql-driver/mysql"
2830
"github.com/uber-go/tally"
2931
pb "github.com/uber/submitqueue/api/stovepipe/gateway/protopb"
32+
"github.com/uber/submitqueue/platform/consumer"
33+
"github.com/uber/submitqueue/platform/errs"
34+
genericerrs "github.com/uber/submitqueue/platform/errs/generic"
35+
mysqlerrs "github.com/uber/submitqueue/platform/errs/mysql"
36+
mysqlcounter "github.com/uber/submitqueue/platform/extension/counter/mysql"
37+
extqueue "github.com/uber/submitqueue/platform/extension/messagequeue"
38+
queueMySQL "github.com/uber/submitqueue/platform/extension/messagequeue/mysql"
39+
"github.com/uber/submitqueue/stovepipe/core/topickey"
40+
mysqlstorage "github.com/uber/submitqueue/stovepipe/extension/storage/mysql"
3041
"github.com/uber/submitqueue/stovepipe/gateway/controller"
42+
logctrl "github.com/uber/submitqueue/stovepipe/gateway/controller/log"
3143
"go.uber.org/zap"
3244
"google.golang.org/grpc"
45+
"google.golang.org/grpc/codes"
3346
"google.golang.org/grpc/reflection"
47+
"google.golang.org/grpc/status"
3448
)
3549

36-
// GatewayServer wraps the controller and implements the gRPC service interface.
50+
// GatewayServer wraps the controllers and implements the gRPC service interface.
3751
type GatewayServer struct {
3852
pb.UnimplementedStovepipeGatewayServer
39-
pingController *controller.PingController
53+
pingController *controller.PingController
54+
ingestController *controller.IngestController
4055
}
4156

4257
// Ping delegates to the controller.
4358
func (s *GatewayServer) Ping(ctx context.Context, req *pb.PingRequest) (*pb.PingResponse, error) {
4459
return s.pingController.Ping(ctx, req)
4560
}
4661

62+
// Ingest delegates to the controller.
63+
func (s *GatewayServer) Ingest(ctx context.Context, req *pb.IngestRequest) (*pb.IngestResponse, error) {
64+
return s.ingestController.Ingest(ctx, req)
65+
}
66+
4767
func main() {
4868
code := 0
4969
if err := run(); err != nil {
@@ -105,19 +125,142 @@ func run() error {
105125
metricsWgDone.Wait()
106126
}()
107127

108-
// Create gRPC server
109-
grpcServer := grpc.NewServer()
128+
// Open application database connection.
129+
// Docker Compose healthchecks ensure MySQL is ready before service starts.
130+
appDSN := os.Getenv("MYSQL_DSN")
131+
if appDSN == "" {
132+
return fmt.Errorf("MYSQL_DSN environment variable is required")
133+
}
134+
appDB, err := sql.Open("mysql", appDSN)
135+
if err != nil {
136+
return fmt.Errorf("failed to open app database: %w", err)
137+
}
138+
defer appDB.Close()
139+
140+
// Initialize counter from shared app database connection. The ingest controller uses it to
141+
// mint a SPID per ingest request.
142+
cnt := mysqlcounter.NewCounter(appDB, scope.SubScope("counter"))
143+
144+
// Open queue database connection
145+
queueDSN := os.Getenv("QUEUE_MYSQL_DSN")
146+
if queueDSN == "" {
147+
return fmt.Errorf("QUEUE_MYSQL_DSN environment variable is required")
148+
}
149+
queueDB, err := sql.Open("mysql", queueDSN)
150+
if err != nil {
151+
return fmt.Errorf("failed to open queue database: %w", err)
152+
}
153+
defer queueDB.Close()
154+
155+
// Initialize queue
156+
mysqlQueue, err := queueMySQL.NewQueue(queueMySQL.Params{
157+
DB: queueDB,
158+
Logger: logger,
159+
MetricsScope: scope.SubScope("queue"),
160+
})
161+
if err != nil {
162+
return fmt.Errorf("failed to create queue: %w", err)
163+
}
164+
defer mysqlQueue.Close()
165+
166+
logger.Info("initialized dependencies",
167+
zap.String("app_dsn", appDSN),
168+
zap.String("queue_dsn", queueDSN),
169+
)
110170

111-
// Create ping controller and wrap it for gRPC
171+
// Subscriber name for the log-topic consumer. It must be unique per running
172+
// instance: SubscriberName identifies a subscriber for partition leases, so
173+
// two gateway processes on the same host (sharing HOSTNAME) would otherwise
174+
// contend for the same lease. Append the PID to keep co-located instances
175+
// distinct; the PID is stable for the life of the process. Offset tracking
176+
// stays keyed on the shared ConsumerGroup ("gateway-log"), not this name.
177+
// Falls back to a time-seeded name when HOSTNAME is unset (e.g. local runs).
178+
hostname := os.Getenv("HOSTNAME")
179+
if hostname == "" {
180+
hostname = fmt.Sprintf("stovepipe-gateway-%d", time.Now().Unix())
181+
}
182+
subscriberName := fmt.Sprintf("%s-%d", hostname, os.Getpid())
183+
184+
// Build the topic registry. The gateway publishes ingest requests to the start of the
185+
// orchestrator pipeline (TopicKeyStart) — publish-only. It additionally consumes the log topic
186+
// (TopicKeyLog): the gateway is the sole writer of the request log, persisting entries that the
187+
// orchestrator publishes there.
188+
registry, err := consumer.NewTopicRegistry([]consumer.TopicConfig{
189+
{Key: topickey.TopicKeyStart, Name: "start", Queue: mysqlQueue},
190+
{
191+
Key: topickey.TopicKeyLog,
192+
Name: "log",
193+
Queue: mysqlQueue,
194+
Subscription: extqueue.DefaultSubscriptionConfig(
195+
subscriberName, "gateway-log",
196+
),
197+
},
198+
})
199+
if err != nil {
200+
return fmt.Errorf("failed to create topic registry: %w", err)
201+
}
202+
203+
// Create gRPC server with a unary interceptor that translates user-input
204+
// validation errors (anything in the chain that matches controller.ErrInvalidRequest)
205+
// into codes.InvalidArgument so gRPC clients can distinguish bad input from
206+
// infrastructure failures. Other errors pass through unchanged.
207+
grpcServer := grpc.NewServer(grpc.UnaryInterceptor(
208+
func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
209+
resp, err := handler(ctx, req)
210+
if err != nil && controller.IsInvalidRequest(err) {
211+
return nil, status.Error(codes.InvalidArgument, err.Error())
212+
}
213+
return resp, err
214+
},
215+
))
216+
217+
// Initialize storage from the shared app database connection. The ingest controller writes the
218+
// accepted entry to the request log directly; the log consumer (registered below) is the sole
219+
// persister of request log entries published by the orchestrator.
220+
store, err := mysqlstorage.NewStorage(appDB, scope.SubScope("storage"))
221+
if err != nil {
222+
return fmt.Errorf("failed to create storage: %w", err)
223+
}
224+
requestLogStore := store.GetRequestLogStore()
225+
226+
// Create controllers and wrap them for gRPC
112227
pingController := controller.NewPingController(logger, scope)
113-
srv := &GatewayServer{
114-
pingController: pingController,
228+
ingestController := controller.NewIngestController(logger.Sugar(), scope, cnt, requestLogStore, registry)
229+
gatewayServer := &GatewayServer{
230+
pingController: pingController,
231+
ingestController: ingestController,
115232
}
116-
pb.RegisterStovepipeGatewayServer(grpcServer, srv)
233+
234+
pb.RegisterStovepipeGatewayServer(grpcServer, gatewayServer)
117235

118236
// Register reflection service for debugging with grpcurl
119237
reflection.Register(grpcServer)
120238

239+
// Create the queue consumer and register the log controller. The gateway is
240+
// the sole persister of the request log: the orchestrator publishes entries
241+
// to the log topic and this consumer writes them to storage.
242+
logConsumer := consumer.New(logger.Sugar(), scope.SubScope("consumer"), registry,
243+
errs.NewClassifierProcessor(
244+
// Storage (stovepipe/extension/storage/mysql) and queue (platform/extension/messagequeue/mysql)
245+
// both run on the same MySQL driver, so a single classifier covers
246+
// errors surfaced from either backend.
247+
genericerrs.Classifier,
248+
mysqlerrs.Classifier,
249+
),
250+
)
251+
252+
logController := logctrl.NewController(logger.Sugar(), scope, requestLogStore, topickey.TopicKeyLog, "gateway-log")
253+
if err := logConsumer.Register(logController); err != nil {
254+
return fmt.Errorf("failed to register log controller: %w", err)
255+
}
256+
257+
if err := logConsumer.Start(ctx); err != nil {
258+
// The error can also be a result of a context cancellation due to SIGINT or SIGTERM.
259+
// This is expected, just propagate it.
260+
return fmt.Errorf("failed to start log consumer: %w", err)
261+
}
262+
logger.Info("log consumer started")
263+
121264
// Listen on configurable port
122265
port := os.Getenv("PORT")
123266
if port == "" {
@@ -137,9 +280,11 @@ func run() error {
137280
serverErrCh <- grpcServer.Serve(listener)
138281
}()
139282

140-
// Wait for interrupt signal or server critical error
141-
// If interruption is signaled, gracefully stop the server
142-
// If an error happens during shutdown, return the actual error, not the context cancellation error
283+
// Wait for interrupt signal or server critical error.
284+
// If interruption is signaled, gracefully stop the server.
285+
// If the server exits with an error, cancel the context to signal the consumer.
286+
// After this, stop the consumer.
287+
// If an error happens during shutdown, return the actual error, not the context cancellation error.
143288
var serverErr error
144289
select {
145290
case <-ctx.Done():
@@ -155,10 +300,27 @@ func run() error {
155300
serverErr = <-serverErrCh
156301
case serverErr = <-serverErrCh:
157302
fmt.Println("Shutting down stovepipe gateway server due to critical GRPC server error...")
303+
304+
// Cancel the context to signal cancellation to the queue consumer
305+
cancel()
158306
}
159307

160308
if serverErr != nil {
161-
err = fmt.Errorf("GRPC server exited with error: %w", serverErr)
309+
serverErr = fmt.Errorf("GRPC server exited with error: %w", serverErr)
310+
}
311+
312+
// Stop the consumer with a 30s timeout; by this time the context should be
313+
// cancelled and the processing threads may already be exiting; recollect them.
314+
errStop := logConsumer.Stop(30000)
315+
if errStop != nil {
316+
errStop = fmt.Errorf("failed to stop consumer: %w", errStop)
317+
}
318+
319+
if errStop != nil || serverErr != nil {
320+
// Override context cancellation error with the shutdown error. The server
321+
// error is the primary/root failure, so it leads; the consumer-stop error
322+
// is secondary cleanup.
323+
err = errors.Join(serverErr, errStop)
162324
}
163325

164326
return err

stovepipe/core/topickey/topickey.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,7 @@ const (
2828
// TopicKeyBatch is the pipeline stage where validated commits are aggregated, since the last
2929
// known green, into a contiguous validation batch.
3030
TopicKeyBatch TopicKey = "batch"
31+
// TopicKeyLog is the gateway-owned sink topic for append-only request log events. The
32+
// orchestrator publishes log entries here; the gateway is the sole consumer that persists them.
33+
TopicKeyLog TopicKey = "log"
3134
)

stovepipe/gateway/controller/BUILD.bazel

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ go_library(
1818
"//platform/metrics",
1919
"//stovepipe/core/topickey",
2020
"//stovepipe/entity",
21+
"//stovepipe/extension/storage",
2122
"@com_github_uber_go_tally//:tally",
2223
"@org_uber_go_zap//:zap",
2324
],
@@ -39,6 +40,7 @@ go_test(
3940
"//platform/extension/messagequeue/mock",
4041
"//stovepipe/core/topickey",
4142
"//stovepipe/entity",
43+
"//stovepipe/extension/storage/mock",
4244
"@com_github_stretchr_testify//assert",
4345
"@com_github_stretchr_testify//require",
4446
"@com_github_uber_go_tally//:tally",

0 commit comments

Comments
 (0)