fix: wire OTelMiddleware as global HTTP middleware so traces appear in Jaeger

[intel352]

Summary

• OTelMiddleware.Process() was never called — the module was instantiated but had no path into the HTTP handler chain, so no traces were emitted.
• Added StandardHTTPRouter.AddGlobalMiddleware() which wraps the entire mux before dispatching, ensuring every request (health probes, metrics, pipeline triggers, static files) gets a trace span.
• Added observability.otel-middleware wiring hook (priority 100) that runs at startup and registers any OTelMiddleware found in the service registry as a global middleware on every StandardHTTPRouter.

Root cause

StandardHTTPRouter.ServeHTTP delegated directly to the inner serveMux with no outer middleware chain. Per-route middlewares (via AddRouteWithMiddleware) were available, but the OTEL middleware module had no wiring hook to attach itself — it just registered as a service and sat unused.

Changes

File	Change
module/http_router.go	Add globalMiddlewares []HTTPMiddleware field; AddGlobalMiddleware() method; ServeHTTP applies global chain before mux
plugins/observability/wiring.go	New wireOTelMiddleware hook at priority 100
plugins/observability/plugin.go	Register observability.otel-middleware in manifest WiringHooks
plugins/observability/plugin_test.go	Update counts: 6 module types, 5 wiring hooks

Test plan

• go test ./... — all packages pass
• Tag v0.2.4, update workflow-cloud to use it, cross-compile and deploy to minikube
• Hit endpoints via curl http://localhost:8080/...
• Confirm workflow-cloud appears in curl -s http://localhost:16686/api/services

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR wires the OpenTelemetry HTTP middleware into the request handling path by adding support for router-level “global” middleware and registering the OTEL middleware on all StandardHTTPRouter instances via an observability wiring hook, so inbound requests produce trace spans (e.g., visible in Jaeger).

Changes:

• Add StandardHTTPRouter.AddGlobalMiddleware() and apply globalMiddlewares around the entire mux in ServeHTTP.
• Add a new observability wiring hook (observability.otel-middleware, priority 100) to register OTelMiddleware globally on all routers.
• Update observability plugin manifest/tests to include the new wiring hook and module type counts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
module/http_router.go	Adds global middleware support and wraps the router’s mux with a global middleware chain in ServeHTTP.
plugins/observability/wiring.go	Introduces wireOTelMiddleware wiring hook to attach OTelMiddleware to all StandardHTTPRouters.
plugins/observability/plugin.go	Registers the new wiring hook name in the observability plugin manifest.
plugins/observability/plugin_test.go	Updates manifest/module type/wiring hook count expectations and hook name list.

[Copilot]

ServeHTTP holds r.mu.RLock for the entire request (due to the deferred RUnlock) and then calls middleware Process() and handler.ServeHTTP while the lock is held. This risks self-deadlock if any handler/middleware tries to mutate the router (e.g., AddRouteWithMiddleware/AddGlobalMiddleware) and also blocks hot-add route updates until the request completes. Consider copying serveMux and globalMiddlewares under the lock, releasing the lock, then building the middleware chain and serving the request without holding r.mu.

[Copilot]

AddGlobalMiddleware/ServeHTTP introduce new global middleware behavior, but there are existing router tests (module/http_test.go) that don't currently validate that global middlewares run (and run in the expected order) for matched routes and 404s. Adding unit tests for AddGlobalMiddleware + ServeHTTP would help prevent regressions in tracing and other cross-cutting middleware.

Suggested change

	defer r.mu.Unlock()
	r.globalMiddlewares = append(r.globalMiddlewares, mw)
	defer r.mu.Unlock()
	r.globalMiddlewares = append(r.globalMiddlewares, mw)
	// Rebuild the mux if we've already started so that newly added global
	// middlewares take effect for all subsequent requests (hot-add support).
	if r.serveMux != nil {
	r.rebuildMuxLocked()
	}

[Copilot]

wireOTelMiddleware currently does a nested iteration over app.SvcRegistry() (OTelMiddleware × routers). Even if the registry is usually small, this is avoidable and makes the hook harder to extend. Consider collecting the routers once (and possibly the OTel middlewares once) and then wiring them in a single pass.

Suggested change

	for _, svc := range app.SvcRegistry() {
	otelMw, ok := svc.(*module.OTelMiddleware)
	if !ok {
	continue
	}
	for _, routerSvc := range app.SvcRegistry() {
	router, ok := routerSvc.(*module.StandardHTTPRouter)
	if !ok {
	continue
	}
	router.AddGlobalMiddleware(otelMw)
	}
	}
	var (
	routers []*module.StandardHTTPRouter
	middlewares []*module.OTelMiddleware
	)
	// Collect all routers and OTel middleware instances in a single pass.
	for _, svc := range app.SvcRegistry() {
	if router, ok := svc.(*module.StandardHTTPRouter); ok {
	routers = append(routers, router)
	continue
	}
	if otelMw, ok := svc.(*module.OTelMiddleware); ok {
	middlewares = append(middlewares, otelMw)
	}
	}
	// Register each OTel middleware as global middleware on every router.
	for _, router := range routers {
	for _, otelMw := range middlewares {
	router.AddGlobalMiddleware(otelMw)
	}
	}