fix(server): rebuild client-aware AI filter factories on config reload

[Artemon-line]

Motivation

AI filters that use the shared SubRequestClient did not receive an updated response-body ceiling after a dynamic config reload.

At startup, register_ai_filters registers the client-aware factories (openai_file_resolve, openai_web_search, anthropic_web_search, and the compact / file-search callout filters), each capturing a clone of the startup SubRequestClient. On reload, reload_pipelines correctly builds an updated_client from the new body_limits.max_response_bytes and threads it into pipeline resolution — but the already-registered factory closures keep handing the frozen startup client to every filter they build. The reload looks successful while the new callout response ceiling never reaches the client-aware AI filters.

Closes #639.

Approach

Introduce ReloadableSubRequestClient, a shared Arc<ArcSwap<SubRequestClient>> handle that mirrors the Arc<ArcSwap<_>> reload idiom already used in filters/src/routing/.

• filters/src/subrequest.rs (new): the handle. new, load() (shared Arc, no client clone — used for rollback pointer identity), current() (owned snapshot at the construction boundary that needs ownership), store().
• Factories (filters/src/register.rs): each client-aware factory now captures a clone of the handle (a cheap Arc bump) and calls handle.current() each time it builds a filter, so a filter rebuilt during reload observes the swapped-in client.
• Reload (server/src/reload.rs): reload_pipelines stores the new client into the handle before resolve_pipelines, so rebuilt filters see the new ceiling. On a build failure it restores the exact prior Arc, leaving live pipelines and their client unchanged.
• Bootstrap (server/src/server.rs, watcher.rs, commands.rs, lib.rs): the handle is created once at startup and threaded through the watcher into the reload path; --validate/--dump build through the same handle.

This covers every filter registered via from_config_with_client, not just openai_file_resolve / openai_web_search.

Testing

• Unit (filters/src/subrequest.rs): handle store/load/shared-cell behavior, asserted via Arc::ptr_eq (the ceiling field is private, so pointer identity is the observable).
• Reload unit (server/src/reload.rs): a valid reload swaps the client in the handle (!ptr_eq); a failed reload (bad filter name) restores the original client (ptr_eq).
• Functional integration (tests/integration/tests/suite/reload_response_ceiling.rs): drives a real reload through the server bootstrap + file watcher via start_reloadable_proxy. A Files API stub serves small metadata but a 4 KiB content body, so a mid-range ceiling passes the metadata callout while rejecting the content callout.
  • tightening_ceiling_on_reload_rejects_oversized_file: start with a large ceiling → probe returns 200; reload with a small ceiling → probe returns 413 with the file_resolve_error envelope. Without the fix this stays 200 (the rebuilt filter keeps the frozen startup client), so the test has teeth.
  • relaxing_ceiling_on_reload_admits_previously_rejected_file: the reverse direction, proving the swap is not one-way.
• make lint passes (clippy -D warnings, fmt --check, and all cargo xtask doc/coverage checks); server unit tests and the two new functional tests are green.

Open questions

• Issue bug(server): rebuild client-aware AI filter factories on config reload #639 also asks to document which subrequest settings are reloadable vs restart-required (e.g. the connector/pool is restart-only; the response ceiling is now reloadable). Happy to add that doc in this PR or a follow-up — guidance welcome.
• Coordination with #695 (centralize subrequest-client construction): this PR keeps client construction where it is and makes it swappable; if bug(server): honor runtime subrequest circuit-breaker configuration #695 lands first the handle can wrap the centralized constructor.

Debugging the bug with LLDB

Create a config in a quiet directory (not /tmp — the watcher watches the config's parent directory and reacts to any file change in it, so /tmp activity retriggers reloads):

# ~/praxis-debug/reload-bug.yaml
body_limits:
  max_response_bytes: 1024

listeners:
  - name: ai-gateway
    address: "127.0.0.1:8080"
    filter_chains: [main]

filter_chains:
  - name: main
    filters:
      - filter: openai_file_resolve
        files_api_url: "http://127.0.0.1:9999"
        allow_private_files_api_url: true
        allow_pre_security_callout: true
        on_missing: reject
        timeout_ms: 10000
      - filter: router
        routes:
          - path: "/v1/responses"
            cluster: "backend"
      - filter: load_balancer
        clusters:
          - name: "backend"
            endpoints: ["127.0.0.1:9998"]

cargo build --bin praxis-ai
rust-lldb ./target/debug/praxis-ai -- --config ~/praxis-debug/reload-bug.yaml

No HTTP request is needed — starting the proxy and editing the config file is enough, because resolve_pipelines rebuilds every filter from the registry factories on each reload.

Before (on main, without this fix)

The factory captures a frozen clone of the startup client, so client at the factory line is a plain SubRequestClient whose ceiling never changes:

# filters/src/register.rs, line where the closure calls:
#   FileResolveFilter::from_config_with_client(config, client.clone())
(lldb) breakpoint set --file register.rs --line 246
(lldb) run
# stops at startup:
(lldb) frame variable client        # max_response_bytes = 1024
(lldb) continue
# now edit the config: change max_response_bytes 1024 -> 65536, save.
# the breakpoint hits again on reload:
(lldb) frame variable client        # BUG: still 1024 (stale startup clone)

After (with this fix)

The factory now reads a swappable handle (client.current()), so at the factory line client is the handle wrapper (not directly readable). Observe the ceiling where it is a plain value instead — in the reload path and in the rebuilt filter:

# server/src/reload.rs, the reload_client.store(...) line:
(lldb) breakpoint set --file reload.rs --line 71
# apis/src/openai/responses/file_resolve/mod.rs, first line of build():
(lldb) breakpoint set --file file_resolve/mod.rs --line 195
(lldb) run
(lldb) continue
# after editing the config to 65536 and saving, the reload fires:
# 1) reload path builds the new client:
(lldb) frame variable new_ceiling updated_client   # both = 65536  (read before stepping; updated_client is moved on this line)
(lldb) continue
# 2) the rebuilt filter receives it:
(lldb) frame variable subrequest_client            # max_response_bytes = 65536  (was 1024 on main)

Caveats: updated_client is move-consumed on reload.rs:71, so read it at the stop, before stepping past; these require a debug (unoptimized) build (under --release the locals can be optimized away); line numbers are tied to this revision, so the code line is named in each comment.

🤖 Generated with Claude Code

[praxis-bot-app]

Unsigned commits: 5c08ac5. Please sign your commits.

[praxis-bot-app]

AI tool authorship detected:

• 7534d98: Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Sorry, this project does not accept commits authored by tools as valid.
Commits need to be authored by and signed-off by the human(s) responsible for the PR, with their name and contact.

[praxis-bot]

Sound fix. The ReloadableSubRequestClient wrapper follows the established Arc<ArcSwap<_>> reload idiom, the store-before-build ordering in the reload path is correct (factories read from the handle during resolve_pipelines, so the swap must precede the build), and the rollback on failure restores the exact prior Arc so live pipelines and their client are unaffected. The integration tests are well-designed -- testing both tightening and relaxing directions with a Files API stub that isolates the ceiling check to the content callout. Two medium convention items noted below.

[praxis-bot]

[Medium] Per project convention, inside mod tests the ordering is: imports, test functions, then test utilities (with a full-width // Test Utilities separator). Move client() below the three #[test] functions.

Also, the inline comment at line 123 (// Smoke check: ...) should be removed or converted to a tracing::debug! call -- project convention prohibits inline comments in test function bodies.

[praxis-bot]

[Medium] Two convention items across the new test code:

1. Doc comments on test functions: tightening_ceiling_on_reload_rejects_oversized_file (here, lines 112-114) and relaxing_ceiling_on_reload_admits_previously_rejected_file (lines 150-151) have /// doc comments. Per convention: "Do not add doc comments on test functions. The function name is the documentation." Remove them. Same applies to valid_reload_swaps_client_in_handle (line 702) and failed_reload_restores_client_in_handle (line 729) in server/src/reload.rs.

2. Inline comments in test bodies: Lines 123, 131-132, 160, and 168-169 use // comments to narrate test steps. Per convention, convert these to tracing::debug! calls or fold the explanation into assertion messages.