test(scenarios): add citizen scenario suite and virtual channel adapter (foundation)#515
Merged
Conversation
Adds two new test-tier projects under tests/scenarios/ that anchor a channel-agnostic acceptance suite for the citizen -> conversation -> session -> adapter loop (plan section 10): - BotNexus.Scenarios.Harness (class library) - reusable harness intended to be referenced by any future per-channel conformance project (Telegram, Teams, Slack, etc.). Built on Domain, Gateway Contracts/Abstractions/Channels, Agent.Providers.Core, and Logging.Abstractions - deliberately no Hosting/DI/Gateway/Memory to keep the public surface lean. - BotNexus.Scenarios.Tests (xUnit + Shouldly) - the scenario specs themselves; only test project in this slice. Carries the Conversations and Sessions production refs needed by the thin behavioural scenario added later in this PR. Refs #514. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Introduces the two core harness primitives plus their conformance tests so any future scenario can plug in a deterministic adapter and a deterministic LLM (plan section 10.3). Harness: - VirtualChannelAdapter implements IChannelAdapter AND IStreamEventChannelAdapter so SendStreamEvent capture is honest against GatewayHost wireup. Uses ConcurrentDictionary<string, ConcurrentQueue<T>> for stream-event buffers (correctness fix caught by rubber-duck review - ConcurrentDictionary<K, List<T>> with .Add() is not thread-safe). XML remarks document the re-listed-interface explicit-implementation pattern used to override the IChannelAdapter.AdapterId default interface member. - VirtualChannelAdapterOptions models per-instance capability flags (steering, streaming, follow-up, thinking, tool calls, inbound images) so capability-gating scenarios can be expressed declaratively. - ScenarioFakeApiProvider is a deterministic IApiProvider that accepts a script ((turnIndex, contextSummary) -> response) so every scenario gets reproducible LLM behaviour without recorded fixtures. Conformance: - 12 [Fact] tests cover the adapter (round-trip dispatch, snapshot isolation, ordering, stream-delta grouping, stream-event grouping by routing key, capability gating on outbound). - 5 [Fact] tests cover the fake provider (script invocation order, context-summary projection, deterministic replay). All 17 tests green. Refs #514. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tionRouter First behavioural scenario in the new suite (plan section 10.4, scenario #2 - minimal user -> agent loop end to end). The test drives a real InboundMessage through the production DefaultConversationRouter wired to InMemoryConversationStore + InMemorySessionStore via a small RouterDispatcher shim, and asserts: - the router creates a Conversation when none exists for the (ChannelKey, ChannelAddress) pair; - the conversation is bound to the virtual channel adapter and carries the originating ChannelBinding; - a session is opened against the new conversation; - the conversation routing result faithfully reports IsNewSession. This is intentionally a thin scenario - the full first-wave inventory (12 scenarios in plan section 10.4) lands incrementally alongside the model-evolution phases that the scenarios validate (compaction-mark, reset-flush, citizen abstraction). Wider behavioural scenarios that need a VirtualWorld harness defer to the PR that introduces it. Refs #514. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds 5 NetArchTest fitness functions (plan section 10.7) that structurally encode the conventions that keep the scenario suite useful as the platform grows: 1. ScenarioTests_DoNotReferenceAnyChannelExtension - the test project must not pull in BotNexus.Extensions.Channels.* (the whole point of the scenario suite is channel-agnostic). 2. ScenarioHarness_DoesNotReferenceAnyChannelExtension - the same rule for the harness; if the harness ever needed a real channel, it would defeat the conformance-suite door we're keeping open for per-channel test projects. 3. VirtualChannelAdapter_ImplementsIChannelAdapter - defends against drift if IChannelAdapter shape changes; the harness has to keep up. 4. ScenarioTests_DoNotDependOnIServiceProvider - scenario tests must use the harness directly, not reach past it into DI. 5. ScenarioHarness_PublicSurface_DoesNotLeakDiPrimitives - reflection-based scan of every harness exported type's constructors / methods / properties for IServiceProvider, IServiceCollection, IServiceScope*, IHost*, IHostedService. This one came out of rubber-duck critique - rule #4 only protected the test project, but a future VirtualWorld helper could still leak DI primitives through the harness API; this rule prevents that statically. All 10 architecture tests pass (5 existing Vogen rules + 5 new scenario rules). Refs #514. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds tests/scenarios/AGENTS.md (the per-folder convention doc) covering the three-layer architecture (production / harness / scenarios), six conventions for scenario tests (prose names, no channel-extension refs, harness-or-nothing, deterministic time, deterministic LLM, capability declarations), the first-wave and deferred scenario inventory, and the phasing alignment with the broader domain-model refactor (plan section 10). Cross-links from the root AGENTS.md under a new 'Scenario Test Suite' subsection so contributors land on the conventions before they add a new scenario. Refs #514. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the foundation for the channel-agnostic citizen scenario suite described in
plan.mdsection 10. Subsequent behavioural scenarios (compaction-mark, reset-flush,multi-binding fan-out, citizen-aware variants, ThreadId-removal regression) ship in
their phase PRs and reuse this harness.
Closes #514
What ships
tests/scenarios/BotNexus.Scenarios.Harness(class library):VirtualChannelAdapter(full
IChannelAdapter+IStreamEventChannelAdapter),VirtualChannelAdapterOptions(per-instance capability flags),
ScenarioFakeApiProvider(deterministic LLM script).tests/scenarios/BotNexus.Scenarios.Tests(xUnit + Shouldly): 17 conformance tests(12 adapter + 5 fake provider) + 1 thin behavioural scenario driving the production
DefaultConversationRouterend to end through the virtual adapter. 18 tests total,all green.
BotNexus.Architecture.Tests(plan section 10.7)IChannelAdapterdrift guard,
IServiceProviderban in scenarios, and a reflection-based public-surfacescan that prevents DI primitives leaking through the harness API.
tests/scenarios/AGENTS.mddocumenting conventions; rootAGENTS.mdcross-link.Why a thin first wave
§10.4 lists 12 first-wave scenarios. Most of them require model changes that have not
shipped yet (
SessionEntry.IsHistoryper F-2a, the/resetREST symmetry per F-2c,the
ChannelAddress-only routing per F-12, theCitizenIdabstraction per Phase 1.5).Writing them now would either (a) lock in today's wrong behaviour as the "spec", or
(b) require a parallel
VirtualWorldhost harness whose right shape only becomesobvious once Phase 3a/3c/6b/1.5 land. Instead, this PR ships the infrastructure plus
the conformance bar that proves the infrastructure is sound, and each model-evolution
PR then adds its own scenarios on top.
Rubber-duck-guided fixes already applied in this PR
VirtualChannelAdapterre-listsIChannelAdapterand implementsIStreamEventChannelAdaptersoSendStreamEventcapture is honest againstGatewayHost's actual interface check.ConcurrentDictionary<string, ConcurrentQueue<T>>rather than...<string, List<T>>(the latter is notthread-safe;
ConcurrentDictionaryonly protects the dictionary, not its values).ScenarioHarness_PublicSurface_DoesNotLeakDiPrimitivesadded so future harnessevolution can't accidentally expose
IServiceProvider/IHost*through aVirtualWorldAPI.Agent.Providers.Core + Logging.Abstractions only - no Hosting/DI/Gateway/Memory.
Validation
dotnet build BotNexus.slnx --nologo --tl:off- clean (0 warnings, 0 errors).dotnet test BotNexus.slnx --nologo --tl:off --no-build- all suites green, fullrun reported
Passed: 1633forBotNexus.Gateway.Testsand 18/18 + 10/10 for thenew scenarios + architecture projects. (One subsequent run produced two flaky failures
in
BotNexus.Gateway.Testsunder concurrentdotnet testparallelism; re-runningthat project in isolation passed 1633/1633. The scenarios harness is fully in-memory -
no SQLite, no temp files - so it can't be the source. Pre-existing flakiness in the
parallel runner, unrelated to this change.)
Reviewer note
If you'd prefer the foundation to land alongside a wider first-wave end-to-end scenario
set built on a real
VirtualWorld(rather than the in-placeDefaultConversationRouterwiring used by the one behavioural scenario here), sayso on the PR and I'll extend before merge - I held back because that work benefits
materially from landing alongside the model-evolution PRs it would exercise.