feat: Safe Array Orchestration & Tool Memoization

[Copilot]

Tools declared at root level share global state, causing race conditions when wired from inside array loops. This PR introduces element-scoped tool declarations (with <tool> as <handle> inside [] as item { }) and request-scoped memoization (memoize keyword) with Promise-based stampede protection.

o <- ctx.catalog[] as cat {
  with std.httpCall as fetchItem memoize

  fetchItem.method = "GET"
  fetchItem.url <- cat.url
  .item <- fetchItem.response.data
}

Parser & Lexer

• MemoizeKw token; optional memoize on bridgeWithDecl, toolBlock, and elementWithDecl
• elementToolWire rule for <handle>.<field> = value / <handle>.<field> <- source inside array blocks
• Element-scoped with declarations assign fork instance IDs (≥100000), matching existing pipe fork pattern
• Cross-scope iterator references: inner array scopes can now read outer iterators (e.g., z reads x.v and y.v) via elementScope on NodeRef

AST Types

• memoize?: boolean on HandleBinding (tool variant) and ToolDef
• elementScoped?: boolean on HandleBinding for serialization routing
• elementScope?: string on NodeRef for cross-scope iterator resolution
• ToolMetadata.memoize accepts boolean | { keyFn: (input) => string } for custom cache keys

Execution Engine

• memoCache = new Map() on ExecutionTree, inherited by shadow() (request-scoped, never global)
• Caches the Promise, not the resolved value — 100 concurrent identical calls attach to one in-flight request
• Three-layer resolution: ToolMetadata.memoize || ToolDef.memoize || HandleBinding.memoize
• Fix: resolveToolDefByName merge chain now propagates memoize flag
• Fix: resolveToolDep now passes toolDef.memoize to callTool (was missing)
• Secure cache keying: callTool accepts nodeKey (trunk key) for scope-qualified cache keys — root-level tools share cache by fnName (deduplication), element-scoped tools (instance ≥100000) use a separate pool, preventing cross-scope cache pollution when inner scopes shadow outer tool names
• Cross-scope element resolution: elementScopeKey on shadow trees enables pullSingle to walk the parent chain and resolve outer iterator data from the correct nesting level

Compiler Codegen

• __memoCache Map + __callMemo(fn, input, toolName, keyFn) helper in generated code
• syncAwareCall dispatches to __callMemo for tools in memoizedTools set
• BridgeCompilerIncompatibleError: thrown for unsupported AST constructs (e.g., elementScope cross-scope references)
• Automatic fallback: compiler's executeBridge catches BridgeCompilerIncompatibleError and falls back to the core runtime interpreter with logger.warn()

Serializer, Syntax Highlighting, Docs

• Round-trip serialization of memoize on handles, tool blocks, and element-scoped declarations
• TextMate grammar updated for memoize on with and tool rules
• User docs in array-mapping.mdx and using-tools-pipes.mdx; playground example added

Scoping Test Suite (scoping.test.ts)

Dedicated variable/iterator scoping test suite using shared-parity runner (runtime + AOT) — 38 tests (38 pass, 0 TODO):

• Single-level arrays: iterator fields + root-level tool reads (1 and 2+ tools)
• Two-level nested arrays: own-iterator reads, root tool visible at depth 2
• Three-level nested arrays: own-iterator at each level, root tool visible at depth 3
• Cross-scope iterators: inner scope reading outer iterator (e.g., z reads x.v and y.v) — fully working in runtime, compiler falls back via BridgeCompilerIncompatibleError
• Aliases and pipes inside single and nested array blocks
• Element-scoped tool declarations: with myTool as t inside array block, isolation per element — fully working
• Request-scoped isolation: each executeBridge call gets a fresh scope

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Safe Array Orchestration & Tool Memoization</issue_title>
<issue_description>Impact: Solves the N+1 problem for expensive API/DB calls inside arrays, and eliminates race conditions caused by global tool state.

Overview

Currently, tools declared at the root level of a bridge (with std.httpCall) share a single global state. If a user wires inputs to this tool from inside an array loop ([] as item { ... }), the loop elements overwrite each other's inputs, causing chaotic race conditions.

This epic solves this by introducing Element-Scoped Tool Declarations (allowing tools to be instantiated locally per-element) combined with Request-Scoped Memoization (preventing duplicate network calls when multiple elements process the same data).

The Target Developer Experience

bridge Query.processCatalog {
  with context as ctx
  with output as o
  
  o <- ctx.catalog[] as cat {
    # 1. LOCAL INSTANCE: Thread-safe, isolated inputs per element
    # 2. MEMOIZE: Duplicate inputs hit a shared request-level cache
    with std.httpCall as fetchItem memoize
    
    fetchItem.method = "GET"
    fetchItem.url <- "https://api.com/items/{cat.id}"
    
    .item <- fetchItem.response.data
  }
}

Element-Scoped Tool Declarations

Allow users to declare fresh, isolated tool instances inside array mapping blocks.

• Parser Update: Update the Chevrotain elementWithDecl rule to accept with <tool> as <handle> alongside the existing alias syntax.
• AST/Engine Integration: Inside processElementLines, intercept these local tool declarations. Assign them a unique instance ID (e.g., 100000 + nextForkSeq++), identically to how Pipe Forks are handled.
• Result: Every element in the runtime shadow tree gets its own isolated ExecutionTree.state trunk, completely preventing input collisions.

The memoize DSL Keyword & Metadata API

Provide both DSL-level (for generic tools) and TypeScript-level (for native tools) controls for opting into memoization.

• TypeScript Metadata (ToolMetadata): Expand the .bridge object on tool functions to accept memoize. This allows tool authors to force memoization by default, and crucially, provide a custom keyFn to bypass the slow JSON.stringify default.
  Implementation Target:

export async function fetchExchangeRate(opts: { base: string, target: string }) {
  // ... expensive DB/API call ...
}

fetchExchangeRate.bridge = {
  // Force memoization by default with a custom, ultra-fast cache key resolver
  memoize: {
    keyFn: (input) => `${input.base}:${input.target}` 
  }
};

• Lexer/Parser: Add a MemoizeKw to the Chevrotain lexer. Allow it at the end of bridgeWithDecl and inside toolBlock definitions.
• AST Types: Add memoize?: boolean to HandleBinding and ToolDef interfaces.
• Resolution Logic: A tool is memoized if any of the three layers request it: TypeScript metadata || ToolDef || Handle Binding.

The Request-Scoped Cache Engine

Implement the actual cache mechanism. Crucial constraint: The cache must live on the request instance, never globally on the Node process, to prevent cross-tenant data leaks.

• Interpreter (ExecutionTree.ts): * Add memoCache = new Map<string, Promise<any>>() to the root execution tree.

• Ensure shadow() explicitly inherits the exact same map reference from the parent.

• In callTool, generate the cache key. If meta.memoize.keyFn exists, invoke it; otherwise fallback to JSON.stringify(input).

• Compiler (codegen.ts): * Inject const __memoCache = new Map(); at the top of the generated async function.

• Pass the memoization flag/keyFn into the __call wrapper to manage the cache lookup.

• Stampede Protection: The engine must cache the Promise, not the resolved result. This ensures that 100 concurrent array elements requesting the same URL instantly attach .then() to the exact same Promise, firing only 1 actual HTTP request.

Scope rules

bridge Query.processCatalog {
  with context as ctx
  with output as o
  with std.httpCall as other

  o <- ctx.catalog1[] as cat {  
    with std.httpCall as outer

    # Assigning inputs is OK as it in current scope
    outer.value <- cat.val

    .inner <- ctx.catalog2[] as cat {
      with std.httpCall as fetchItem memoize
      
      # Assigning inputs is OK as it in current scope
      fetchItem.method = "GET"
      fetchItem.url <- "https://api.com/items/{cat.id}"
  
      # Assign THROWS as you assign to non-scoped node
      other.value = "What"
      outer.value = "Cant do this"
  
      # You can pull from any "above nested" scopes
      .more <- other.result
      .item <- fetchItem.response.data
      .alsoOk <- outer.value
    }
  }
}

```</issue_description>

<agent_instructions>This is a large change ... create a plan first then execute step by s...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes stackables/bridge#105

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/stackables/bridge/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2642b66

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	bridge	2642b66	Commit Preview URL Branch Preview URL	Mar 07 2026, 10:00 AM

[aarne]

@copilot

there is a minor bug in how you are determining shouldMemoize for tools backed by a ToolDef.

Look closely at scheduleToolDef:

  // Memoize if ToolDef or handle binding requests it
  const handleMemoize = ctx.bridge?.handles.some(
    (h) => h.kind === "tool" && h.memoize && h.name === toolName,
  );
  const shouldMemoize = toolDef.memoize || handleMemoize;
  try {
    return await ctx.callTool(toolName, toolDef.fn!, fn, input, shouldMemoize);
  }

Now look at resolveToolDep:

  const promise = (async () => {
    // ...
    // BUG: You forgot to pass the memoize flag here!
    try {
      return await ctx.callTool(toolName, toolDef.fn!, fn, input); 
    } 
    // ...

=====

i would want to see a dedicated test suite for variable scoping ... we have auto-isolation wright now and its gets more and more complicated lets create a separate suite that we can validate agains both runtimes. it this is human readable then we can also link to it from the README

some examples with multiple nested loops to understand at which level the caching is done and what scoped noes are availble

[Copilot]

@copilot

there is a minor bug in how you are determining shouldMemoize for tools backed by a ToolDef.
...

Fixed in 0f1a82f:

1. Bug fix: resolveToolDep now passes toolDef.memoize to callTool. The ToolLookupContext interface doesn't expose bridge (handles), so we use the ToolDef-level flag directly — which covers the tool block memoize keyword case.

2. Scoping test suite: Added packages/bridge/test/scoping.test.ts with 25 tests using the same shared-parity runner pattern (runtime + AOT). Covers:

   • Root-level tool reads from inside single/nested arrays
   • Two-level and three-level nested array mappings
   • Alias and pipe scoping inside array blocks
   • ToolMetadata, DSL keyword, and ToolDef memoization deduplication (with call count assertions)
   • Request-scoped cache isolation across executions

Tests that involve array mapping + tool reads inside arrays are marked aotSupported: false where the AOT compiler doesn't yet handle that pattern.

[aarne]

@copilot

scoping tests are not good enough ... even if they fail right now we need to have a proper overview of what works ... dont create sjut happy path tests

bridge Query.test {
with input as i
with output as o

o.l1 <- i.a[] as x {
.xv <- x.v
.l2 <- x.children[] as y {
.yv <- y.v
.xv <- x.v
.l3 <- y.children[] as z {
.zv <- z.v
.xv <- x.v
}
}
}
}

the assumption is that every nested inner scope can access outer scope tools (z sees y and x and global, etc).

then there is the with tool syntax that explicitly creates a "scoped clone" but i think we also support automatic clone but the tests are not clear about how and when this happens

in scoping tests memoization adds not extra value ... unless we want to express at chich level the results re shared ?

=====

what happens if the AOT does not supprot a feature? we have a similar pattern in graphql where we have a fallback for the user with an warning message

[Copilot]

@copilot

scoping tests are not good enough ... even if they fail right now we need to have a proper overview of what works ... dont create sjut happy path tests
...

Rewrote scoping.test.ts in 5887104 and 10a8a61:

Scoping focus — removed all memoization tests (those stay in memoize.test.ts). The suite now covers:

1. Single-level: iterator fields + root-level tool reads (1 and 2+ tools)
2. Two-level: own-iterator reads, root tool at depth 2
3. Three-level: own-iterator at each level, root tool at depth 3
4. Cross-scope iterators (TODO): y reads x.v, z reads x.v and y.v — the exact pattern from your example. Currently fails at parse time ("Undeclared handle x"), documented with full assertion bodies that will auto-validate when the parser is updated
5. Aliases and pipes inside single and nested blocks
6. Element-scoped tools (TODO): with myTool as t inside array block — parses but runtime doesn't execute the tool wires yet
7. Request-scoped isolation: fresh scope per executeBridge call

36 tests total: 26 pass, 10 todo with ⚠ warnings showing exact failures.

Re: AOT fallback — there's already a pattern in bridge-graphql: assertBridgeGraphQLCompatible() throws BridgeGraphQLIncompatibleError, which the transform catches and falls back to standalone execution with logger.warn(). The direct executeBridge API (non-GraphQL) doesn't have a fallback layer — it either compiles or throws. Adding a similar try/catch-fallback wrapper for the compiler API would be a separate change.

[aarne]

@copilot

1. implement all scoping features described in the tests but not working in execution tree
2. make sure compiler does not fail silently but throws for incompatible AST variants
3. implement the fallback for compiler executeBridge to core version in case the incompatible error is thrown

====

If an inner scope shadows an outer variable (e.g., with getUser as usr outside, and with deleteUser as usr inside), standard programming language rules dictate that the inner usr is a fundamentally new, isolated lexical declaration.

If the interpreter relies solely on the string "usr" to build the cache key, it breaches lexical scope boundaries. An inner array loop could accidentally hit the cache pool of an outer scope just because the human-readable names match. That is completely insecure.

The Secure Fix: Keying by Graph Node, Not Name

To make the cache mathematically secure, we must stop using the human-readable toolName for the cache pool, and instead use the Unique Graph Coordinate (Trunk Key).

Because your engine already assigns unique instance IDs to shadowed/forked handles during AST construction (e.g., Tools:usr:1 for the outer scope and Tools:usr:2 for the inner scope), we can use the Trunk as the ultimate, tamper-proof cache boundary.

Here is the exact change to make the cache strictly adhere to lexical scoping.

1. Update callTool Signature

In ExecutionTree.ts (and ToolLookupContext interface), change toolName to nodeKey.

  callTool(
    nodeKey: string,  // <--- Use the unique graph node coordinate
    fnName: string,
    fnImpl: (...args: any[]) => any,
    input: Record<string, any>,
    memoize?: boolean,
  ): MaybePromise<any> {
    // ...

2. Update the Cache Key Generation

Inside callTool, use the nodeKey to partition the cache.

    const shouldMemoize = !!(memoMeta || memoize) && !isSyncTool;
    if (shouldMemoize) {
      const keyFn = memoMeta?.keyFn ?? JSON.stringify;
      
      // PERFECT ISOLATION: "Tools:usr:2:deleteUser:{"id":1}"
      const cacheKey = `${nodeKey}:${fnName}:${keyFn(input)}`;
      
      const cached = this.memoCache.get(cacheKey);
      if (cached) return cached;
// ...

3. Pass the TrunkKey from the Scheduler

In scheduleTools.ts, instead of passing the human-readable toolName, pass the unique trunkKey(target).

In scheduleFinish:

  if (directFn) {
    const handleMemoize = ctx.bridge?.handles.some(
      (h) => h.kind === "tool" && h.memoize && h.name === toolName,
    );
    // SECURE: Pass trunkKey(target) which includes the scope instance ID
    return ctx.callTool(trunkKey(target), toolName, directFn, input, handleMemoize);
  }

In scheduleToolDef:

  const handleMemoize = ctx.bridge?.handles.some(
    (h) => h.kind === "tool" && h.memoize && h.name === toolName,
  );
  const shouldMemoize = toolDef.memoize || handleMemoize;
  try {
    // SECURE: Pass trunkKey(target)
    return await ctx.callTool(trunkKey(target), toolDef.fn!, fn, input, shouldMemoize);
  }

Why this guarantees 100% Security:

Let's trace your exact shadowing scenario:

with getUser as usr memoize
o.items <- i.list[] as item {
  with deleteUser as usr memoize  # Shadowed!
}

1. Outer Scope Execution:

• The scheduler targets Trunk: module: SELF, type: Tools, field: usr, instance: 1.
• trunkKey(target) = "Tools:usr:1".
• Cache Key = "Tools:usr:1:getUser:{"id":1}".

2. Inner Scope Execution (Element 1):

• The array mapping creates a new lexical scope. The AST assigned it a new instance.
• The scheduler targets Trunk: module: SELF, type: Tools, field: usr, instance: 2.
• trunkKey(target) = "Tools:usr:2".
• Cache Key = "Tools:usr:2:deleteUser:{"id":1}".

3. Inner Scope Execution (Element 2 with duplicate ID):

• Uses the same inner trunk: "Tools:usr:2".
• Cache Key = "Tools:usr:2:deleteUser:{"id":1}".
• Result: Hits the cache of Element 1 safely!

[Copilot]

@copilot

1. implement all scoping features described in the tests but not working in execution tree
   ...

Implemented in commits b44a9e6 through 2642b66:

1. Scoping features: Element-scoped tools now execute correctly inside array blocks (b44a9e6). Cross-scope iterator references work — inner scopes can read outer iterators via elementScope on NodeRef (0056933). All 38 scoping tests pass (0 todo).

2. Compiler throws for incompatible AST: BridgeCompilerIncompatibleError is thrown when the compiler encounters unsupported constructs like elementScope (2642b66).

3. Compiler fallback: The compiler's executeBridge catches BridgeCompilerIncompatibleError and falls back to the core runtime with logger.warn() (2642b66).

4. Secure cache keying: Cache keys use scope-qualified prefixes (root: vs elem:) combined with fnName to prevent cross-scope cache pollution. The callTool interface now accepts nodeKey (trunk key) for scope detection while keeping toolName for human-readable tracing (fed188b). Same-function handles at root level share the cache (deduplication), while element-scoped tools (instance ≥ 100000) use a separate pool.