diff --git a/docs/thesis/ai-authorship.md b/docs/thesis/ai-authorship.md index 03185d8e..36b7c666 100644 --- a/docs/thesis/ai-authorship.md +++ b/docs/thesis/ai-authorship.md @@ -48,6 +48,8 @@ The thesis argues that FFL separates *domain programmers* (who write workflows) This is higher-leverage than authorship: one reviewer can approve what would have taken ten authors to write. The review is tractable because FFL was designed to be readable by domain experts, which happens also to make it readable by reviewers of AI-generated workflows, and because the FFL compiler catches the boring class of errors automatically. +This shift is no longer hypothetical in the small. A recent development cycle redesigned FFL's scoping model end to end — grammar, validator, runtime, and a scope-aware migration tool (`fw ffl migrate-scoping`) — then migrated the in-repo workflow corpus and one external domain's FFL onto the new model, flipped it to the default, and carried the change through a live rollout to a three-machine runner fleet. The work was AI-authored throughout, with the human supervisor confined to the director and reviewer roles above: specifying the intent, adjudicating the design (see §5.2), and approving the result. The division of labour §3 describes is the one that actually produced a language-level change to Facetwork itself — not merely a workflow written in it. + ## 4. Sealed skills Once generated, a workflow can be **sealed** — frozen into a reusable, inspectable, versioned artifact. The concept parallels Docker images, WASM modules, and signed smart contracts, adapted to workflows. @@ -94,6 +96,10 @@ Those properties are independent of *who* authors the language. They are what gi **What is worth adding.** Specific features — sum types, refinement types, structural row polymorphism, better composition primitives — are worth considering on their own merits because they add expressiveness *without* sacrificing static checkability. What is not worth doing is lifting restrictions (recursion, unbounded dynamic graphs, arbitrary control flow) on the grounds that AI can handle the complexity. The compiler cannot. +**A concrete data point.** A subsequent redesign of FFL's scoping model bears the prediction out directly. The model moved from a *flat* scheme — where `$.name` reached the enclosing workflow or facet's inputs, and a step body could name its containing step by name (e.g. `resolved.regions`) — to a *relative* one: `$` denotes the immediate lexical container and reads that container's parameters *and* returns uniformly (no input/output distinction; an unassigned read yields the default or NONE), while `$$`/`$$$` walk up the enclosing containers and overrunning the outermost is a compile error. Tellingly, the change *added* expressive machinery — up-level references and step-body clause chaining (`s = F() andThen {…} andThen foreach … andThen when …`) — while *simultaneously narrowing* what is legal: a block may now reference only its `$…` container attributes and its same-block steps, no longer its containing step by name (that is reached through `$`) nor a sibling block's step, and `andThen when` lost its former cross-block exemption and obeys the same rule. Expressiveness and constraint moved together, exactly as this section argues they should — the additions buy new well-typed programs; the restrictions let the validator reject a larger class of ill-scoped ones before any handler runs. + +The same episode shows the language *nudging authors toward explicit structure* rather than cleverness. To gate a step on several prior ones, the idiom that survives is to pass those steps as facet-typed parameters to a small gating facet: every reference then reads `$.param.field`, the gated steps stay parallel, and the facet's own signature *names its dependencies* — reusable, self-documenting, and valid under either scoping model. And one tempting extension was deliberately *resisted*: a deeper reading of `$$` as a *contract* — a facet's `$$.x` reaching whatever *instantiates* it, checked at each use site — was considered and parked, because every motivating case turned out clearer as an explicit parameter. That is this section's thesis in miniature: the feature that would have added a dynamic, use-site-dependent surface lost to the one that keeps each facet's contract local and statically checkable. The lesson was not that AI authorship licensed more cleverness in the language; it was that the sharper compile-time contract, plus a parameter-passing idiom, served the AI author better than a cleverer scope would have. + ## 6. Concrete design proposals With the false paths ruled out, the constructive direction follows. Given that AI agents are the primary authors and humans are reviewers and operators, the following specific changes serve the resulting system best. None relax the static discipline; most strengthen it. @@ -104,6 +110,8 @@ With the false paths ruled out, the constructive direction follows. Given that A - **Refinement types and sum types.** Return fields typed as `Int where x >= 0 and x <= 100` rather than bare `Int`. Sum types (`Result = Ok(T) | Err(E)`) replace loose `error: dict | None` conventions. AI generates these more precisely than humans do, and the compiler catches more. - **Pre- and post-conditions on facet signatures.** Lightweight contracts: `requires x > 0`, `ensures result.count >= 0`. A small theorem prover or SMT backend at compile time catches whole classes of AI hallucination before execution. +The relative-scoping redesign (§5.2) is a *shipped* instance of this strengthening, applied to references rather than types. Under the relative model the validator enforces that a block names only its `$…` container attributes and its same-block steps: the containing step is not nameable (it is reached through `$`), cross-block references are rejected, and walking past the outermost container is a compile error. The references a well-formed block may make are now fixed by its lexical position, so the compiler resolves — or rejects — every reference structurally, before any handler runs. It is the same discipline the bullets above pursue through effect and refinement types, carried into the scope rules themselves, and it strengthened the compile-time contract without adding any dynamic surface. + ### 6.2 Make generation itself a first-class concern - **Schema inference from examples.** Instead of the AI hallucinating a schema, the author provides two or three concrete example payloads; the compiler infers the most specific schema that accepts them and rejects generated workflows that then violate it. Closes the "guessed at the shape" failure mode. diff --git a/docs/thesis/ai-authorship.pdf b/docs/thesis/ai-authorship.pdf index 295c5f79..a191fd0b 100644 Binary files a/docs/thesis/ai-authorship.pdf and b/docs/thesis/ai-authorship.pdf differ diff --git a/docs/thesis/defense.md b/docs/thesis/defense.md index b42a7bed..12180266 100644 --- a/docs/thesis/defense.md +++ b/docs/thesis/defense.md @@ -127,10 +127,44 @@ It is visually noisy and I dislike it as much as you do. I will say what it buys Without `$.`, the alternative is to give outer-container attributes the same namespace as block-local bindings. This creates shadowing: an inner `region = ...` would shadow the outer `region: String` parameter. Shadowing is the source of an enormous number of bugs in imperative languages and I did not want to pay for that in FFL. The alternatives are (a) prohibit shadowing, which is restrictive and unnatural, or (b) use explicit syntactic distinction between "my scope" and "my container's scope," which is what `$.` does. -Better-known languages make similar choices. Scala's `this.` disambiguates class attributes from locals. JavaScript's explicit `this.` does the same. Python's `self.` is the same pattern. `$.` is Facetwork's version, and I picked `$` because it is visually distinct from identifier characters and cannot appear as a bare word in any reasonable surface syntax. A future version of FFL might use `outer.` or `container.` instead; the semantics would be unchanged. +Better-known languages make similar choices. Scala's `this.` disambiguates class attributes from locals. JavaScript's explicit `this.` does the same. Python's `self.` is the same pattern. `$.` is Facetwork's version, and I picked `$` because it is visually distinct from identifier characters and cannot appear as a bare word in any reasonable surface syntax. Since this defense the sigil has stayed but its meaning has sharpened: what was a single flat container level is now a *relative* scope, in which `$` names the immediate container and `$$`, `$$$` walk outward. I had expected any change here to be cosmetic — swap `$.` for `outer.`, semantics untouched. The opposite happened: the syntax survived and the semantics moved. I take that redesign up in Q8a. I accept that `$` carries Perl baggage for readers of a certain age. The syntax is stable now and, on the evidence of the examples I have shown, authors acclimate to it quickly. +### Q8a (Prof. Pike) + +> *You have just told us the scoping model changed after the defense. That is unusual — a language's scope rules are its spine. Walk us through what changed, and then justify the up-level operators `$$` and `$$$`. They look like exactly the Perl-noise you apologised for a moment ago.* + +The old model was flat. Every block resolved names against one implicit container — the workflow's inputs — plus its own local bindings, and `$.` reached that single container level. It worked for shallow workflows and it was easy to explain. It was also a lie about the structure of the programs authors were actually writing, because facets and steps nest, and a flat model hands every nesting level the same one outer scope. + +The relative model tells the truth about nesting. `$` names the *immediate* lexical container — the enclosing step, facet, or workflow — and `$.attr` reads that container's parameters *and* its return fields. `$$` names the container one level out, `$$$` the one beyond that, and walking past the outermost container is a compile error, not a silent `null`. A block sees exactly two things: the `$…` attributes of its enclosing containers, and the steps declared in its *own* block. It cannot see a sibling block's steps, and — this is the part that surprised me — it cannot name the step that contains it. The containing step is reached only through `$`. `andThen when` obeys the same rule: you attach the guard to the step it gates and resolve names from there. + +Now, the up-level operators, which you are right to be suspicious of. I resisted them. I wanted `$` and nothing else. The example that forced my hand is nested step bodies that each declare a same-named parameter: + +``` +o1 = One(input = $.input) andThen { + o2 = One(input = $.input + $$.input) andThen { + o3 = One(input = $.input + $$.input + $$$.input) andThen { … } + } +} +``` + +Here `o3` legitimately needs three different `input`s: its own container's, `o2`'s, and the workflow's. Under the flat model those three names collided and only the outermost was reachable. Under the relative model each is spelled unambiguously — `$.input`, `$$.input`, `$$$.input` — and there is *no other spelling available*, because, as I said, the containing step is not nameable. So the up-level operator is not noise I added for symmetry; it is the minimum mechanism that makes a legitimate program expressible at all. I would rather have a slightly Perl-ish sigil that can say the thing than a clean syntax that cannot. + +There is a deeper version of this idea that I considered and deliberately parked, and I want it on the record because it is a road I chose not to take. The relative operators walk *lexical* containers. One could imagine `$$` in a facet body reaching not its lexical parent but its *caller* — the container that instantiates the facet, as though the facet's body were inlined at each call site, with the compiler checking at every use-site that the instantiating container actually provides the named field. That would make a facet a kind of open term parameterised by its context. It is expressible and it is checkable. I parked it for one reason: every concrete case I found where I wanted it was better served by an explicit parameter. A facet that reaches into its caller's scope is a facet whose contract is invisible at the call site, and invisible contracts are precisely what the DSL exists to prevent. So the up-level operators are lexical only, and the caller-reaching version stays in the drawer with a note explaining why. + +### Q8b (Dr. Yegge) + +> *Two practical things. First: relative nesting sounds like it pushes authors toward deep pyramids — if I have to gate one step on five earlier ones, do I end up five `$` deep? Second: you changed the scope rules of a language that already had live workflows running against it. How did you do that without a flag day?* + +On the first: yes, naive nesting pushes you toward pyramids, and I hit exactly the failure you are describing. I had a workflow where a final step needed to see five prior results, and the obvious relative-scoping transcription nested them five deep — a `$$$$$` tower — which was unreadable, and worse, it *serialised* five steps that had no data dependency on each other simply to stack them in one another's scope. The nesting bought visibility at the cost of parallelism, which is a bad trade. + +The idiom that fixes it is decomposition, not deeper nesting: pass the prior steps as *facet-typed parameters* to a small gating facet. The five steps stay siblings in one block — parallel, no artificial ordering — and the gating facet receives them as typed inputs and refers to each as `$.param.field`. Every reference is one level deep. The facet is reusable, and its signature documents exactly what it gates on, which the pyramid never did. The nicest property is that this idiom is model-agnostic: it validates identically under the old flat scoping and the new relative scoping, because it never reaches across scopes at all — it passes data explicitly. When I found a construction that was clean under *both* models, that was a strong signal it was the right one, and it is now the pattern I steer authors toward the moment a pyramid starts to grow. + +On the second — the live migration — this is the part I am most pleased with operationally, because we did it without a flag day. The relative runtime was built to still run legacy flat FFL unchanged: at top level `$.x` still means a workflow input, step references still resolve the same way, and there is an injected-inputs fallback for the constructs that relied on the flat container. So the two models are not a hard fork; the new runtime is a strict superset of the old behaviour on old programs. That let us ship it flag-gated — dual-model behind `FW_FFL_RELATIVE_SCOPING` — run both interpretations side by side, provide a scope-aware migration tool that rewrote workflows into the relative idiom, flip the default once the corpus was migrated and green, and only then remove the old path. A live language migration on a running system, staged rather than switched. + +One implementation note the committee touched on earlier. The cross-block deferral mechanism — the `_StepNotReady` signal I raise when a block references a step that has not produced its value yet — was not deleted in the rework. It was *repurposed*: it is now what resolves `$.field` references to a container's *return* fields, which are not available until the container's body has run. The same machinery that used to paper over cross-block ordering now does honest work resolving relative container-return references. I mention it because it is a case where a mechanism I might have been tempted to rip out during a scope-model change turned out to be exactly the primitive the new model needed. + ### Q9 (Prof. Pike) > *No recursion. A workflow author who needs to walk a tree — a dependency graph, a hierarchical resource — cannot express that in FFL. You call this "a deliberate restriction that makes topology statically tractable." What do you say to the author who has a recursive problem?* diff --git a/docs/thesis/defense.pdf b/docs/thesis/defense.pdf index a6757df1..876666be 100644 Binary files a/docs/thesis/defense.pdf and b/docs/thesis/defense.pdf differ diff --git a/docs/thesis/future-thoughts-ai-native.md b/docs/thesis/future-thoughts-ai-native.md index 43dec47b..baa39372 100644 --- a/docs/thesis/future-thoughts-ai-native.md +++ b/docs/thesis/future-thoughts-ai-native.md @@ -20,6 +20,8 @@ Human languages optimize for **readability, stability, and cognitive chunking**: So the "language" is really two things: a **canonical IR** (what executes) and a **projection layer** (what a human sees when verifying). FFL-style surface syntax becomes one of many projections, alongside a graph view, a natural-language paraphrase, and a diff-against-prior-run view. +A recent, small data point cuts both ways here (§9). FFL's relative-scoping redesign added deliberately terse, positional operators — `$` for the immediate container's parameters and returns, `$$`/`$$$` to walk up N enclosing containers — plus step-body clause chaining. To a human these read as cryptic; to a generator they are precise and positional, exactly the terse, machine-oriented notation this section argues an agent actually wants in place of readable keywords. But the same redesign *parked* an even more implicit feature — `$$`-as-contract, where a facet reaches its instantiating container's scope by name, checked at the use site — because for human authors an explicit parameter was clearer. So on the one decision where terseness and legibility genuinely traded off, the human-first bias still won: the notation bent toward the agent, but the semantics stayed anchored to the human reader. + ## 2. Plans as data, stored and fetched dynamically Agents don't ship code; they **write rows**. A plan is a set of content-addressed nodes in a store (Mongo, Postgres, a Merkle store — doesn't matter). Consequences: @@ -87,7 +89,7 @@ A plausible incremental path: add effect annotations to existing event facets ## 9. Concrete first steps already taken -Three pieces of recent work line up with the speculation above closely enough to be worth flagging. None of them are the agent-native endpoint; each is a first step that the design exploration above would, in retrospect, have predicted. +Several pieces of recent work line up with the speculation above closely enough to be worth flagging. None of them are the agent-native endpoint; each is a first step that the design exploration above would, in retrospect, have predicted. **Standalone packages as proto-sealed-skills (§4).** Eight `fwh_*` repositories now exist as pip-installable Facetwork example packages — `fwh_osm`, `fwh_osm_lz`, `fwh_noaa_weather`, `fwh_jenkins`, `fwh_census_us`, `fwh_genomics`, `fwh_sensor_monitoring`, and `fwh_anthropic`. Each ships its own FFL sources, its own handlers, and its own pinned dependencies; each declares a `facetwork.examples` entry point that Facetwork runners discover at start time. Two pieces are missing relative to a true sealed skill: the content-addressable identifier (the unit today is a Git commit, not a hash of the FFL + handler set), and the formal provenance metadata (model, prompt, reviewer, tests). Both are additive — a sealing tool that produced `(content_hash, fingerprint, manifest)` from an `fwh_*` directory would convert the existing packages into sealed skills without changing their internal structure. The substrate is the FFL + entry-point pair; sealing is a layer on top. @@ -99,6 +101,10 @@ Three pieces of recent work line up with the speculation above closely enough to The same package also makes one weakness visible: nested-list payloads are carried through FFL today as JSON-encoded strings (`tools_json`, `messages_json`, `trace_json`, `results_json`, `files_json`) because the current type system does not natively model arbitrary lists of records as facet attributes. This is exactly the desugar-everything-to-{node, dependency, effect, schema} pressure §1 anticipated. Either FFL grows typed records-in-lists (a tractable addition that does not disturb the rest of the algebra), or the next-generation IR §1 sketches takes over as the canonical form and FFL becomes one projection of it. The first option is incremental and likely; the second is the agent-native endpoint and longer-horizon. Either way, the JSON-bridge fields are evidence that the language is already feeling the constraint the speculation anticipates. +A second, sharper data point arrived after the initial draft of this section: an end-to-end redesign of FFL's scoping rules that was authored, migrated, and deployed almost entirely by the AI collaborator, with the human acting as reviewer and arbiter of a few genuinely contested decisions. The change replaced flat container scoping with a *relative* model — `$` names the immediate lexical container, `$$`/`$$$` walk outward N levels, the containing step is deliberately not nameable, and cross-block references are rejected by the validator with a documented rule id. It is worth flagging what this exercise confirms about §1–§8 and what it does not. On the confirming side: the notation drifted toward exactly the terse, positional, machine-oriented operators §1 predicted a generator would prefer over readable keywords; the migration was staged behind a flag (`FW_FFL_RELATIVE_SCOPING`) with a scope-aware automated rewriter and a backward-compatible runtime, so a live corpus of workflows moved models without a flag day — the kind of mechanical, verifiable, reversible transformation §5 argues is the natural unit of AI-driven language evolution; and each new constraint shipped as a structured validator `rule_id` with paired wrong/right docs, i.e. the machine-checkable contract §6 wants in place of prose convention. On the not-confirming side: the one decision where terseness and human legibility genuinely traded off — `$$`-as-caller-contract, letting a facet reach its instantiating container's scope by name — was *parked* in favour of an explicit facet-typed parameter, precisely because an invisible cross-scope contract is the kind of thing the DSL exists to prevent. The agent proposed the more implicit, more powerful, less legible feature; the human-first bias declined it. + +So the redesign is a real instance of §5's thesis — AI authoring a non-trivial language change and carrying it through migration and fleet deployment — while its single most consequential judgement call landed on the human-legibility side of the line. That is the honest shape of the current moment: the mechanics of language evolution are already substantially automatable, the taste about which semantics to admit is not yet delegated, and the redesign is evidence for both halves at once. + None of this is the agent-native system. It is what the *human-first system* looks like when AI-author and cross-package distribution start to be load-bearing on its existing primitives. The interesting question, in light of this, is which §1–§8 ideas are reachable from here as additive layers (sealing, effect typing, plan registry) and which require the canonical IR + projection split §1 calls for. The first set is plausibly an incremental roadmap; the second set is, on this evidence, still a redesign rather than a refactor. --- diff --git a/docs/thesis/future-thoughts-ai-native.pdf b/docs/thesis/future-thoughts-ai-native.pdf index 19cf1d77..d7ac081f 100644 Binary files a/docs/thesis/future-thoughts-ai-native.pdf and b/docs/thesis/future-thoughts-ai-native.pdf differ diff --git a/docs/thesis/future-thoughts-positioning-dissent.md b/docs/thesis/future-thoughts-positioning-dissent.md index f118ae85..616349d2 100644 --- a/docs/thesis/future-thoughts-positioning-dissent.md +++ b/docs/thesis/future-thoughts-positioning-dissent.md @@ -103,3 +103,36 @@ First, the JSON-bridge fields that `fwh_anthropic` uses for nested-list payloads Second, the runtime issues that surfaced during the `fwh_anthropic` work — three of them, detailed in thesis §14.4 — are the kind of bug a typed-DSL approach catches because the surface forces the question. `resume_step` not re-queueing stuck siblings of a dirtied block, and `get_completed_step_by_name` rejecting steps with populated returns but not-yet-complete status, are bugs that exist only because the runtime distinguishes between *a step has returns to read* and *a step has finished its lifecycle*. A purely procedural orchestrator does not have that distinction to get wrong, but it also does not have it to use: the inline-andThen pattern `g0 = Gather(...) andThen { f0 = Extract(sources = g0.sources) }` is not expressible at all in a system whose composition primitives are imperative function calls. The dissent's broader point — that the load-bearing properties live in the algebra, not in the surface — is illustrated by both directions. The algebra makes hard-to-catch bugs visible, and it makes hard-to-write compositions trivial. The earlier claim — that FFL's semantic restrictions and its readable surface are paired — extends from the source layer (the validator + docs-by-rule_id of §5) to the package layer (vendor wrappers as facet catalogs) and to the runtime layer (composition bugs discovered through real workflow execution). Three layers, one substrate. None of this is incompatible with the "AI-native control layer" framing of §3; it is a refinement of what that framing *is*. The control layer happens to be a language, the language happens to compose vendor surfaces cleanly, and the vendor surfaces happen to be the most natural carriers for the effect typing that the broader AI-agent agenda eventually wants. The dominant reframing was right that Facetwork's value sits in the control layer. The working evidence keeps suggesting that the surface is part of the control layer, not separate from it. + +## 7. Working evidence: an AI-authored redesign meets an un-automatable fleet + +The most direct test yet of the *optimistic* framing — the one this note dissents from — is not a document but an episode. An agent carried out an end-to-end redesign of FFL's scoping model: it changed the grammar, the validator, and the runtime; wrote a scope-aware migration tool; ran the migration across the existing FFL corpus; flipped the default to the new model; and then deployed the result live across a three-machine runner fleet. That is the "agents author the system" claim in its strongest form — not agents filling in a function body, but an agent moving a language and its runtime from one semantic model to another and shipping it. On its face it supports the AI-native optimists. It is worth being honest that it does. But three details from the same episode cut in the dissent's direction, and they are the more interesting part. + +### 7.1 The deployment stalled on exactly the moat §1 conceded + +§1 grants that mature engines have a runtime moat built from "the operational trust accumulated over years of production usage," and that a new runtime cannot shortcut it. This episode is a concrete instance of what that moat is made of, and of the fact that AI-authorship does not buy any of it. + +The language redesign went cleanly. The *deployment* stalled repeatedly — not on anything the agent had written, and not on anything a cleverer language would have prevented, but on unglamorous, host-specific operational reality: + +- The infra host's DHCP lease drifted, leaving `afl-mongodb` / `afl-minio` stale in every machine's `/etc/hosts`. The whole fleet silently disconnected from MongoDB — zero live runners, and no alert. Nothing in the FFL, the validator, or the runtime was wrong; the *names* underneath them had moved. +- The fleet-agent failed because it ran under the system Python, which lacked `pymongo`, rather than under the project venv. A dependency-resolution accident, invisible to the language layer. +- A Mongo-discovery timeout was set too tight, so a slow-but-healthy host read as dead. + +Each was separate, subtle, and cost real debugging. This is precisely the hidden operational cost that the "your team's own machines as the cluster" / informal-fleet model carries and that a managed platform *hides* — DNS/service-discovery stability, dependency isolation on every heterogeneous host, liveness thresholds tuned to real network variance. A managed runtime absorbs all three behind an SLA; the informal fleet exposes them to whoever is on call. The point for this note is narrow but firm: **AI-authorship removes the cost of writing the system, not the cost of operating it.** The redesign demonstrates the former dramatically and leaves the latter entirely untouched. The optimistic framing tends to let the second cost disappear behind the first; this episode is a reminder that they are unrelated quantities, and that the second is the one §1 already identified as the durable moat. + +### 7.2 Even under an AI author, the pressure ran toward *more* explicitness + +A sub-current of the optimistic view is that syntax can now afford to be terse and cryptic-to-humans, because the author is a machine — readability is a legacy tax. The redesign is a clean natural experiment on that claim, and it comes out against it. + +The new model did add a terse, positional up-level operator (`$$` / `$$$` — reach one or two scopes up), the sort of cryptic sigil the "AI writes it now" position happily endorses. But two of the redesign's own decisions ran the other way, and they were made *by* the same AI-driven process that could have chosen otherwise: + +- A still-more-implicit feature — `$$`-as-contract, letting a facet reach its caller's scope *by name* — was designed and then **parked**, on the grounds that an explicit parameter was clearer for humans. The more magical option lost to the more legible one. +- The emergent best practice was a *facet-typed-parameter decomposition* that makes a step's dependencies **explicit in its signature** rather than implicit in its position. The house style the redesign settled into pushed dependencies up into readable declarations, not down into terse scope-walking. + +So the observed pressure, even with an agent doing the authoring, was toward explicitness and readability, not away from them. This is §2.2 and §4's "keep the readable surface" restated from an unexpected angle: the readable surface earned its keep not because a human insisted on it against the machine, but because the machine's own workflow was clearer with it. The cryptic operator survives as an escape hatch; it did not become the idiom. "The language can get more cryptic now that AI writes it" does not survive contact with an AI actually writing it. + +### 7.3 The "simple" migration was safe because of the boring discipline around it + +Finally, the episode is a corrective to the "just let the agent rewrite it" gloss. The migration did not land as a big-bang replacement. It shipped flag-gated behind a dual model, backward-compatible — the new runtime still runs old FFL — with layered verification at each stage and a live rollout across the fleet rather than an in-place swap. That is competent, ordinary engineering discipline: compatibility windows, staged rollout, a rollback path. It is also exactly the discipline that the optimistic phrase elides. The agent's fluency at *writing* the redesign is real; what made the redesign *safe to deploy* was the same set of practices any careful team would have insisted on, and none of them were made unnecessary by the fact that an agent produced the diff. The headline ("an agent redesigned the language and shipped it") is true. The subtext ("and it was safe because it was dual-modeled, compat-preserving, staged, and reversible") is where the actual assurance lives — and that subtext is human-invariant. + +Taken together, §7 neither refutes nor confirms the optimistic positioning; it partitions it. The claim that *agents can author the system* survives — it was demonstrated at full scale. The claims that ride along with it in the optimistic framing — that operation gets cheaper, that the surface can get more cryptic, that rewrites can be casual — do not. The dissent's standing bet is that the load-bearing value sits in the constrained, readable, disciplined *control* layer; an AI author reaching for that same constraint, readability, and discipline of its own accord is, if anything, the strongest evidence for it so far. diff --git a/docs/thesis/future-thoughts-positioning-dissent.pdf b/docs/thesis/future-thoughts-positioning-dissent.pdf index 3a04bbfb..0740060a 100644 Binary files a/docs/thesis/future-thoughts-positioning-dissent.pdf and b/docs/thesis/future-thoughts-positioning-dissent.pdf differ diff --git a/docs/thesis/thesis.md b/docs/thesis/thesis.md index 5a1ced06..eca22af3 100644 --- a/docs/thesis/thesis.md +++ b/docs/thesis/thesis.md @@ -265,14 +265,14 @@ namespace osm.ops { } ``` -A workflow's signature is `=> (name: Type, ...)` declaring its typed return fields, followed by an `andThen { ... }` block that is the workflow's body. Inside that body, parameters of the enclosing workflow are reached through the **container-attribute syntax** `$.name`; locally-defined assignments (`c`, `s`) are reached by bare name. The `$` distinguishes *reaching out of the current block into its container* from *reaching across statements within the current block*, and the distinction is enforced by the compiler. This is the simple case. The power of FFL comes in composition. +A workflow's signature is `=> (name: Type, ...)` declaring its typed return fields, followed by an `andThen { ... }` block that is the workflow's body. Inside a block, the `$` sigil denotes the block's **immediate container** — the step, facet, or workflow whose body or clause the block is — and `$.name` reads one of that container's attributes: its parameters and, once produced, its declared return fields alike, treated uniformly as fields on the container's record. In this top-level body the container is the workflow itself, so `$.region` and `$.force` are the workflow's parameters; locally-defined assignments (`c`, `s`) are reached by bare name. The `$` thus distinguishes *reaching out of the current block into its container* from *reaching across statements within the current block*, and the distinction is enforced by the compiler. A block may also reach further out: `$$` denotes the container's container, `$$$` one beyond that, and so on, one lexical hop per extra `$` — a generalisation that only becomes visible once blocks nest inside one another (§4.3.2). This is the simple case; the power of FFL comes in composition. ### 4.3 Composition: `andThen`, `foreach`, and `when` `andThen { ... }` introduces a **block** — a lexical scope containing a set of assignments. Two rules govern blocks: 1. **Statements within a block share a scope.** Any assignment in the block may reference any other assignment in the *same* block by name, and the runtime orders execution according to the resulting data-dependency partial order. -2. **Statements cannot reach across block boundaries.** An assignment in one block cannot name an assignment in a sibling block or an outer block. To reference values from the enclosing workflow or facet — its parameters, its declared return fields — the block uses the container-attribute syntax `$.name`. +2. **Statements cannot reach across block boundaries.** An assignment in one block cannot name an assignment in a sibling block or an outer block. To reference values from the block's immediate container — its parameters, its declared return fields — the block uses the container-attribute syntax `$.name`, and to reach a container further out it adds one `$` per level (`$$`, `$$$`, …; §4.3.2). What it cannot do is name a *step* in another block by that step's binding. Sibling blocks — two `andThen { ... }` blocks attached to the same container, or two `andThen foreach` expansions in the same scope — execute concurrently with each other. Consequently, a workflow that wants analysis to happen *after* extraction cannot place the two in separate sibling blocks; it must either place them in the *same* block (letting data flow order the work) or chain the blocks so that the later block sees the earlier block's outputs through the container scope. @@ -402,6 +402,74 @@ result = s andThen when { The `andThen when` expression evaluates exactly one branch. Together, `andThen`, `andThen foreach`, and `andThen when` provide enough control flow for all the workflows we have encountered without needing general-purpose loops. The absence of unbounded loops is not an accident; it is a deliberate restriction that makes topology statically tractable. Bounded iteration (foreach) is sufficient because the collections it iterates over are themselves either the result of upstream facets (and thus have known size at the iteration point) or parameters to the workflow. +#### 4.3.2 Nested blocks and the up-level operators + +Everything so far has needed only `$` (the immediate container) and bare names (same-block siblings). Both suffice as long as blocks do not nest. But FFL blocks *do* nest: a step may carry its own body — `s = F(...) andThen { ... }`, `andThen foreach`, or `andThen when` attached to a step — and those bodies can themselves contain steps that carry bodies. Each level of nesting introduces a new immediate container, and `$` always binds to the *nearest* one. To reach an enclosing container from inside a nested block, FFL adds one `$` per level: `$$` is the container's container, `$$$` one beyond that, and so on. Each extra `$` is exactly one lexical hop outward; there is no operator for hopping *down* into an inner block or *sideways* into a sibling one, because neither is ever in scope. + +A real workflow makes the mechanism concrete. This facet resolves a list of region names and then, for each resolved subregion in parallel, downloads its data and extracts populated places: + +```ffl +facet ExtractPlacesAcrossSubregions( + region_names: [String], min_population: Long, cache_policy: String +) => (place_paths: [String]) andThen { + resolved = osm.Region.ResolveRegions(names = $.region_names, expand = "subregions") + andThen foreach r in $.regions { + cache = osm.cache.Download(region = $.r, cache_policy = $$.cache_policy) + places = osm.Population.AllPopulatedPlaces(cache = cache.cache, + min_population = $$.min_population) + yield ExtractPlacesAcrossSubregions(place_paths = [places.result.output_path]) + } +} +``` + +Read the sigils level by level: + +- In the outer block, `$` is the facet, so `$.region_names` reads the facet's parameter. +- The `foreach` is attached to the step `resolved`, so *inside the loop body the immediate container is `resolved`*. There, `$.regions` reads `resolved`'s own output field (`ResolveRegions` returns `regions`), and `$.r` reads the loop variable — the iteration variable lives on the loop's immediate frame alongside the container step's attributes. +- `$$.cache_policy` and `$$.min_population` walk one level out, past `resolved`, to the enclosing facet's parameters — which is where those names actually live. `resolved` has no `cache_policy`, so `$` would not find it; the up-level is not stylistic, it is the only spelling that resolves. + +That last point is the crux: **the up-level operators are load-bearing, not sugar.** They earn their place when a name is shadowed — when an inner container and an outer one both declare an attribute of the same name and a step genuinely needs the outer one. The following minimal program is the case that forces them: + +```ffl +event facet One(input: Int) => (output: Int) + +workflow w1(input: Int) => (output: Int) andThen { + o1 = One(input = $.input) andThen { + o2 = One(input = $.input) andThen { + o3 = One(input = $$$.input) + } + } + yield w1(output = o1.output) +} +``` + +Three containers each expose an attribute named `input`: the workflow `w1`, the step `o1`, and the step `o2`. Inside `o2`'s body, `o3` can name all three, and only the up-level count distinguishes them: `$.input` is `o2`'s, `$$.input` is `o1`'s, `$$$.input` is the workflow's. Without the operators the three would collide and only one would be reachable; with them, each is spelled unambiguously and there is *no other spelling available*, precisely because the containing step is not nameable. Walking past the outermost container is a compile-time error (`REF_DOLLAR_OVERFLOW`): `$$$$.input` here has nowhere to resolve and is rejected by the validator. + +**The pyramid is a smell, and the fix is decomposition, not deeper nesting.** Nesting steps merely to stack them in one another's scope is almost always the wrong reason to nest. It buys visibility at the cost of parallelism: five steps stacked five deep so the innermost can see all four outer results are thereby *serialised*, even when they share no data dependency. When a step needs to see several prior results, the idiom is to pass those priors as **facet-typed parameters** (§4.4.2) to a small gating facet, keeping the producers as flat siblings: + +```ffl +event facet BuildImage(ref: String) => (image: String) +event facet RunTests(ref: String) => (passed: Boolean) +event facet AnalyzeRisk(ref: String) => (level: String) +event facet Ship(image: String, passed: Boolean, level: String) => (result: String) + +facet Deploy(build: BuildImage, tests: RunTests, risk: AnalyzeRisk) + => (status: String) andThen { + s = Ship(image = $.build.image, passed = $.tests.passed, level = $.risk.level) + yield Deploy(status = s.result) +} + +workflow Release(ref: String) => (status: String) andThen { + b = BuildImage(ref = $.ref) + t = RunTests(ref = $.ref) + r = AnalyzeRisk(ref = $.ref) + d = Deploy(build = b, tests = t, risk = r) + yield Release(status = d.status) +} +``` + +`b`, `t`, and `r` stay siblings in one block — the runtime runs them concurrently — and `Deploy` gates on all three by *naming them in its signature*, reading each as `$.build.image`, `$.tests.passed`, `$.risk.level`, every reference exactly one level deep. The dependency is documented in the type rather than buried in the shape of the nesting; the gating facet is reusable; and, tellingly, this construction validates identically under flat and relative scoping because it never reaches across scopes at all. The up-level operators exist for the genuine shadowing case above; the decomposition idiom is what keeps them rare. + ### 4.4 Data-dependency semantics: concurrency by default A defining property of FFL, and one that distinguishes it sharply from imperative workflow-as-code, is that **the textual order of statements in an `andThen` block does not determine execution order**. Execution order is determined solely by data dependencies inferred from the expressions. @@ -455,7 +523,7 @@ The practical effect of these rules is that `yield`, viewed from inside the lang #### 4.4.2 Pass-by-step: parameters typed as facets -The reference forms presented so far carry a single value — `s1.field` selects one attribute, `$.field` reads one of the enclosing facet's bound inputs. FFL admits a third form: when a parameter's declared type is itself a facet name, the argument bound to that parameter is **a reference to a whole step** rather than a copy of one of its fields. +The reference forms presented so far carry a single value — `s1.field` selects one attribute, `$.field` reads one of the immediate container's attributes (and `$$.field` one of an outer container's). FFL admits a third form: when a parameter's declared type is itself a facet name, the argument bound to that parameter is **a reference to a whole step** rather than a copy of one of its fields. ```ffl facet Value(input: String) => (output: String) @@ -547,8 +615,8 @@ facet Gather(key: String) // step-level andThen attached to the call site. workflow Report(key: String) => (message: String) andThen { g = Gather(key = $.key) andThen { - c = LoadC(key = $.key) - f = Format(a = g.a, b = g.b, c = c.c, doubled = g.doubled, tagged = g.tagged) + c = LoadC(key = $$.key) + f = Format(a = $.a, b = $.b, c = c.c, doubled = $.doubled, tagged = $.tagged) yield Report(message = f.result) } } @@ -560,13 +628,13 @@ When `g = Gather(...)` is evaluated, the runtime proceeds in three distinct phas **Phase 2 — Primary facet phase.** The primary facet's `andThen` blocks are now eligible to run. Crucially, at this point `$.a` and `$.b` *are* readable from inside Phase 2, because the mixin phase has committed them to the container. The two primary `andThen` blocks execute concurrently with each other (they are sibling blocks per §4.3), each reading `$.a` and `$.b` and contributing a different output field (`doubled` and `tagged`). Neither primary block can read the other's yields — scalar outputs yielded in sibling blocks do not become visible to each other mid-phase — but both can freely read mixin outputs. When both primary blocks complete, their yields are atomically applied to the container, so that by the end of Phase 2, `Gather`'s four output fields (`a`, `b`, `doubled`, `tagged`) are all populated. -**Phase 3 — Step-level block (at the caller's site).** The `andThen { ... }` appended to the call `g = Gather(key = $.key)` is the *step-level* block. It executes after Phases 1 and 2 have fully completed, which means it sees the fully-assembled step output through the step binding `g`: `g.a`, `g.b`, `g.doubled`, `g.tagged`. In this example it also performs a fresh `LoadC` call and then calls `Format` using all of the step's outputs plus the newly-loaded `c`. The step-level block belongs lexically to the *caller* (`Report`), not to `Gather`, so it can read the caller's own container attributes (`$.key`) and any earlier bindings in the caller's block. +**Phase 3 — Step-level block (attached to the step).** The `andThen { ... }` appended to the call `g = Gather(key = $.key)` is the *step-level* block. (The call's own argument, `key = $.key`, is evaluated in the workflow's top-level block, where `$` is `Report`.) It executes after Phases 1 and 2 have fully completed. Under relative scoping the block's **immediate container is the step `g` itself**: `$` denotes `g`, so the fully-assembled step output is read as `$.a`, `$.b`, `$.doubled`, `$.tagged` — the container step is not nameable from inside its own body, so one writes `$.a`, never `g.a`. The enclosing workflow `Report` is one hop further out, reached with `$$`; that is why `LoadC(key = $$.key)` walks up past `g` (a `Gather`, whose own `key` `$.key` would also name, but the intent here is the workflow's input) to `Report`'s parameter. In this example the block performs a fresh `LoadC` call — its own sibling, read by bare name `c.c` — and then calls `Format` using all of the container step's outputs plus the newly-loaded `c`. Three visibility rules follow, which taken together capture the whole model: 1. **Phase 1 sees only inputs.** Mixins cannot see the primary facet's yields or other mixins' yields. They are pure functions of the container's inputs. 2. **Phase 2 sees inputs plus Phase-1 yields.** Primary `andThen` blocks can read every attribute yielded by any mixin, because Phase 1 has already committed. They cannot read sibling primary-block yields — those are still accumulating — and they cannot read attributes that will only be produced by the step-level block. -3. **Phase 3 sees the completed step.** The step-level block at the caller's site reads the step's attributes as `stepName.fieldName`, and only runs after every field has been assigned by Phase 1 or Phase 2. +3. **Phase 3 sees the completed step.** The step-level block's immediate container is the step it is attached to, so it reads that step's assembled attributes as `$.fieldName` (the step is not nameable from inside its own body) and reaches the enclosing workflow's attributes with `$$`. It only runs after every field has been assigned by Phase 1 or Phase 2. The model composes cleanly with everything said earlier. The atomicity described in §4.4.1 — "yields are collected and applied atomically at block completion" — now has a richer story: it applies at every phase boundary, not just at the end of a single block. Phase 1's aggregated yields are committed atomically to the container; Phase 2's aggregated yields are committed atomically; Phase 3 sees the committed total. No phase ever observes a half-populated step, and no concurrent writers within a phase can race because the phase's yields are accumulated and applied only at its boundary. @@ -601,18 +669,18 @@ FFL makes error handling part of the flow language rather than relegating it to ```ffl s = DownloadPBF(region = $.region) catch when { - case err.type == "NetworkError" => { - retried = DownloadPBF(region = $.region, mirror = "backup") + case $.error_type == "NetworkError" => { + retried = DownloadPBF(region = $$.region, mirror = "backup") } - case err.type == "DiskFullError" => { + case $.error_type == "DiskFullError" => { cleared = CleanCache() - retried = DownloadPBF(region = $.region) + retried = DownloadPBF(region = $$.region) } - case _ => { FailWorkflow(reason = err.message) } + case _ => { failed = FailWorkflow(reason = $.error) } } ``` -`catch when` cases are statically matched against a typed error schema; the compiler verifies that every case is reachable and that the final catch-all is present if the cases are not exhaustive. This lifts error handling from a handler-internal `try/except` — invisible to the orchestrator — into a topology-visible construct that the dashboard renders and that the compiler can type-check. +A `catch when` clause is itself a block, and under relative scoping (§4.3.2) its immediate container is the **caught step** — here `s` — so `$` denotes that step. On top of the step's ordinary attributes the runtime exposes two synthetic fields inside a catch clause: `$.error_type` (the error's type) and `$.error` (its message). The enclosing workflow is one hop further out, so its own inputs are reached with `$$` — that is why the retries call `DownloadPBF(region = $$.region)`. `catch when` cases are statically matched against a typed error schema; the compiler verifies that every case is reachable and that the final catch-all is present if the cases are not exhaustive. This lifts error handling from a handler-internal `try/except` — invisible to the orchestrator — into a topology-visible construct that the dashboard renders and that the compiler can type-check. ### 4.7 Prompt blocks: first-class LLM integration diff --git a/docs/thesis/thesis.pdf b/docs/thesis/thesis.pdf index 76d7727e..e3f1fae9 100644 Binary files a/docs/thesis/thesis.pdf and b/docs/thesis/thesis.pdf differ