diff --git a/Claude.md b/Claude.md index d0cd25717..4627801c1 100644 --- a/Claude.md +++ b/Claude.md @@ -197,9 +197,15 @@ Every platform trait extends the shared marker `InteropApi` and must exist on every platform the class compiles to (empty is fine). **Stability:** interop packages are the MiMa-stable surface; route churn-prone -additions into `*.internal` subpackages. `InteropSurfaceTest` mechanically -enforces the rules. Internal/compiler/prelude code is out of scope — stays fully -idiomatic Scala. +additions into `*.internal` subpackages. (`InteropSurfaceTest` is planned for M2 +to mechanically enforce the rules; today the gate is MiMa.) Compiler-internal +packages are exempt from the MiMa check via wildcard filters in build.sbt: +`scalus.compiler.sir.lowering`, `scalus.compiler.sir.linking`, +`scalus.compiler.intrinsics`, `scalus.uplc.builtin.internal`. MIXED packages +(`scalus.uplc`, `scalus.uplc.eval`, `scalus.compiler.sir`, +`scalus.serialization.flat`, `scalus.utils`, `scalus.cardano.ledger.rules`) +take per-symbol filters only, never wildcards. Internal/compiler/prelude code +is out of scope for interop style rules and stays fully idiomatic Scala. ## Commit Guidelines diff --git a/README.md b/README.md index 0a31578a6..14b9c6867 100644 --- a/README.md +++ b/README.md @@ -63,7 +63,7 @@ On JVM, Scalus provides the full Scala developer experience: compiler plugin, tr ## Versioning and stability -Starting with 1.0.0-M1, `scalus-core`, `scalus-cardano-ledger`, and `scalus-bloxbean-cardano-client-lib` form the stable API surface, checked with [MiMa](https://github.com/lightbend-labs/mima) on every build. Most APIs will remain binary compatible across the 1.x line; some parts will likely still see breaking changes in 1.x releases, always following a deprecation cycle. `scalus-testkit` is best-effort, and `*.internal` packages and compiler internals carry no compatibility promise. Upgrading from 0.18.x? See the [migration guide](https://scalus.org/docs/get-started/migrating-to-1.0). +Starting with 1.0.0-M1, `scalus-core`, `scalus-cardano-ledger`, and `scalus-bloxbean-cardano-client-lib` form the stable API surface, checked with [MiMa](https://github.com/lightbend-labs/mima) on every build. Most APIs will remain binary compatible across the 1.x line; some parts will likely still see breaking changes in 1.x releases, always following a deprecation cycle. `scalus-testkit` is best-effort, and `*.internal` packages and compiler internals carry no compatibility promise (concretely: `scalus.compiler.sir.lowering`, `scalus.compiler.sir.linking`, `scalus.compiler.intrinsics` and `scalus.uplc.builtin.internal` are exempt from the MiMa check; see `mimaBinaryIssueFilters` in build.sbt). Upgrading from 0.18.x? See the [migration guide](https://scalus.org/docs/get-started/migrating-to-1.0). --- diff --git a/build.sbt b/build.sbt index 31a22460f..ab22e1adb 100644 --- a/build.sbt +++ b/build.sbt @@ -413,6 +413,20 @@ lazy val scalus = crossProject(JSPlatform, JVMPlatform, NativePlatform) }, // scalacOptions += "-Yretain-trees", mimaPreviousArtifacts := Set(organization.value %%% name.value % scalusCompatibleVersion), + mimaBinaryIssueFilters ++= Seq( + // Compiler-internal packages: no supported external implementors or instantiators; + // excluded from the binary-compat promise (README: "compiler internals carry no + // compatibility promise"; interop style guide: SIR compiler out of scope). Everything + // user-facing stays checked - the `scalus.compiler` entry points, the `scalus.compiler.sir` + // types appearing in `compile`'s signature and in plugin-generated bytecode, and all + // MIXED packages (scalus.uplc, scalus.uplc.eval, scalus.serialization.flat, scalus.utils) + // get per-symbol filters only, never wildcards. Known caveat: the wildcard also hides + // deletion of the `lowering.simple` backend objects referenced by `sir.toUplc`. + ProblemFilters.exclude[Problem]("scalus.compiler.sir.lowering.*"), + ProblemFilters.exclude[Problem]("scalus.compiler.sir.linking.*"), + ProblemFilters.exclude[Problem]("scalus.compiler.intrinsics.*"), + ProblemFilters.exclude[Problem]("scalus.uplc.builtin.internal.*") + ), // enable when debug compilation of tests Test / scalacOptions += "-color:never", diff --git a/docs/superpowers/plans/2026-07-31-uplc-source-view.md b/docs/superpowers/plans/2026-07-31-uplc-source-view.md new file mode 100644 index 000000000..bc1e72247 --- /dev/null +++ b/docs/superpowers/plans/2026-07-31-uplc-source-view.md @@ -0,0 +1,1412 @@ +# UPLC Source View Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Emit a `.uplc.json` artifact (UPLC text + text-range → source-position span map) with profile reports, and add a bidirectional side-by-side UPLC view to the Scalus Profiler VS Code extension. + +**Architecture:** Scalus stamps `functionName` into `UplcAnnotation` during V3 lowering, renders the evaluated in-memory `Term` with the existing paiges printer plus zero-width markers to recover exact text offsets, and writes the artifact via `ProfileReportWriter` into the existing `profile-manifest.json` (`format: "uplc"`). The extension parses the artifact and syncs cursor/highlights between the Scala editor and a read-only virtual UPLC document. + +**Tech Stack:** Scala 3 (scalus-core shared), paiges 0.4.4 (`Doc.zeroWidth`), jsoniter-scala, TypeScript VS Code extension (zero runtime deps, esbuild). + +**Spec:** `docs/superpowers/specs/2026-07-31-uplc-source-view-design.md` + +## Global Constraints + +- Two repos: `scalus` (this repo) and `/Users/nau/projects/lantr/scalus-vscode-extension`. +- Branch name in BOTH repos: `feature/uplc-source-view` (create from current master/main HEAD; do NOT commit to master). +- Conventional commits (`feat:`, `fix:`, `test:`, `docs:`). NEVER add a `Co-Authored-By: Claude` (or similar) trailer. +- No em dashes (—) in any authored text (docs, comments, commit messages); use en dash (–) if needed. +- Scalus repo: run `sbtn scalafmtAll` before EVERY commit (CI fails on one unformatted file). +- Scalus code style: Scala 3, `{}` for top-level defs, indentation syntax for small `if`/`match`, `then`/`do` keywords. 4-space indent (scalafmt enforces). +- MiMa: only ADD public API; never change existing public signatures. New public members on existing types are OK. `private[scalus]` additions are safe. +- Artifact schema constants (copy verbatim): file name `"$scriptHash-$redeemerTag-$redeemerIndex.uplc.json"`, manifest format string `"uplc"`, `schemaVersion: 1`, span fields `s`,`e`,`n`,`file`,`sl`,`sc`,`el`,`ec`,`fn` (0-based lines AND columns, character offsets, end-exclusive `e`), top-level fields `schemaVersion`,`uplc`,`files`,`functions`,`spans`. +- Render width: 80 (same as `Term.show`). +- Extension: zero runtime dependencies; new pure logic goes in vscode-free modules tested by `test/smoke.ts`; `npm run typecheck && npm test` must pass before each commit. + +## Deviation from spec (approved during planning) + +The spec says "New `ProfileFormat.Uplc` in `EvaluatorReportConfig`". Do NOT add that enum case: `ProfileFormat` values are rendered from `ProfilingData` by `ProfileReporting.render`, which cannot produce UPLC (it has no `Term`). Instead `ProfileReportWriter.write` takes the term as a new optional parameter and writes the artifact directly when the profile level is `Full`. Task 5 updates the spec file accordingly. + +## Key file map (scalus) + +| File | Role | +|---|---| +| `scalus-core/shared/src/main/scala/scalus/uplc/UplcAnnotation.scala` | annotation type (pos + functionName) | +| `scalus-core/shared/src/main/scala/scalus/uplc/Term.scala` | `Term` enum, fill passes (line ~111/198), `given Pretty[Term]` (line ~561) | +| `scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweringContext.scala` | mutable lowering context (add `currentFunction`) | +| `scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweredValue.scala` | ~40 `UplcAnnotation(pos)` stamp sites | +| `scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/Lowering.scala` | `lowerSIR`, `SIR.Let`/`SIR.Decl` cases (set `currentFunction`) | +| `scalus-core/shared/src/main/scala/scalus/uplc/eval/ProfileReportWriter.scala` | artifact + manifest writer | +| `scalus-cardano-ledger/shared/.../PlutusScriptEvaluator.scala:382,722` | `renderProfile` + call site (has `plutusScript`) | +| `scalus-testkit/shared/.../ScalusTest.scala:94` | `runWithProfileReport` (has `self: Program`) | +| `scalus-core/shared/src/main/scala/scalus/compiler/Compiled.scala:89-134` | `toUplc` fill-pass call site | + +Facts verified during planning: +- paiges 0.4.4 has `Doc.zeroWidth(s: String): Doc` (zero layout width, emitted in render output). The printer already uses paiges styling (zero-width ANSI) for XTerm mode. +- The `Apply` printer case flattens chains via `a.applyToList`; inner `Apply` nodes of a chain never pass through `prettyTermWithDepth`, so they get no spans. Accepted: the outermost application's span covers the chain. +- Annotated terms reach the ledger evaluator via `Script.PlutusV3(program)` factories caching `_cachedProgram` (`scalus-core/shared/.../cardano/ledger/Script.scala:26-39`); CBOR-decoded scripts have empty annotations, so the writer must skip the artifact when the term carries no source info. +- `Pretty[Term].pretty` calls `TermSanitizer.sanitizeNames` (annotation-preserving, structure-preserving) before printing. + +--- + +# Part A: scalus repo + +### Task 1: Branch + `functionName` stamping during lowering + +**Files:** +- Modify: `scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweringContext.scala` +- Modify: `scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweredValue.scala` +- Modify: `scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/Lowering.scala` +- Test: `scalus-core/shared/src/test/scala/scalus/compiler/sir/lowering/FunctionNameAnnotationTest.scala` (create) + +**Interfaces:** +- Produces: `LoweringContext.currentFunction: String` (var, default `""`), `LoweringContext.ann(pos: SIRPosition): UplcAnnotation` returning `UplcAnnotation(pos, currentFunction)`. Lowered terms carry `annotation.functionName` for code inside named `Let`-bound lambdas / top-level defs. + +- [ ] **Step 1: Create the branch** + +```bash +cd /Users/nau/projects/lantr/scalus && git checkout -b feature/uplc-source-view +``` + +- [ ] **Step 2: Write the failing test** + +Look at existing lowering tests in `scalus-core/shared/src/test/scala/scalus/compiler/sir/lowering/` for the established way to compile-and-lower in tests (most use `scalus.Compiler.compile { ... }` then `.toUplc()` via `import scalus.*`). Follow that pattern: + +```scala +package scalus.compiler.sir.lowering + +import org.scalatest.funsuite.AnyFunSuite +import scalus.* +import scalus.uplc.Term + +class FunctionNameAnnotationTest extends AnyFunSuite { + + private def collectFunctionNames(t: Term): Set[String] = { + def go(t: Term, acc: Set[String]): Set[String] = { + val acc1 = if t.annotation.functionName.nonEmpty then acc + t.annotation.functionName else acc + t match + case Term.LamAbs(_, body, _) => go(body, acc1) + case Term.Apply(f, arg, _) => go(arg, go(f, acc1)) + case Term.Force(b, _) => go(b, acc1) + case Term.Delay(b, _) => go(b, acc1) + case Term.Constr(_, args, _) => args.foldLeft(acc1)((a, x) => go(x, a)) + case Term.Case(arg, cases, _) => cases.foldLeft(go(arg, acc1))((a, x) => go(x, a)) + case _ => acc1 + } + go(t, Set.empty) + } + + test("lowered terms carry the enclosing function name") { + val sir = Compiler.compile { + def double(x: BigInt): BigInt = x + x + double(21) + } + val term = sir.toUplc() + val names = collectFunctionNames(term) + assert(names.contains("double"), s"expected 'double' in $names") + } +} +``` + +Note: the compiled `def double` becomes a `SIR.Let` binding whose name may be qualified or suffixed. If the assert fails only because of a name prefix (e.g. `"...double"`), relax to `names.exists(_.endsWith("double"))` and keep that as the contract. + +- [ ] **Step 3: Run the test, verify it fails** + +```bash +sbtn "scalusJVM/testOnly scalus.compiler.sir.lowering.FunctionNameAnnotationTest" +``` +Expected: FAIL (empty set – nothing populates `functionName` today). + +- [ ] **Step 4: Implement** + +1. `LoweringContext.scala` – add to the class body (not the constructor): + +```scala + /** Name of the innermost enclosing user function being lowered. Stamped into + * [[scalus.uplc.UplcAnnotation.functionName]] by [[ann]] so tooling (VS Code + * UPLC source view) can group compiled UPLC by source function. Empty when + * lowering code outside any named binding. + */ + var currentFunction: String = "" + + /** Annotation for a lowered term: position plus the enclosing function name. */ + def ann(pos: SIRPosition): UplcAnnotation = UplcAnnotation(pos, currentFunction) + + /** Run `body` with [[currentFunction]] set to `name`, restoring the previous value. */ + def withFunction[A](name: String)(body: => A): A = { + val saved = currentFunction + currentFunction = name + try body + finally currentFunction = saved + } +``` + +Add `import scalus.uplc.UplcAnnotation` to the file's imports. + +2. `Lowering.scala` – find where `SIR.Let` bindings are lowered (grep `case SIR.Let` / `Binding(`). For each binding whose rhs is lowered, wrap the rhs lowering in `lctx.withFunction(binding.name) { ... }` ONLY when the rhs is a lambda (`SIR.LamAbs`) – value bindings keep the enclosing function. Also find where top-level module definitions are lowered (the driver that iterates `Module.defs` or lowers the root `SIR.Decl`/`Let` chain produced by the plugin) and do the same there. Use the binding's simple name: `binding.name.split('.').last` if names are dot-qualified. + +3. `LoweredValue.scala` – mechanically replace `UplcAnnotation(pos)` / `UplcAnnotation()` term-construction sites with `lctx.ann(pos)` (the enclosing methods have `(using lctx: LoweringContext)` or a `lctx` in scope; check each of the ~40 sites; where no context is in scope, leave `UplcAnnotation(pos)` unchanged and note it in the commit message). + +- [ ] **Step 5: Run the test, verify it passes; run the lowering test suite** + +```bash +sbtn "scalusJVM/testOnly scalus.compiler.sir.lowering.*" +``` +Expected: PASS, no regressions. + +- [ ] **Step 6: Format and commit** + +```bash +sbtn scalafmtAll +git add -A && git commit -m "feat(compiler): stamp enclosing function name into UplcAnnotation during V3 lowering" +``` + +--- + +### Task 2: Annotation-preserving fill passes + +**Files:** +- Modify: `scalus-core/shared/src/main/scala/scalus/uplc/Term.scala` (fill passes, lines ~111-230) +- Modify: `scalus-core/shared/src/main/scala/scalus/compiler/Compiled.scala` (line ~133) +- Test: `scalus-core/shared/src/test/scala/scalus/uplc/FillAnnotationsTest.scala` (create) + +**Interfaces:** +- Consumes: `UplcAnnotation(pos, functionName)` from Task 1. +- Produces: `private[scalus] def fillEmptyAnnotationsBottomUp: (Term, UplcAnnotation)` and `private[scalus] def fillEmptyAnnotationsTopDown(inherited: UplcAnnotation): Term` on `Term`. Existing public `fillEmptyPosBottomUp`/`fillEmptyPosTopDown` keep their exact signatures and become thin delegates. + +- [ ] **Step 1: Write the failing test** + +```scala +package scalus.uplc + +import org.scalatest.funsuite.AnyFunSuite +import scalus.utils.ScalusSourcePos + +class FillAnnotationsTest extends AnyFunSuite { + private val pos = ScalusSourcePos("Foo.scala", 10, 0, 10, 20) + private val ann = UplcAnnotation(pos, "validate") + + test("bottom-up fill propagates functionName to spine nodes") { + val leaf = Term.Var(NamedDeBruijn("x"), ann) + val spine = Term.Force(Term.Delay(leaf)) // spine has empty annotations + val (filled, _) = spine.fillEmptyAnnotationsBottomUp + assert(filled.annotation.functionName == "validate") + assert(filled.annotation.pos == pos) + } + + test("top-down fill propagates functionName downward") { + val inner = Term.Delay(Term.Var(NamedDeBruijn("x"))) + val filled = inner.fillEmptyAnnotationsTopDown(ann) + assert(filled.annotation.functionName == "validate") + val Term.Delay(v, _) = filled: @unchecked + assert(v.annotation.functionName == "validate") + } + + test("existing annotations are never overwritten") { + val other = UplcAnnotation(ScalusSourcePos("Bar.scala", 1, 0, 1, 5), "other") + val leaf = Term.Var(NamedDeBruijn("x"), other) + val filled = leaf.fillEmptyAnnotationsTopDown(ann) + assert(filled.annotation == other) + } +} +``` + +- [ ] **Step 2: Run, verify it fails to compile** (methods don't exist) + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.FillAnnotationsTest" +``` + +- [ ] **Step 3: Implement** + +In `Term.scala`, generalize the two existing fill passes from `ScalusSourcePos` to `UplcAnnotation`: + +- `fillEmptyAnnotationsBottomUp: (Term, UplcAnnotation)` – same traversal as `fillEmptyPosBottomUp`, but `firstNonEmpty` picks the first annotation whose `pos.effectivePos` is not effectively empty (preserving that annotation's `functionName`), and `stamp` writes the whole representative annotation: + +```scala + def firstNonEmpty(as: UplcAnnotation*): UplcAnnotation = + as.iterator + .map(a => a.copy(pos = a.pos.effectivePos)) + .find(!_.pos.isEffectivelyEmpty) + .getOrElse(UplcAnnotation.empty) + def stamp(t: Term, rep: UplcAnnotation): UplcAnnotation = + if t.annotation.isEffectivelyEmpty && !rep.pos.isEffectivelyEmpty then rep + else t.annotation +``` + +Each case mirrors the existing one, with `t.annotation` in place of `t.annotation.pos` for the recursion results. + +- `fillEmptyAnnotationsTopDown(inherited: UplcAnnotation): Term` – same as `fillEmptyPosTopDown` with `UplcAnnotation` threaded instead of `ScalusSourcePos`. + +- Rewrite the two existing public methods as delegates (signatures unchanged): + +```scala + def fillEmptyPosBottomUp: (Term, ScalusSourcePos) = + val (t, a) = fillEmptyAnnotationsBottomUp + (t, a.pos) + + def fillEmptyPosTopDown(inherited: ScalusSourcePos): Term = + fillEmptyAnnotationsTopDown(UplcAnnotation(inherited)) +``` + +- `Compiled.scala:133` – replace + `optimized.fillEmptyPosBottomUp._1.fillEmptyPosTopDown(scalus.utils.ScalusSourcePos.empty)` + with + `optimized.fillEmptyAnnotationsBottomUp._1.fillEmptyAnnotationsTopDown(UplcAnnotation.empty)` (import `scalus.uplc.UplcAnnotation`). + +- [ ] **Step 4: Run tests** + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.FillAnnotationsTest scalus.uplc.eval.CekSourcePosTest" +``` +Expected: PASS (CekSourcePosTest guards the existing pos-fill behavior). + +- [ ] **Step 5: Format and commit** + +```bash +sbtn scalafmtAll +git add -A && git commit -m "feat(uplc): annotation-preserving fill passes carrying functionName" +``` + +--- + +### Task 3: Decorator hook in the Pretty[Term] printer + +**Files:** +- Modify: `scalus-core/shared/src/main/scala/scalus/uplc/Term.scala` (`given Pretty[Term]`, lines ~560-655) +- Test: `scalus-core/shared/src/test/scala/scalus/uplc/PrettyDecoratedTest.scala` (create) + +**Interfaces:** +- Produces: `private[scalus] object TermPrinter { def prettySanitized(term: Term, style: Style, decorate: (Term, Doc) => Doc): Doc }` where `term` must already be name-sanitized. `Term.pretty` behavior is byte-identical to before. + +- [ ] **Step 1: Write the failing test** + +```scala +package scalus.uplc + +import org.scalatest.funsuite.AnyFunSuite +import org.typelevel.paiges.Doc +import scalus.utils.Style +import scalus.uplc.DefaultFun.AddInteger + +class PrettyDecoratedTest extends AnyFunSuite { + private val term = Term.Apply( + Term.Apply(Term.Builtin(AddInteger), Term.Const(Constant.Integer(1))), + Term.Const(Constant.Integer(2)) + ) + + test("identity decorator renders identically to pretty") { + val sanitized = TermSanitizer.sanitizeNames(term) + val doc = TermPrinter.prettySanitized(sanitized, Style.Normal, (_, d) => d) + assert(doc.render(80) == term.show) + } + + test("decorator wraps every printed node") { + var count = 0 + val sanitized = TermSanitizer.sanitizeNames(term) + TermPrinter + .prettySanitized(sanitized, Style.Normal, (_, d) => { count += 1; d }) + .render(80) + // builtin + 2 consts + outermost Apply of the flattened chain = 4 + assert(count == 4) + } +} +``` + +- [ ] **Step 2: Run, verify compile failure** (`TermPrinter` doesn't exist) + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.PrettyDecoratedTest" +``` + +- [ ] **Step 3: Implement** + +Move the body of the `given Pretty[Term]`'s `prettyTermWithDepth` into a new `private[scalus] object TermPrinter` in `Term.scala` (same file, below the given): + +```scala +private[scalus] object TermPrinter { + /** Pretty-print an already-sanitized term, passing every printed node's Doc through + * `decorate`. `(term, doc) => doc` reproduces `Term.pretty` exactly. Inner `Apply` + * nodes of a flattened application chain are not printed individually and are not + * decorated. + */ + def prettySanitized(term: Term, style: Style, decorate: (Term, Doc) => Doc): Doc = + prettyTermWithDepth(term, style, depth = 0, decorate) + + private def prettyTermWithDepth( + term: Term, + style: Style, + depth: Int, + decorate: (Term, Doc) => Doc + ): Doc = { ... existing body, every recursive call passes decorate, + and the final Doc of each case is wrapped: decorate(term, doc) } +} +``` + +Concretely: each `case` in the existing match builds its `Doc` exactly as today; bind it to `val doc = ...` and return `decorate(term, doc)`. Recursive calls become `prettyTermWithDepth(x, style, depth + 1, decorate)`. + +The `given Pretty[Term]` becomes: + +```scala + given Pretty[Term] with + def pretty(term: Term, style: Style): Doc = + TermPrinter.prettySanitized(TermSanitizer.sanitizeNames(term), style, (_, d) => d) +``` + +- [ ] **Step 4: Run the test plus printer-sensitive suites** + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.PrettyDecoratedTest scalus.uplc.*Pretty* scalus.uplc.UplcParserTest" +``` +Expected: PASS. Also run `sbtn "scalusJVM/testQuick"` to catch golden-output tests that assert on `show` strings. + +- [ ] **Step 5: Format and commit** + +```bash +sbtn scalafmtAll +git add -A && git commit -m "refactor(uplc): extract TermPrinter with a per-node decorator hook" +``` + +--- + +### Task 4: UplcSourceMapRenderer + +**Files:** +- Create: `scalus-core/shared/src/main/scala/scalus/uplc/eval/UplcSourceMap.scala` +- Test: `scalus-core/shared/src/test/scala/scalus/uplc/eval/UplcSourceMapRendererTest.scala` + +**Interfaces:** +- Consumes: `TermPrinter.prettySanitized` (Task 3), `Term.annotation` with `functionName` (Tasks 1-2). +- Produces: + +```scala +case class UplcSpan(s: Int, e: Int, n: Int, file: Int, sl: Int, sc: Int, el: Int, ec: Int, fn: Option[Int]) +case class UplcSourceMap(schemaVersion: Int, uplc: String, files: Seq[String], functions: Seq[String], spans: Seq[UplcSpan]) +object UplcSourceMapRenderer { + val SchemaVersion = 1 + def hasSourceInfo(term: Term): Boolean + def render(term: Term): UplcSourceMap + def toJson(map: UplcSourceMap): Array[Byte] // jsoniter, indented +} +``` + +- [ ] **Step 1: Write the failing test** + +```scala +package scalus.uplc.eval + +import org.scalatest.funsuite.AnyFunSuite +import scalus.uplc.* +import scalus.uplc.DefaultFun.AddInteger +import scalus.utils.ScalusSourcePos + +class UplcSourceMapRendererTest extends AnyFunSuite { + private val posA = ScalusSourcePos("/src/Foo.scala", 10, 2, 10, 7) + private val posB = ScalusSourcePos("/src/Foo.scala", 12, 4, 12, 9) + private val annA = UplcAnnotation(posA, "validate") + private val annB = UplcAnnotation(posB, "") + + private val term = Term.Apply( + Term.Apply(Term.Builtin(AddInteger, annA), Term.Const(Constant.Integer(1), annB)), + Term.Const(Constant.Integer(2)), + annA + ) + + test("uplc text equals plain show (markers fully stripped)") { + val map = UplcSourceMapRenderer.render(term) + assert(map.uplc == term.show) + } + + test("spans point at the printed node text") { + val map = UplcSourceMapRenderer.render(term) + val builtinSpan = map.spans.find(sp => map.uplc.substring(sp.s, sp.e).contains("addInteger")).get + assert(map.files(builtinSpan.file) == "/src/Foo.scala") + assert(builtinSpan.sl == 10 && builtinSpan.sc == 2 && builtinSpan.el == 10 && builtinSpan.ec == 7) + assert(builtinSpan.fn.map(map.functions) == Some("validate")) + } + + test("nodes without positions produce no spans") { + val map = UplcSourceMapRenderer.render(term) + // the '2' const has an empty annotation + assert(!map.spans.exists(sp => map.uplc.substring(sp.s, sp.e) == "(con integer 2)")) + } + + test("spans nest and offsets are within bounds") { + val map = UplcSourceMapRenderer.render(term) + map.spans.foreach { sp => + assert(sp.s >= 0 && sp.e <= map.uplc.length && sp.s < sp.e) + } + } + + test("post-order indices are stable under Apply wrapping") { + val wrapped = Term.Apply(term, Term.Const(Constant.Integer(3))) + val base = UplcSourceMapRenderer.render(term) + val wrap = UplcSourceMapRenderer.render(wrapped) + val baseByPos = base.spans.map(sp => (sp.sl, sp.sc, sp.n)).toSet + // every base span keeps its node index in the wrapped program + baseByPos.foreach { case (sl, sc, n) => + assert(wrap.spans.exists(sp => sp.sl == sl && sp.sc == sc && sp.n == n)) + } + } + + test("hasSourceInfo") { + assert(UplcSourceMapRenderer.hasSourceInfo(term)) + assert(!UplcSourceMapRenderer.hasSourceInfo(Term.Const(Constant.Integer(1)))) + } + + test("json round-trip") { + val map = UplcSourceMapRenderer.render(term) + val json = new String(UplcSourceMapRenderer.toJson(map), "UTF-8") + assert(json.contains("\"schemaVersion\": 1") || json.contains("\"schemaVersion\":1")) + assert(json.contains("\"uplc\"")) + } +} +``` + +- [ ] **Step 2: Run, verify compile failure** + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.eval.UplcSourceMapRendererTest" +``` + +- [ ] **Step 3: Implement `UplcSourceMap.scala`** + +```scala +package scalus.uplc.eval + +import com.github.plokhotnyuk.jsoniter_scala.core.* +import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker +import org.typelevel.paiges.Doc +import scalus.uplc.{Term, TermPrinter, TermSanitizer} +import scalus.utils.Style + +/** One mapped region of rendered UPLC text. + * + * Offsets `s`/`e` are character offsets into [[UplcSourceMap.uplc]] (end-exclusive). + * `n` is the node's post-order index in the term tree (children before parent, fields + * in declaration order) – stable when the program is later wrapped in `Apply` nodes. + * `sl`/`sc`/`el`/`ec` are 0-based source lines/columns (raw ScalusSourcePos values; + * note profile.json uses 1-based lines). `file`/`fn` index [[UplcSourceMap.files]] / + * [[UplcSourceMap.functions]]. + */ +case class UplcSpan(s: Int, e: Int, n: Int, file: Int, sl: Int, sc: Int, el: Int, ec: Int, fn: Option[Int]) + +/** The `.uplc.json` document consumed by the Scalus VS Code extension. */ +case class UplcSourceMap( + schemaVersion: Int, + uplc: String, + files: Seq[String], + functions: Seq[String], + spans: Seq[UplcSpan] +) + +object UplcSourceMapRenderer { + val SchemaVersion = 1 + + private given JsonValueCodec[UplcSourceMap] = JsonCodecMaker.make + + private val MarkerStart = '\u0001' + private val MarkerEnd = '\u0002' + + /** True when at least one node carries a usable source position. */ + def hasSourceInfo(term: Term): Boolean = + !term.annotation.pos.effectivePos.isEffectivelyEmpty || (term match + case Term.LamAbs(_, b, _) => hasSourceInfo(b) + case Term.Apply(f, a, _) => hasSourceInfo(f) || hasSourceInfo(a) + case Term.Force(b, _) => hasSourceInfo(b) + case Term.Delay(b, _) => hasSourceInfo(b) + case Term.Constr(_, as, _) => as.exists(hasSourceInfo) + case Term.Case(a, cs, _) => hasSourceInfo(a) || cs.exists(hasSourceInfo) + case _ => false + ) + + def render(term: Term): UplcSourceMap = { + val sanitized = TermSanitizer.sanitizeNames(term) + + // Post-order index per node (identity-based: the tree may contain equal subterms). + val postOrder = new java.util.IdentityHashMap[Term, Integer]() + var next = 0 + def index(t: Term): Unit = { + t match + case Term.LamAbs(_, b, _) => index(b) + case Term.Apply(f, a, _) => index(f); index(a) + case Term.Force(b, _) => index(b) + case Term.Delay(b, _) => index(b) + case Term.Constr(_, as, _) => as.foreach(index) + case Term.Case(a, cs, _) => index(a); cs.foreach(index) + case _ => () + postOrder.put(t, next) + next += 1 + } + index(sanitized) + + // Collect annotations per marker id; decorate with zero-width markers. + val nodes = scala.collection.mutable.ArrayBuffer.empty[Term] + val doc = TermPrinter.prettySanitized( + sanitized, + Style.Normal, + (t, d) => + if t.annotation.pos.effectivePos.isEffectivelyEmpty then d + else { + val id = nodes.length + nodes += t + Doc.zeroWidth(s"$MarkerStart$id$MarkerEnd") + d + + Doc.zeroWidth(s"$MarkerStart/$id$MarkerEnd") + } + ) + val marked = doc.render(80) + + // Strip markers, recording clean offsets. + val clean = new StringBuilder(marked.length) + val starts = new Array[Int](nodes.length) + val ends = new Array[Int](nodes.length) + var i = 0 + while i < marked.length do { + val c = marked.charAt(i) + if c == MarkerStart then { + val stop = marked.indexOf(MarkerEnd, i + 1) + val body = marked.substring(i + 1, stop) + if body.startsWith("/") then ends(body.drop(1).toInt) = clean.length + else starts(body.toInt) = clean.length + i = stop + 1 + } else { + clean.append(c) + i += 1 + } + } + + val files = scala.collection.mutable.LinkedHashMap.empty[String, Int] + val functions = scala.collection.mutable.LinkedHashMap.empty[String, Int] + def intern(m: scala.collection.mutable.LinkedHashMap[String, Int], s: String): Int = + m.getOrElseUpdate(s, m.size) + + val spans = nodes.indices.map { id => + val t = nodes(id) + val pos = t.annotation.pos.effectivePos + val fn = t.annotation.functionName + UplcSpan( + s = starts(id), + e = ends(id), + n = postOrder.get(t), + file = intern(files, pos.file), + sl = pos.startLine, + sc = pos.startColumn, + el = pos.endLine, + ec = pos.endColumn, + fn = if fn.isEmpty then None else Some(intern(functions, fn)) + ) + } + + UplcSourceMap(SchemaVersion, clean.toString, files.keys.toSeq, functions.keys.toSeq, spans) + } + + def toJson(map: UplcSourceMap): Array[Byte] = + writeToArray(map, WriterConfig.withIndentionStep(2)) +} +``` + +Check the actual `ScalusSourcePos` field names (`startLine`, `startColumn`, `endLine`, `endColumn`) and `effectivePos`/`isEffectivelyEmpty` before compiling; adjust if they differ. + +- [ ] **Step 4: Run the test, verify PASS** + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.eval.UplcSourceMapRendererTest" +``` + +- [ ] **Step 5: Add an integration invariant test on a real compiled program** + +Append to the same test file: + +```scala + test("invariant holds for a compiled program") { + import scalus.* + val sir = Compiler.compile { + def double(x: BigInt): BigInt = x + x + double(21) + } + val t = sir.toUplc() + val map = UplcSourceMapRenderer.render(t) + assert(map.uplc == t.show) + assert(map.spans.nonEmpty) + assert(map.functions.exists(_.endsWith("double"))) + } +``` + +Run again; expected PASS. + +- [ ] **Step 6: Format and commit** + +```bash +sbtn scalafmtAll +git add -A && git commit -m "feat(uplc): UplcSourceMapRenderer – UPLC text with source-position span map" +``` + +--- + +### Task 5: Write the artifact with profile reports + +**Files:** +- Modify: `scalus-core/shared/src/main/scala/scalus/uplc/eval/ProfileReportWriter.scala` +- Modify: `scalus-cardano-ledger/shared/src/main/scala/scalus/cardano/ledger/PlutusScriptEvaluator.scala` (renderProfile ~382, call site ~722) +- Modify: `scalus-testkit/shared/src/main/scala/scalus/testing/kit/ScalusTest.scala` (~94) +- Modify: `docs/superpowers/specs/2026-07-31-uplc-source-view-design.md` (remove the `ProfileFormat.Uplc` sentence, describe the writer-parameter approach) +- Test: `scalus-core/shared/src/test/scala/scalus/uplc/eval/ProfileReportWriterUplcTest.scala` (create; put it next to any existing ProfileReportWriter test and follow its temp-dir pattern if one exists) + +**Interfaces:** +- Consumes: `UplcSourceMapRenderer` (Task 4). +- Produces: `ProfileReportWriter.write(data, report, scriptHash, language, redeemerTag, redeemerIndex, onConsole, uplcTerm: Option[Term] = None)`. When `report.profile == ProfileLevel.Full`, `uplcTerm` is defined, and `UplcSourceMapRenderer.hasSourceInfo(term)`, writes `.uplc.json` under `report.outputDir` and adds `("uplc", fileName)` to the manifest run's files. + +- [ ] **Step 1: Write the failing test** + +```scala +package scalus.uplc.eval + +import com.github.plokhotnyuk.jsoniter_scala.core.* +import org.scalatest.funsuite.AnyFunSuite +import scalus.cardano.ledger.{EvaluatorReportConfig, ProfileLevel} +import scalus.uplc.* +import scalus.uplc.DefaultFun.AddInteger +import scalus.utils.ScalusSourcePos + +import java.nio.file.{Files, Path} + +class ProfileReportWriterUplcTest extends AnyFunSuite { + private def annotated: Term = + Term.Builtin(AddInteger, UplcAnnotation(ScalusSourcePos("/src/A.scala", 3, 0, 3, 5), "f")) + + private def emptyProfile: ProfilingData = ProfilingData.empty // if no such member exists, + // construct the minimal ProfilingData the same way existing ProfileReportWriter/Formatter + // tests do – check those tests first. + + test("uplc.json is written and indexed in the manifest") { + val dir = Files.createTempDirectory("scalus-uplc-test") + val report = EvaluatorReportConfig( + enabled = true, + outputDir = dir.toString, + profile = ProfileLevel.Full + ) + ProfileReportWriter.write( + emptyProfile, report, "cafe01", "PlutusV3", "Spend", 0, _ => (), Some(annotated) + ) + val uplcFile = dir.resolve("cafe01-Spend-0.uplc.json") + assert(Files.exists(uplcFile)) + val manifest = new String(Files.readAllBytes(dir.resolve("profile-manifest.json")), "UTF-8") + assert(manifest.contains("\"uplc\"")) + assert(manifest.contains("cafe01-Spend-0.uplc.json")) + } + + test("no artifact for a term without source info") { + val dir = Files.createTempDirectory("scalus-uplc-test2") + val report = EvaluatorReportConfig(enabled = true, outputDir = dir.toString, profile = ProfileLevel.Full) + ProfileReportWriter.write( + emptyProfile, report, "cafe02", "PlutusV3", "Spend", 0, _ => (), + Some(Term.Const(Constant.Integer(1))) + ) + assert(!Files.exists(dir.resolve("cafe02-Spend-0.uplc.json"))) + } +} +``` + +Before running: check how existing tests build a `ProfilingData` (grep `ProfilingData(` in test sources) and use that instead of the `ProfilingData.empty` placeholder if it does not exist. Note: `ProfilingData` totals feed the manifest budget; zeros are fine. + +- [ ] **Step 2: Run, verify failure** (no such parameter) + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.eval.ProfileReportWriterUplcTest" +``` + +- [ ] **Step 3: Implement** + +In `ProfileReportWriter.write`, add the parameter `uplcTerm: Option[scalus.uplc.Term] = None` (last, after `onConsole`). After the `outputs.foreach` loop and before `val files = written.result()`: + +```scala + uplcTerm.foreach { term => + if report.profile == ProfileLevel.Full && UplcSourceMapRenderer.hasSourceInfo(term) + then { + val file = s"$key.uplc.json" + platform.createDirectories(report.outputDir) + platform.writeFile( + reportPath(report, file), + UplcSourceMapRenderer.toJson(UplcSourceMapRenderer.render(term)) + ) + written += "uplc" -> file + } + } +``` + +Add `import scalus.cardano.ledger.ProfileLevel` if missing. + +`PlutusScriptEvaluator.scala`: add a `term: Term` parameter to `renderProfile` and pass it through: + +```scala + private def renderProfile( + result: Result, + scriptHash: ScriptHash, + redeemer: Redeemer, + language: Language, + uplcTerm: => Term + ): Unit = result.profile.foreach { data => + ProfileReportWriter.write( + data, report, scriptHash.toHex, language.toString, + redeemer.tag.toString, redeemer.index, log.info(_), + Some(uplcTerm) + ) + } +``` + +Call site (~line 722): pass `plutusScript.program.term` as the new argument. (By-name so the CBOR decode only happens when a profile was actually produced.) + +`ScalusTest.runWithProfileReport` (~line 97): add `Some(self.term)` as the last `write` argument. + +- [ ] **Step 4: Run tests** + +```bash +sbtn "scalusJVM/testOnly scalus.uplc.eval.ProfileReportWriterUplcTest" +sbtn "scalusJVM/testQuick" +``` +Expected: PASS. + +- [ ] **Step 5: Update the spec file** + +In `docs/superpowers/specs/2026-07-31-uplc-source-view-design.md`, replace the two lines + +> - New `ProfileFormat.Uplc` in `EvaluatorReportConfig`. `ProfileLevel.Full` writes it. + +with + +> - `ProfileReportWriter.write` takes the evaluated term as an optional parameter and +> writes the artifact when the profile level is `Full` and the term carries source +> info (no new `ProfileFormat` case: those are rendered from `ProfilingData`, which +> has no `Term`). + +- [ ] **Step 6: Full check, format and commit** + +```bash +sbtn scalafmtAll +sbtn quick +git add -A && git commit -m "feat(profiler): write .uplc.json UPLC source map with profile reports" +``` + +- [ ] **Step 7: Verify MiMa** + +```bash +sbtn mima +``` +Expected: clean (all changed members are private/`private[scalus]`; `Term` additions are additive). If `fillEmptyAnnotations*` additions are flagged (they should not be – additions are compatible), report back instead of adding filters. + +--- + +# Part B: scalus-vscode-extension repo + +All paths below are relative to `/Users/nau/projects/lantr/scalus-vscode-extension`. + +### Task 6: uplcMap.ts (pure model + queries) with tests + +**Files:** +- Create: `src/uplcMap.ts` +- Modify: `test/smoke.ts` (append a test section; follow its existing plain-assert style) + +**Interfaces:** +- Produces: + +```ts +export const UPLC_MAP_SCHEMA_VERSION = 1; +export interface UplcSpan { s: number; e: number; n: number; file: number; sl: number; sc: number; el: number; ec: number; fn?: number } +export interface UplcSourceMap { schemaVersion: number; uplc: string; files: string[]; functions: string[]; spans: UplcSpan[] } +export function parseUplcMap(text: string): UplcSourceMap; // throws on malformed/wrong version +export function spansAtSource(map: UplcSourceMap, file: number, line0: number, col0: number): UplcSpan[]; +export function innermostSpanAt(map: UplcSourceMap, offset: number): UplcSpan | undefined; +export function spansForFunction(map: UplcSourceMap, fn: number): UplcSpan[]; +``` + +- [ ] **Step 1: Create the branch** + +```bash +cd /Users/nau/projects/lantr/scalus-vscode-extension && git checkout -b feature/uplc-source-view +``` + +- [ ] **Step 2: Write failing tests in `test/smoke.ts`** + +Follow the file's existing pattern (plain `assert` helpers, no vscode import). Append: + +```ts +// --- uplcMap --- +import { + parseUplcMap, + spansAtSource, + innermostSpanAt, + spansForFunction, +} from "../src/uplcMap"; + +{ + const map = parseUplcMap( + JSON.stringify({ + schemaVersion: 1, + uplc: "(program 1.1.0 [(builtin addInteger) (con integer 1) (con integer 2)])", + files: ["/src/Foo.scala"], + functions: ["validate"], + spans: [ + { s: 15, e: 74, n: 3, file: 0, sl: 10, sc: 0, el: 12, ec: 5 }, + { s: 16, e: 36, n: 0, file: 0, sl: 10, sc: 2, el: 10, ec: 7, fn: 0 }, + ], + }) + ); + assertEq(map.spans.length, 2, "uplcMap parses spans"); + + // spansAtSource: line 10 col 3 hits both (outer covers 10..12, inner covers 10:2-10:7) + assertEq(spansAtSource(map, 0, 10, 3).length, 2, "spansAtSource hits nested spans"); + // line 11 hits only the outer span + assertEq(spansAtSource(map, 0, 11, 0).length, 1, "spansAtSource line containment"); + // line 10 col 1 is before the inner span's start column + assertEq(spansAtSource(map, 0, 10, 1).length, 1, "spansAtSource column boundary"); + + // innermost: offset 20 is inside both spans; the smaller one wins + assertEq(innermostSpanAt(map, 20)?.n, 0, "innermostSpanAt picks smallest"); + assertEq(innermostSpanAt(map, 40)?.n, 3, "innermostSpanAt falls back to outer"); + assertEq(innermostSpanAt(map, 0), undefined, "innermostSpanAt outside all spans"); + + assertEq(spansForFunction(map, 0).length, 1, "spansForFunction"); + + let threw = false; + try { + parseUplcMap(JSON.stringify({ schemaVersion: 99, uplc: "", files: [], functions: [], spans: [] })); + } catch { + threw = true; + } + assertEq(threw, true, "parseUplcMap rejects wrong schemaVersion"); +} +``` + +Adapt `assertEq` to whatever helper `test/smoke.ts` actually defines (read it first). + +- [ ] **Step 3: Run, verify failure** + +```bash +npm test +``` +Expected: compile error (module missing). + +- [ ] **Step 4: Implement `src/uplcMap.ts`** + +```ts +// TypeScript mirror of .uplc.json written by Scalus UplcSourceMapRenderer +// (scalus-core/.../uplc/eval/UplcSourceMap.scala). Offsets s/e are character offsets +// into `uplc` (end-exclusive); sl/sc/el/ec are 0-BASED source lines/columns (unlike +// profile.json, which is 1-based); n is the node's post-order index in the term tree. + +export const UPLC_MAP_SCHEMA_VERSION = 1; + +export interface UplcSpan { + s: number; + e: number; + n: number; + file: number; + sl: number; + sc: number; + el: number; + ec: number; + fn?: number; +} + +export interface UplcSourceMap { + schemaVersion: number; + uplc: string; + files: string[]; + functions: string[]; + spans: UplcSpan[]; +} + +/** Parse and validate a .uplc.json document. Throws on malformed input or an + * unsupported schema version. */ +export function parseUplcMap(text: string): UplcSourceMap { + const raw = JSON.parse(text) as Partial; + if (!raw || typeof raw.uplc !== "string" || !Array.isArray(raw.spans)) { + throw new Error("not a Scalus UPLC source map"); + } + if (raw.schemaVersion !== UPLC_MAP_SCHEMA_VERSION) { + throw new Error( + `unsupported UPLC map schemaVersion ${raw.schemaVersion} (expected ${UPLC_MAP_SCHEMA_VERSION})` + ); + } + return { + schemaVersion: raw.schemaVersion, + uplc: raw.uplc, + files: raw.files ?? [], + functions: raw.functions ?? [], + spans: raw.spans.filter(validSpan), + }; +} + +function validSpan(sp: unknown): sp is UplcSpan { + const x = sp as UplcSpan; + return ( + !!x && + typeof x.s === "number" && + typeof x.e === "number" && + typeof x.file === "number" && + typeof x.sl === "number" + ); +} + +/** True when the 0-based source position (line0, col0) falls inside the span's range. */ +function containsSource(sp: UplcSpan, line0: number, col0: number): boolean { + if (line0 < sp.sl || line0 > sp.el) { + return false; + } + if (line0 === sp.sl && col0 < sp.sc) { + return false; + } + if (line0 === sp.el && col0 > sp.ec) { + return false; + } + return true; +} + +/** All spans of `file` whose source range contains the cursor. */ +export function spansAtSource( + map: UplcSourceMap, + file: number, + line0: number, + col0: number +): UplcSpan[] { + return map.spans.filter((sp) => sp.file === file && containsSource(sp, line0, col0)); +} + +/** The smallest span containing the UPLC text offset, or undefined. */ +export function innermostSpanAt(map: UplcSourceMap, offset: number): UplcSpan | undefined { + let best: UplcSpan | undefined; + for (const sp of map.spans) { + if (offset >= sp.s && offset < sp.e) { + if (!best || sp.e - sp.s < best.e - best.s) { + best = sp; + } + } + } + return best; +} + +/** Every span attributed to the given function index. */ +export function spansForFunction(map: UplcSourceMap, fn: number): UplcSpan[] { + return map.spans.filter((sp) => sp.fn === fn); +} +``` + +- [ ] **Step 5: Run tests + typecheck, verify PASS** + +```bash +npm run typecheck && npm test +``` + +- [ ] **Step 6: Commit** + +```bash +git add src/uplcMap.ts test/smoke.ts && git commit -m "feat: uplcMap model and span queries for the UPLC source view" +``` + +--- + +### Task 7: UPLC virtual document + Show UPLC command + +**Files:** +- Modify: `src/profileStore.ts` (add `uplcUri` getter after `htmlUri`, ~line 60) +- Create: `src/uplcView.ts` +- Modify: `src/extension.ts` (wire UplcView + command) +- Modify: `package.json` (command contribution) + +**Interfaces:** +- Consumes: `parseUplcMap`, `UplcSourceMap` (Task 6); `ProfileStore` (`run`, `onDidChange`, existing getters); `runFile`/`resolveManifestFile` from `src/manifest.ts`. +- Produces: `class UplcView implements vscode.Disposable` with `constructor()`, `setSource(uri: vscode.Uri | undefined): Promise` (loads + parses the artifact, refreshes an open document), `show(): Promise` (opens the virtual doc beside), `get map(): UplcSourceMap | undefined`, `readonly docUri: vscode.Uri`. `ProfileStore.uplcUri: vscode.Uri | undefined`. Command `scalusProfile.showUplc` ("Scalus Profile: Show Compiled UPLC"). + +- [ ] **Step 1: Add `uplcUri` to ProfileStore** (mirror of `htmlUri`, lines 52-60): + +```ts + /** The selected run's UPLC source map, when the manifest lists one. */ + get uplcUri(): vscode.Uri | undefined { + if (!this._run || !this._manifestDir) { + return undefined; + } + const uplc = runFile(this._run, "uplc"); + return uplc + ? vscode.Uri.file(resolveManifestFile(this._manifestDir.fsPath, uplc)) + : undefined; + } +``` + +- [ ] **Step 2: Implement `src/uplcView.ts`** + +```ts +import * as vscode from "vscode"; +import { UplcSourceMap, parseUplcMap } from "./uplcMap"; + +export const UPLC_SCHEME = "scalus-uplc"; + +/** Read-only virtual document showing the compiled UPLC of the selected profile run, + * backed by the run's .uplc.json source map. */ +export class UplcView implements vscode.Disposable { + readonly docUri = vscode.Uri.parse(`${UPLC_SCHEME}:/compiled.uplc`); + private _map: UplcSourceMap | undefined; + private readonly onDidChangeEmitter = new vscode.EventEmitter(); + private readonly providerReg: vscode.Disposable; + + constructor() { + this.providerReg = vscode.workspace.registerTextDocumentContentProvider(UPLC_SCHEME, { + onDidChange: this.onDidChangeEmitter.event, + provideTextDocumentContent: () => this._map?.uplc ?? "", + }); + } + + get map(): UplcSourceMap | undefined { + return this._map; + } + + /** Load (or clear) the source map from the run's .uplc.json. */ + async setSource(uri: vscode.Uri | undefined): Promise { + if (!uri) { + this._map = undefined; + } else { + try { + const bytes = await vscode.workspace.fs.readFile(uri); + this._map = parseUplcMap(Buffer.from(bytes).toString("utf8")); + } catch (e) { + this._map = undefined; + console.warn(`Scalus Profile: ignoring UPLC map ${uri.fsPath}: ${(e as Error).message}`); + } + } + this.onDidChangeEmitter.fire(this.docUri); + } + + /** Open the UPLC document beside the active editor. */ + async show(): Promise { + if (!this._map) { + vscode.window.showInformationMessage( + "Scalus Profile: no UPLC map for this run. Re-run a profiled test with a Scalus version that emits it (format \"uplc\" in profile-manifest.json)." + ); + return; + } + const doc = await vscode.workspace.openTextDocument(this.docUri); + await vscode.window.showTextDocument(doc, { + viewColumn: vscode.ViewColumn.Beside, + preserveFocus: true, + preview: false, + }); + } + + dispose(): void { + this.providerReg.dispose(); + this.onDidChangeEmitter.dispose(); + } +} +``` + +- [ ] **Step 3: Wire into `src/extension.ts`** + +- `const uplcView = new UplcView();` after the other components; push onto `context.subscriptions`. +- Inside `refreshViews()` add: `void uplcView.setSource(store.uplcUri);` +- Register the command with the others: + +```ts + vscode.commands.registerCommand("scalusProfile.showUplc", () => uplcView.show()), +``` + +- [ ] **Step 4: package.json** – add to `contributes.commands`: + +```json +{ "command": "scalusProfile.showUplc", "title": "Show Compiled UPLC", "category": "Scalus Profile" } +``` + +- [ ] **Step 5: Verify** + +```bash +npm run typecheck && npm test && npm run compile +``` +Expected: all pass. + +- [ ] **Step 6: Commit** + +```bash +git add -A && git commit -m "feat: Show Compiled UPLC command with scalus-uplc virtual document" +``` + +--- + +### Task 8: Bidirectional cursor sync + +**Files:** +- Modify: `src/uplcView.ts` +- Modify: `src/extension.ts` + +**Interfaces:** +- Consumes: `spansAtSource`, `innermostSpanAt` (Task 6), `bestMatchingFile` from `src/pathMatch.ts` (existing: `bestMatchingFile(candidates: string[], editorPath: string): string | undefined`). +- Produces: `UplcView.onSelectionChanged(e: vscode.TextEditorSelectionChangeEvent): void` – the single sync entry point wired to `vscode.window.onDidChangeTextEditorSelection`. + +- [ ] **Step 1: Add decorations and sync to `UplcView`** + +Add to the class: + +```ts + private readonly highlight = vscode.window.createTextEditorDecorationType({ + backgroundColor: new vscode.ThemeColor("editor.findMatchHighlightBackground"), + }); + + private uplcEditor(): vscode.TextEditor | undefined { + return vscode.window.visibleTextEditors.find((e) => e.document.uri.scheme === UPLC_SCHEME); + } + + /** Sync highlights on any selection change: Scala -> UPLC or UPLC -> Scala. */ + onSelectionChanged(e: vscode.TextEditorSelectionChangeEvent): void { + const map = this._map; + if (!map) { + return; + } + if (e.textEditor.document.uri.scheme === UPLC_SCHEME) { + void this.syncToSource(e.textEditor, map); + } else if (e.textEditor.document.languageId === "scala") { + this.syncToUplc(e.textEditor, map); + } + } + + /** Scala cursor -> highlight matching UPLC spans. */ + private syncToUplc(editor: vscode.TextEditor, map: UplcSourceMap): void { + const target = this.uplcEditor(); + if (!target) { + return; + } + const match = bestMatchingFile(map.files, editor.document.uri.fsPath); + const file = match ? map.files.indexOf(match) : -1; + if (file < 0) { + target.setDecorations(this.highlight, []); + return; + } + const pos = editor.selection.active; + const spans = spansAtSource(map, file, pos.line, pos.character); + const ranges = spans.map( + (sp) => + new vscode.Range( + target.document.positionAt(sp.s), + target.document.positionAt(sp.e) + ) + ); + target.setDecorations(this.highlight, ranges); + if (ranges.length > 0) { + // Reveal the tightest (innermost) match. + const tightest = ranges.reduce((a, b) => + b.end.character - b.start.character + (b.end.line - b.start.line) * 1e6 < + a.end.character - a.start.character + (a.end.line - a.start.line) * 1e6 + ? b + : a + ); + target.revealRange(tightest, vscode.TextEditorRevealType.InCenterIfOutsideViewport); + } + } + + /** UPLC cursor -> highlight the originating Scala range. */ + private async syncToSource(editor: vscode.TextEditor, map: UplcSourceMap): Promise { + const offset = editor.document.offsetAt(editor.selection.active); + const span = innermostSpanAt(map, offset); + if (!span) { + return; + } + const mapped = map.files[span.file]; + const source = vscode.window.visibleTextEditors.find( + (ed) => + ed.document.languageId === "scala" && + bestMatchingFile(map.files, ed.document.uri.fsPath) === mapped + ); + if (!source) { + return; // v1: only highlight already-visible Scala editors + } + const last = source.document.lineCount - 1; + const clamp = (line: number) => Math.max(0, Math.min(line, last)); + const range = new vscode.Range(clamp(span.sl), span.sc, clamp(span.el), span.ec); + source.setDecorations(this.highlight, [range]); + source.revealRange(range, vscode.TextEditorRevealType.InCenterIfOutsideViewport); + } +``` + +Imports to add in `uplcView.ts`: `spansAtSource`, `innermostSpanAt` from `./uplcMap`; `bestMatchingFile` from `./pathMatch`. Dispose the decoration type in `dispose()`: `this.highlight.dispose();`. + +- [ ] **Step 2: Wire the listener in `extension.ts`** + +```ts + context.subscriptions.push( + vscode.window.onDidChangeTextEditorSelection((e) => uplcView.onSelectionChanged(e)) + ); +``` + +- [ ] **Step 3: Verify + manual smoke** + +```bash +npm run typecheck && npm test && npm run compile +``` +Then a quick manual test in the Extension Development Host (F5) against a scalus checkout with a generated artifact – see Task 10 Step 3 for how to generate one. Verify: cursor in the Scala file highlights UPLC; cursor in UPLC highlights Scala. + +- [ ] **Step 4: Commit** + +```bash +git add -A && git commit -m "feat: bidirectional cursor sync between Scala source and compiled UPLC" +``` + +--- + +### Task 9: Function-level highlight command + +**Files:** +- Modify: `src/uplcView.ts` +- Modify: `src/extension.ts`, `package.json` + +**Interfaces:** +- Consumes: `spansForFunction`, `spansAtSource` (Task 6). +- Produces: command `scalusProfile.showUplcForFunction` ("Show Compiled UPLC for Function"): from the cursor (Scala or UPLC editor), resolves the function of the span under the cursor and highlights ALL its spans in the UPLC editor. + +- [ ] **Step 1: Add to `UplcView`:** + +```ts + /** Highlight every UPLC span belonging to the function under the cursor. */ + async showFunction(): Promise { + const map = this._map; + if (!map) { + await this.show(); + return; + } + const editor = vscode.window.activeTextEditor; + if (!editor) { + return; + } + let fn: number | undefined; + if (editor.document.uri.scheme === UPLC_SCHEME) { + fn = innermostSpanAt(map, editor.document.offsetAt(editor.selection.active))?.fn; + } else { + const match = bestMatchingFile(map.files, editor.document.uri.fsPath); + const file = match ? map.files.indexOf(match) : -1; + const pos = editor.selection.active; + const withFn = file < 0 ? [] : spansAtSource(map, file, pos.line, pos.character) + .filter((sp) => sp.fn !== undefined) + .sort((a, b) => (a.el - a.sl) - (b.el - b.sl)); // innermost (smallest) first + fn = withFn[0]?.fn; + } + if (fn === undefined) { + vscode.window.showInformationMessage("Scalus Profile: no function found at the cursor."); + return; + } + await this.show(); + const target = this.uplcEditor(); + if (!target) { + return; + } + const ranges = spansForFunction(map, fn).map( + (sp) => new vscode.Range(target.document.positionAt(sp.s), target.document.positionAt(sp.e)) + ); + target.setDecorations(this.highlight, ranges); + if (ranges.length > 0) { + target.revealRange(ranges[0], vscode.TextEditorRevealType.InCenterIfOutsideViewport); + } + vscode.window.setStatusBarMessage( + `UPLC: ${ranges.length} region(s) from ${map.functions[fn]}`, + 5000 + ); + } +``` + +- [ ] **Step 2: Register command** in `extension.ts`: + +```ts + vscode.commands.registerCommand("scalusProfile.showUplcForFunction", () => + uplcView.showFunction() + ), +``` + +and in `package.json` `contributes.commands`: + +```json +{ "command": "scalusProfile.showUplcForFunction", "title": "Show Compiled UPLC for Function", "category": "Scalus Profile" } +``` + +- [ ] **Step 3: Verify and commit** + +```bash +npm run typecheck && npm test && npm run compile +git add -A && git commit -m "feat: highlight all compiled UPLC of the function under the cursor" +``` + +--- + +### Task 10: UPLC syntax grammar, docs, end-to-end check + +**Files:** +- Create: `syntaxes/uplc.tmLanguage.json` +- Modify: `package.json` (`contributes.languages`, `contributes.grammars`), `src/uplcView.ts` (set language), `README.md`, `CHANGELOG.md` + +- [ ] **Step 1: Grammar + contributions** + +`syntaxes/uplc.tmLanguage.json`: + +```json +{ + "$schema": "https://raw.githubusercontent.com/martinring/tmlanguage/master/tmlanguage.json", + "name": "Untyped Plutus Core", + "scopeName": "source.uplc", + "patterns": [ + { "match": "\\b(program|lam|delay|force|error|constr|case|con|builtin)\\b", "name": "keyword.control.uplc" }, + { "match": "(?<=\\(builtin\\s)[A-Za-z0-9_']+", "name": "support.function.uplc" }, + { "match": "(?<=\\(con\\s)[A-Za-z]+(\\s*\\([^)]*\\))?", "name": "storage.type.uplc" }, + { "match": "-?\\b\\d+\\b", "name": "constant.numeric.uplc" }, + { "match": "#[0-9a-fA-F]*", "name": "constant.other.uplc" }, + { "match": "\\b(True|False)\\b", "name": "constant.language.uplc" }, + { "match": "\"(\\\\.|[^\"])*\"", "name": "string.quoted.double.uplc" }, + { "match": "[\\[\\]()]", "name": "punctuation.section.uplc" } + ] +} +``` + +`package.json`: + +```json + "contributes": { + "languages": [ + { "id": "uplc", "aliases": ["Untyped Plutus Core", "UPLC"], "extensions": [".uplc"] } + ], + "grammars": [ + { "language": "uplc", "scopeName": "source.uplc", "path": "./syntaxes/uplc.tmLanguage.json" } + ] + } +``` + +(merge into the existing `contributes` object). In `UplcView.show()`, after `openTextDocument`, set the language: + +```ts + await vscode.languages.setTextDocumentLanguage(doc, "uplc"); +``` + +- [ ] **Step 2: README + CHANGELOG** + +README: add a "Compiled UPLC view" section documenting the two commands, the sync behavior, and the requirement (a Scalus version that writes `format: "uplc"` into `profile-manifest.json`, profile level `full`). CHANGELOG: add an Unreleased entry. + +- [ ] **Step 3: End-to-end verification (both repos)** + +In the scalus repo (branch `feature/uplc-source-view`), generate a real artifact by running any profiled example test, e.g.: + +```bash +cd /Users/nau/projects/lantr/scalus +SCALUS_PROFILE=full sbtn "scalusExamplesJVM/testOnly *HtlcValidator*" +ls scalus-examples/jvm/target/scalus/ | grep uplc +``` + +(If that suite does not produce profile reports, grep for `runWithProfileReport` usages in test sources and run one of those suites instead.) Expected: a `*.uplc.json` next to the profile files, and `"uplc"` inside `profile-manifest.json`. + +Then launch the Extension Development Host (F5) on the scalus workspace and verify the full flow: Show Compiled UPLC, both sync directions, function highlight, syntax colors. + +- [ ] **Step 4: Final verify + commit** + +```bash +npm run typecheck && npm test && npm run compile +git add -A && git commit -m "feat: uplc grammar, docs for the compiled UPLC view" +``` + +--- + +## Completion + +Both branches stay unmerged; report back with: +1. scalus branch: commits + `sbtn quick` and `sbtn mima` results. +2. extension branch: commits + `npm test` results + what was manually verified end-to-end. +3. Any deviations from this plan (annotate which task and why). diff --git a/docs/superpowers/specs/2026-07-31-uplc-source-view-design.md b/docs/superpowers/specs/2026-07-31-uplc-source-view-design.md new file mode 100644 index 000000000..dcfb393a5 --- /dev/null +++ b/docs/superpowers/specs/2026-07-31-uplc-source-view-design.md @@ -0,0 +1,187 @@ +# UPLC Source View – Design + +Date: 2026-07-31 +Status: Approved design, pending implementation plan +Repos: `scalus` (producer), `scalus-vscode-extension` (consumer) + +## Goal + +Show the compiled UPLC for a given Scala source line, definition, or function in the +Scalus Profiler VS Code extension. A side-by-side, bidirectionally synced view: +moving the cursor in the Scala file highlights the matching UPLC text, and moving +the cursor in the UPLC view highlights the originating Scala range. + +## Feasibility summary (verified) + +- Every UPLC `Term` node carries `UplcAnnotation(pos: ScalusSourcePos, functionName: String)` + (`scalus-core/shared/src/main/scala/scalus/uplc/UplcAnnotation.scala`). +- `ScalusSourcePos` has file, 0-based start/end line and column, and an `inlinedFrom` chain. +- Positions survive V3 lowering (~40 stamp sites in `LoweredValue.scala`), all UPLC + optimizer passes, `DeBruijn`, and `TermSanitizer`. Two fill passes in + `CompiledPlutus.toUplc` (`Compiled.scala`) give near-total coverage on the final term. +- Positions are erased at flat/CBOR encoding. The mapping must be extracted from the + in-memory `Term` before serialization. +- `UplcAnnotation.functionName` is currently never populated (dead field). +- The `Pretty[Term]` printer (`Term.scala`, paiges) discards annotations today. + +Ecosystem check: neither Aiken (shipped) nor Plutus emits a source-map file; both use +custom in-band/JSON approaches, not JS Source Map v3. Aiken's unmerged +`pi/source-maps` branch uses a custom JSON keyed by post-order node index; we adopt +that index as a forward-compatible join key. + +## Decisions + +| Decision | Choice | +|---|---| +| Primary UX | Side-by-side synced view (Compiler Explorer style) | +| Artifact timing | Written with profile reports (evaluation time) | +| Sync direction | Bidirectional | +| Function granularity | Populate `functionName` during lowering; spans carry it | +| Renderer | Existing paiges printer + decorator hook + zero-width markers | +| Format | Custom JSON (no ecosystem standard exists); includes post-order node index | + +## Architecture + +### 1. Scalus: function names in annotations + +- Add a `currentFunction: String` field to the V3 lowering context. + Set it when lowering a top-level binding and when lowering a `Let`-bound lambda. +- Annotation construction sites in `LoweredValue.scala` go through one helper that + builds `UplcAnnotation(pos, ctx.currentFunction)`. +- Extend `fillEmptyPosBottomUp` / `fillEmptyPosTopDown` to back-fill the whole + annotation (pos and functionName), not only pos. Method signatures do not change, + so MiMa is unaffected. + +### 2. Scalus: span-aware rendering (hook + markers) + +Why not record offsets in `Pretty[Term]` directly: the printer builds a paiges `Doc`, +a layout tree. Text offsets exist only after `doc.render(width)`, and paiges provides +no render callback and no annotation channel (unlike Haskell's `prettyprinter`). +The printer already depends on paiges zero-width output for ANSI styling +(`d.style(...)` in `Term.scala`), so zero-width markers use a supported mechanism and +provably do not change layout. + +Design: + +- Refactor the `Pretty[Term]` printer to accept a decorator hook + `(Term, Doc) => Doc`, default identity. Default output stays byte-identical. +- New `UplcSourceMapRenderer`: + - The hook wraps each node that has a non-empty effective position in + `Doc.zeroWidth` markers: `` before, `/` after. + `id` indexes an array of collected annotations. + - Render at the same width the plain `show` uses. + - One post-render scan strips markers and records `(startOffset, endOffset)` per id + in the clean text. Spans nest; nesting is expected and used for innermost-match. + - During the same traversal, assign each node its post-order index (children + visited in declaration order, then the parent). Post-order keeps existing + indices stable when a program is later wrapped in `Apply` nodes for parameter + application, matching Aiken's convention. +- Invariant (tested): marker-stripped output equals the plain pretty output. + +The renderer runs on the same in-memory `Term` the CEK machine evaluated. That term's +positions already feed the profiler, so it is available at report time by construction. + +### 3. Artifact and manifest + +New file per run, next to the profile files (default `target/scalus/`): +`--.uplc.json` + +```json +{ + "schemaVersion": 1, + "uplc": "(program 1.1.0 ...)", + "files": ["/abs/path/Validator.scala"], + "functions": ["validate", "checkSig"], + "spans": [ + { "s": 120, "e": 245, "n": 17, + "file": 0, "sl": 16, "sc": 4, "el": 18, "ec": 20, "fn": 0 } + ] +} +``` + +- `s`/`e`: character offsets into `uplc` (start inclusive, end exclusive). +- `n`: post-order node index of the term node. Forward-compatible join key for + future consumers that work on decoded on-chain scripts (debuggers, coverage). +- `file`/`fn`: indices into the `files`/`functions` string tables. `fn` optional. +- `sl`/`sc`/`el`/`ec`: 0-based source lines and columns (raw `ScalusSourcePos`). + Note: `profile.json` uses 1-based lines; this artifact is 0-based and documents it. +- `inlinedFrom` is omitted in v1. +- Spans are emitted only for nodes with a non-empty effective position. + +Wiring: + +- `ProfileReportWriter.write` takes the evaluated term as an optional parameter and + writes the artifact when the profile level is `Full`, the term carries source info, + and the run rendered at least one profile file (a console-only report stays off + disk, and never replaces a manifest run that indexes profile files). No new + `ProfileFormat` case: those are rendered from `ProfilingData`, which has no `Term`. +- The file registers in the existing `profile-manifest.json` run as + `{ "format": "uplc", "file": "..." }`. Manifest `schemaVersion` stays 1; the + extension's `parseManifest` ignores unknown formats, so old extension versions are + unaffected. +- Writer plumbing follows `ProfileReportWriter` (jsoniter-scala codec, existing + locked read-merge-write for the manifest). +- Producers: `PlutusScriptEvaluator.renderProfile` and + `ScalusTest.runWithProfileReport`, i.e. both existing manifest writers. + +### 4. Extension: side-by-side synced view + +- New pure module `src/uplcMap.ts`: parse the artifact, offset-to-position helpers, + span queries (spans intersecting a source range; innermost span containing a UPLC + offset; all spans of a function). No `vscode` imports, testable in `test/smoke.ts`. +- New `UplcContentProvider` (`TextDocumentContentProvider`, scheme `scalus-uplc`). + Document content is the `uplc` text of the active run. Read-only by construction. +- Commands: + - `scalusProfile.showUplc`: opens the UPLC document beside the active editor + (`ViewColumn.Beside`) for the run selected in `ProfileStore`. + - `scalusProfile.showUplcForFunction`: highlights every span whose `fn` matches the + function under the cursor. +- Cursor sync via `window.onDidChangeTextEditorSelection`: + - Scala to UPLC: match the file with the existing `bestMatchingFile`; find spans + whose source range contains the cursor; decorate those UPLC ranges + (`TextDocument.positionAt(offset)`); reveal the first. + - UPLC to Scala: find the innermost span containing the cursor offset; decorate + and reveal its source range in the matching Scala editor. +- Highlight decoration uses a theme color (e.g. `editor.findMatchHighlightBackground`). +- The UPLC document follows the selected run; `ProfileStore.onDidChange` refreshes it. +- Nice-to-have: contribute a minimal TextMate grammar for language id `uplc` so the + view is not plain text. + +### 5. Error handling + +- Run has no `uplc` file: info message "No UPLC map found. Re-run a profiled test + with a Scalus version that emits it." +- Source drift after edits: highlights may be off until the profile is regenerated. + Same limitation as the existing cost decorations; accepted for v1. +- Path mismatch (artifact produced on CI or another checkout): handled by the + existing trailing-segment `bestMatchingFile` matching. +- Malformed or wrong `schemaVersion` artifact: treated as absent, log to the output + channel. + +### 6. Testing + +Scalus: +- Invariant test: marker-stripped render equals plain pretty render for a corpus of + compiled programs. +- Span correctness: compile a small validator, assert selected spans map to expected + source lines and function names. +- Manifest test: `uplc` entry merges into an existing manifest without dropping runs. + +Extension: +- Unit tests in the vscode-free smoke harness: artifact parsing, innermost-span + query, source-range intersection, function-span query. + +### Out of scope (future work) + +- Cost overlay inside the UPLC document (per-span cpu/mem from `profile.json`). +- `inlinedFrom` chains in the artifact and hover. +- Build-time sbt task emitting the artifact without running tests. +- Standard Source Map v3 emission for external tooling. + +## Delivery + +Two-repo rollout, producer first: + +1. `scalus`: annotations, renderer, artifact writer, tests. Ships in the next release. +2. `scalus-vscode-extension`: consumer feature; degrades gracefully (info message) + when the artifact is absent. diff --git a/scalus-cardano-ledger/shared/src/main/scala/scalus/cardano/ledger/PlutusScriptEvaluator.scala b/scalus-cardano-ledger/shared/src/main/scala/scalus/cardano/ledger/PlutusScriptEvaluator.scala index f79517719..0cb9c1b61 100644 --- a/scalus-cardano-ledger/shared/src/main/scala/scalus/cardano/ledger/PlutusScriptEvaluator.scala +++ b/scalus-cardano-ledger/shared/src/main/scala/scalus/cardano/ledger/PlutusScriptEvaluator.scala @@ -355,24 +355,15 @@ object PlutusScriptEvaluator { private def budgetLogPath: String = reportPath("budget.log") - /** Render a script's profile to each configured [[ProfileOutput]] (console / files). File - * destinations are prefixed with the script key so per-redeemer profiles don't collide, - * and are also recorded in `profile-manifest.json` (see [[writeProfileManifest]]). The - * actual rendering is delegated to the platform-specific [[ProfileReporting]] so that - * [[scalus.uplc.eval.ProfileFormatter]] (HTML/CSS/JS templates, Tarjan pass) stays out of - * the JS bundle; HTML output annotates source lines when the source file is readable from - * the CWD (JVM only — [[ProfileReporting]] returns `None` on JS). - * - * @note - * This is fed by a *separate* profiling evaluation of the script (see the call site), so - * enabling profiling roughly doubles evaluation cost. That profiling pass counts budget - * but does not enforce the redeemer's execution-unit limit, so it is only run after the - * real (budget-enforcing) evaluation has already succeeded. - */ /** Render a script's profile to the configured destinations and index the written files in * `profile-manifest.json`. Delegated to [[ProfileReportWriter]], which is shared with * test-side profiling (`ScalusTest.runWithProfileReport`) so both produce the same layout. * + * @param uplcTerm + * the script's term, for the `.uplc.json` source map. By name because it can force + * a CBOR decode, which is wasted work when no profile was produced. A decoded term + * carries no annotations, so scripts that did not come from an in-memory `Program` get + * no source map. * @note * This is fed by a *separate* profiling evaluation of the script (see the call site), so * enabling profiling roughly doubles evaluation cost. That profiling pass counts budget @@ -383,7 +374,8 @@ object PlutusScriptEvaluator { result: Result, scriptHash: ScriptHash, redeemer: Redeemer, - language: Language + language: Language, + uplcTerm: => Term ): Unit = result.profile.foreach { data => ProfileReportWriter.write( data, @@ -392,7 +384,8 @@ object PlutusScriptEvaluator { language.toString, redeemer.tag.toString, redeemer.index, - log.info(_) + log.info(_), + Some(uplcTerm) ) } @@ -724,7 +717,8 @@ object PlutusScriptEvaluator { vm.evaluateScriptProfile(applied), plutusScript.scriptHash, redeemer, - vm.language + vm.language, + plutusScript.program.term ) Result.Success(resultTerm, spender.getSpentBudget, Map.empty, logger.getLogs.toSeq) catch diff --git a/scalus-core/jvm/src/test/scala/scalus/uplc/eval/ProfileReportWriterUplcTest.scala b/scalus-core/jvm/src/test/scala/scalus/uplc/eval/ProfileReportWriterUplcTest.scala new file mode 100644 index 000000000..b51a28f88 --- /dev/null +++ b/scalus-core/jvm/src/test/scala/scalus/uplc/eval/ProfileReportWriterUplcTest.scala @@ -0,0 +1,128 @@ +package scalus.uplc.eval + +import org.scalatest.funsuite.AnyFunSuite +import scalus.cardano.ledger.{EvaluatorReportConfig, ExUnits, ProfileDestination, ProfileFormat, ProfileLevel, ProfileOutput} +import scalus.uplc.* +import scalus.uplc.DefaultFun.AddInteger +import scalus.utils.ScalusSourcePos + +import java.nio.file.Files + +/** The `.uplc.json` artifact [[ProfileReportWriter]] writes next to a full profile, and its + * `"uplc"` entry in `profile-manifest.json`. + */ +class ProfileReportWriterUplcTest extends AnyFunSuite { + + private val annotated: Term = + Term.Builtin(AddInteger, UplcAnnotation(ScalusSourcePos("/src/A.scala", 3, 0, 3, 5), "f")) + + private val emptyProfile: ProfilingData = ProfilingData( + bySourceLocation = Nil, + byFunction = Nil, + byLocationFunction = Nil, + transitions = Nil, + totalBudget = ExUnits(memory = 0, steps = 0) + ) + + private def fullReport(dir: java.nio.file.Path) = EvaluatorReportConfig( + enabled = true, + outputDir = dir.toString, + profile = ProfileLevel.Full + ) + + test("uplc.json is written and indexed in the manifest") { + val dir = Files.createTempDirectory("scalus-uplc-test") + ProfileReportWriter.write( + emptyProfile, + fullReport(dir), + "cafe01", + "PlutusV3", + "Spend", + 0, + _ => (), + Some(annotated) + ) + val uplcFile = dir.resolve("cafe01-Spend-0.uplc.json") + assert(Files.exists(uplcFile)) + val json = new String(Files.readAllBytes(uplcFile), "UTF-8") + assert(json.contains("\"schemaVersion\":1") || json.contains("\"schemaVersion\": 1")) + assert(json.contains("addInteger")) + val manifest = + new String(Files.readAllBytes(dir.resolve("profile-manifest.json")), "UTF-8") + assert(manifest.contains("\"uplc\"")) + assert(manifest.contains("cafe01-Spend-0.uplc.json")) + } + + test("no artifact for a term without source info") { + val dir = Files.createTempDirectory("scalus-uplc-test2") + ProfileReportWriter.write( + emptyProfile, + fullReport(dir), + "cafe02", + "PlutusV3", + "Spend", + 0, + _ => (), + Some(Term.Const(Constant.Integer(1))) + ) + assert(!Files.exists(dir.resolve("cafe02-Spend-0.uplc.json"))) + } + + test("no artifact below profile level Full") { + val dir = Files.createTempDirectory("scalus-uplc-test3") + val summary = EvaluatorReportConfig( + enabled = true, + outputDir = dir.toString, + profile = ProfileLevel.Summary + ) + ProfileReportWriter.write( + emptyProfile, + summary, + "cafe03", + "PlutusV3", + "Spend", + 0, + _ => (), + Some(annotated) + ) + assert(!Files.exists(dir.resolve("cafe03-Spend-0.uplc.json"))) + } + + test("no artifact, and no manifest, for a console-only report") { + val dir = Files.createTempDirectory("scalus-uplc-test5") + val consoleOnly = EvaluatorReportConfig( + enabled = true, + outputDir = dir.toString, + profile = ProfileLevel.Full, + profileOutputs = Seq(ProfileOutput(ProfileFormat.Text, ProfileDestination.Console)) + ) + var consoleOutput = "" + ProfileReportWriter.write( + emptyProfile, + consoleOnly, + "cafe05", + "PlutusV3", + "Spend", + 0, + line => consoleOutput += line, + Some(annotated) + ) + assert(consoleOutput.nonEmpty, "the console rendering itself must still happen") + assert(!Files.exists(dir.resolve("cafe05-Spend-0.uplc.json"))) + assert(!Files.exists(dir.resolve("profile-manifest.json"))) + } + + test("no artifact when no term is passed") { + val dir = Files.createTempDirectory("scalus-uplc-test4") + ProfileReportWriter.write( + emptyProfile, + fullReport(dir), + "cafe04", + "PlutusV3", + "Spend", + 0, + _ => () + ) + assert(!Files.exists(dir.resolve("cafe04-Spend-0.uplc.json"))) + } +} diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweredValue.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweredValue.scala index 8c71b7ab3..d84cfb8a6 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweredValue.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweredValue.scala @@ -35,6 +35,20 @@ trait LoweredValue { val createdEx = new RuntimeException("Lowered value created here") var debugMark = "i" + /** Simple name of the user function that was being lowered when this value was created, or `""` + * when it was created outside any named binding. Captured from + * [[LoweringContext.currentFunction]] at construction time, because term generation runs as a + * separate pass after lowering has finished and no longer knows where a value came from. + * + * Pure metadata: it is copied into the [[scalus.uplc.UplcAnnotation]] of every term this value + * emits (see [[ann]]) so tooling can group the compiled UPLC by source function. It never + * influences the generated code. + */ + val functionName: String = LoweringContext.currentFunctionName + + /** Annotation for a term emitted by this value: `p` plus [[functionName]]. */ + def ann(p: SIRPosition): UplcAnnotation = UplcAnnotation(p, functionName) + def sirType: SIRType def pos: SIRPosition @@ -255,7 +269,7 @@ case class ConstantLoweredValue( override def isEffortLess: Boolean = true override def isConstant: Boolean = true override def termInternal(gctx: TermGenerationContext): Term = - Term.Const(sir.uplcConst, UplcAnnotation(pos)) + Term.Const(sir.uplcConst, ann(pos)) override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { (Pretty[Term].pretty(Term.Const(sir.uplcConst), ctx.style) + Doc.text(":") + Doc.text( @@ -432,7 +446,7 @@ class VariableLoweredValue( Set(this) ++ optRhs.map(rhs => rhs.usedUplevelVars).getOrElse(Set.empty) override def termInternal(gctx: TermGenerationContext): Term = { - if gctx.generatedVars.contains(id) then Term.Var(NamedDeBruijn(id), UplcAnnotation(pos)) + if gctx.generatedVars.contains(id) then Term.Var(NamedDeBruijn(id), ann(pos)) else optRhs match { case Some(rhs) => @@ -446,7 +460,7 @@ class VariableLoweredValue( s"VariableLoweredValue: generating term for undefined variable $name with id $id" ) } - Term.Var(NamedDeBruijn(id), UplcAnnotation(pos)) + Term.Var(NamedDeBruijn(id), ann(pos)) } else throw new IllegalStateException( s"Variable $name with id $id is not defined and has no rhs to generate term." @@ -548,7 +562,7 @@ case class DependendVariableLoweredValue( override def optRhs: Option[LoweredValue] = Some(rhs) override def termInternal(gctx: TermGenerationContext): Term = { - if gctx.generatedVars.contains(id) then Term.Var(NamedDeBruijn(id), UplcAnnotation(pos)) + if gctx.generatedVars.contains(id) then Term.Var(NamedDeBruijn(id), ann(pos)) else rhs.termWithNeededVars(gctx) } @@ -623,7 +637,7 @@ case class DelayLoweredValue(input: LoweredValue, override val pos: SIRPosition) extends ProxyLoweredValue(input) { override def termInternal(gctx: TermGenerationContext): Term = { - Term.Delay(input.termWithNeededVars(gctx), UplcAnnotation(pos)) + Term.Delay(input.termWithNeededVars(gctx), ann(pos)) } override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -638,7 +652,7 @@ case class ForceLoweredValue(input: LoweredValue, override val pos: SIRPosition) extends ProxyLoweredValue(input) { override def termInternal(gctx: TermGenerationContext): Term = { - Term.Force(input.termWithNeededVars(gctx), UplcAnnotation(pos)) + Term.Force(input.termWithNeededVars(gctx), ann(pos)) } override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -768,7 +782,7 @@ case class LambdaLoweredValue(newVar: VariableLoweredValue, body: LoweredValue, Term.LamAbs( newVar.id, body.termWithNeededVars(gctx.addGeneratedVar(newVar.id)), - UplcAnnotation(pos) + ann(pos) ) } @@ -849,7 +863,7 @@ case class BuilinApply1LoweredVale( Term.Apply( fun.bn.tpf, arg.termWithNeededVars(gctx), - UplcAnnotation(pos) + ann(pos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -874,10 +888,10 @@ case class BuilinApply2LoweredVale( Term.Apply( Lowering.forcedBuiltin(fun.bn), arg1.termWithNeededVars(gctx), - UplcAnnotation(pos) + ann(pos) ), arg2.termWithNeededVars(gctx), - UplcAnnotation(pos) + ann(pos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -904,7 +918,7 @@ case class ApplyLoweredValue( Term.Apply( f.termWithNeededVars(gctx), arg.termWithNeededVars(gctx), - UplcAnnotation(pos) + ann(pos) ) } @@ -947,13 +961,13 @@ case class LetNonRecLoweredValue( val bodyTerm = body.termWithNeededVars(bodyGctx) bindings.foldRight(bodyTerm) { case ((varVal, rhs), term) => Term.Apply( - Term.LamAbs(varVal.id, term, UplcAnnotation(pos)), + Term.LamAbs(varVal.id, term, ann(pos)), rhs.termWithNeededVars( gctx.copy( generatedVars = gctx.generatedVars + varVal.id ) ), - UplcAnnotation(pos) + ann(pos) ) } } @@ -1004,14 +1018,14 @@ case class LetRecLoweredValue( Term.LamAbs( newVar.id, rhs.termWithNeededVars(nGctx), - UplcAnnotation(pos) + ann(pos) ), - UplcAnnotation(pos) + ann(pos) ) Term.Apply( - Term.LamAbs(newVar.id, body.termWithNeededVars(nGctx), UplcAnnotation(pos)), + Term.LamAbs(newVar.id, body.termWithNeededVars(nGctx), ann(pos)), fixed, - UplcAnnotation(pos) + ann(pos) ) } @@ -1096,7 +1110,7 @@ case class CaseBooleanLoweredValue( falseBranch.termWithNeededVars(gctx), trueBranch.termWithNeededVars(gctx) ), - UplcAnnotation(pos) + ann(pos) ) } @@ -1139,7 +1153,7 @@ case class CaseIntegerLoweredValue( Term.Case( scrutinee.termWithNeededVars(gctx), branches.map(_.termWithNeededVars(gctx)), - UplcAnnotation(pos) + ann(pos) ) } @@ -1194,15 +1208,15 @@ case class CaseListLoweredValue( val consCtx = gctx.copy(generatedVars = gctx.generatedVars + consHead.id + consTail.id) val consTerm = Term.LamAbs( consHead.id, - Term.LamAbs(consTail.id, consBranch.termWithNeededVars(consCtx), UplcAnnotation(pos)), - UplcAnnotation(pos) + Term.LamAbs(consTail.id, consBranch.termWithNeededVars(consCtx), ann(pos)), + ann(pos) ) val branches = optNilBranch match case Some(nilBranch) => scala.collection.immutable.List(consTerm, nilBranch.termWithNeededVars(gctx)) case None => scala.collection.immutable.List(consTerm) - Term.Case(scrutinee.termWithNeededVars(gctx), branches, UplcAnnotation(pos)) + Term.Case(scrutinee.termWithNeededVars(gctx), branches, ann(pos)) } override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -1252,11 +1266,11 @@ case class CasePairLoweredValue( scala.collection.immutable.List( Term.LamAbs( fstVar.id, - Term.LamAbs(sndVar.id, body.termWithNeededVars(bodyCtx), UplcAnnotation(pos)), - UplcAnnotation(pos) + Term.LamAbs(sndVar.id, body.termWithNeededVars(bodyCtx), ann(pos)), + ann(pos) ) ), - UplcAnnotation(pos) + ann(pos) ) } @@ -1343,28 +1357,28 @@ case class CaseDataLoweredValue( Term.LamAbs( constrArgsVar.id, constrBranch.termWithNeededVars(constrCtx), - UplcAnnotation(pos) + ann(pos) ), - UplcAnnotation(pos) + ann(pos) ), // Map branch (index 1): λentries.body Term.LamAbs( mapEntriesVar.id, mapBranch.termWithNeededVars(mapCtx), - UplcAnnotation(pos) + ann(pos) ), // List branch (index 2): λelements.body Term.LamAbs( listElementsVar.id, listBranch.termWithNeededVars(listCtx), - UplcAnnotation(pos) + ann(pos) ), // I branch (index 3): λvalue.body - Term.LamAbs(iValueVar.id, iBranch.termWithNeededVars(iCtx), UplcAnnotation(pos)), + Term.LamAbs(iValueVar.id, iBranch.termWithNeededVars(iCtx), ann(pos)), // B branch (index 4): λvalue.body - Term.LamAbs(bValueVar.id, bBranch.termWithNeededVars(bCtx), UplcAnnotation(pos)) + Term.LamAbs(bValueVar.id, bBranch.termWithNeededVars(bCtx), ann(pos)) ), - UplcAnnotation(pos) + ann(pos) ) } @@ -1417,7 +1431,7 @@ case class ChooseListLoweredValue( import scalus.compiler.sir.lowering.Lowering.tpf (!(DefaultFun.ChooseList.tpf $ listInput.termWithNeededVars(gctx) $ ~nilBody.termWithNeededVars(gctx) - $ ~consBody.termWithNeededVars(gctx))).withPosIfEmpty(pos) + $ ~consBody.termWithNeededVars(gctx))).withAnnotationIfEmpty(ann(pos)) } override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -1493,7 +1507,7 @@ case class ChooseDataLoweredValue( $ ~mapBranch.termWithNeededVars(gctx) $ ~listBranch.termWithNeededVars(gctx) $ ~iBranch.termWithNeededVars(gctx) - $ ~bBranch.termWithNeededVars(gctx))).withPosIfEmpty(pos) + $ ~bBranch.termWithNeededVars(gctx))).withAnnotationIfEmpty(ann(pos)) } override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/Lowering.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/Lowering.scala index 083f54182..462e3ac31 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/Lowering.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/Lowering.scala @@ -7,7 +7,6 @@ import scalus.compiler.sir.lowering.typegens.SirTypeUplcGenerator import scalus.cardano.onchain.plutus.prelude.List as PList import scalus.pretty import scalus.uplc.* -import scalus.uplc.UplcAnnotation import scala.util.control.NonFatal @@ -496,7 +495,7 @@ object Lowering { ) val loweredMsg = lowerSIR(msg, Some(SIRType.String)) val errorTerm = - ErrorLoweredValue(sirError, ~Term.Error(UplcAnnotation(anns.pos))) + ErrorLoweredValue(sirError, ~Term.Error(lctx.ann(anns.pos))) lvForce( lvBuiltinApply2( SIRBuiltins.trace, @@ -508,11 +507,41 @@ object Lowering { ), anns.pos ) - else ErrorLoweredValue(sirError, Term.Error(UplcAnnotation(anns.pos))) + else ErrorLoweredValue(sirError, Term.Error(lctx.ann(anns.pos))) lctx.nestingLevel -= 1 retval } + /** Trailing `-` that the plugin's `VariableKey` appends to local binding names so + * shadowed variables stay distinct. + */ + private[lowering] val LocalBindingIdSuffix = """^(.+)-\d+$""".r + + /** The source-level name of a binding, for display in the UPLC source view. Strips the package + * and owner prefix that linked top-level defs carry (`scalus.examples.Foo$.bar` -> `bar`) and + * the symbol id that local defs carry (`double-432208` -> `double`). + */ + private[lowering] def simpleBindingName(name: String): String = { + val dotIdx = name.lastIndexOf('.') + val simple = if dotIdx >= 0 then name.substring(dotIdx + 1) else name + simple match + case LocalBindingIdSuffix(base) => base + case _ => simple + } + + /** Lower `body` as the right-hand side of the binding `name`, so every value created while + * lowering it is stamped with `name` (see [[LoweringContext.currentFunction]]). + * + * Only function-shaped right-hand sides open a new scope. A plain value binding is part of the + * code of whatever function encloses it, so it keeps the enclosing name. + */ + private[lowering] def loweringBinding[A](name: String, rhs: SIR)( + body: => A + )(using lctx: LoweringContext): A = + rhs match + case _: SIR.LamAbs => lctx.withFunction(simpleBindingName(name))(body) + case _ => body + private def lowerLet(sirLet: SIR.Let)(using lctx: LoweringContext): LoweredValue = { val retval = sirLet match case SIR.Let(bindings, body, flags, anns) => @@ -520,7 +549,8 @@ object Lowering { if !flags.isRec then val bindingValues = bindings.map { b => var prevDebug = lctx.debug - val loweredRhs = lowerSIR(b.value, Some(b.tp)) + val loweredRhs = + loweringBinding(b.name, b.value)(lowerSIR(b.value, Some(b.tp))) if lctx.debug then lctx.log( s"[LET binding] ${b.name}: lowered RHS type ${loweredRhs.sirType.show}, repr ${loweredRhs.representation}, target type ${b.tp.show}" @@ -574,9 +604,11 @@ object Lowering { lctx.scope = lctx.scope.add(newVar) val loweredRhs = - lowerSIR(rhs) - .maybeUpcast(tp, anns.pos) - .toRepresentation(rhsRepr, anns.pos) + loweringBinding(name, rhs) { + lowerSIR(rhs) + .maybeUpcast(tp, anns.pos) + .toRepresentation(rhsRepr, anns.pos) + } val loweredBody = lowerSIR(body) lctx.scope = prevScope @@ -1186,17 +1218,17 @@ object Lowering { v match case dv: DependendVariableLoweredValue => Term.Apply( - Term.LamAbs(dv.id, term, UplcAnnotation(dv.pos)), + Term.LamAbs(dv.id, term, dv.ann(dv.pos)), dv.rhs.termWithNeededVars(nGctx), - UplcAnnotation(dv.pos) + dv.ann(dv.pos) ) case v: VariableLoweredValue => v.optRhs match case Some(rhs) => Term.Apply( - Term.LamAbs(v.id, term, UplcAnnotation(v.pos)), + Term.LamAbs(v.id, term, v.ann(v.pos)), rhs.termWithNeededVars(nGctx), - UplcAnnotation(v.pos) + v.ann(v.pos) ) case None => throw LoweringException( @@ -1234,17 +1266,17 @@ object Lowering { v match case dv: DependendVariableLoweredValue => Term.Apply( - Term.LamAbs(dv.id, term, UplcAnnotation(dv.pos)), + Term.LamAbs(dv.id, term, dv.ann(dv.pos)), dv.rhs.termWithNeededVars(nGctx), - UplcAnnotation(dv.pos) + dv.ann(dv.pos) ) case v: VariableLoweredValue => v.optRhs match case Some(rhs) => Term.Apply( - Term.LamAbs(v.id, term, UplcAnnotation(v.pos)), + Term.LamAbs(v.id, term, v.ann(v.pos)), rhs.termWithNeededVars(nGctx), - UplcAnnotation(v.pos) + v.ann(v.pos) ) case None => throw LoweringException( diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweringContext.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweringContext.scala index 8f96d90e6..558fe2d78 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweringContext.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/LoweringContext.scala @@ -2,6 +2,7 @@ package scalus.compiler.sir.lowering import scalus.cardano.ledger.{Language, MajorProtocolVersion} import scalus.compiler.sir.* +import scalus.uplc.UplcAnnotation import scala.collection.mutable.Map as MutableMap @@ -39,6 +40,32 @@ class LoweringContext( val supportModules: Map[String, Module] = Map.empty, ) { + /** Simple name of the innermost enclosing user function being lowered, or `""` outside any + * named binding. Stamped into [[scalus.uplc.UplcAnnotation.functionName]] so tooling (the VS + * Code UPLC source view) can group compiled UPLC by the source function it came from. It is + * pure metadata: it never influences the generated code. + * + * Backed by a thread-local in the companion rather than by a plain field, because every + * [[LoweredValue]] captures it at construction time (see [[LoweredValue.functionName]]) and + * lowered values are built deep inside helpers that do not all carry a `LoweringContext`. + * Several lowerings can run at once in one JVM (parallel sbt subprojects, parallel test + * suites), so the state has to be per-thread. Always change it through [[withFunction]]. + */ + def currentFunction: String = LoweringContext.currentFunctionName + + def currentFunction_=(name: String): Unit = LoweringContext.currentFunctionName = name + + /** Annotation for a term built at lowering time: position plus the enclosing function name. */ + def ann(pos: SIRPosition): UplcAnnotation = UplcAnnotation(pos, currentFunction) + + /** Run `body` with [[currentFunction]] set to `name`, restoring the previous value after. */ + def withFunction[A](name: String)(body: => A): A = { + val saved = currentFunction + currentFunction = name + try body + finally currentFunction = saved + } + private val bindingCache = MutableMap.empty[(String, String), Option[Binding]] /** Annotation-keyed cache of pre-lowered values. Indexed by Int. @@ -244,6 +271,17 @@ class LoweringContext( object LoweringContext { + /** Enclosing function name for the lowering running on the current thread. See + * [[LoweringContext.currentFunction]] for why this is thread-local state instead of a field. + */ + private val currentFunctionTL: ThreadLocal[String] = new ThreadLocal[String] { + override def initialValue(): String = "" + } + + def currentFunctionName: String = currentFunctionTL.get() + + def currentFunctionName_=(name: String): Unit = currentFunctionTL.set(name) + /** Process-wide trace facility for `pendingTopLevelLetRecs` add/hit events. Gated by * `SCALUS_TRACE_LETREC` env var (or `-Dscalus.trace.letrec=true` JVM prop). Emits a monotonic * counter so events across multiple `compile { }` calls in the same JVM can be interleaved and diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/ScalusRuntime.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/ScalusRuntime.scala index ad3389052..3132769b5 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/ScalusRuntime.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/ScalusRuntime.scala @@ -4,7 +4,7 @@ import org.typelevel.paiges.Doc import scalus.cardano.ledger.MajorProtocolVersion import scalus.compiler.sir.lowering.LoweredValue.Builder.* import scalus.compiler.sir.* -import scalus.uplc.{Term, UplcAnnotation} +import scalus.uplc.Term import scalus.compiler.sir.lowering.typegens.SumUplcConstrOps object ScalusRuntime { @@ -47,7 +47,10 @@ object ScalusRuntime { val prevFlag = lctx.inUplcConstrListScope if isNativeConstr then lctx.inUplcConstrListScope = true try - val lowered = Lowering.lowerSIR(d.value) + // Same attribution as a user binding: everything lowered here belongs to the + // support def, so the UPLC source view can name prelude/support code too. + val lowered = + Lowering.loweringBinding(d.name, d.value)(Lowering.lowerSIR(d.value)) LoweredValue.Builder.lvNewLazyNamedVar( d.name, d.tp, @@ -653,7 +656,7 @@ object ScalusRuntime { Term.Constr( scalus.cardano.ledger.Word64(0L), scala.List.empty, - UplcAnnotation(constrPos) + ann(constrPos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = Doc.text("Constr(0)") @@ -721,7 +724,7 @@ object ScalusRuntime { convertedHead.termWithNeededVars(gctx), recCall.termWithNeededVars(gctx) ), - UplcAnnotation(AnnotationsDecl.empty.pos) + ann(AnnotationsDecl.empty.pos) ) override def docDef( ctx: LoweredValue.PrettyPrintingContext diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProdUplcConstrOps.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProdUplcConstrOps.scala index c21e41ab8..0a00590ec 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProdUplcConstrOps.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProdUplcConstrOps.scala @@ -4,7 +4,7 @@ package typegens import org.typelevel.paiges.Doc import scalus.compiler.sir.lowering.ProductCaseClassRepresentation.* import scalus.compiler.sir.* -import scalus.uplc.{Term, UplcAnnotation} +import scalus.uplc.Term /** Emitter for `ProductCaseClassRepresentation.ProdUplcConstr` — native UPLC * `Constr(tag, [t1, t2, ...])` emission for product values. @@ -89,7 +89,7 @@ object ProdUplcConstrOps { Term.Constr( scalus.cardano.ledger.Word64(constrIndex.toLong), adoptedArgs.map(_.termWithNeededVars(gctx)).toList, - UplcAnnotation(constr.anns.pos) + ann(constr.anns.pos) ) } diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseEmitter.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseEmitter.scala index 965afdea9..87c2730cc 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseEmitter.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseEmitter.scala @@ -6,7 +6,7 @@ import scalus.cardano.ledger.MajorProtocolVersion import scalus.compiler.sir.lowering.LoweredValue.Builder.* import scalus.compiler.sir.lowering.ProductCaseClassRepresentation.* import scalus.compiler.sir.* -import scalus.uplc.{Term, UplcAnnotation} +import scalus.uplc.Term /** Product with one element without parent, represented as an element. */ @@ -317,7 +317,7 @@ object ProductCaseEmitter extends SirTypeUplcGenerator { Term.Constr( scalus.cardano.ledger.Word64(puc.tag.toLong), fields.map(_.termWithNeededVars(gctx)).toList, - UplcAnnotation(inPos) + ann(inPos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext) = Doc.text("DataList→UplcConstr") @@ -429,15 +429,15 @@ object ProductCaseEmitter extends SirTypeUplcGenerator { gctx.copy(generatedVars = gctx.generatedVars ++ fieldVars.map(_.id)) val body = dataList.termWithNeededVars(innerCtx) val branch = fieldVars.foldRight(body) { (fv, inner) => - Term.LamAbs(fv.id, inner, UplcAnnotation(pos)) + Term.LamAbs(fv.id, inner, ann(pos)) } // Pad with Error branches for tags < this constructor's tag val errorBranches = - scala.List.fill(puc.tag)(Term.Error(UplcAnnotation(pos))) + scala.List.fill(puc.tag)(Term.Error(ann(pos))) Term.Case( input.termWithNeededVars(gctx), errorBranches :+ branch, - UplcAnnotation(pos) + ann(pos) ) } override def docDef(ctx: LoweredValue.PrettyPrintingContext) = @@ -488,17 +488,17 @@ object ProductCaseEmitter extends SirTypeUplcGenerator { val body = Term.Constr( scalus.cardano.ledger.Word64(outPuc.tag.toLong), convertedFields.map(_.termWithNeededVars(innerCtx)).toList, - UplcAnnotation(inPos) + ann(inPos) ) val branch = fieldVars.foldRight(body: Term) { (fv, inner) => - Term.LamAbs(fv.id, inner, UplcAnnotation(inPos)) + Term.LamAbs(fv.id, inner, ann(inPos)) } val errorBranches = - scala.List.fill(inPuc.tag)(Term.Error(UplcAnnotation(inPos))) + scala.List.fill(inPuc.tag)(Term.Error(ann(inPos))) Term.Case( input.termWithNeededVars(gctx), errorBranches :+ branch, - UplcAnnotation(inPos) + ann(inPos) ) } override def docDef(ctx: LoweredValue.PrettyPrintingContext) = diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseUplcConstrOnlyEmitter.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseUplcConstrOnlyEmitter.scala index c5d856d72..145912977 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseUplcConstrOnlyEmitter.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/ProductCaseUplcConstrOnlyEmitter.scala @@ -42,7 +42,7 @@ object ProductCaseUplcConstrOnlyEmitter extends SirTypeUplcGenerator { override def genSelect(sel: SIR.Select, loweredScrutinee: LoweredValue)(using lctx: LoweringContext ): LoweredValue = { - import scalus.uplc.{Term, UplcAnnotation} + import scalus.uplc.Term val pos = sel.anns.pos val constrDecl = ProductCaseEmitter.retrieveConstrDecl( @@ -96,15 +96,15 @@ object ProductCaseUplcConstrOnlyEmitter extends SirTypeUplcGenerator { val innerCtx = gctx.copy(generatedVars = gctx.generatedVars ++ fieldNames) val body = Term.Var(scalus.uplc.NamedDeBruijn(selectedFieldName)) val branch = fieldNames.foldRight(body: Term) { (name, inner) => - Term.LamAbs(name, inner, UplcAnnotation(sel.anns.pos)) + Term.LamAbs(name, inner, ann(sel.anns.pos)) } // Pad with Error branches for tags < this constructor's tag val errorBranches = - scala.List.fill(puc.tag)(Term.Error(UplcAnnotation(sel.anns.pos))) + scala.List.fill(puc.tag)(Term.Error(ann(sel.anns.pos))) Term.Case( loweredScrutinee.termWithNeededVars(gctx), errorBranches :+ branch, - UplcAnnotation(sel.anns.pos) + ann(sel.anns.pos) ) } diff --git a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/SumUplcConstrOps.scala b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/SumUplcConstrOps.scala index 6fee872ae..7ef59db52 100644 --- a/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/SumUplcConstrOps.scala +++ b/scalus-core/shared/src/main/scala/scalus/compiler/sir/lowering/typegens/SumUplcConstrOps.scala @@ -7,7 +7,7 @@ import scalus.compiler.sir.* import scalus.compiler.sir.SIR.Pattern import scalus.compiler.sir.lowering.LoweredValue.Builder.* import scalus.compiler.sir.lowering.SumCaseClassRepresentation.* -import scalus.uplc.{Term, UplcAnnotation} +import scalus.uplc.Term /** Type generator for sum types using UplcConstr representation. * @@ -38,7 +38,7 @@ object SumUplcConstrOps { override def termInternal(gctx: TermGenerationContext): Term = { val innerCtx = gctx.copy(generatedVars = gctx.generatedVars + fieldVar.id) - Term.LamAbs(fieldVar.id, inner.termWithNeededVars(innerCtx), UplcAnnotation(casePos)) + Term.LamAbs(fieldVar.id, inner.termWithNeededVars(innerCtx), ann(casePos)) } override def toRepresentation( @@ -329,9 +329,9 @@ object SumUplcConstrOps { Term.LamAbs( tailVar.id, consResult.termWithNeededVars(ngctx), - UplcAnnotation(constrPos) + ann(constrPos) ), - UplcAnnotation(constrPos) + ann(constrPos) ) } override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = @@ -347,7 +347,7 @@ object SumUplcConstrOps { Term.Case( scrutinee.termWithNeededVars(gctx), scala.List(nilBody.termWithNeededVars(gctx), consBranch.termWithNeededVars(gctx)), - UplcAnnotation(constrPos) + ann(constrPos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = Doc.text( @@ -435,7 +435,7 @@ object SumUplcConstrOps { val ngctx = fieldVars.foldLeft(gctx)((g, v) => g.addGeneratedVar(v.id)) val innerTerm = branchBody.termWithNeededVars(ngctx) fieldVars.foldRight(innerTerm) { (v, body) => - Term.LamAbs(v.id, body, UplcAnnotation(constrPos)) + Term.LamAbs(v.id, body, ann(constrPos)) } } override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -455,7 +455,7 @@ object SumUplcConstrOps { Term.Case( scrutinee.termWithNeededVars(gctx), branches.map(_.termWithNeededVars(gctx)).toList, - UplcAnnotation(constrPos) + ann(constrPos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { val branchDocs = branches.zipWithIndex.map { case (b, i) => @@ -540,7 +540,7 @@ object SumUplcConstrOps { Term.Case( scrutinee.termWithNeededVars(gctx), branchesList.map(_.termWithNeededVars(gctx)), - UplcAnnotation(matchPos) + ann(matchPos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext): Doc = { @@ -876,7 +876,7 @@ object SumUplcConstrOps { Term.Constr( scalus.cardano.ledger.Word64(idx.toLong), convertedFields.map(_.termWithNeededVars(innerCtx)).toList, - UplcAnnotation(inPos) + ann(inPos) ) } override def docDef(ctx: LoweredValue.PrettyPrintingContext) = @@ -893,7 +893,7 @@ object SumUplcConstrOps { Term.LamAbs( fv.id, inner.termWithNeededVars(ngctx), - UplcAnnotation(inPos) + ann(inPos) ) } override def docDef(ctx: LoweredValue.PrettyPrintingContext) = @@ -913,7 +913,7 @@ object SumUplcConstrOps { Term.Case( inputVal.termWithNeededVars(gctx), branches.map(_.termWithNeededVars(gctx)).toList, - UplcAnnotation(pos) + ann(pos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext) = Doc.text("UplcConstr→UplcConstr(") + inputVal.docRef(ctx) + Doc.text(")") @@ -1025,7 +1025,7 @@ object SumUplcConstrOps { Term.Constr( scalus.cardano.ledger.Word64(idx.toLong), fields.map(_.termWithNeededVars(gctx)).toList, - UplcAnnotation(inPos) + ann(inPos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext) = Doc.text(s"PairIntDataList→UplcConstr($idx)") @@ -1116,7 +1116,7 @@ object SumUplcConstrOps { override def pos = inPos override def termInternal(gctx: TermGenerationContext) = { val ctx = gctx.copy(generatedVars = gctx.generatedVars + fv.id) - Term.LamAbs(fv.id, inner.termWithNeededVars(ctx), UplcAnnotation(inPos)) + Term.LamAbs(fv.id, inner.termWithNeededVars(ctx), ann(inPos)) } override def docDef(ctx: LoweredValue.PrettyPrintingContext) = Doc.text(s"λ${fv.name}.") + inner.docRef(ctx) @@ -1133,7 +1133,7 @@ object SumUplcConstrOps { Term.Case( input.termWithNeededVars(gctx), branches.map(_.termWithNeededVars(gctx)).toList, - UplcAnnotation(inPos) + ann(inPos) ) override def docDef(ctx: LoweredValue.PrettyPrintingContext) = Doc.text("UplcConstr→DataConstr(") + input.docRef(ctx) + Doc.text(")") diff --git a/scalus-core/shared/src/main/scala/scalus/uplc/Compiled.scala b/scalus-core/shared/src/main/scala/scalus/uplc/Compiled.scala index 6ec68ba9b..9a1d5a150 100644 --- a/scalus-core/shared/src/main/scala/scalus/uplc/Compiled.scala +++ b/scalus-core/shared/src/main/scala/scalus/uplc/Compiled.scala @@ -124,13 +124,14 @@ sealed abstract class CompiledPlutus[A]( options.uplcOptimizers.foldLeft(uplc)((term, opt) => opt(term)) else if options.optimizeUplc then optimizer(uplc) else uplc - // Give every still-position-less node a source location, so profiling and source-traces can - // attribute the cost of generated/optimized spines (the UPLC optimizer rebuilds Apply/Case/ - // Constr nodes without positions). Run on the FINAL term, after optimization: bottom-up so a - // spine node inherits the location of the leaf it operates on (where positions actually sit), - // then top-down to fill any node with no positioned descendant from its nearest positioned - // ancestor. Positions never affect flat encoding, budget, or evaluation — only diagnostics. - optimized.fillEmptyPosBottomUp._1.fillEmptyPosTopDown(scalus.utils.ScalusSourcePos.empty) + // Give every still-un-annotated node a source location and an enclosing function name, so + // profiling and source-traces can attribute the cost of generated/optimized spines (the UPLC + // optimizer rebuilds Apply/Case/Constr nodes without annotations). Run on the FINAL term, + // after optimization: bottom-up so a spine node inherits the annotation of the leaf it + // operates on (where annotations actually sit), then top-down to fill any node with no + // annotated descendant from its nearest annotated ancestor. Annotations never affect flat + // encoding, budget, or evaluation – only diagnostics. + optimized.fillEmptyAnnotationsBottomUp._1.fillEmptyAnnotationsTopDown(UplcAnnotation.empty) } } diff --git a/scalus-core/shared/src/main/scala/scalus/uplc/Term.scala b/scalus-core/shared/src/main/scala/scalus/uplc/Term.scala index ad5c1906f..0b47ece95 100644 --- a/scalus-core/shared/src/main/scala/scalus/uplc/Term.scala +++ b/scalus-core/shared/src/main/scala/scalus/uplc/Term.scala @@ -63,34 +63,42 @@ enum Term: /** Returns a copy of this term with the given source position. */ def withPos(pos: ScalusSourcePos): Term = withAnnotation(UplcAnnotation(pos)) - /** Sets the annotation on this term and recursively on subterms, but only where the annotation - * is currently empty. Recursion stops at terms that already have an annotation. + /** Sets the annotation on this term and recursively on subterms, but only where the position is + * currently empty. Recursion stops at terms that already have a position. + * + * Keys on the position alone, and keeps a `functionName` the term already carries: lowering + * stamps the enclosing function on terms that have no position of their own, and those still + * need a position. Keying on whole-annotation emptiness would treat them as already annotated + * and stop the recursion into their whole subtree. */ def withAnnotationIfEmpty(ann: UplcAnnotation): Term = - if !annotation.isEmpty then this + if !annotation.pos.isEmpty then this else + val self = + if annotation.functionName.isEmpty then ann + else annotation.copy(pos = ann.pos) this match - case t: Var => t.copy(annotation = ann) + case t: Var => t.copy(annotation = self) case t: LamAbs => - t.copy(term = t.term.withAnnotationIfEmpty(ann), annotation = ann) + t.copy(term = t.term.withAnnotationIfEmpty(ann), annotation = self) case t: Apply => t.copy( f = t.f.withAnnotationIfEmpty(ann), arg = t.arg.withAnnotationIfEmpty(ann), - annotation = ann + annotation = self ) - case t: Force => t.copy(term = t.term.withAnnotationIfEmpty(ann), annotation = ann) - case t: Delay => t.copy(term = t.term.withAnnotationIfEmpty(ann), annotation = ann) - case t: Const => t.copy(annotation = ann) - case t: Builtin => t.copy(annotation = ann) - case t: Error => t.copy(annotation = ann) + case t: Force => t.copy(term = t.term.withAnnotationIfEmpty(ann), annotation = self) + case t: Delay => t.copy(term = t.term.withAnnotationIfEmpty(ann), annotation = self) + case t: Const => t.copy(annotation = self) + case t: Builtin => t.copy(annotation = self) + case t: Error => t.copy(annotation = self) case t: Constr => - t.copy(args = t.args.map(_.withAnnotationIfEmpty(ann)), annotation = ann) + t.copy(args = t.args.map(_.withAnnotationIfEmpty(ann)), annotation = self) case t: Case => t.copy( arg = t.arg.withAnnotationIfEmpty(ann), cases = t.cases.map(_.withAnnotationIfEmpty(ann)), - annotation = ann + annotation = self ) /** Sets the source position on this term and recursively on subterms, but only where the @@ -98,74 +106,80 @@ enum Term: */ def withPosIfEmpty(pos: ScalusSourcePos): Term = withAnnotationIfEmpty(UplcAnnotation(pos)) - /** Bottom-up pass: gives every position-less subterm the source position of its nearest - * positioned *descendant*, and returns that representative position for this subtree (its own - * if positioned, else the first positioned child, preferring the function/scrutinee side). + /** Bottom-up pass: gives every un-annotated subterm the annotation of its nearest annotated + * *descendant*, and returns that representative annotation for this subtree (its own if + * annotated, else the first annotated child, preferring the function/scrutinee side). + * + * This is the main filler for lowered code: annotations sit on the leaves (the `Var`/`Const`/ + * `Builtin` a value references), while the `Apply`/`Case`/`Constr` spine that combines them is + * built un-annotated. A spine node here inherits the location and enclosing function name of + * what it operates on – e.g. an application inherits the annotation of the function being + * applied – so its cost is attributed to that code rather than vanishing. * - * This is the main filler for lowered code: source positions sit on the leaves (the `Var`/ - * `Const`/`Builtin` a value references), while the `Apply`/`Case`/`Constr` spine that combines - * them is built position-less. A spine node here inherits the location of what it operates on - * — e.g. an application inherits the position of the function being applied — so its cost is - * attributed to that code rather than vanishing. Never overwrites an existing position. + * Fields are filled independently and an existing one is never overwritten: a term that + * lowering stamped with its enclosing function name (but no position) still gets a position, + * and keeps its own name. */ - def fillEmptyPosBottomUp: (Term, ScalusSourcePos) = + private[scalus] def fillEmptyAnnotationsBottomUp: (Term, UplcAnnotation) = // Resolve each candidate's effective position first (a synthetic compile-boundary root - // becomes the real user call it was inlined from), then take the first real one — so - // provenance wins over the structural descendant/ancestor fallback. - def firstNonEmpty(ps: ScalusSourcePos*): ScalusSourcePos = - ps.iterator - .map(_.effectivePos) - .find(!_.isEffectivelyEmpty) - .getOrElse(ScalusSourcePos.empty) - def stamp(t: Term, rep: ScalusSourcePos): UplcAnnotation = - if t.annotation.isEffectivelyEmpty && !rep.isEffectivelyEmpty then UplcAnnotation(rep) - else t.annotation + // becomes the real user call it was inlined from), then take the first candidate that + // resolves to a real position – so provenance wins over the structural descendant/ancestor + // fallback. The winner's *whole* annotation becomes the representative, so a filled node's + // function name always describes the same code as its position. + def firstNonEmpty(as: UplcAnnotation*): UplcAnnotation = + as.iterator + .map { a => + val p = a.pos.effectivePos + if p eq a.pos then a else a.copy(pos = p) + } + .find(!_.pos.isEffectivelyEmpty) + .getOrElse(UplcAnnotation.empty) this match - case t: Var => (t, t.annotation.pos) - case t: Const => (t, t.annotation.pos) - case t: Builtin => (t, t.annotation.pos) - case t: Error => (t, t.annotation.pos) + case t: Var => (t, t.annotation) + case t: Const => (t, t.annotation) + case t: Builtin => (t, t.annotation) + case t: Error => (t, t.annotation) case t: LamAbs => - val (b, bp) = t.term.fillEmptyPosBottomUp - val rep = firstNonEmpty(t.annotation.pos, bp) - val ann = stamp(t, rep) + val (b, ba) = t.term.fillEmptyAnnotationsBottomUp + val rep = firstNonEmpty(t.annotation, ba) + val ann = Term.fillEmptyFields(t.annotation, rep) ( if (b eq t.term) && (ann eq t.annotation) then t else t.copy(term = b, annotation = ann), rep ) case t: Force => - val (b, bp) = t.term.fillEmptyPosBottomUp - val rep = firstNonEmpty(t.annotation.pos, bp) - val ann = stamp(t, rep) + val (b, ba) = t.term.fillEmptyAnnotationsBottomUp + val rep = firstNonEmpty(t.annotation, ba) + val ann = Term.fillEmptyFields(t.annotation, rep) ( if (b eq t.term) && (ann eq t.annotation) then t else t.copy(term = b, annotation = ann), rep ) case t: Delay => - val (b, bp) = t.term.fillEmptyPosBottomUp - val rep = firstNonEmpty(t.annotation.pos, bp) - val ann = stamp(t, rep) + val (b, ba) = t.term.fillEmptyAnnotationsBottomUp + val rep = firstNonEmpty(t.annotation, ba) + val ann = Term.fillEmptyFields(t.annotation, rep) ( if (b eq t.term) && (ann eq t.annotation) then t else t.copy(term = b, annotation = ann), rep ) case t: Apply => - val (f, fp) = t.f.fillEmptyPosBottomUp - val (arg, ap) = t.arg.fillEmptyPosBottomUp - val rep = firstNonEmpty(t.annotation.pos, fp, ap) - val ann = stamp(t, rep) + val (f, fa) = t.f.fillEmptyAnnotationsBottomUp + val (arg, aa) = t.arg.fillEmptyAnnotationsBottomUp + val rep = firstNonEmpty(t.annotation, fa, aa) + val ann = Term.fillEmptyFields(t.annotation, rep) ( if (f eq t.f) && (arg eq t.arg) && (ann eq t.annotation) then t else t.copy(f = f, arg = arg, annotation = ann), rep ) case t: Constr => - val processed = t.args.map(_.fillEmptyPosBottomUp) - val rep = firstNonEmpty((t.annotation.pos +: processed.map(_._2))*) - val ann = stamp(t, rep) + val processed = t.args.map(_.fillEmptyAnnotationsBottomUp) + val rep = firstNonEmpty((t.annotation +: processed.map(_._2))*) + val ann = Term.fillEmptyFields(t.annotation, rep) val args = processed.map(_._1) ( if args.corresponds(t.args)(_ eq _) && (ann eq t.annotation) then t @@ -173,10 +187,10 @@ enum Term: rep ) case t: Case => - val (arg, ap) = t.arg.fillEmptyPosBottomUp - val processed = t.cases.map(_.fillEmptyPosBottomUp) - val rep = firstNonEmpty((t.annotation.pos +: ap +: processed.map(_._2))*) - val ann = stamp(t, rep) + val (arg, aa) = t.arg.fillEmptyAnnotationsBottomUp + val processed = t.cases.map(_.fillEmptyAnnotationsBottomUp) + val rep = firstNonEmpty((t.annotation +: aa +: processed.map(_._2))*) + val ann = Term.fillEmptyFields(t.annotation, rep) val cases = processed.map(_._1) ( if (arg eq t.arg) && cases.corresponds(t.cases)(_ eq _) && (ann eq t.annotation) @@ -185,55 +199,67 @@ enum Term: rep ) - /** Top-down pass: stamps every position-less subterm with the source position of its nearest - * enclosing positioned ancestor (`inherited` at the root). Positioned subterms keep their own - * position and become the inherited position for their descendants. + /** Top-down pass: stamps every un-annotated subterm with the annotation of its nearest + * enclosing annotated ancestor (`inherited` at the root). Annotated subterms keep what they + * have and become the inherited annotation for their descendants; as in the bottom-up pass, + * position and function name are filled independently and never overwritten. * - * This completes [[fillEmptyPosBottomUp]]: per-value stamping during lowering can only place a - * position a lowered value actually knows, but many `Apply`/`Let` SIR nodes carry no position - * at all (the plugin doesn't stamp them), so the spines they build stay position-less. Here - * those nodes inherit the source location of the surrounding code — which is exactly what - * profiling and source traces should attribute their cost to. + * This completes [[fillEmptyAnnotationsBottomUp]]: per-value stamping during lowering can only + * place an annotation a lowered value actually knows, but many `Apply`/`Let` SIR nodes carry + * no position at all (the plugin doesn't stamp them), so the spines they build stay + * un-annotated. Here those nodes inherit the source location of the surrounding code – which + * is exactly what profiling and source traces should attribute their cost to. */ - def fillEmptyPosTopDown(inherited: ScalusSourcePos): Term = - val eff = if annotation.isEffectivelyEmpty then inherited else annotation.pos - val selfAnn = - if annotation.isEffectivelyEmpty && !inherited.isEffectivelyEmpty then - UplcAnnotation(inherited) - else annotation + private[scalus] def fillEmptyAnnotationsTopDown(inherited: UplcAnnotation): Term = + // What this term ends up with is also what its descendants inherit: its own fields where it + // has them, the ancestor's where it does not. + val selfAnn = Term.fillEmptyFields(annotation, inherited) this match case t: Var => if selfAnn eq t.annotation then t else t.copy(annotation = selfAnn) case t: Const => if selfAnn eq t.annotation then t else t.copy(annotation = selfAnn) case t: Builtin => if selfAnn eq t.annotation then t else t.copy(annotation = selfAnn) case t: Error => if selfAnn eq t.annotation then t else t.copy(annotation = selfAnn) case t: LamAbs => - val b = t.term.fillEmptyPosTopDown(eff) + val b = t.term.fillEmptyAnnotationsTopDown(selfAnn) if (selfAnn eq t.annotation) && (b eq t.term) then t else t.copy(term = b, annotation = selfAnn) case t: Force => - val b = t.term.fillEmptyPosTopDown(eff) + val b = t.term.fillEmptyAnnotationsTopDown(selfAnn) if (selfAnn eq t.annotation) && (b eq t.term) then t else t.copy(term = b, annotation = selfAnn) case t: Delay => - val b = t.term.fillEmptyPosTopDown(eff) + val b = t.term.fillEmptyAnnotationsTopDown(selfAnn) if (selfAnn eq t.annotation) && (b eq t.term) then t else t.copy(term = b, annotation = selfAnn) case t: Apply => - val f = t.f.fillEmptyPosTopDown(eff) - val arg = t.arg.fillEmptyPosTopDown(eff) + val f = t.f.fillEmptyAnnotationsTopDown(selfAnn) + val arg = t.arg.fillEmptyAnnotationsTopDown(selfAnn) if (selfAnn eq t.annotation) && (f eq t.f) && (arg eq t.arg) then t else t.copy(f = f, arg = arg, annotation = selfAnn) case t: Constr => - val args = t.args.map(_.fillEmptyPosTopDown(eff)) + val args = t.args.map(_.fillEmptyAnnotationsTopDown(selfAnn)) if (selfAnn eq t.annotation) && args.corresponds(t.args)(_ eq _) then t else t.copy(args = args, annotation = selfAnn) case t: Case => - val arg = t.arg.fillEmptyPosTopDown(eff) - val cases = t.cases.map(_.fillEmptyPosTopDown(eff)) + val arg = t.arg.fillEmptyAnnotationsTopDown(selfAnn) + val cases = t.cases.map(_.fillEmptyAnnotationsTopDown(selfAnn)) if (selfAnn eq t.annotation) && (arg eq t.arg) && cases.corresponds(t.cases)(_ eq _) then t else t.copy(arg = arg, cases = cases, annotation = selfAnn) + /** Bottom-up position fill: see [[fillEmptyAnnotationsBottomUp]], of which this is the + * position-only view. Returns the representative position of this subtree. + */ + def fillEmptyPosBottomUp: (Term, ScalusSourcePos) = + val (t, rep) = fillEmptyAnnotationsBottomUp + (t, rep.pos) + + /** Top-down position fill: see [[fillEmptyAnnotationsTopDown]], of which this is the + * position-only view. The root inherits `inherited` and no function name. + */ + def fillEmptyPosTopDown(inherited: ScalusSourcePos): Term = + fillEmptyAnnotationsTopDown(UplcAnnotation(inherited)) + /** Applies the argument to the term. */ infix def $(rhs: Term): Term = Term.Apply(this, rhs) @@ -414,6 +440,24 @@ enum Term: object Term { + /** Returns `own` with each of its empty fields filled from `rep`, leaving the fields it already + * has untouched. Used by both annotation fill passes, so they never overwrite anything the + * lowering knew: a term stamped with only its enclosing function name gains a position, and a + * term that has a position of its own keeps it. + * + * Returns `own` itself (reference-equal) when there is nothing to fill, so callers can detect + * that with `eq` and keep the node as is instead of rebuilding it. + */ + private def fillEmptyFields(own: UplcAnnotation, rep: UplcAnnotation): UplcAnnotation = + val fillPos = own.pos.isEffectivelyEmpty && !rep.pos.isEffectivelyEmpty + val fillName = own.functionName.isEmpty && rep.functionName.nonEmpty + if !fillPos && !fillName then own + else + UplcAnnotation( + if fillPos then rep.pos else own.pos, + if fillName then rep.functionName else own.functionName + ) + /** Truncate a string to a maximum length, showing only first line if multiline */ private[uplc] def truncateForDisplay(s: String, maxLength: Int = 60): String = val firstLine = s.linesIterator.nextOption().getOrElse("") @@ -560,96 +604,132 @@ object Term { /** Pretty[Term] instance with rainbow brackets based on nesting depth */ given Pretty[Term] with def pretty(term: Term, style: Style): Doc = - prettyTermWithDepth(TermSanitizer.sanitizeNames(term), style, depth = 0) + TermPrinter.prettySanitized(TermSanitizer.sanitizeNames(term), style, (_, d) => d) - private def prettyTermWithDepth(term: Term, style: Style, depth: Int): Doc = - import Pretty.{kw, rainbowChar} +} - // Local extension that captures 'style' from enclosing scope - extension (d: Doc) - def styled(s: paiges.Style): Doc = - if style == Style.XTerm then d.style(s) else d +/** Pretty-printer for [[Term]] with a per-node decorator hook. + * + * This is the implementation behind `given Pretty[Term]`, exposed so callers that need to attach + * extra information to individual nodes (source positions, cost annotations, ...) can reuse the + * exact same layout. + */ +private[scalus] object TermPrinter { + import Term.* - term match - case Var(name, _) => text(name.name) + /** Pretty-print an already-sanitized term, passing every printed node's Doc through `decorate`. + * + * `(term, doc) => doc` reproduces `Term.pretty` exactly. Inner `Apply` nodes of a flattened + * application chain are not printed individually and are not decorated. + * + * @param term + * a term whose names are already sanitized (see [[TermSanitizer.sanitizeNames]]); this + * method does not sanitize + * @param style + * plain or XTerm-highlighted output + * @param decorate + * applied to the Doc of every printed node, together with that node + */ + def prettySanitized(term: Term, style: Style, decorate: (Term, Doc) => Doc): Doc = + prettyTermWithDepth(term, style, depth = 0, decorate) + + private def prettyTermWithDepth( + term: Term, + style: Style, + depth: Int, + decorate: (Term, Doc) => Doc + ): Doc = + import Pretty.{kw, rainbowChar} + + // Local extension that captures 'style' from enclosing scope + extension (d: Doc) + def styled(s: paiges.Style): Doc = + if style == Style.XTerm then d.style(s) else d + + val doc = term match + case Var(name, _) => text(name.name) + + case LamAbs(name, body, _) => + // (lam name body) with rainbow parens at current depth + val openP = rainbowChar('(', depth, style) + val closeP = rainbowChar(')', depth, style) + val prefix = openP + kw("lam", style) & text(name) + ((prefix / prettyTermWithDepth(body, style, depth + 1, decorate)) + .nested(2) + .grouped + closeP).grouped + + case a @ Apply(_, _, _) => + // [f arg1 arg2 ...] with rainbow brackets + val (t, args) = a.applyToList + val openB = rainbowChar('[', depth, style) + val closeB = rainbowChar(']', depth, style) + val allTerms = + (t :: args).map(prettyTermWithDepth(_, style, depth + 1, decorate)) + val body = intercalate(lineOrSpace, allTerms) + ((openB + body).nested(2).grouped + closeB).grouped + + case Force(t, _) => + // (force term) + val openP = rainbowChar('(', depth, style) + val closeP = rainbowChar(')', depth, style) + (openP + kw("force", style) & prettyTermWithDepth( + t, + style, + depth + 1, + decorate + )).grouped + closeP + + case Delay(t, _) => + // (delay term) + val openP = rainbowChar('(', depth, style) + val closeP = rainbowChar(')', depth, style) + (openP + kw("delay", style) & prettyTermWithDepth( + t, + style, + depth + 1, + decorate + )).grouped + closeP + + case Const(const, _) => + // (con type value) - no depth increase, leaf node + val openP = rainbowChar('(', depth, style) + val closeP = rainbowChar(')', depth, style) + openP + kw("con", style) & const.pretty.styled(Fg.colorCode(64)) + closeP + + case Builtin(bn, _) => + val openP = rainbowChar('(', depth, style) + val closeP = rainbowChar(')', depth, style) + openP + kw("builtin", style) & PrettyPrinter + .pretty(bn) + .styled(Fg.colorCode(176)) + closeP + + case Error(_) => + kw("(error)", style) - case LamAbs(name, body, _) => - // (lam name body) with rainbow parens at current depth - val openP = rainbowChar('(', depth, style) - val closeP = rainbowChar(')', depth, style) - val prefix = openP + kw("lam", style) & text(name) - ((prefix / prettyTermWithDepth(body, style, depth + 1)) + case Constr(tag, args, _) => + // (constr tag arg1 arg2 ...) + val openP = rainbowChar('(', depth, style) + val closeP = rainbowChar(')', depth, style) + val argDocs = args.map(prettyTermWithDepth(_, style, depth + 1, decorate)) + val body = kw("constr", style) & str(tag.value) + if argDocs.isEmpty then openP + body + closeP + else + ((openP + body & intercalate(lineOrSpace, argDocs)) .nested(2) .grouped + closeP).grouped - case a @ Apply(_, _, _) => - // [f arg1 arg2 ...] with rainbow brackets - val (t, args) = a.applyToList - val openB = rainbowChar('[', depth, style) - val closeB = rainbowChar(']', depth, style) - val allTerms = (t :: args).map(prettyTermWithDepth(_, style, depth + 1)) - val body = intercalate(lineOrSpace, allTerms) - ((openB + body).nested(2).grouped + closeB).grouped - - case Force(t, _) => - // (force term) - val openP = rainbowChar('(', depth, style) - val closeP = rainbowChar(')', depth, style) - (openP + kw("force", style) & prettyTermWithDepth( - t, - style, - depth + 1 - )).grouped + closeP - - case Delay(t, _) => - // (delay term) - val openP = rainbowChar('(', depth, style) - val closeP = rainbowChar(')', depth, style) - (openP + kw("delay", style) & prettyTermWithDepth( - t, - style, - depth + 1 - )).grouped + closeP - - case Const(const, _) => - // (con type value) - no depth increase, leaf node - val openP = rainbowChar('(', depth, style) - val closeP = rainbowChar(')', depth, style) - openP + kw("con", style) & const.pretty.styled(Fg.colorCode(64)) + closeP - - case Builtin(bn, _) => - val openP = rainbowChar('(', depth, style) - val closeP = rainbowChar(')', depth, style) - openP + kw("builtin", style) & PrettyPrinter - .pretty(bn) - .styled(Fg.colorCode(176)) + closeP - - case Error(_) => - kw("(error)", style) - - case Constr(tag, args, _) => - // (constr tag arg1 arg2 ...) - val openP = rainbowChar('(', depth, style) - val closeP = rainbowChar(')', depth, style) - val argDocs = args.map(prettyTermWithDepth(_, style, depth + 1)) - val body = kw("constr", style) & str(tag.value) - if argDocs.isEmpty then openP + body + closeP - else - ((openP + body & intercalate(lineOrSpace, argDocs)) - .nested(2) - .grouped + closeP).grouped - - case Case(arg, cases, _) => - // (case scrutinee branch1 branch2 ...) - val openP = rainbowChar('(', depth, style) - val closeP = rainbowChar(')', depth, style) - val argDoc = prettyTermWithDepth(arg, style, depth + 1) - val caseDocs = cases.map(prettyTermWithDepth(_, style, depth + 1)) - val body = kw("case", style) & argDoc - if caseDocs.isEmpty then openP + body + closeP - else - ((openP + body & intercalate(lineOrSpace, caseDocs)) - .nested(2) - .grouped + closeP).grouped + case Case(arg, cases, _) => + // (case scrutinee branch1 branch2 ...) + val openP = rainbowChar('(', depth, style) + val closeP = rainbowChar(')', depth, style) + val argDoc = prettyTermWithDepth(arg, style, depth + 1, decorate) + val caseDocs = cases.map(prettyTermWithDepth(_, style, depth + 1, decorate)) + val body = kw("case", style) & argDoc + if caseDocs.isEmpty then openP + body + closeP + else + ((openP + body & intercalate(lineOrSpace, caseDocs)) + .nested(2) + .grouped + closeP).grouped + decorate(term, doc) } diff --git a/scalus-core/shared/src/main/scala/scalus/uplc/eval/ProfileReportWriter.scala b/scalus-core/shared/src/main/scala/scalus/uplc/eval/ProfileReportWriter.scala index 10e82bd83..ad87881ee 100644 --- a/scalus-core/shared/src/main/scala/scalus/uplc/eval/ProfileReportWriter.scala +++ b/scalus-core/shared/src/main/scala/scalus/uplc/eval/ProfileReportWriter.scala @@ -2,7 +2,8 @@ package scalus.uplc.eval import com.github.plokhotnyuk.jsoniter_scala.core.* import com.github.plokhotnyuk.jsoniter_scala.macros.JsonCodecMaker -import scalus.cardano.ledger.{EvaluatorReportConfig, ProfileDestination, ProfileFormat} +import scalus.cardano.ledger.{EvaluatorReportConfig, ProfileDestination, ProfileFormat, ProfileLevel} +import scalus.uplc.Term import scalus.uplc.builtin.platform import scala.util.control.NonFatal @@ -57,6 +58,14 @@ private[scalus] object ProfileReportWriter { * @param onConsole * sink for [[scalus.cardano.ledger.ProfileDestination.Console]] output, so callers keep * their own logger. + * @param uplcTerm + * the evaluated program's term, when the caller has it in annotated form. With + * [[scalus.cardano.ledger.ProfileLevel.Full]], and only alongside at least one rendered + * profile file, it adds a `.uplc.json` source map (the UPLC text plus text-range → + * Scala-source spans) to the report, indexed as format `"uplc"`. It is not a + * [[scalus.cardano.ledger.ProfileFormat]] because those are rendered from [[ProfilingData]], + * which carries no term. Terms decoded from CBOR carry no annotations, and nothing is + * written for them. */ def write( data: ProfilingData, @@ -65,7 +74,8 @@ private[scalus] object ProfileReportWriter { language: String, redeemerTag: String, redeemerIndex: Int, - onConsole: String => Unit + onConsole: String => Unit, + uplcTerm: Option[Term] = None ): Unit = { val key = s"$scriptHash-$redeemerTag-$redeemerIndex" val outputs = report.effectiveProfileOutputs @@ -91,7 +101,25 @@ private[scalus] object ProfileReportWriter { written += formatLabel(out.format) -> path } } - val files = written.result() + val profileFiles = written.result() + // The source map only ever accompanies rendered profile files. A run that wrote none (a + // console-only report) must stay off disk entirely: writing one would create the output + // directory unasked and, worse, replace this script's manifest run – keyed by + // (scriptHash, tag, index) – with an entry listing the source map alone, hiding the + // profile files an earlier run had indexed there. + val uplcFiles = uplcTerm match + case Some(term) + if profileFiles.nonEmpty && report.profile == ProfileLevel.Full && + UplcSourceMapRenderer.hasSourceInfo(term) => + val file = s"$key.uplc.json" + platform.createDirectories(report.outputDir) + platform.writeFile( + reportPath(report, file), + UplcSourceMapRenderer.toJson(UplcSourceMapRenderer.render(term)) + ) + Seq("uplc" -> file) + case _ => Nil + val files = profileFiles ++ uplcFiles if files.nonEmpty then writeManifest( report, diff --git a/scalus-core/shared/src/main/scala/scalus/uplc/eval/UplcSourceMap.scala b/scalus-core/shared/src/main/scala/scalus/uplc/eval/UplcSourceMap.scala new file mode 100644 index 000000000..22c28bf89 --- /dev/null +++ b/scalus-core/shared/src/main/scala/scalus/uplc/eval/UplcSourceMap.scala @@ -0,0 +1,267 @@ +package scalus.uplc.eval + +import com.github.plokhotnyuk.jsoniter_scala.core.* +import com.github.plokhotnyuk.jsoniter_scala.macros.{CodecMakerConfig, JsonCodecMaker} +import org.typelevel.paiges.Doc +import scalus.uplc.{Term, TermPrinter, TermSanitizer} +import scalus.utils.Style + +import scala.collection.mutable + +/** One mapped region of rendered UPLC text. + * + * @param s + * start character offset into [[UplcSourceMap.uplc]], inclusive + * @param e + * end character offset into [[UplcSourceMap.uplc]], exclusive + * @param n + * the node's post-order index in the term tree (children before parent, fields in declaration + * order). Post-order is used because it keeps the indices of an already-rendered program stable + * when that program is later wrapped in `Apply` nodes to apply parameters. + * @param file + * index into [[UplcSourceMap.files]] + * @param sl + * 0-based start line of the source position + * @param sc + * 0-based start column of the source position + * @param el + * 0-based end line of the source position + * @param ec + * 0-based end column of the source position + * @param fn + * index into [[UplcSourceMap.functions]], absent when the node carries no enclosing function + * name + * @note + * lines are 0-based here, as in [[scalus.utils.ScalusSourcePos]]. The `profile.json` report uses + * 1-based lines; do not mix the two. + */ +final case class UplcSpan( + s: Int, + e: Int, + n: Int, + file: Int, + sl: Int, + sc: Int, + el: Int, + ec: Int, + fn: Option[Int] +) + +/** The `.uplc.json` document consumed by the Scalus VS Code extension: the UPLC text of a + * compiled program plus a table mapping ranges of that text back to Scala source positions. + * + * `files` and `functions` are string tables referenced by [[UplcSpan.file]] and [[UplcSpan.fn]]. + * + * @param spans + * the mapped regions, in the order the printer finished the nodes: a node's children precede it, + * and [[UplcSpan.s]] is **not** monotonic. Do not assume document order – a consumer that wants + * to binary-search by offset, or to resolve a cursor to the innermost containing span, must sort + * first (by `s` ascending, then `e` descending, which orders enclosing spans before the spans + * they contain). The spans are properly nested, so that resolution is well defined. The table is + * empty when nothing could be mapped; [[uplc]] is always the full program text. + */ +final case class UplcSourceMap( + schemaVersion: Int, + uplc: String, + files: Seq[String], + functions: Seq[String], + spans: Seq[UplcSpan] +) + +/** Renders a [[scalus.uplc.Term]] to UPLC text together with the text-range to source-position map + * that the UPLC source view needs. + * + * The text comes from the ordinary pretty-printer at the same width as `Term.show`, so the + * rendered UPLC is exactly what every other Scalus output shows. Offsets are recovered by passing + * zero-width markers through the printer ([[org.typelevel.paiges.Doc.zeroWidth]], the same + * mechanism the printer already uses for ANSI styling): markers take part in no layout decision, + * are emitted verbatim into the rendered string, and are stripped afterwards while recording where + * each node's text starts and ends. + */ +object UplcSourceMapRenderer { + + /** Schema version of the `.uplc.json` document. Bump on any incompatible change to its + * shape so consumers (e.g. the Scalus VS Code extension) can detect and reject documents they + * don't understand. + */ + val SchemaVersion = 1 + + /** The width `Term.show` renders at. Used here too, so the artifact holds the familiar text. */ + private val RenderWidth = 80 + + // Always emit `files`/`functions`/`spans`, even when empty, so consumers can index them without + // a presence check. `fn` is still omitted when absent (jsoniter's transientNone default). + private given JsonValueCodec[UplcSourceMap] = + JsonCodecMaker.make(CodecMakerConfig.withTransientEmpty(false)) + + // Control characters that printed UPLC does not contain: names are sanitized identifiers, byte + // strings are hex, numbers are digits. A `string` constant is printed verbatim and could in + // principle hold them; `render` verifies the stripped text against an unmarked render, so such + // a program loses its spans rather than getting corrupted text. + private val MarkerStart = '\u0001' + private val MarkerEnd = '\u0002' + + /** True when at least one node of `term` carries a usable source position, i.e. rendering it + * would produce a non-empty span table. Terms decoded from CBOR carry no annotations at all, + * and a source map for them would map nothing. + */ + def hasSourceInfo(term: Term): Boolean = + !term.annotation.pos.effectivePos.isEffectivelyEmpty || (term match + case Term.LamAbs(_, b, _) => hasSourceInfo(b) + case Term.Apply(f, a, _) => hasSourceInfo(f) || hasSourceInfo(a) + case Term.Force(b, _) => hasSourceInfo(b) + case Term.Delay(b, _) => hasSourceInfo(b) + case Term.Constr(_, as, _) => as.exists(hasSourceInfo) + case Term.Case(a, cs, _) => hasSourceInfo(a) || cs.exists(hasSourceInfo) + case _ => false) + + /** Renders `term` and maps every positioned node to the text it printed. + * + * The rendered text is always identical to `term.show`; this is checked, not merely intended + * (see below). Nodes without an effective source position get no span, and neither do the + * inner `Apply` nodes of an application chain: the printer flattens `[[[f a] b] c]` to + * `[f a b c]` and prints only the outermost `Apply`, whose span covers the whole chain. + * + * A `string` constant is printed verbatim, so a program can contain text indistinguishable + * from a marker. Rather than reason about which spoofs are recoverable, the marked render is + * verified against an unmarked one and the whole span table is dropped when they disagree: a + * source view with no spans is a degraded view, one with wrong text is a wrong one. + */ + def render(term: Term): UplcSourceMap = { + // Name sanitization is what the printer does anyway, and it preserves both the tree + // structure and the annotations, so the post-order indices computed here are equally valid + // for the caller's term. + val sanitized = TermSanitizer.sanitizeNames(term) + + // Post-order index per node. Identity-based, because the tree may contain structurally + // equal subterms. A node instance shared by several parents (as common-subexpression + // elimination produces) is printed once per occurrence, and every occurrence then reports + // the index this numbering gave its last visit. + val postOrder = new java.util.IdentityHashMap[Term, Integer]() + var next = 0 + def index(t: Term): Unit = { + t match + case Term.LamAbs(_, b, _) => index(b) + case Term.Apply(f, a, _) => index(f); index(a) + case Term.Force(b, _) => index(b) + case Term.Delay(b, _) => index(b) + case Term.Constr(_, as, _) => as.foreach(index) + case Term.Case(a, cs, _) => index(a); cs.foreach(index) + case _ => () + postOrder.put(t, next) + next += 1 + } + index(sanitized) + + // Collect the positioned nodes in printing order, wrapping each one's text in markers. + val nodes = mutable.ArrayBuffer.empty[Term] + val doc = TermPrinter.prettySanitized( + sanitized, + Style.Normal, + (t, d) => + if t.annotation.pos.effectivePos.isEffectivelyEmpty then d + else { + val id = nodes.length + nodes += t + Doc.zeroWidth(s"$MarkerStart$id$MarkerEnd") + d + + Doc.zeroWidth(s"$MarkerStart/$id$MarkerEnd") + } + ) + val (uplc, starts, ends) = stripMarkers(doc.render(RenderWidth), nodes.length) + + // Ground truth for the text, produced by the same printer with no markers at all. Markers + // are zero-width, so they cannot change a layout decision, and stripping them must restore + // this string exactly. If it does not, the program's own text collided with the marker + // encoding; report the clean text and no spans. + val plain = TermPrinter + .prettySanitized(sanitized, Style.Normal, (_, d) => d) + .render(RenderWidth) + if uplc != plain then return UplcSourceMap(SchemaVersion, plain, Nil, Nil, Nil) + + val files = mutable.LinkedHashMap.empty[String, Int] + val functions = mutable.LinkedHashMap.empty[String, Int] + def intern(table: mutable.LinkedHashMap[String, Int], s: String): Int = + table.getOrElseUpdate(s, table.size) + + val spans = nodes.indices.flatMap { id => + // Defensive: a node whose markers did not both survive the scan is dropped rather than + // reported at a wrong offset. + if starts(id) < 0 || ends(id) <= starts(id) then None + else { + val t = nodes(id) + val pos = t.annotation.pos.effectivePos + val fn = t.annotation.functionName + Some( + UplcSpan( + s = starts(id), + e = ends(id), + n = postOrder.get(t).intValue, + file = intern(files, pos.file), + sl = pos.startLine, + sc = pos.startColumn, + el = pos.endLine, + ec = pos.endColumn, + fn = if fn.isEmpty then None else Some(intern(functions, fn)) + ) + ) + } + } + + UplcSourceMap(SchemaVersion, uplc, files.keys.toSeq, functions.keys.toSeq, spans) + } + + /** Serializes a source map to the `.uplc.json` bytes, indented for readability. */ + def toJson(map: UplcSourceMap): Array[Byte] = + writeToArray(map, WriterConfig.withIndentionStep(2)) + + /** Removes the markers [[render]] injected, returning the clean text plus, per marker id, the + * offsets into that text where the node's text starts and ends (`-1` when the marker was not + * found). + * + * A marker is consumed only when it parses completely: start char, an optional `/`, digits + * forming an id below `count` that this scan has not seen yet, end char. Anything else is + * content and is copied through, so a control character inside a `string` constant can never + * make this throw or index out of bounds. + * + * It can still mislead this scan, though: a `string` constant holding the exact encoding of a + * marker that is printed *before* the node it names is consumed as that marker, which both + * eats the constant's characters and leaves the genuine marker to be copied through as text. + * Detecting that here would mean re-deriving what the text should have been, so [[render]] + * checks the result against an unmarked render instead and discards the spans on a mismatch. + */ + private def stripMarkers(marked: String, count: Int): (String, Array[Int], Array[Int]) = { + val clean = new StringBuilder(marked.length) + val starts = Array.fill(count)(-1) + val ends = Array.fill(count)(-1) + var i = 0 + while i < marked.length do { + var consumed = false + if marked.charAt(i) == MarkerStart then { + var j = i + 1 + val closing = j < marked.length && marked.charAt(j) == '/' + if closing then j += 1 + var id = 0 + var digits = 0 + while j < marked.length && marked.charAt(j).isDigit && id < count do { + id = id * 10 + (marked.charAt(j) - '0') + digits += 1 + j += 1 + } + if digits > 0 && id < count && j < marked.length && marked.charAt(j) == MarkerEnd + then { + val slots = if closing then ends else starts + if slots(id) < 0 then { + slots(id) = clean.length + consumed = true + i = j + 1 + } + } + } + if !consumed then { + clean.append(marked.charAt(i)) + i += 1 + } + } + (clean.toString, starts, ends) + } +} diff --git a/scalus-core/shared/src/test/scala/scalus/compiler/sir/lowering/FunctionNameAnnotationTest.scala b/scalus-core/shared/src/test/scala/scalus/compiler/sir/lowering/FunctionNameAnnotationTest.scala new file mode 100644 index 000000000..d34ef107e --- /dev/null +++ b/scalus-core/shared/src/test/scala/scalus/compiler/sir/lowering/FunctionNameAnnotationTest.scala @@ -0,0 +1,62 @@ +package scalus.compiler.sir.lowering + +import org.scalatest.funsuite.AnyFunSuite +import scalus.* +import scalus.cardano.ledger.MajorProtocolVersion +import scalus.compiler.{compile, Compile, Options} +import scalus.compiler.sir.TargetLoweringBackend +import scalus.uplc.Term + +@Compile +object FunctionNameAnnotationFixtures { + def triple(x: BigInt): BigInt = x + x + x +} + +/** V3 lowering stamps the enclosing source function into every UPLC term annotation, so tooling + * (the VS Code UPLC source view) can group compiled UPLC by the function it came from. + * + * The UPLC optimizer is disabled here: this suite is about what the lowering emits, not about how + * later passes preserve annotations. + */ +class FunctionNameAnnotationTest extends AnyFunSuite { + + private given Options = Options( + targetLoweringBackend = TargetLoweringBackend.SirToUplcV3Lowering, + targetProtocolVersion = MajorProtocolVersion.vanRossemPV, + optimizeUplc = false + ) + + private def collectFunctionNames(t: Term): Set[String] = { + def go(t: Term, acc: Set[String]): Set[String] = { + val acc1 = + if t.annotation.functionName.nonEmpty then acc + t.annotation.functionName else acc + t match + case Term.LamAbs(_, body, _) => go(body, acc1) + case Term.Apply(f, arg, _) => go(arg, go(f, acc1)) + case Term.Force(b, _) => go(b, acc1) + case Term.Delay(b, _) => go(b, acc1) + case Term.Constr(_, args, _) => args.foldLeft(acc1)((a, x) => go(x, a)) + case Term.Case(arg, cases, _) => cases.foldLeft(go(arg, acc1))((a, x) => go(x, a)) + case _ => acc1 + } + go(t, Set.empty) + } + + test("lowered terms carry the enclosing function name") { + val sir = compile { + def double(x: BigInt): BigInt = x + x + double(21) + } + val term = sir.toUplc() + val names = collectFunctionNames(term) + assert(names.contains("double"), s"expected 'double' in $names") + } + + test("linked top-level module functions carry their simple name") { + val sir = compile { + FunctionNameAnnotationFixtures.triple(7) + } + val names = collectFunctionNames(sir.toUplc()) + assert(names.contains("triple"), s"expected 'triple' in $names") + } +} diff --git a/scalus-core/shared/src/test/scala/scalus/uplc/FillAnnotationsTest.scala b/scalus-core/shared/src/test/scala/scalus/uplc/FillAnnotationsTest.scala new file mode 100644 index 000000000..de4654d03 --- /dev/null +++ b/scalus-core/shared/src/test/scala/scalus/uplc/FillAnnotationsTest.scala @@ -0,0 +1,69 @@ +package scalus.uplc + +import org.scalatest.funsuite.AnyFunSuite +import scalus.utils.ScalusSourcePos + +class FillAnnotationsTest extends AnyFunSuite { + private val pos = ScalusSourcePos("Foo.scala", 10, 0, 10, 20) + private val ann = UplcAnnotation(pos, "validate") + + /** An annotation carrying only a function name, as lowering stamps it on terms that have no + * position of their own. + */ + private def nameOnly(n: String) = UplcAnnotation(ScalusSourcePos.empty, n) + + test("bottom-up fill propagates functionName to spine nodes") { + val leaf = Term.Var(NamedDeBruijn("x"), ann) + val spine = Term.Force(Term.Delay(leaf)) // spine has empty annotations + val (filled, _) = spine.fillEmptyAnnotationsBottomUp + assert(filled.annotation.functionName == "validate") + assert(filled.annotation.pos == pos) + } + + test("top-down fill propagates functionName downward") { + val inner = Term.Delay(Term.Var(NamedDeBruijn("x"))) + val filled = inner.fillEmptyAnnotationsTopDown(ann) + assert(filled.annotation.functionName == "validate") + val Term.Delay(v, _) = filled: @unchecked + assert(v.annotation.functionName == "validate") + } + + test("existing annotations are never overwritten") { + val other = UplcAnnotation(ScalusSourcePos("Bar.scala", 1, 0, 1, 5), "other") + val leaf = Term.Var(NamedDeBruijn("x"), other) + val filled = leaf.fillEmptyAnnotationsTopDown(ann) + assert(filled.annotation == other) + } + + test("bottom-up fills a missing position without touching an existing functionName") { + val leaf = Term.Var(NamedDeBruijn("x"), ann) + val spine = Term.Apply(leaf, Term.Const(Constant.Integer(1)), nameOnly("helper")) + val (filled, rep) = spine.fillEmptyAnnotationsBottomUp + assert(filled.annotation == UplcAnnotation(pos, "helper")) + assert(rep == ann) // the representative is the whole annotation of the positioned leaf + } + + test("top-down fills a missing position without touching an existing functionName") { + val leaf = Term.Var(NamedDeBruijn("x"), nameOnly("helper")) + val filled = leaf.fillEmptyAnnotationsTopDown(ann) + assert(filled.annotation == UplcAnnotation(pos, "helper")) + } + + test("top-down: an inner functionName wins for its own subtree") { + val inner = Term.Delay( + Term.Var(NamedDeBruijn("x")), + UplcAnnotation(ScalusSourcePos("Bar.scala", 1, 0, 1, 5), "inner") + ) + val filled = Term.Force(inner).fillEmptyAnnotationsTopDown(ann) + assert(filled.annotation == ann) // the Force spine node inherits the root annotation + val Term.Force(Term.Delay(v, _), _) = filled: @unchecked + assert(v.annotation.functionName == "inner") + } + + test("fills leave a fully-annotated term untouched (identity, no realloc)") { + val t = + Term.Apply(Term.Var(NamedDeBruijn("x"), ann), Term.Const(Constant.Integer(0), ann), ann) + assert(t.fillEmptyAnnotationsBottomUp._1 eq t) + assert(t.fillEmptyAnnotationsTopDown(UplcAnnotation.empty) eq t) + } +} diff --git a/scalus-core/shared/src/test/scala/scalus/uplc/PrettyDecoratedTest.scala b/scalus-core/shared/src/test/scala/scalus/uplc/PrettyDecoratedTest.scala new file mode 100644 index 000000000..f22c2f05d --- /dev/null +++ b/scalus-core/shared/src/test/scala/scalus/uplc/PrettyDecoratedTest.scala @@ -0,0 +1,28 @@ +package scalus.uplc + +import org.scalatest.funsuite.AnyFunSuite +import scalus.utils.Style +import scalus.uplc.DefaultFun.AddInteger + +class PrettyDecoratedTest extends AnyFunSuite { + private val term = Term.Apply( + Term.Apply(Term.Builtin(AddInteger), Term.Const(Constant.Integer(1))), + Term.Const(Constant.Integer(2)) + ) + + test("identity decorator renders identically to pretty") { + val sanitized = TermSanitizer.sanitizeNames(term) + val doc = TermPrinter.prettySanitized(sanitized, Style.Normal, (_, d) => d) + assert(doc.render(80) == term.show) + } + + test("decorator wraps every printed node") { + var count = 0 + val sanitized = TermSanitizer.sanitizeNames(term) + TermPrinter + .prettySanitized(sanitized, Style.Normal, (_, d) => { count += 1; d }) + .render(80) + // builtin + 2 consts + outermost Apply of the flattened chain = 4 + assert(count == 4) + } +} diff --git a/scalus-core/shared/src/test/scala/scalus/uplc/eval/UplcSourceMapRendererTest.scala b/scalus-core/shared/src/test/scala/scalus/uplc/eval/UplcSourceMapRendererTest.scala new file mode 100644 index 000000000..c1dd07346 --- /dev/null +++ b/scalus-core/shared/src/test/scala/scalus/uplc/eval/UplcSourceMapRendererTest.scala @@ -0,0 +1,187 @@ +package scalus.uplc.eval + +import org.scalatest.funsuite.AnyFunSuite +import scalus.* +import scalus.compiler.compile +import scalus.uplc.* +import scalus.uplc.DefaultFun.AddInteger +import scalus.utils.ScalusSourcePos + +class UplcSourceMapRendererTest extends AnyFunSuite { + private val posA = ScalusSourcePos("/src/Foo.scala", 10, 2, 10, 7) + private val posB = ScalusSourcePos("/src/Foo.scala", 12, 4, 12, 9) + private val annA = UplcAnnotation(posA, "validate") + private val annB = UplcAnnotation(posB, "") + + private val term = Term.Apply( + Term.Apply(Term.Builtin(AddInteger, annA), Term.Const(Constant.Integer(1), annB)), + Term.Const(Constant.Integer(2)), + annA + ) + + test("uplc text equals plain show (markers fully stripped)") { + val map = UplcSourceMapRenderer.render(term) + assert(map.uplc == term.show) + } + + test("spans point at the printed node text") { + val map = UplcSourceMapRenderer.render(term) + val builtinSpan = + map.spans.find(sp => map.uplc.substring(sp.s, sp.e) == "(builtin addInteger)").get + assert(map.files(builtinSpan.file) == "/src/Foo.scala") + assert( + builtinSpan.sl == 10 && builtinSpan.sc == 2 && builtinSpan.el == 10 && builtinSpan.ec == 7 + ) + assert(builtinSpan.fn.map(map.functions) == Some("validate")) + } + + test("nodes without positions produce no spans") { + val map = UplcSourceMapRenderer.render(term) + // the '2' const has an empty annotation + assert(!map.spans.exists(sp => map.uplc.substring(sp.s, sp.e) == "(con integer 2)")) + } + + test("a node with a position but no function name gets a span without fn") { + val map = UplcSourceMapRenderer.render(term) + val constSpan = + map.spans.find(sp => map.uplc.substring(sp.s, sp.e) == "(con integer 1)").get + assert(constSpan.fn.isEmpty) + assert(constSpan.sl == 12 && constSpan.sc == 4) + } + + test("spans nest and offsets are within bounds") { + val map = UplcSourceMapRenderer.render(term) + assert(map.spans.nonEmpty) + map.spans.foreach { sp => + assert(sp.s >= 0 && sp.e <= map.uplc.length && sp.s < sp.e) + } + // the root application encloses every other span + val root = map.spans.maxBy(sp => sp.e - sp.s) + map.spans.foreach(sp => assert(sp.s >= root.s && sp.e <= root.e)) + } + + test("post-order indices are stable under Apply wrapping") { + // How a compiled script gets its parameters applied: the script is a lambda and the + // wrapper Apply is added on top, so every node of the original term is still printed. + val wrapped = Term.Apply(Term.LamAbs("p", term), Term.Const(Constant.Integer(3))) + val base = UplcSourceMapRenderer.render(term) + val wrap = UplcSourceMapRenderer.render(wrapped) + val baseByPos = base.spans.map(sp => (sp.sl, sp.sc, sp.n)).toSet + assert(baseByPos.size == base.spans.size) + // every base span keeps its node index in the wrapped program + baseByPos.foreach { case (sl, sc, n) => + assert( + wrap.spans.exists(sp => sp.sl == sl && sp.sc == sc && sp.n == n), + s"($sl, $sc, $n) missing from ${wrap.spans}" + ) + } + } + + test("an Apply-rooted term absorbed into an enclosing chain keeps its children's indices") { + // The printer flattens application chains, so wrapping an Apply in another Apply drops + // the inner node's own span. Every other node keeps its post-order index. + val wrapped = Term.Apply(term, Term.Const(Constant.Integer(3))) + val base = UplcSourceMapRenderer.render(term) + val wrap = UplcSourceMapRenderer.render(wrapped) + def spanOf(map: UplcSourceMap, text: String): Option[UplcSpan] = + map.spans.find(sp => map.uplc.substring(sp.s, sp.e) == text) + assert( + spanOf(base, "(builtin addInteger)").map(_.n) == spanOf(wrap, "(builtin addInteger)") + .map(_.n) + ) + assert(spanOf(base, "(con integer 1)").map(_.n) == spanOf(wrap, "(con integer 1)").map(_.n)) + assert(wrap.uplc == wrapped.show) + } + + test("hasSourceInfo") { + assert(UplcSourceMapRenderer.hasSourceInfo(term)) + assert(!UplcSourceMapRenderer.hasSourceInfo(Term.Const(Constant.Integer(1)))) + assert( + UplcSourceMapRenderer.hasSourceInfo(Term.LamAbs("x", Term.Builtin(AddInteger, annA))) + ) + } + + test("a term without any source info renders to text and no spans") { + val plain = Term.Apply(Term.Builtin(AddInteger), Term.Const(Constant.Integer(1))) + val map = UplcSourceMapRenderer.render(plain) + assert(map.uplc == plain.show) + assert(map.spans.isEmpty) + assert(map.files.isEmpty && map.functions.isEmpty) + } + + test("markers survive multi-line layout") { + // Force the printer to break the application across lines. + val long = (1 to 40).foldLeft[Term](Term.Builtin(AddInteger, annA)) { (acc, i) => + Term.Apply(acc, Term.Const(Constant.Integer(i), annB), annA) + } + val map = UplcSourceMapRenderer.render(long) + assert(map.uplc == long.show) + assert(map.uplc.contains("\n")) + map.spans.foreach { sp => + assert(sp.s >= 0 && sp.e <= map.uplc.length && sp.s < sp.e) + } + assert(map.spans.exists(sp => map.uplc.substring(sp.s, sp.e) == "(con integer 40)")) + } + + test("a string constant that spoofs a marker degrades to no spans, never to corrupt text") { + // UPLC string constants are printed verbatim, so a constant can contain the renderer's + // marker characters. Here the constant spoofs the *start* marker of node id 2 and is + // printed before that node, so the scanner's "already recorded" guard cannot help: it + // would record a bogus offset, drop the constant's characters and emit spans that + // partially overlap. The renderer must notice and give up the spans rather than hand the + // view corrupted UPLC. + val markerStart = 0x01.toChar + val markerEnd = 0x02.toChar + val spoof = Term.Const(Constant.String(s"${markerStart}2$markerEnd"), annB) + val term = Term.Apply( + Term.Apply(Term.Builtin(AddInteger, annA), spoof, annB), + Term.Const(Constant.Integer(7), annB), + annA + ) + val map = UplcSourceMapRenderer.render(term) + assert(map.uplc == term.show) + assert(map.spans.isEmpty) + } + + test("json round-trip") { + val map = UplcSourceMapRenderer.render(term) + val json = new String(UplcSourceMapRenderer.toJson(map), "UTF-8") + assert(json.contains("\"schemaVersion\": 1") || json.contains("\"schemaVersion\":1")) + assert(json.contains("\"uplc\"")) + assert(json.contains("\"spans\"")) + assert(json.contains("\"fn\"")) + } + + test("json always carries the string tables, even when empty") { + val plain = Term.Const(Constant.Integer(1)) + val json = new String(UplcSourceMapRenderer.toJson(UplcSourceMapRenderer.render(plain))) + assert(json.contains("\"files\"")) + assert(json.contains("\"functions\"")) + assert(json.contains("\"spans\"")) + } + + test("invariant holds for a compiled program") { + val sir = compile { + def double(x: BigInt): BigInt = x + x + double(21) + } + val t = sir.toUplc() + val map = UplcSourceMapRenderer.render(t) + assert(map.uplc == t.show) + assert(map.spans.nonEmpty) + assert(map.functions.exists(_.endsWith("double")), s"functions: ${map.functions}") + map.spans.foreach { sp => + assert(sp.s >= 0 && sp.e <= map.uplc.length && sp.s < sp.e) + assert(sp.file >= 0 && sp.file < map.files.size) + sp.fn.foreach(i => assert(i >= 0 && i < map.functions.size)) + } + // Spans are properly nested: the view resolves a cursor to the innermost span containing + // it, which is only well defined when no two spans partially overlap. + var open = List.empty[UplcSpan] + map.spans.sortBy(sp => (sp.s, -sp.e)).foreach { sp => + open = open.dropWhile(_.e <= sp.s) + open.headOption.foreach(o => assert(sp.e <= o.e, s"partial overlap: $o and $sp")) + open = sp :: open + } + } +} diff --git a/scalus-testkit/shared/src/main/scala/scalus/testing/kit/ScalusTest.scala b/scalus-testkit/shared/src/main/scala/scalus/testing/kit/ScalusTest.scala index 8c8a10f51..241ae0d90 100644 --- a/scalus-testkit/shared/src/main/scala/scalus/testing/kit/ScalusTest.scala +++ b/scalus-testkit/shared/src/main/scala/scalus/testing/kit/ScalusTest.scala @@ -22,6 +22,9 @@ import scalus.compiler.sir.SIR import scalus.uplc.* import scalus.uplc.eval.* +// `prelude.Option.*` above shadows the standard Some; alias it for the off-chain APIs that take one. +import scala.Some as ScalaSome + trait ScalusTest extends ArbitraryInstances, Assertions { protected def plutusVM: PlutusVM = PlutusVM.makePlutusV3VM() protected given PlutusVM = plutusVM @@ -90,6 +93,10 @@ trait ScalusTest extends ArbitraryInstances, Assertions { * calling this asks for the full set, and `SCALUS_PROFILE` / `SCALUS_PROFILE_OUT` / * `SCALUS_DUMP_DIR` override it as they do for the ledger. Manifest entries are merged by * (scriptHash, tag, index), so several profiled tests accumulate rather than overwrite. + * + * A `.uplc.json` source map is written alongside them when this program still carries + * the compiler's source annotations, i.e. it was compiled in this process rather than + * decoded from CBOR. */ def runWithProfileReport(scriptContext: ScriptContext)(using vm: PlutusVM): Result = { val result = runWithProfile(scriptContext) @@ -101,7 +108,8 @@ trait ScalusTest extends ArbitraryInstances, Assertions { Language.PlutusV3.toString, redeemerTag(scriptContext.scriptInfo), 0, - println + println, + ScalaSome(self.term) ) } result